name: inverse layout: true class: center, middle, inverse background-image: url(https://camo.githubusercontent.com/98ed65187a84ecf897273d9fa18118ce45845057/68747470733a2f2f7261772e6769746875622e636f6d2f676f6c616e672d73616d706c65732f676f706865722d766563746f722f6d61737465722f676f706865722e706e67) --- # Writing microservices in Go --- name: inverse layout: true class: center, middle, inverse --- ## What is it and why should I be using it? --- layout: false .left-column[ ## What is it? ] .right-column[ A simple, statically typed, C-like, 'less is more', batteries included programming language invented by Rob Pike and Ken Thompson while working at Google: - Opinionated - Performance oriented - System language - Garbage collected - Type inference - Higher-order functions - Composition over inheritance - No generics - CSP based concurrency - Extensive standard lib - GIT based dependency mgmt - Nice tooling ] --- layout: false .left-column[ ## What is it? ## Why use it? ] .right-column[ Simple tool for simple problems: - SELECT/marshal/HTTP kind of services - CPU intensive tasks - CLI commands ] --- template: inverse ## How does it work? --- name: how .left-column[ ## How does it work? ### - SELECTing ] .right-column[ First import a db driver ```go import ( "database/sql" _ "github.com/go-sql-driver/mysql" ) ``` Then open a db ```go db, err := sql.Open("mysql", "user:password@tcp(127.0.0.1:3306)/hello") if err != nil { log.Fatal(err) } defer db.Close() ``` Query and parse the rows ```go age := 27 rows, err := db.Query("SELECT name FROM users WHERE age=?", age) if err != nil { log.Fatal(err) } defer rows.Close() for rows.Next() { var name string if err := rows.Scan(&name); err != nil { log.Fatal(err) } fmt.Printf("%s is %d\n", name, age) } if err := rows.Err(); err != nil { log.Fatal(err) } ``` ] --- name: how .left-column[ ## How does it work? ### - SELECTing ### - JSON Marshalling ] .right-column[ Import needed package ```go import ( "encoding/json" ) ``` JSON annotate your structs ```go type Movie struct { Title string Year int `json:"released"` Color bool `json:"color,omitempty"` Actors []string } ``` Create some Movies ```go var movies = []Movie{ {Title: "Casablanca", Year: 1942, Color: false, Actors: []string{"Humphrey Bogart", "Ingrid Bergman"}}, {Title: "Cool Hand Luke", Year: 1967, Color: true, Actors: []string{"Paul Newman"}}, {Title: "Bullitt", Year: 1968, Color: true, Actors: []string{"Steve McQueen", "Jacqueline Bisset"}} } ``` Marshal in JSON ```go data, err := json.Marshal(movies) if err != nil { log.Fatalf("JSON marshaling failed: %s", err) } fmt.Printf("%s\n", data) ``` ] --- name: how .left-column[ ## How does it work? ### - SELECTing ### - JSON Marshalling ### - HTTP server ] .right-column[ Needed imports ```go import ( "fmt" "log" "net/http" ) ``` Define an handler fn ```go // handler echoes the Path component of the request URL r. func handler(w http.ResponseWriter, r *http.Request) { fmt.Fprintf(w, "URL.Path = %q\n", r.URL.Path) } ``` Serve your handler ```go func main() { http.HandleFunc("/", handler) // each request calls handler log.Fatal(http.ListenAndServe("localhost:8000", nil)) } ``` ] --- template: inverse ## How do you test? --- name: testing .left-column[ ## How do you test? ### - Ginkgo ] .right-column[ .center[] Ginkgo provides a fluent DSL to write BDD style unit tests ] --- name: testing .left-column[ ## How do you test? ### - Ginkgo ### - Mountebank ] .right-column[ .center[] Ginkgo provides a fluent DSL to write BDD style unit tests .mountebank-img.center[] Mountebank enables easy test doubles over the wire enabling end-to-end integration testing ] --- name: ginkgo .left-column[ ## Unit testing with Ginkgo ### - Quick start ] .right-column[ Bootstrapping a test suite ```go ginkgo bootstrap ``` This will create all the needed scaffolding ```go package service_test import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" "testing" ) func TestService(t *testing.T) { RegisterFailHandler(Fail) RunSpecs(t, "Service Suite") } ``` Go enforces 'black box` testing Unit tests *are not* in the same package of tested code Orthogonal test code resulting in less brittle test suites ] --- name: ginkgo .left-column[ ## Unit testing with Ginkgo ### - Quick start ### - Nice tooling ] .right-column[ Automatic generation of test skeleton ```shell $ ginkgo generate binders ├── service │ ├── binders.go │ ├── binders_test.go │ ├── export_test.go │ ├── service.go │ ├── service_integration_test.go │ └── service_suite_test.go ``` A test skeleton gets generated in the same folder *but* in a different package Running a test suite ```shell ginkgo Running Suite: Service Suite ============================ Random Seed: 1490796972 Will run 10 of 10 specs •••••••••• Ran 10 of 10 Specs in 0.001 seconds SUCCESS! -- 10 Passed | 0 Failed | 0 Pending | 0 Skipped PASS Ginkgo ran 1 suite in 978.257274ms Test Suite Passed ``` ] --- name: ginkgo .left-column[ ## Unit testing with Ginkgo ### - Quick start ### - Nice tooling ### - Fluent DSL ] .right-column[ Setup / Teardown ```go BeforeEach(func() { ... }) AfterEach(func() { ... }) ``` Descriptive BDD style ```go var _ = Describe("UserStore", func() { var testDb *sql.DB var userStore UserStore BeforeEach(func() { testDb = mkTestDb("user_store_test.dml.sql") userStore = NewUserStore(testDb) }) AfterEach(func() { testDb.Close() }) Context("when trying to load an user by its guid", func() { It("should find and load an user from its guid iff is on the db", func() { Expect(userStore.LoadUserIdByGuid("96f9efe0-30c2-012c-8f71-0015177442e6")) .Should(Equal(2)) Expect(userStore.LoadUserIdByGuid("9472ae70-30c2-012c-8f71-0015177442e6")) .Should(Equal(3)) }) It("should return an error if no user is present with the given guid", func() { _, err := userStore.LoadUserIdByGuid("367aee50-2fb4-012c-4444-001517734d92") Expect(err).To(HaveOccurred()) }) }) }) ``` ] --- name: ginkgo .left-column[ ## Testing with Mountebank ### - What is it? ] .right-column[ Enables the creation of 'test doubles' Test Double == A fake version of a remote dependency (e.g. another service you depend upon) .imposter-img.center[] Your service will run against a set of imposters mocking a predefined set of behaviors Test against - Slow responses - Different failure scenarios (timeouts, 500s, ...) - Invalid responses - Dynamically generated behaviour (in Js) ] --- name: ginkgo .left-column[ ## Testing with Mountebank ### - What is it? ### - Define an Imposter ] .right-column[ Initial Imposter in 'proxyOnce' mode ```json "stubs": [ { "responses": [ { "proxy": { "to": "http://origin-server.com", "mode": "proxyOnce", "predicateGenerators": [ { "matches": { "method": true, "path": true, "query": true } } ] } } ] } ] ``` Run the first time and record the interaction ] --- name: ginkgo .left-column[ ## Testing with Mountebank ### - What is it? ### - Define an Imposter ] .right-column[ Imposter with recorded interaction ```json "stubs": [ { "predicates": [{ "deepEquals': { "method": "GET", "path": "/test", "query": { "q": "mountebank" } } }], "responses": [ { "is": ...saved response } ] } { "responses": [ { "proxy": { "to": "http://origin-server.com", "mode": "proxyOnce", "predicateGenerators": [ { "matches": { "method": true, "path": true, "query": true } } ] } } ] } ] ``` Extract the Imposter configuration and use it in your integration tests ] --- template: inverse ## Yeah, but how I deploy this thing? --- name: deploy .center[] .left-column[ ## How to deploy? ### - Build ] .right-column[ Go will bake a ready to run executable for the target platform ```go go install github.com/gilt/svc-shipping-address ``` Statically linked Just copy over the binary and run - Only if same architecture ] --- name: deploy .center[] .left-column[ ## How to deploy? ### - Build ### - Docker image ] .right-column[ Build the executable *inside* the official Docker Go image ```go FROM golang ADD . /go/src/github.com/gilt/svc-shipping-address ADD ./config /go/config RUN go install github.com/gilt/svc-shipping-address ``` Publish your port ```go ENTRYPOINT ./bin/svc-shipping-address EXPOSE 6501 ``` ] --- name: deploy .center[] .left-column[ ## How to deploy? ### - Build ### - Docker image ### - Push to Amazon ] .right-column[ Tag & push to ECR ```go docker tag svc-shipping-address:$(PREVIOUS_HASH) 905260852223.dkr.ecr.us-east-1.amazonaws.com/svc-shipping-address:$(PREVIOUS_HASH) docker push 905260852223.dkr.ecr.us-east-1.amazonaws.com/svc-shipping-address:$(PREVIOUS_HASH) ``` ] --- name: deploy .center[] .left-column[ ## How to deploy? ### - Deploy to ECS ] .right-column[ There is a bit of setup up front to build an ECS cluster Your Go service will be an ECS task Configure an ECS service that uses your task Run the ECS service! ] --- name: ecs-service .left-column[ ## Running in ECS ### - Service configuration ] .right-column[ ```json { "family": "svc-shipping-address", "containerDefinitions": [ { "name": "svc-shipping-address", "image": "905260852223.dkr.ecr.us-east-1.amazonaws.com/svc-shipping-address:c475fef", "portMappings": [ { "containerPort": 6501 } ], "cpu": 100, "memory": 64, "memoryReservation": 8, "environment": [ { "name": "ENVIRONMENT", "value": "production" } ], "mountPoints": [ { "sourceVolume": "application-logs", "containerPath": "/go/log", "readOnly": false } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "awslogs-svc-shipping-address", "awslogs-region": "us-east-1" } }, "ulimits": [ { "name": "nofile", "softLimit": 1024, "hardLimit": 102400 } ] } ], "volumes": [ { "name": "application-logs", "host": { "sourcePath": "/var/log/svc-shipping-address" } } ] } ``` ] --- name: ecs-service .left-column[ ## Running in ECS ### - Service configuration ### - Live ] .right-column[ Running Service .center[] Tasks .center[] ] --- name: monitoring .left-column[ ## Running in ECS ### - Service configuration ### - Live ### - Monitoring ] .right-column[ NewRelic is fully supported .center[] ] --- name: monitoring .left-column[ ## Running in ECS ### - Service configuration ### - Live ### - Monitoring ] .right-column[ NewRelic is fully supported .center[] ] --- name: monitoring .left-column[ ## Running in ECS ### - Service configuration ### - Live ### - Monitoring ### - Distributed tracing ] .right-column[ Making sense of what is going on Exponentially hard in a microservices environment NewRelic does not help that much Other solutions - Zipkin (Twitter) - Dapper (Google) Support for propagating a ```Context``` across API boundaries - Baked in the ```context``` package - Generate a ```Context``` - Accept, decorate and pass on an incoming ```Context``` - Collect all this ```Context```(s) and extract a picture of your system Examples - [Phosphor](https://phosphor.io/) - [Hailo Trace](https://sudo.hailoapp.com/services/2015/03/09/journey-into-a-microservice-world-part-3/) ] --- template: inverse ## Cool stuff --- name: cool-stuff .left-column[ ## Cool stuff ### - Revel ] .right-column[ .big-img.center[] ] --- name: cool-stuff .left-column[ ## Cool stuff ### - Revel ### - Gorm ] .right-column[ .big-img.center[] ] --- name: cool-stuff .left-column[ ## Cool stuff ### - Revel ### - Gorm ### - CockroachDB ] .right-column[ .big-img.center[] ] --- name: cool-stuff .left-column[ ## Cool stuff ### - Revel ### - Gorm ### - CockroachDB ### - Consul ] .right-column[ .big-img.center[] ] --- template: inverse ## You need only this book .center[] --- template: inverse ## Questions? .center[]