Go Hex Arch

In an earlier post I wrote about how to structure Go programs to be leveraged by both a FaaS and a standalone application. This simple application showed the power of what a Hex architecture can do by limiting the need to rewrite logic when an external resource has changed. In that example we looked at changing communication between an HTTP call from an AWS Lambda to a direct application call.

This pattern can be used in many different ways to help ensure that the business logic remains in tact while other architecture decisions may change. By writing applications specific to certain technologies the portability of the code will decrease. In this day and age many people don’t care about being tied to a specific cloud vendor or being coupled to a certain database. I believe this is short sighted. It is not your job as a developer to make those decisions all of the time and it is incumbent upon you to make sure that you can meet the needs of the business at any time.

Domain Driven Design has been a huge influence on design patterns and implementations schemes in technology in recent years. It can be attributed to the structuring of microservices and separation of duties in organizations. The concept being that smaller applications specific to a domain, which has an understanding of its business and specific elements of the business, know their needs and implementation structures.

Thinking in this pattern we can understand that core business logic is the key towards any sort of development and that its the job of the developer to write the code independent of how inputs and outputs will be transmitted. The key is to structure your code in such a way that it has the inputs and outputs abstracted to protect the core business logic, and that’s where Hex Architecture comes in.

Hex architecture

I’ve briefly covered Hexagon (Clean) architecture before but I believe it’s important to go a little deeper to understand some of the thoughts and reasonings behind this design. Later I will show you how we can structure our code to easily leverage this design in Go but for now let’s cover the basics.

At the core of all applications is business logic. Whether you are Twitter or a large bank there is some form of logic that transforms an input into an output. The point of Hexagonal Architecture is to insulate that logic from the inputs and outputs. By doing this you decrease the possibility of code duplication or bugs in the code by ensuring that the logic is in one place.

The core business logic should have an abstraction layer above it for any inputs to call and consume abstractions below for any sort of output that is necessary. This will allow the logic to not be tied to a specific system for output and not have to accommodate for all types of input. These layers of abstractions are often interfaces and provide the ability to provide different implementations without changing the core logic.

The gains from this architecture are mostly the ability to reuse business logic in different environments. In the one post I wrote we allowed for the same logic to be used given two separate calling implementations (Lambda versus Server). This was because we leveraged Go’s http interface and found a special library that translates the two.

Quick Note On Dependency Injection

Go does not have a set of rules on how you should implement a system. Patterns emerge over time and one that I’ve migrated towards in recent months is the dependency injection model. This felt comfortable for me given my background in Java and Spring which leveraged dependency injection quite heavily.

Peter Bourgon wrote a blog post on “A theory of modern go” which argues agains the use of globals for service and database registration. The “magic” as he describes it tends to confuse the consumer of a library and relies on strange configuration which often becomes hard to test.

So in the following example I will leverage this model and show how it combines well with a Hex architecture.

Go Example

Disclaimer: This original idea came from a talk about this talk on Domain Driven Design in Go and his code examples using Gokit

Domain

Our Project Management department wants to implement an in house ticketing solution. The current ticketing system is slow, cumbersome, and not written in Go. So they want you to write a new system called Gira. Tickets must have a title, description, assignee, creator, some information around creation and update, and a status.

For the initial phase they just want to be able to create and get tickets (all and by id). Since we are the only company using it we don’t have to worry about others and for the time being we don’t have to worry about updating because we just want to get a POC out there (yay agile!).

You are chosen to be the developer and given carte blanche on technologies. Only tech requirements are it needs to be written in Go and have an HTTP interface.

Model

So first let’s put on our Domain Driven Design hats. We have one domain: Tickets. So we’ll create a directory for that domain.

Next we need to come up with a model that describes the tickets based on the requirements. Luckily our PM was pretty explicit on the fields. We will include encoding information just in case. It’s important to realize that this is not a way for us to explicitly tie to one technology or another, luckily Go is flexible with this field and will allow us to control how this model is encoded or decoded by calling methods, much like an interface.

package ticket

import "time"

type Ticket struct {
	ID          string    `json:"id" db:"id"`
	Creator     string    `json:"creator" db:"creator"`
	Assigned    string    `json:"assigned" db:"assigned"`
	Title       string    `json:"title" db:"title"`
	Description string    `json:"description" db:"description"`
	Status      string    `json:"status" db:"status"`
	Points      int       `json:"points" db:"points"`
	Created     time.Time `json:"created" db:"created"`
	Updated     time.Time `json:"updated" db:"updated"`
	Deleted     time.Time `json:"deleted" db:"deleted"`
}

Now that we have a model we can start to build logic around it. We know that we need a way of fetching and saving the model. I know this comment is extremely vague, and it should be. Hexagonal architecture requires all actions on the domain to be inward facing towards the domain. So in this case we aren’t sure what will happen to the data other than we need a way to access and save it. Our PM told us that we need to be able to create and get tickets. So we’ll define a repository that allows us to FindAll, FindById, and Create.

package ticket

type TicketRepository interface {
	Create(ticket *Ticket) error
	FindById(id string) (*Ticket, error)
	FindAll() ([]*Ticket, error)
}

Notice here that this is again within the domain package and is just an interface. Interfaces provide the abstraction layers needed for the domain to function without implementation details that will be built by the running application.

Service

So we have an abstract repository to allow us to get and save domains, now we need a way to apply business logic. The business logic here is pretty straight forward because the use case we are building around is quite simple.

package ticket

import (
	"github.com/google/uuid"
	"time"
)

type TicketService interface {
	CreateTicket(ticket *Ticket) error
	FindTicketById(id string) (*Ticket, error)
	FindAllTickets() ([]*Ticket, error)
}

type ticketService struct {
	repo TicketRepository
}

func NewTicketService(repo TicketRepository) TicketService {
	return &ticketService{
		repo,
	}
}

func (s *ticketService) CreateTicket(ticket *Ticket) error {
	ticket.ID = uuid.New().String()
	ticket.Created = time.Now()
	ticket.Updated = time.Now()
	ticket.Status = "open"
	return s.repo.Create(ticket)
}


func (s *ticketService) FindTicketById(id string) (*Ticket, error){
	return s.repo.FindById(id)
}

func (s *ticketService) FindAllTickets() ([]*Ticket, error) {
	return s.repo.FindAll()
}

Mostly the methods are passthrough with the exception of Create. Here we want to make sure that we always use a UUID string for the id and make sure that we use the system date for created and updated fields. This removes any sort of control from the callers end on those fields. If this were more detailed we may also do some custom error handling or validation.

Caution! Here be dragons! The Hex architecture allows us to abstract a lot to protect business level logic but we need to be careful on where we make our assumptions. In our repository we have a method called FindAll, which tells whoever will be implementing our interface that we just need to return all entities. But if the business wants some specific logic around the corresponding service method we may be in trouble. For example: if the business wants tickets returned in descending order by created date we will need to change our service or change the repository interface to be “FindAllOrderByCreatedDesc”. It’s up to you how this should be implemented but my warning is to make sure you don’t make any assumptions, especially when it comes to defining your interface.

Handlers

If you look at the hexagon diagram again you’ll see the adapter layer. In this example we’ll place this in the same package as the domain. This is a grey area because technically this is not part of the domain but is an adapter so it could be part of a handler package instead. This is a consideration for when you build larger applications.

package ticket

import (
	"encoding/json"
	"github.com/gorilla/mux"
	"net/http"
)

type TicketHandler interface {
	Get(w http.ResponseWriter, r *http.Request)
	GetById(w http.ResponseWriter, r *http.Request)
	Create(w http.ResponseWriter, r *http.Request)
}

type ticketHandler struct {
	ticketService TicketService
}

func NewTicketHandler(ticketService TicketService) TicketHandler {
	return &ticketHandler{
		ticketService,
	}
}

func (h *ticketHandler) Get(w http.ResponseWriter, r *http.Request) {
	tickets, _ := h.ticketService.FindAllTickets()

	response, _ := json.Marshal(tickets)

	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusOK)
	_, _ = w.Write(response)
}

func (h *ticketHandler) GetById(w http.ResponseWriter, r *http.Request) {
	vars := mux.Vars(r)
	id := vars["id"]
	ticket, _ := h.ticketService.FindTicketById(id)

	response, _ := json.Marshal(ticket)

	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusOK)
	_, _ = w.Write(response)
}

func (h *ticketHandler) Create(w http.ResponseWriter, r *http.Request) {

	var ticket Ticket
	decoder := json.NewDecoder(r.Body)
	_ = decoder.Decode(&ticket)
	_ = h.ticketService.CreateTicket(&ticket)

	response, _ := json.Marshal(ticket)
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusOK)
	_, _ = w.Write(response)

}

The handler allows us to push the information towards the center of the Hexagon, meaning that the inputs and outputs are solely based on the business logic. These layers of communication toward the domain are called “ports”. You can imagine that if someone wanted to create a listener on a queue or invoke a GRPC call the setups would be similar and you would expect the same response from the business layer.

Database Adapter

Now we have an architecture decision to make, which database do we use? You do some research and decide you like the performance of Redis and really believe that scaling shouldn’t be an issue. You also figure that since this is internal, we don’t care about the durability of data. So you go ahead and start writing.

Redis

Redis for those who don’t know is a really big hash table. It’s speed is unmatched but runs (moslty) on memory so there is a possiblity to lose your data. In this example we’ll set up a new port for Redis that fulfils the TicketRepository interface we created. We’ll place this in the database/redis directory.

package redis

import (
	"encoding/json"
	"github.com/go-redis/redis"
	"hex-example/ticket"
)

const table = "tickets"

type ticketRepository struct {
	connection *redis.Client
}


func NewRedisTicketRepository(connection *redis.Client) ticket.TicketRepository {
	return &ticketRepository{
		connection,
	}
}

func (r *ticketRepository)	Create(ticket *ticket.Ticket) error {
	encoded, err := json.Marshal(ticket)

	if err != nil {
		return err
	}

	r.connection.HSet(table, ticket.ID, encoded) //Don't expire
	return nil
}

func (r *ticketRepository) FindById(id string) (*ticket.Ticket, error) {
	b, err := r.connection.HGet(table, id).Bytes()

	if err != nil {
		return nil, err
	}

	t := new(ticket.Ticket)
	err = json.Unmarshal(b, t)

	if err != nil {
		return nil, err
	}

	return t, nil
}

func (r *ticketRepository) FindAll() (tickets []*ticket.Ticket, err error) {
	ts := r.connection.HGetAll(table).Val()
	for key, value := range ts {
		t := new(ticket.Ticket)
		err = json.Unmarshal([]byte(value), t)

		if err != nil {
			return nil, err
		}

		t.ID = key
		tickets = append(tickets, t)
	}
	return tickets, nil
}

As you can see we store all of the information into a hash table titled “tickets” and fulfill all the methods required by the interface.

Postgres

So now you have an implementation that you like done. Time for standup. You tell everyone that you have the API done and are ready to go. You present the architecture to the team and someone questions your choice in database. Have you considered something more durable? What if we want to sell this later on down the road? Is it possible to do BOTH?

Sure, why not?

Let’s create a Postgres port to use. Postgres is a great structured database that is used throughout the industry. So we will create another directory called database/psql and create a repository that fulfils the TicketRepostory interface.

package psql

import (
	"database/sql"
	"hex-example/ticket"
	"log"

	_ "github.com/lib/pq"
)

type ticketRepository struct {
	db *sql.DB
}

func NewPostgresTicketRepository(db *sql.DB) ticket.TicketRepository {
	return &ticketRepository{
		db,
	}
}

func (r *ticketRepository) Create(ticket *ticket.Ticket) error {
	r.db.QueryRow("INSERT INTO tickets(creator, assigned, title, description, status, points, created, updated) "+
		"VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING id",
		ticket.Creator, ticket.Assigned, ticket.Title, ticket.Description, ticket.Status, ticket.Points, ticket.Created, ticket.Updated).Scan(&ticket.ID)
	return nil
}

func (r *ticketRepository) FindById(id string) (*ticket.Ticket, error) {
	ticket := new(ticket.Ticket)
	err := r.db.QueryRow("SELECT id, creator, assigned, title, description, status, points, created, updated FROM tickets where id=$1", id).Scan(&ticket.ID, &ticket.Creator, &ticket.Assigned, &ticket.Title, &ticket.Description, &ticket.Status, &ticket.Points, &ticket.Created, &ticket.Updated)
	if err != nil {
		panic(err)
	}
	return ticket, nil
}

func (r *ticketRepository) FindAll() (tickets []*ticket.Ticket, err error) {
	rows, err := r.db.Query("SELECT id, creator, assigned, title, description, status, points, created, updated FROM tickets")
	defer rows.Close()

	for rows.Next() {
		ticket := new(ticket.Ticket)
		if err = rows.Scan(&ticket.ID, &ticket.Creator, &ticket.Assigned, &ticket.Title, &ticket.Description, &ticket.Status, &ticket.Points, &ticket.Created, &ticket.Updated); err != nil {
			log.Print(err)
			return nil, err
		}

		tickets = append(tickets, ticket)

	}
	return tickets, nil
}

The database and table are now setup and everything looks great. Let’s now build the main part of the application that supports both databases so we test which one we prefer. Or even better, if we decide to open source this we can let the user decide.

Main Application

Let’s use flags to determine which database to use. You can see we have two connection helper methods that will be passed into the new repository depending on the flag. It then will inject the repository into the service and then the service into the handler. Finally the handler is served and we are a go!

package main

import (
	"database/sql"
	"flag"
	"fmt"
	"hex-example/database/psql"
	redisdb "hex-example/database/redis"
	"hex-example/ticket"
	"log"
	"net/http"
	"os"
	"os/signal"
	"syscall"

	"github.com/go-redis/redis"
	"github.com/gorilla/mux"
	_ "github.com/lib/pq"
)

func main() {

	dbType := flag.String("database", "redis", "database type [redis, psql]")
	flag.Parse()

	var ticketRepo ticket.TicketRepository

	switch *dbType {
    case "psql":
  		pconn := postgresConnection("postgresql://postgres@localhost/ticket?sslmode=disable")
  		defer pconn.Close()
  		ticketRepo = psql.NewPostgresTicketRepository(pconn)
  	case "redis":
  		rconn := redisConnection("localhost:6379")
  		defer rconn.Close()
  		ticketRepo = redisdb.NewRedisTicketRepository(rconn)
	default:
		panic("Unknown database")
	}

	ticketService := ticket.NewTicketService(ticketRepo)
	ticketHandler := ticket.NewTicketHandler(ticketService)

	router := mux.NewRouter().StrictSlash(true)
	router.HandleFunc("/tickets", ticketHandler.Get).Methods("GET")
	router.HandleFunc("/tickets/{id}", ticketHandler.GetById).Methods("GET")
	router.HandleFunc("/tickets", ticketHandler.Create).Methods("POST")

	http.Handle("/", accessControl(router))

	errs := make(chan error, 2)
	go func() {
		fmt.Println("Listening on port :3000")
		errs <- http.ListenAndServe(":3000", nil)
	}()
	go func() {
		c := make(chan os.Signal, 1)
		signal.Notify(c, syscall.SIGINT)
		errs <- fmt.Errorf("%s", <-c)
	}()

	fmt.Printf("terminated %s", <-errs)

}

func redisConnection(url string) *redis.Client {
	fmt.Println("Connecting to Redis DB")
	client := redis.NewClient(&redis.Options{
		Addr:     url,
		Password: "", // no password set
		DB:       0,  // use default DB
	})
	err := client.Ping().Err()

	if err != nil {
		panic(err)
	}
	return client
}

func postgresConnection(database string) *sql.DB {
	fmt.Println("Connecting to PostgreSQL DB")
	db, err := sql.Open("postgres", database)
	if err != nil {
		log.Fatalf("%s", err)
		panic(err)
	}
	return db
}

func accessControl(h http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Access-Control-Allow-Origin", "*")
		w.Header().Set("Access-Control-Allow-Methods", "GET, OPTIONS")
		w.Header().Set("Access-Control-Allow-Headers", "Origin, Content-Type")

		if r.Method == "OPTIONS" {
			return
		}

		h.ServeHTTP(w, r)
	})
}

Note: I did not do much error handling in this code but this is an example and not for production use

Lets take it for a spin:

go build main.go
./main

Now it’s running in Redis mode.

curl -X POST   http://localhost:3000/tickets  -H 'Cache-Control: no-cache' -H 'Content-Type: application/json' -d '{
"creator" : "Joel",
"title" : "Test ticket",
"description" : "A test ticket",
"points": 5
}'

curl http://localhost:3000/tickets

Okay lets try Postgres:

./main --database psql

Run the same curl requests and you should see similar responses.

Conclusion

In the end you can see how flexible your code can be. If other adapters need added the logic stays the same. If your microservice gets very large you can just rip out a domain. All of this flexibility allows you to prevent rewriting a bunch of code across the domain. If a bug occures in the business logic itself you will have fixed it in a central location. This level of code isolation also allows for easier tests to be written and identify problem areas.

When I started Go development I wanted to understand the patterns to build applications. This was the first pattern I found that made sense to me. It can be done in a simlar way in other languages. Given the package structure of Go it becomes easy to isolate domains from the ports. In the next section I’ll show how it will make testing easier as well.

Additional Resources:

  • Source code can be found here
  • This talk was published when I was writing this post and is a great example of all of the different styles and reasonings to structure your Go code.