How to write a REST API in go with dingo

This tutorial shows how to use sarulabs/dingo to create a basic rest api in go (golang). Dingo is a code generator. It can generate dependency injection containers. Basically it is a dependency injection framework for go programs. Its role is to handle the life cycle of the services in your application.

This article assumes that you are already familiar with go and the creation of an http server. It is a good complement to the dingo documentation. The code of the api is available at github.com/sarulabs/dingo-example. Most of the code is explained here, but you still will need to go to the github repository if you want to see the whole thing.

API description

The role of the api is to manage a list a car. The cars are stored in mongodb (easier for this tutorial because schemaless). The api implements the following basic CRUD operations:

Method	URL	Role	JSON Body example
GET	/cars	List cars
POST	/cars	Insert car	{"brand": "audi", "color": "black"}
GET	/cars/{id}	Get car
PUT	/cars/{id}	Update car	{"brand": "audi", "color": "black"}
DELETE	/cars/{id}	Delete car

The request and response bodies are encoded in json. The api handles the following error codes:

400 - Bad Request : the parameters of the request are not valid
404 - Not Found : the car does not exist
500 - Internal Error : an unexpected error occurred (eg: the database connection failed)

The api can be tested with curl of applications like postman. The github repository includes a docker-compose file that can be used to start the api without much effort.

Project structure

The project structure is as simple as this:

├─ app
│  ├─ handlers
│  ├─ middlewares
│  └─ models
│     ├─ garage
│     └─ helpers
│
├─ config
│  ├─ logging
│  └─ services
│
└─ main.go

The main.go file is the entrypoint of the application. Its role is to create a web server that can handle the api routes.

The app/handler and app/middlewares directories are, as their names say, where the handlers and the middlewares of the application are defined. They represent the controller part of an MVC application, nothing more.

app/models/garage contains the business logic. Put another way, it defines what is a car and how to manage them.

app/models/helpers consists of functions that can assist the handlers. For example, they can read the body of an http request, or write a json response. The package also include two errors type: ErrValidation and ErrNotFound. They are used to facilitate the error handling in the http handlers.

In the config/logging directory, a logger is defined as a global variable. The logger is a special object. That is because you need to have a logger as soon as possible in your application. And you also want to keep it until the application stops.

config/services is where you can find the dingo definitions. They describe how the services are created, and how they should be closed.

Model

The model is where you can find the business logic of the application. In our case, the model should be able to handle CRUD operations for cars. In the models/garage package, you can find the following elements:

type Car struct {
    ID    string `json:"id" bson:"_id"`
    Brand string `json:"brand" bson:"brand"`
    Color string `json:"color" bson:"color"`
}

func ValidateCar(car *Car) error

type CarManagerInterface interface {
    GetAll() ([]*Car, error)
    Get(id string) (*Car, error)
    Create(car *Car) (*Car, error)
    Update(id string, car *Car) (*Car, error)
    Delete(id string) error
}

type CarRepositoryInterface interface {
    FindAll() ([]*Car, error)
    FindByID(id string) (*Car, error)
    Insert(car *Car) error
    Update(car *Car) error
    Delete(id string) error
    IsNotFoundErr(err error) bool
    IsAlreadyExistErr(err error) bool
}

Car is the structure saved in the database. It represents a very simple car with only two fields, a brand and a color. The ValidateCar function can check if the brand and the color are available. If the combination is not allowed, it returns a validation error explaining what is wrong with the given car. It is used in the creation and the update of a car.

There are two more structures in the model. A CarManager and a CarRepository. Their interfaces CarManagerInterface and CarRepositoryInterface do not exist in the application code but are given here to illustrate what they do.

The CarManager is the structure used by the handlers to execute the CRUD operations. There is one method for each operation, or in other words, one by http handler. The CarManager needs a CarRepository to execute the mongo queries.

Separating the database queries in a repository allows to easily list all the database queries. In this situation, it is easy to replace the database. For example you can create another repository using postgres instead of mongo, or you can create a mock repository for your tests.

The CarManager adds parameter validation, logging and some error management to the repository code.

Services

The services package is where dingo comes into play. You declare how to instantiate the structures from the model, as well as their dependencies. It will allow the creation of a dependency injection container that can be used to retrieve the objects.

Actually, in our case the handlers only need a CarManager to handle the requests. But it should be created with its dependencies:

CarManager
|- Logger
|- CarRepository
   |- Mongo connection

The definitions can be found in config/services/services.go. Let’s start with the CarManager dependencies.

The logger is a global variable in the logging package. But it still should be declared in the container, so it can be injected in other services.

The declaration is as follow:

var Services = []dingo.Def{
    {
        Name:  "logger",
        Scope: dingo.App,
        Build: func() (*zap.Logger, error) {
            return logging.Logger, nil
        },
    },
    // other services
}

The logger is in the App scope. It means it is only created once for the whole application.

Now we need a mongo connection. What we want first, is a pool of connections. Then each http request will use that pool to retrieve its own connection.

So we will create two services. mongo-pool in the App scope, and mongo in the Request scope:

    {
        Name:  "mongo-pool",
        Scope: dingo.App,
        Build: func() (*mgo.Session, error) {
            return mgo.DialWithTimeout(os.Getenv("MONGO_URL"), 5*time.Second)
        },
        Close: func(s *mgo.Session) {
            s.Close()
        },
        NotForAutoFill: true,
    },
    {
        Name:  "mongo",
        Scope: dingo.Request,
        Build: func(pool *mgo.Session) (*mgo.Session, error) {
            return pool.Copy(), nil // retrieve a connection from the pool
        },
        Params: dingo.Params{
            "0": dingo.Service("mongo-pool"), // inject mongo-pool as parameter 0 in the build function
        },
        Close: func(s *mgo.Session) {
            s.Close()
        },
    },

The mongo service uses the mongo-pool service to retrieve the connection. The mongo Build function is helped by the Params field. The key “0” is the index of the parameter in the build function. The value is the one used as parameter. Using the dingo.Service type allows to use another service from the container.

The NotForAutoFill field in the mongo-pool definition is important. Dingo can guess dependencies. If the dependency is not specified with an entry in the Params fields, dingo tries to use a service with the same type. But dingo is only able to do that if there is exactly one service for the type of the dependency.

In our container, there are two mgo.Session services. But in the other services, we only want to use the mongo service, and not the connection pool.

By using NotForAutoFill: true, the mongo-pool service is no longer a candidate when dingo tries to guess the dependency. The mongo service will be used by default. If at some point you need to use the mongo-pool in a service, you would have to use the Params field, like in the mongo definition.

Note that it is also important to close the mongo connection in both cases. This can be done using the Close field of the dingo definition. The Close function is called when the container is deleted. It happens at the end of each http request for the Request containers, and when the program stops for the App container.

For the CarRepository and CarManager definitions, this is really easy. Their dependencies are already defined in the container and they are public fields. This mean you can use a pointer to a structure as Build field in the dingo definition.

    {
        Name:  "car-repository",
        Scope: dingo.Request,
        Build: (*garage.CarRepository)(nil),
    },
    {
        Name:  "car-manager",
        Scope: dingo.Request,
        Build: (*garage.CarManager)(nil),
    },

With public fields, and the dependencies guessed by dingo, you end up with very short definitions.

Now that you have the definitions, all you need to do is to generate the code.

dingo -src=config/services/ -dest=var/lib/services/

The code is generated in the var/lib/services/ directory. It is committed in the github repository. The most interesting file is var/lib/services/dic/container.go. It contains what you can use from your application, like the NewContainer function to create a container.

Handlers

The role of an http handler is simple. It must parse the incoming request, retrieve and call the suitable service and write the formatted response. All the handlers are more or less the same. For example the GetCarHandler looks like this:

func GetCarHandler(w http.ResponseWriter, r *http.Request) {
    id := mux.Vars(r)["carId"]

    car, err := dic.CarManager(r).Get(id)

    if err == nil {
        helpers.JSONResponse(w, 200, car)
        return
    }

    switch e := err.(type) {
    case *helpers.ErrNotFound:
        helpers.JSONResponse(w, 404, map[string]interface{}{
            "error": e.Error(),
        })
    default:
        helpers.JSONResponse(w, 500, map[string]interface{}{
            "error": "Internal Error",
        })
    }
}

mux.Vars is just the way to retrieve the carId Parameter from the url with gorilla/mux, the routing library that has been used for this project.

The purpose of the end of this code snippet is to format and write the response depending on what happened in the CarManager. The type switch allows to have different responses depending on the nature of the error.

The interesting part of the handler, is how the CarManager is retrieved from the dependency injection container. It is done with dic.CarManager(r). For this to work, the container should be included in the http.Request. You have to use a middleware to achieve that.

Middlewares

The api uses two middlewares. The first one is used to catch the possible panic that could occur in the handler, and log the errors. It is really important because dic.CarManager(r).Get(id) can panic if the CarManager can not be retrieved from the container. In this case it is nil, and calling the Get method on nil makes the program panic.

The second middleware allows dic.CarManager(r) to work by injecting the dingo container in the request.

func DingoMiddleware(h http.HandlerFunc, app *dic.Container) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // create a request container from tha app container
        ctn, err := app.SubContainer()
        if err != nil {
            panic(err)
        }
        defer ctn.Delete()

        // store the container in the http.Request
        ctx := context.WithValue(r.Context(), dingo.ContainerKey("dingo"), ctn)
        req := r.WithContext(ctx)

        h(w, req)
    }
}

It takes the App container and creates a sub-container in the Request scope. Then it injects the container in the http.Request context.

dic.CarManager(r) uses the generated dic.C function to retrieve the container in the request. Then it tries to retrieve the CarManager with container.CarManager().

At the end of the request, the request container is deleted. That closes all the services that it created. In our case its major role is to close the mongo connection.

Main

The main.go file is the entrypoint of the application.

First it should ensure that the logger will write everything before the program ends:

defer logging.Logger.Sync()

Then the dependency injection container can be created:

app, err := dic.NewContainer()
if err != nil {
    logging.Logger.Fatal(err.Error())
}
defer app.Delete()

dic.ErrorCallback = func(err error) {
    logging.Logger.Error("Dingo error: " + err.Error())
}

It is important to redefine dic.ErrorCallback to log the build errors. We saw that if the CarManager can not be retrieved from the container, dic.CarManager(r) can be nil and make the handler panic. But the only log you get from the PanicRecoveryMiddleware, only tells you that dic.CarManager(r) is nil. But it does not explain why. dic.ErrorCallback gives you the opportunity to find the reason.

Last interesting thing the this part:

m := func(h http.HandlerFunc) http.HandlerFunc {
    return middlewares.PanicRecoveryMiddleware(
        middlewares.DingoMiddleware(h, app),
        logging.Logger,
    )
}

The m function combines the two middlewares. It can be use to apply the middlewares to the handlers.

The rest of the main file is just the configuration of the gorilla mux router and the creation of the web server.

Conclusion

Creating this small api was really easy. It would also be simple to extend it to handle more routes.

Using dingo allows you to separate the declaration of the services from the business logic. It happens in a single place, which is part of the application configuration. The services can be retrieved in the handlers by using the generated functions. Writing the service definitions is not very troublesome. Because most of the time, the dependencies can be guessed allowing short definitions.

The major downside of using dingo is that code need to be generated. During development you will need to use a file watcher to regenerate the container when the services are updated. You can use realize which written in go and allows you to stay in the same ecosystem. Watchman can also be a good choice if you do not mind installing it.