Nameko - Quick Microservices in Python

Frameworks for microservices seem to be a dime a dozen. They provide the abstraction needed for a developer to quickly develop a service that can be exposed to others who may need it. Some languages like Go provide enough from the standard library that the setup for a microservice mostly comes down to using built in libraries and structuring data properly. However, some other languages have some libraries that allow you to rapidly develop services without having to worry too much about setup. In the Java world Spring tends to be the “go to” library and many in the Python world use Flask. However, Flask is not a microservice framework, it’s a “micro web” framework. This means it can be more than just an API endpoint and can also serve static webpages as well as other content. So if you want to write just microservices quickly in Python are there others?

Nameko seems to be a great library if you want to get some services up and running. It relies heavily on RabbitMQ which is a message broker that does both AMQP and RPC calls. By leveraging this underlying technology Nameko allows you to build systems that can communicate over a reliable message broker or allow you to communicate through REST endpoints. I find this extremely useful because all to often people say microservice and think only of REST APIs when in reality communication through other channels can fit a better pattern.

Nameko makes the developer create service modules which can be run independently and with little configuration. Let’s look at an example:

We’ll build a “Hello World” API which will respond with JSON.

First let’s create a project and a virtual environment

mkdir micro
cd micro
python -m virtualenv env
source ./env/bin/activate

Then we will install Nameko

pip install nameko

Great, let’s get to programming. Nameko uses modules to run the applications so we will create api.py with the following code.

import json

from nameko.web.handlers import http

class API:
    name = 'api'
    @http('GET', '/hello')
    def get_method(self):
        return json.dumps({'hello': 'world'})

There you go, API! Now you need to define a config file config.yml:

WEB_SERVER_ADDRESS: '0.0.0.0:8080'

Now to run you just need to do

nameko run --config config.yml micro.api

Open another terminal and type:

curl localhost:8080/hello

Hopefully you’ll see the results come up!

Now we want to do some event driven portions of our code. Event driven code (link) allows you to do some asynchronous tasks responding to a particular event happening within your system. We can think of scenarios where we may want to have some sort of audit logging or write to the database that the originator of the event doesn’t really care about.

So in our example we will create two types of event handling. The first is a pub/sub model and the second is a more complex use case where we use an RabbitMQ exchange. Pub/sub sound straight forward where applications will subscribe to an event and do something when this happens. Nameko utilizes RabbitMQ to handle this:

So lets say we want an audit logger to write down every time someone POSTs a name to our api but don’t want the API service to worry about the details of the logging. In true microservice fashion we are separating the duties of the system. Right now the worker we build will just write to stdout but you could imagine in the future maybe writing to a database or doing additional business logic.

We’ll change our API to add a POST endpoint and build a publisher.

import json
from nameko.web.handlers import http
from nameko.events import EventDispatcher
class API:
    name = 'api'
    dispatch = EventDispatcher()
    @http('GET', '/hello')
    def get_method(self):
        return json.dumps({'hello': 'world'})
    @http('POST', '/hello')
    def do_post(self, request):
        name = request.get_data(as_text=True)
        self.dispatch("say_hello", name)
        return json.dumps({'hello': name})

Our dispatcher will publish the name to anyone who subscribes to “say_hello”. So next we’ll create a new package called workers and a file named subscribers.py. Our subscribers will listen for an event to be published and then will just print the message.

from nameko.events import event_handler
class WorkerSubscriber:
    name = 'worker_subscriber'
    @event_handler("api", "say_hello")
    def handle_event(self, payload):
        print("{0} said hello!".format(payload))

Great! We’ll have to change our configuration file now so we can add our RabbitMQ connection. To do this lets spin up a RabbitMQ container:

docker run -p 5672:5672 rabbit-mq:latest

And then change our configuration to add the connection.

AMQP_URI: 'amqp://guest:guest@localhost'

Finally lets run both the API and the worker (on separate instances) and see what happens.

nameko run --config config.yml micro.ap
nameko run --config config.yml micro.workers.subscribers

See how it logs! Next we will add another worker, this time it will be a queue so we can bind to a given exchange. This is similar to how the pub/sub model works under the hood with Nameko but is more explicit. This is so you can define exchanges that other applications can use or bind to existing exchanges from other systems that may not be using Nameko.

So first lets update our worker to publish a message to an exchange whenever we receive an event.

from nameko.events import event_handler
from nameko.messaging import Publisher
from kombu.messaging import Exchange
class WorkerSubscriber:
    name = 'worker_subscriber'
    test = Exchange('test', type='direct')
    publish = Publisher(exchange=test)
    @event_handler("api", "say_hello")
    def handle_event(self, payload):
        print("{0} said hello!".format(payload))
        self.publish("Goodbye {0}".format(payload))

And we will create a new worker that now creates a queue that points to this same exchange and will receive events published there.

We’ll call this consumer.py:

from nameko.messaging import consume
from kombu.messaging import Exchange, Queue
class Consumer:
    name = 'worker_consumer'
    test = Exchange('test', type='direct')
    tq = Queue('q1', exchange=test)
    @consume(tq)
    def handle_consume(self, body):
        print("Received message: {0}".format(body))

Here we build a queue that ties to the test exchange. You can read about the different types of exchanges on RabbitMQ’s site. Essentially a direct exchange will publish to whatever queue is bound to it no matter what. Run these and see how it works.

nameko run --config config.yml micro.api
nameko run --config config.yml micro.workers.subscribers
nameko run --config config.yml micro.workers.consumer

Now you should see the messages come on the screen.

This was a very quick walk through of this framework and there is a lot more you could dive into. But frameworks like this are great to get rapid microservice development underway for any organization. Built on a solid event architecture along with easy to write code Nameko can take you from monolith to microservice rapidly.

Avatar
Joel Holmes
Software Developer

Related