In the first post of this series I looked at how kubernetes employs a combination of virtual network devices and routing rules to allow a pod running on one cluster node to communicate with a pod running on another, as long as the sender knows the receiver’s pod network IP address. If you aren’t already familiar with how pods communicate then it’s worth a read before continuing. Pod networking in a cluster is neat stuff, but by itself it is insufficient to enable the creation of durable systems. That’s because pods in kubernetes are ephemeral. You can use a pod IP address as an endpoint but there is no guarantee that the address won’t change the next time the pod is recreated, which might happen for any number of reasons.
This post is going to attempt to demystify the several layers of networking operating in a kubernetes cluster. Kubernetes is a powerful platform embodying many intelligent design choices, but discussing the way things interact can get confusing: pod networks, service networks, cluster IPs, container ports, host ports, node ports… I’ve seen a few eyes glaze over. We mostly talk about these things at work, cutting across all layers at once because something is broken and someone wants it fixed. If you take it a piece at a time and get clear on how each layer works it all makes sense in a rather elegant way.
HashiCorp’s terraform is a powerful and extensible tool for defining and creating cloud infrastructure in a repeatable way. At Olark we use it to manage a number of different environments on Google Cloud Platform. On the journey from imperative to declarative infrastructure we’ve learned a few things. Here are five that I feel are particularly important. What follows are entirely my own opinions.
Workloads running in kubernetes pods commonly need access to services outside the cluster. In heterogeneous architectures where some services run in kubernetes and others are implemented on cloud VMs this often means resolving private DNS names that point to either specific hosts or to internal load balancers that provide ingress to groups of hosts.
Chatbots are the hot topic of conversation among developers and investors alike right now. Advances in natural language processing (NLP) and wide adoption of a few messaging platforms like Facebook Messenger, Slack, Snapchat and Whatsapp have presented developers with a unique opportunity. Would you rather navigate to, say, Nordstrom’s website, find the shoe section, figure out the categories, search, choose a pair, add them to a cart, etc., or would you rather message the Nordstrom’s bot: “I need a pair of brown dress shoes, size 10 1/2, leather, and I like Johnson and Murphy” and have it respond with a selection which you can order from simply by tapping? Yeah, thought so. Me too.
Chatbots have a number of advantages. The chat platform (FB for example) knows who you are so that whole create account/auth business can be dispensed with. You can save your personal details like shipping address and payment methods once, and have them shared with companies whose bots you interact with. You don’t have to learn the organization of a new website with a unique information flow. You don’t even have to remember their domain name, and there are still plenty of opportunities for branding and positive customer interactions.
So bots are cool. How do you build one? I’ve just embarked on some experiments to find that out, and I will be posting a little bit here and there about the experience. I chose to build a bot for Facebook Messenger for some simple reasons: they have 900 million users, and it’s Facebook. Everyone uses Facebook, so bots developed on their framework should have very broad reach. I had never done any development on FB so the first thing I had to do was register as a developer. That literally took one button click and I was in the developer console. First step is to create a page and an app. The page profile photo and title are used as the public identity of the bot once it is deployed, but they don’t have to be public during development. I created a page titled MarkBot and left it unpublished. Next I created the app, skipping the canned templates and just proceeding to the console to create an App ID. Facebook interacts with apps via webhook callbacks, so the first thing I had to do was set one up and verify it. Verification consists of FB calling the hook with a challenge param that you need to send back, so the upshot was I needed to deploy some code somewhere to handle this request.
I have an AWS account so my first thought was just to write a quick flask app and slide the code into a docker container, deploy it to a free tier instance, set up an ELB… yeah even with docker this is all still a lot of work to handle a web callback. I only had to do it once and then I would write makefiles to automate the whole thing, but still… had to be a better way. That better way is AWS Lambda. I heard about lambda when it first came out, of course, but I had given no further thought to it since. I work on real systems, with containers, and servers and stuff! Why would I want to simply define a python function, attach an endpoint to it, and be able to respond to http requests… er, yeah. Turns out lambda is pretty damn awesome.
At a high level you write some code in java, python, or node.js. You can create the code right in the lambda console, or create it as an external package that you upload. For the purposes of this simple verification hook I did the former, but will definitely transition to the latter for the bot message handlers, so I will have more to say about that in future posts. Your code gets called with some defined parameters either when you run it manually, or when certain events occur. The definition of “event” includes a whole bunch of interesting sources, such as S3 buckets changing, CloudWatch logs or DynamoDB streams matching a content filter, RDS database tables getting updated, etc. It’s a long list that includes most of AWS’s services, including the one I was interested in: API Gateway.
API Gateway is basically a programmable http request handler that you can use to set up an endpoint path and define the methods it supports. When you’re done you get a url that you can use to hit the gateway. On the back end API Gateway hands off the requests it receives to either another http service, an AWS service proxy or a lambda function. So that is the other big piece of the puzzle: API gateway + lambda function == instant http service. Getting to that realization was the easy part. As with most of Amazon’s AWS offerings there is a lot of complexity lurking just beneath the surface of the busy-looking console and rapidly outdated web documentation. One confusing thing is that you can approach this build from two independent directions: you can create a lambda function and use the lambda console to create an API Gateway endpoint to attach it to; or you can create an API Gateway endpoint and use that console to attach it to an existing lambda function. I imagine both approaches will get you to the same place. I chose to perform the setup in the lambda console, but still had to jump over to the API Gateway console a couple of times.
The two main things that slowed me down were: understanding the deploy cycle for the API, and mapping querystring parameters into my lambda function. On the deploy point it all boils down to one important fact: changes you make to the API endpoint, such as the mapping changes I’ll touch on below, don’t take effect until you deploy the API. You can change and test but until you deploy the new stuff isn’t visible to consumers. That caused me a few rounds of puzzling over Cloudwatch logs (did I mention that lambda logs to Cloudwatch? Slick.). Once I realized what deploy did it all made sense.
The parameter mapping thing is a little more complex, and I don’t have space left to go into it in detail here. The basic point is that you have API Gateway receiving an http request, and your lambda function expecting an event object (and a context object but that’s another topic), and so you have to tell API Gateway how to map the path params, headers, querystring, form variables, or body into the event. You do this in the method execution section of the endpoint method. There are four steps in the processing pipeline where you can affect how requests are handled: method request, integration request, integration response and method response. Since the Facebook webhook verification uses querystring args and expects a json response I had to make three changes here: add the two querystring params to the method request; add a mapping template to the integration request to transform the params into a json object; and add a mapping to the integration response to transform the string I returned into an application/json response.
Once that was done the call from Facebook succeeded and my webhook was verified. Next step is to understand the interaction model between the chatbot and my API. More on that after I get there.
If you design and implement APIs for a living then you’re probably already familiar with swagger. Swagger is a specification for defining API endpoints and the model objects they transact. Once you have a swagger definition of your API written in either json or yaml you can do quite a few useful things with it: generate HTML documentation on the fly; generate client and server code; generate a postman collection for endpoint testing; and more to the point of this piece, use the spec to validate incoming objects at request time, resulting in meaningful error responses to clients.
Python function decorators are encountered in many advanced programming contexts, including the use of frameworks like twisted and flask. In this post on Medium I deconstruct the decorator syntax and show how they are used to perform operations at both module import and function call time.
In my latest article on medium.com I discuss using a Queue to dispatch events across threads in python 2.
Containers are awesome, but sometimes it can feel like your code has been shut up in a black box, stuck off in the cloud where you can’t see what it’s doing. When your app runs the way it’s supposed to this isn’t a problem: containers come, containers go, everyone’s happy. But then there’s that moment when you push a new version of your image and check the cluster status only to see it crashlooping. What’s going on? In my latest post on medium.com I list ten tips that will make debugging your docker containers easier.
Over the last two years docker containers have completely changed the way I look at software builds, deployment and infrastructure. Orchestration platforms like Amazon’s Elastic Container Service and Google Container Engine, built on their open source kubernetes framework, are closing the gap between what we have been able to do with software in containers, and what we want to be able to do with infrastructure and deployments in the cloud. My first piece on medium.com is a comparison of these two “container orchestration” platforms.