Microservices are becoming increasingly popular on the landscape of Service Oriented Architecture. More companies are becoming enamored with them and with good reason. Microservices provide a degree of granularity and flexibility that allows modern developers to adapt to the ever increasing demands of the marketplace. Microservices are discrete and when your CI/CD process is adjusted to accommodate the small size of microservices, you’ll find them very suited for scaling. Also, smaller development teams are needed to create, deploy and support them. It’s easier for a new developer to become productive on microservices as opposed to full blown, monolithic applications.

There’s a lot to be said for using microservices provided you have a testing policy and procedures in place that ensures the quality of the given microservices. Microservices that are not well tested are accidents waiting to happen. You just can’t put a bunch of of URLs out on the Internet and hope for the best. Believe me, most times working in software, anything that can go wrong will go wrong. But, I suspect I preach to the choir.

The people that know a thing or two about large scale system testing advocate using a variety of strategies to ensure that your microservices work as you intend them to. No argument here. I am a big supporter of comprehensive testing, all the way from automated unit testing to the higher level, human based, exploratory testing.

What is Contract Testing?

One strategy that I find particularly useful for testing microservices is Contract Testing. Contract Testing has become quite mature and is covered extensively in the book, Growing Object-Oriented Software, Guided by Tests. But the short version is this: Contract Testing is writing tests to ensure that the explicit and implicit contracts of your microservices work as advertised.

There are two perspectives when it comes to Contract Tests, consumer and provider. As you might suspect, the consumer perspective is that of an entity using the microservices. The provider perspective is that of the entity providing the service. These perspectives have been in cultural conflict since the first punch card rolled through an IBM 700. It’s the difference between, “your service isn’t working the way that the spec said it would” (consumer) and “The service is working according to spec, RTFM” (provider). Thus, you can infer that it all comes down to the spec, in other words, the contract.

Making the Contract Known

Before any testing can take place, consumer or producer, a contract must be available to all parties. The more common way that microservice contracts are known is by way of the API documentation. Please know there are some more advanced API publishers that have embraced the self-discovery and hypermedia control aspects of Richardson Maturity Model (RMM) Level 3 and thus can expose contract declaration via endpoint responses, whether real or mocked. But, for most of us grunting it out making APIs at RMM, Level 2, on the order of Etsy, NYTimes, Twitter and Shopify, the documentation is all we have.

Microservices need to be documented to a fine degree of detail so they're very clear to the consumer as well as the team testing the structure and use of the endpoint. For example, when specifying the input and output for a POST method on a given endpoint, just declaring the input/output data as object is going to create work down the line. Test designers will have to guess about what structures to create to test the API of your microservices. Rather, you will do well to specify each attribute of the request and response objects, as well as the datatype of the given attribute therein.

When it comes to creating the documentation for your microservices, life will be a lot easier when you use a standard API specification format. There are many tools out there that will automatically generate documentation when an API is defined against a known standard such as RAML, OpenAPI (neé Swagger by Tony Tam) or API Blueprint.

You will find these tools helpful, no doubt. But the documentation produced will only be as good as the specification you create. If the specification lacks granularity in terms of well-defined request and response objects, you’ll find consumers submitting anything under the sun and then getting upset when the microservices’ API responds with the typically vague 400 error code. So again, put as much meaningful detail into your specification as you can.

So What Do We Test and When?

First let’s start with what not to test. The focus of a contract test is to make sure that things work as advertised in terms of the agreement. Typically, agreements are confined to the API specification presented by the microservices. Thus, do not plan to test service availability, load tolerance or deployment integrity. All we care about is testing against the contract. So far, so good. But, before I go on, let me admit two things.

First, my perspective is that of a microservices provider.  It’s rare for me to release the microservices specification independent of the microservices themselves. I am a big proponent of Specification First, using a standard API specification format when I design an API (in my case Swagger/OpenAPI). I can definitely imagine larger enterprises that have many teams working in parallel needing to have a specification in hand before any provider code is delivered. But, my experience is more limited to delivering code along with the documentation.

Since I am taking the perspective of a microservices provider, please know I test directly against an active endpoint, usually in Development or QA environments. When in DEV, I might mock an internal collaborator component or service within the service. But, the entry point of my testing will always be against an active METHOD/URL using HTTP.

Second, when it comes to test sensibilities, I have a bias to testing in terms of state as described by the Classic School of TDD. What is the Classic School of TDD approach? Imagine you have an endpoint api/customers/ that takes firstname, lastname and email as body parameters on a POST. When you POST the data, you expect back an id that references an unique identifier for the resource item. Thus, taking the Classic approach, I send in the data and get back the id. If I want to know that the data stuck, I make a subsequent call, GET to api/customers/{id}. Still, all I care about is what I put in is good and what I get back is expected. Thus, I test the changing state of the microservice using a sequence of calls to various resource endpoints.

There is another school, the London School of TDD, that promotes testing the entire behavior of the microservices contract including all collaborator components and services behind the endpoint. Internal components and services must be exercised and verified in terms of roles, responsibilities and interactions. Granted, the London School is much more comprehensive. Given that I think you can never have too much testing, the London School approach has very little downside provided the team has the appetite and the required staffing, time and expertise required for it.

Because I am taking the perspective of a microservices provider in the Classic School, I am going write tests that confirm both the happy and unhappy paths for all the given endpoints and associated HTTP methods defined for the microservice. I will set up tests, execute, assert against the response and then teardown. When the documentation implies a sequence, for example, POSTing customer data the returns a customer_id that will allow me to GET the customer information later on, I will perform that sequence along the happy path. Also, I will test using a bogus customer_id to meet the unhappy path testing requirement. Granted, unhappy path testing can become a quest requiring infinite labor in order to imagine and test everything that can go wrong with your microservices.

The shops I’ve worked in have adopted DevOps principles and thus have time constraints. It is rare to have unlimited time to test.

I am pragmatic. I think that essential testing must be done. I think that all code written must be designed to be testable. I think that any test written needs to pass. Also, I think that at some point in the entire testing process, there must be a set of passing tests that can prove that code, particularly functional code, has been adequately covered.

Thus, at the least, you should test using ranges of values against a given attribute value. Also test beyond the supported ranges of value and expected data types.  I am a big supporter of adding tests, happy and unhappy path during each development iteration. Also, I am a big believer in using tools when possible.

Tools

One of the benefits of using a standard specification such as RAML, Swagger/OpenAPI or API Blueprint when designing an API for your microservices is that a lot has been done to create tools that will do test autogeneration against specification. For example, you use Vigia to create tests against a RAML specification.  If your spec is written in Swagger/OpenAPI, you can autogen tests using Swagger Test Templates. For API Blueprint there’s Dredd.

Having said this, when you find yourself in a position where you cannot use automated test generation to create tests, you can use any test environment that supports HTTP calls. Examples are mocha/chai running supertest in a NodeJs environment, SoapUI which offers support for defining tests graphically, or the good old standby, JMeter. You can run Mocha/Chai, SoapUI and JMeter all from the command line, which makes for seamless integration into your CI/CD process.

When to Test

Tests take time to run, even for microservices. Microservices that have one or two endpoints, with minimal response and request objects, will not take as long as tests for services that support very complex data structures. Also, network latency will always contribute for the time it takes for a set of tests to run.

If you have small microservices that support limited data structures, running the contract tests associated with the service each time you deploy the microservices should not cause a great burden on your CI/CD pipeline. For microservices that support substantial data structures and do a lot of heavy lifting in terms of computing, bottlenecks can occur.

I am a pessimistic developer. I believe that anything that can go wrong, will go wrong. Thus, from a microservices provider’s perspective, I advocate always running Contract Tests on each deployment. I’d rather slow down the assembly line to ensure that all products rolling off the end are of the highest quality.

Typically source control environment such as GitHub allow you to discover when the spec has changed, provided you are using a standard specification technology such as RAML, Swagger/OpenAPI or API Blueprint. Thus, your automated CI/CD process will be tipped off when it’s time to run the Contract Tests. If you are rolling-your-own in terms of specification definition, make sure you build in a way into your homegrown technique to detect API changes from source code.

Putting It All Together

As mentioned at the beginning of this piece, Contract Testing is but one of the many strategies that need to be incorporated into a comprehensive microservices release process. Performing Contract Tests in combination with other test strategies such as automated unit testing, integration testing and load testing combined with manual, human based exploratory testing will provide the overall quality assurance process you need to make the microservices your business provides reliable and desirable. Some of you might adhere to the Classic School of TDD, others to the London School. The important thing to remember is that Contract Testing counts. In the world of architectures that use microservices, the contract is the foundation upon which overall system integrity stands. Getting it right matters.