Third part of our Consumer-Driven Contact series of articles.

Being an eMobility leader in various projects across multiple startups, Piotr was responsible for OCPI and OCPP integration as well as EMP and CPO platforms development. With extensive expertise in the open standards field and eMobility tech solutions, he is leading teams of experienced developers and contributing with invaluable knowledge and experience.

Piotr Majcher

Consumer-Driven Contract (1) - how to increase stability in distributed systems.

Consumer-Driven Contract (2) - Pact-based implementation

 we described the base principles and implementation of the Consumer-Driven Contract without any automation.

In this article, we’re going to orchestrate CDC and make them part of the existing CI/CD pipelines. We will refer to the code from this post, so if you haven’t read it, some parts of this post may be unclear.

We’ve already chosen Pact as a tool that will help us introduce CDC. Naturally, we also decided to use the Pactflow platform that plays the role of a Pact Broker. This time, we will focus much more on the platform than on our code. We will prepare a simple pipeline using Gitlab CI/CD cooperating with a Pact Broker. Let’s dive into the details now!

This Picture describes the Customer CI/CD flow responsible for validating contracts.

Everything starts with a change in the Consumer codebase. It triggers the entire pipeline and, as a part of it, we’re going to check whether the Consumer assumptions of what can be delivered by the producer are valid. We described the process of preparing the contract and uploading it to the broker 

, but just to recall it: we need two commands here. One for generating the contract:

Before we move those two commands into the pipeline, let’s focus on the important part of publishing contract which is contract versioning.

The contract version is the internal Pact value that is not exposed to the users. But each contract may be identified by the consumer version and provider version. So instead of thinking in terms of contract versioning, we should switch to versioning of consumer and provider apps.

According to Pact best practices - which are similar to the best practices of versioning in general - we should be able to refer from the app version to some point in SCM easily.

The easiest way to do it is by appending the commit hash to the version. In real life, we would need to prepare some release scripts for our projects, but for the sake of simplicity, we’re going to use the feature of the pact Gradle plugin that allows specifying the consumer version explicitly.

As you can see, the parameter name is “providerVersion”. It’s a bit tricky because it's the consumer, not the provider, who publishes the contract. But maybe designers decided to name the parameter that way because the API consumer actually acts as a contract provider.

CONSUMER_VERSION will be populated by the GitLab pipeline with the current value during the pipeline run. We’re ready to prepare the first box from the diagram.

Here is the part of the GitLab pipeline responsible for generating and pushing the contract.

There are two steps we need to take here:

running a tests suite that builds the contract

and invoking the pact gradle plugin for publishing it.

Triggering the producer to validate the contract

Pact Flow offers two main types of webhooks:

A webhook has a form of an HTTP POST call. When a new contract is published, what we want to do is trigger the API Provider pipeline able to validate the contract. We’re going to prepare the Provider pipeline in the next section.

Let’s focus now on triggering the Provider pipeline. The Gitlab platform has an HTTP API allowing us to integrate the service with other tools. We can easily generate an API token to trigger project pipelines:

We have the url now, so we can configure the Pack flow webhook:

The code responsible for validating the contract has been presented in a previous 

. Let’s just go back to the command used to execute the check.

As we can see, it looks like something we can move to the CI/CD pipeline easily.

There some additional commands required by test containers used for testing, but functionally, it’s just invoking a single Gradle task. As we build the contract validation logic using the Pact library, the verification result is shared with the Pact Broker automatically. Now it’s time to push the customer’s pipeline waiting for information about the finished contract validation.

Here’s where Pact Flow webhooks come into play again. But it’s going to be a bit more complicated this time. Gitlab’s API allows us to continue a stopped pipeline, but we need to know the id of the job we want to push. Any time a job runs, a new instance with a new id of the job is created. We can’t prepare a static URL to continue the consumer’s pipeline. Since you can run two pipelines at the same time, referring to a specific pipeline for running it further is impossible. How to determine which pipeline should be unlocked?

Fortunately, there are few features and tools that make it possible.

First, there are the dynamic variables that can be used to construct a Pact Flow webhook url or body. So our endpoint may contain, for instance, consumer’s name and version, producer’s name and version, and more different variables. We decided to use the commit’s short id as a part of the customer version. So the webhook url may contain a commit identifier. But that’s not enough to trigger a blocked pipeline job. We have to find the job id.

As we already mentioned, Gitlab exposes the HTTP API. A job is a first-class entity, so we can perform a useful operation on it. For instance, we can look for all project’s jobs with a specific scope. So we can have something like this:

Note: projectId can be found in the Gitlab frontend.

We’re interested in jobs with a manual scope because that’s how we implemented the job waiting to be triggered. A manual job is a job that can be triggered by the click or API call. As a response, we get a detailed description of jobs. I removed 90% of the response to focus on the parts that are most important for us:

can_i_deploy

Good news: we have the commit short id and the job’s id! So in theory, we found a way to map the commit short id to the job id (or multiple ids). Now we have to trigger the the job. Here’s the Gitlab API method we can use:

We have the theory ready. But where to put all this code? Pact Flow allows entering a string as a webhook url. It sounds like a good candidate for AWS Lambda. And that’s how we implemented it. Lambda exposes the endpoint accepting the commit short id (acting as consumer version) and creates a really simple algorithm:

Get all jobs from a specified project with scope "manual,"

Find jobs with the commit short id matching requested one,

Trigger jobs based on job ids from step 2.

Lambda is exposed to the world using the AWS API Gateway and can be called by the Pact Flow webhook. Here is the most important part of the Lambda querying jobs and filtering the commit short id:

We’re not going to discuss the topic of security at the moment but, of course, such an endpoint needs to be secured somehow.

The code above is simple, but someone may say that it’s not the best option to maintain such a custom code. And actually, who should maintain it? But if you think about how many different CI/CD platforms companies use, there are some concepts in one platform that don’t exist in others, or have different definitions.

A successful introduction of CDC requires work on both sides, consumer and producer. In my opinion, this specific code should be maintained by the consumer. The consumer should know how to trigger its pipeline.

There’s another very important reason to keep this code on the consumer side which is security. Giving external company access to the API of our Gitlab may not be an option. Exposing one very specific method sounds much safer.

The webhook is triggered when contract is verified, no matter the result of the verification. The HTTP POST call to the registered url will be performed even if the contract doesn’t pass verification. It’s the customer’s responsibility to decide whether the new version should be deployed or not. Pact prepares a cli tool that ensures we can deploy some version of our code to the server safely. The tool has a self-descriptive name “can-I-deploy”. We have to specify a few parameters and, as a result, we get a go/no go decision. The invocation in our case looks as follows:

We can install cli manually, but Pact also delivers a Docker image containing all required dependencies. That makes wrapping this invocation in a pipeline job really straightforward:

This crowns the introduction of Customer-Driven Contracts into the Consumer’s CI/CD pipeline. Here’s the full Consumer’s pipeline diagram:

Customer-Driven Contracts/Testing is a pattern that is relatively expensive to set up. Once set up, it also requires cooperation between different teams, systems, and tools - which can be time-consuming. But in the end, we can have much more confidence when applying any changes to our API.

How to build consumer CI/CD pipelines with Gitlab and Pactflow. Click to read the whole article.

Our introduction and the first article of the series about Consumer-Driven Contact.

Some time ago, we had a discussion in one of our projects whether it was the right time to introduce the Consumer-Driven Contract (CDC) approach. After a short debate, we came to the conclusion that while CDC may be a very useful tool, it’s not a silver bullet. It’s extra to what we already have and makes some processes much faster and less error-prone.

Normally, as a part of the API developers prepare some documentation like a swagger or wiki page. They specify all the possible input values with information on whether a particular parameter is mandatory or not. They do the same with the returned value. All values returned by the API are described in this document. We share documentation with third parties (or with other teams inside a company), and that’s all.

There is no space for feedback from the API’s user. We don’t know how our API is used on the data level. We probably have some monitoring in place, and we know how often users are calling our service. But do they use the entire response? Or maybe just one field? As in many other places, a short feedback loop may bring value here.

CDC is nothing else than the feedback from users about how they’re going to use our API. CDC is all about reversing the most common approach I described above. With CDC, the contract is not documentation written by the provider, but information from the consumer describing how they use the API.

Contracts over communication and documentation?

Does it mean that someday instead of receiving an email from someone who is going to use our service or ask us for something custom we’ll receive just a formal contract? Or does it mean that the consumer will define the API and we will just have to implement it? Of course, in most cases, that’s not going to happen.

The CDC doesn’t remove the need for good documentation. And for sure, it doesn’t replace face-to-face communication with a formal contract. Especially the second element - communication - is crucial for implementing the CDC successfully. In most cases, the CDC is only about feedback on how people use our API. We’re still responsible for providing documentation and discussing the details if the API we’re providing is something new. Sometimes not only the contract but the entire API may be driven by the consumer, using the same tools (some kind of formal contract). It all depends on our process and relations between the consumer team and the producer team.

How risky is refactoring code without test coverage? We would have to go to each place where the refactored function is used and check whether everyone is working in some way. Sometimes, it’s too expensive and risky so developers might give up.

Here we have a similar situation but in the HTTP world. Since we may not work with all the API consumers in the same office, such verification is usually even more expensive. Even in the same location, asking all developers to check the code because we are going to fix a typo in one of many fields returned by our API (which may be never used by someone) doesn’t sound good. By keeping contracts from all consumers in one place, we can quickly check whether anyone uses a specific parameter. We can safely modify our code without thinking about backward compatibility. The CI won’t pass any element that breaks any of the existing contracts.

It may sound as if we’re looking for a quick fix instead of proper API versioning. But API versioning is very expensive and complicated. We should avoid it for as long as possible (

). The CDC may allow us postpone that moment when versioning is the only solution.

Another aspect, in some cases an even more important one, is testing distributed systems. When we’re writing integration tests for a single service, we have two options - set up all dependencies or mock them. Since a contract consists of input and expected output, it can be used for automatic stubs generation. Having stubs for all the dependent services, we can easily prepare an isolated test that doesn’t take hours to start or need as many resources as integration test setting up many services.

What if we don’t know our consumers? Of course, we can’t ask them to prepare contracts. It means the CDC isn’t going to work in our case. In such a situation, we have to treat our documentation as a contract and be very careful when modifying anything resulting from changes to API. We have to assume that all the data we return are consumed by someone. We need to keep backward compatibility on the basis of our documentation, not specific use cases.

The second case when the CDC may not be a good fit is when we’re working with a public API. Even if we know all of our consumers, it may be really difficult, or in some cases impossible, to force them to spend time on preparing a contract and keeping it updated. It’s sometimes the matter of a paper-based contract signed by a business that states the expectations of both parties. In such cases, the CDC may not be the best solution. The biggest benefit of using the CDC will be visible if both sides of the contract work at the same company and share the same development practices.

One of the most important steps in CDC is sharing the contract written by the consumer with the producer. It can be done, for instance, through a merge request containing a contract in a format supported by the producer. Contracts may be also shared using a special tool, for instance, Pact (Pact Broker). Pack is a great tool that offers much more than just sharing contracts. It supports the entire CDC flow, including testing consumers using stubs created based on contract, sharing the contract, and verifying the contract on the producer side. Another tool that may be helpful here is the Spring Cloud Contract. You may also use Pact Broker as a contract repository.

Contract-Driven Design and contract testing are two useful tools that bring stability to distributed systems. They allow getting rid of late validation of problems regarding the integration between services. It doesn’t require deploying a service and checking integration manually. The feedback loop is quite short and can stop the building process before an artifact is created. It’s also good documentation showing who uses the API we provide and how they do it.

Our introduction and the first article of the series about Consumer-Driven Contact. Click to learn more!

Second article of our series. An example of CDC implementation with the usage of Pact. 

, we described the principles of the Consumer-Driven Contract approach. We went through the potential use cases of CDC and scenarios where the CDC doesn’t fit. Since the Consumer-Driven Contract is a concept, it may be implemented in different ways.

This article focuses on the implementation based on Pact and using the following tools/frameworks:

Spring Boot-based applications as API provider and consumer,

Pact libraries used by the API provider and API consumer, and responsible for publishing (consumer) and verifying (provider) Contracts,

For the purpose of this article, we prepared a sample API. It’s a simple HTTP API exposing sensors data. Three methods are exposed here:

GET: /api/sensors/{sensorId} - get single sensor

Each sensor has two properties: id and name.

Everything starts from a technical perspective here. Before the agreement between provider and consumer is made, the documentation defined by the provider needs to be shared with the consumer. But the CDC flow begins with the consumer defining the contract. Before we start writing code, we have to add the following dependency to our project:

Defining the contract takes the form of an integration test. Let’s stop thinking about the CDC for a moment and consider preparing an integration test for the component that makes HTTP calls. You need two things: the stub for the API and the test. Let’s start with the stub.

Almost everything is self-explanatory. It’s an API stub definition prepared using the Pact fluent API. We haven’t touched on the topic of the CDC yet. It’s very similar to what we can do with HTTP-based API simulators like Wiremock. We can define the input which is HTTP GET method against the /api/sensors path and the output - in this case, following the JSON body (sensors.json):

Sensor in the kitchen

Sensor in the bedroom

SensorsFacade is a proxy class that translates methods invocation into an HTTP call using the RestTemplate. After removing boilerplate code, it’s as simple as that:

As above, if we skip the @PactTestFor annotation, it’s just a simple test. It has the standard “given, when, then” test structure (the given is the stub we have prepared earlier). We can stop here, and it will still bring us some value. Pact can be used as a mock server, similarly to Wiremock, which we mentioned earlier.

Let’s focus on two annotations now. The first one (@Pact) defines the consumer name and marks the method as a Pack method. The second annotation (@PactTestFor) connects the Pact method with a test case. That’s how the test knows how to stub API.

The last thing we need to add before we run the tests is the annotations used to let the test class know that we want to bring up the Spring context and enable Pact.

The test passed, and that’s the last point that CDC testing and testing with a Mock Server have in common. That’s because apart from the verification of our test case, the JSON file containing a contract has been generated in the target directory (target/pacts).

sensor_management

sensor_provider

As you can see, the contract is quite concise. Apart from json boilerplate, it contains only useful information about the expected API behavior.

Note that this contract has been simplified. Usually, the interactions node has many more children, each one describing a single Pact test case. I choose one GET and one POST request. Every interaction entry has:

provider state - it allows the provider to set up its state when validating the contract

response - expected, returned by the provider

Everything until this moment happened on the consumer's side. Now we need to send the contract to the provider and offer them a chance to validate it.

There are many ways we can share our contracts. It’s possible to use almost any way that includes S3/FTP/SCP. But Pact offers a component for storing and sharing contracts called Pact Broker. It provides many features out-of-the-box, which we would need to implement manually in other solutions (for example, S3).

We can use several different options for implementing the Pact Broker into the CDC flow. It can be deployed into our existing infrastructure because it’s an open-source project. It can also be used like a Software-as-as-Service available on https://pactflow.io/. In this article, we’re going to use the SaaS version of Pact Broker. After the registration, we get our own Pact Broker URL and API tokens (read and read/write).

Pact Broker exposes HTTP API we can use to manage contracts. But there are also plugins that make the process of sharing contracts easy to integrate with the existing build process.

And we’re ready to generate a contract:

That concludes everything we need to do on the consumer side. Now it’s time for the provider part.

The consumer generates a contract that describes what they want rather than referring to the provider’s real capacity to fulfill it. Each contract has to be verified by the provider to be treated as applicable. In our case, the provider is a simple 

 app that exposes two HTTP endpoints. It has the logic needed to validate the contract by design because is has some domain implemented. But we need to tweak the provider a bit to be able to fetch the contract and use it as an input to a test suite.

We need to do two things. First, we have to add a dependency to the Pact provider library (in our case, it’s a version for Junit5, but there are many more options):

Then we need to prepare code capable of verifying the contract.

This is the Junit5 test that can be divided into two parts: lines 1-5 are responsible for setting up Pact. We need to define how to connect with Pact Broker. We have also to tell Pact where it can expect the provider API (lines 16-19). Finally, we have also enabled the publishing verification process result to Pact Broker.

The second part manages the test flow. We inform the JUnit of how to perform the test in lines 26-30. It’s one line delivered by the Pact library we have added.

The next three methods are responsible for setting up the system to the state expected by the contract. Each case written in a contract defines the state in which the system has to be to act in some way. For instance, the contract we have prepared started with the following lines:

We now have to define this state so that the two sensors are stored and then returned by the API.

Some states don’t require any specific set up steps because the system is able to handle some behaviors on its own - for example, validating requests and managing access to non-existing resources.

Let’s verify whether the consumer expectations and producer capabilities have met.

The Pact junit library has quite good logs. Among them, the most important one for us is the following:

Let’s briefly check the pactflow.io dashboard to check if everything we did is reflected here:

As you can see, the contract has been shared and verified by the provider.

You may have noticed that even though we used a few tools, we still several manual steps we needed to take to produce, share, and verify the contract. More importantly, we didn’t do anything with the result yet. The ultimate goal of the CDC is to integrate it with the CI/CD pipelines and make everything work automatically.

In the end, the most important outcome of this process is the decision to go/no go to production. We’re going to explore this topic further in the next article of this short series about Consumer-Driven Contracts. Stay tuned!

Consumer-Driven Contract (2)

Overview of CI and CD pipelines - why is it so important?

Tomasz Sławiński

With outgrowing technology and amount of services around us manual integration and deployment process is no longer the right approach to build and ship your projects safely. Small and large businesses builds and ships their products more often than it was years ago. It is said that each day there is over a billion builds triggered. Can you imagine each of them started manually? I certainly can’t.

That’s when CI/CD pipelines come handy. CI/CD is a concept of delivering any application to a production environment with safety, speed and reliability and as least as possible impact on the customer side. It can be a manual process but in most cases, it’s fully automized and sometimes it will require pressing just one button in some CI/CD tool’s dashboard.

For any software that is built these days, there should be a CI/CD pipeline that will consist of individual steps responsible for different elements of the integration and deployment process.

When working in teams or even individually it is crucial to know that the code that is getting produced is reliable and stable. That’s where CI practice comes in handy. That’s a part of the CI/CD process which gets triggered each time the developer pushes a code change to the repository. That is when verification of newly created code should happen. It’s responsible for proving that the new code doesn’t break existing functionalities and integrates with current business requirements or alarming developers that there are some issues with it. Thanks to it the developers can take action before the code gets to production. Usually, it consists of two or more steps which are: build the project -> run unit tests -> run integration tests. More sophisticated pipelines would integrate with additional tools (such as SonarQube) in further steps to provide a detailed report on the quality of the code. A best practice is that the CI pipeline is getting triggered whenever any push to the repository happens.

Continuous Delivery is a follow-up of the Continuous Integration step and is responsible for getting the code changes deployed into the live environment. Thanks to this step being after CI we have certainty that the code is in the deployable state even if there were thousands of code commits done by multiple developers. In most cases, CD pipeline will get triggered only when push happens to certain branches (master/develop/tag) but it can vary based on companies' standards. Also in some cases, CD pipeline will be executed automatically or manually.

Nowadays there are numbers of tools that can be used for automated CI/CD process for any software. This can be done in Jenkins, Gitlab CICD, Bitbucket pipelines or if the code is hosted on cloud infrastructure Azure Pipelines, AWS Code Build and Code deploy or Code Pipeline. Most of those tools have numerous plugins and integrates with currently used project management tools. When all of that is properly configured life of any person involved in the development becomes much simpler as those plugins are capable of relating code commits with tasks in a way that the whole development history can be accessed from one place.

Let’s get some hands-on CI/CD process. Let’s say that you’ve been asked to set up CI/CD pipeline for a corporation that has several microservices that integrates with other various products and you are responsible for preparing CI/CD pipeline for one backend microservice written in 

How should then such pipeline look like? Let’s have a look at some most popular approaches here:

For this use case, we will be using GitLab CI and will focus only on the account microservice and has pom configured with appropriate execution goals. It’s a quite popular tool these days especially because it comes along with the GitLab repository and the CI/CD pipeline configuration sits within the project code in .gitlab-ci.yml file.

For this case we want that CI pipeline consist of three steps:

So how can we achieve it with GitLab CI? Let’s have a look:

The configuration above defines three stepped CI pipeline. The most interesting one is the integration test step, so let's have a bit closer look. For integration testing, we need some sort of data source and microservices that our account microservice talks to. The assumption here is that each microservice is dockerised and the docker images are held within GitLab docker registry. Within the .yml file we define that as a part of this step we want three other services running alongside – for that GitLab runner will pull latest images and start them up. After all that we can pass those to the integration-test goal as an environment variable. Those stages will be run for any code commit as we don’t specify a branch on which given step should be run. Below is a graphical representation of how will it look like on GitLab console:

As a next step, this CI pipeline could generate code quality report and push it to third-party provider tools (such as SonarQube).

Let’s assume that the business has DEV, QA and PROD environment. For each deployment we want it to be manually triggered by the developer/DevOps. In such case our pipeline would consist of four steps:

build docker image and push it to the repository with appropriate tag on it

For given CD pipeline only the build-image-and-push is an automatic step and will be trigger only when integration-test step passes. The deploy steps are manual and branch-specific. We won’t go into the details how does the deploy-* scripts look like as they will be different per infrastructure type.

When working with microservices it is crucial to have CI pipelines that could tell us if the whole system is consistent across all the microservices that builds it.

For that, we should have a separate project that would consist of a set of end to end tests and its own CI pipeline that would get triggered when a push to any of the microservices projects that build the solutions happens. Such CI pipeline can be set up on Gitlab Ci or Jenkins or any other standalone tool such as 

Consumer-Driven Contract (3)

03 JANUARY 2020 • 15 MIN READ