Second article of our series. An example of CDC implementation with the usage of Pact. 

Being an eMobility leader in various projects across multiple startups, Piotr was responsible for OCPI and OCPP integration as well as EMP and CPO platforms development. With extensive expertise in the open standards field and eMobility tech solutions, he is leading teams of experienced developers and contributing with invaluable knowledge and experience.

Piotr Majcher

, we described the principles of the Consumer-Driven Contract approach. We went through the potential use cases of CDC and scenarios where the CDC doesn’t fit. Since the Consumer-Driven Contract is a concept, it may be implemented in different ways.

This article focuses on the implementation based on Pact and using the following tools/frameworks:

Spring Boot-based applications as API provider and consumer,

Pact libraries used by the API provider and API consumer, and responsible for publishing (consumer) and verifying (provider) Contracts,

For the purpose of this article, we prepared a sample API. It’s a simple HTTP API exposing sensors data. Three methods are exposed here:

GET: /api/sensors/{sensorId} - get single sensor

Each sensor has two properties: id and name.

Everything starts from a technical perspective here. Before the agreement between provider and consumer is made, the documentation defined by the provider needs to be shared with the consumer. But the CDC flow begins with the consumer defining the contract. Before we start writing code, we have to add the following dependency to our project:

Defining the contract takes the form of an integration test. Let’s stop thinking about the CDC for a moment and consider preparing an integration test for the component that makes HTTP calls. You need two things: the stub for the API and the test. Let’s start with the stub.

Almost everything is self-explanatory. It’s an API stub definition prepared using the Pact fluent API. We haven’t touched on the topic of the CDC yet. It’s very similar to what we can do with HTTP-based API simulators like Wiremock. We can define the input which is HTTP GET method against the /api/sensors path and the output - in this case, following the JSON body (sensors.json):

Sensor in the kitchen

Sensor in the bedroom

SensorsFacade is a proxy class that translates methods invocation into an HTTP call using the RestTemplate. After removing boilerplate code, it’s as simple as that:

As above, if we skip the @PactTestFor annotation, it’s just a simple test. It has the standard “given, when, then” test structure (the given is the stub we have prepared earlier). We can stop here, and it will still bring us some value. Pact can be used as a mock server, similarly to Wiremock, which we mentioned earlier.

Let’s focus on two annotations now. The first one (@Pact) defines the consumer name and marks the method as a Pack method. The second annotation (@PactTestFor) connects the Pact method with a test case. That’s how the test knows how to stub API.

The last thing we need to add before we run the tests is the annotations used to let the test class know that we want to bring up the Spring context and enable Pact.

The test passed, and that’s the last point that CDC testing and testing with a Mock Server have in common. That’s because apart from the verification of our test case, the JSON file containing a contract has been generated in the target directory (target/pacts).

sensor_management

sensor_provider

As you can see, the contract is quite concise. Apart from json boilerplate, it contains only useful information about the expected API behavior.

Note that this contract has been simplified. Usually, the interactions node has many more children, each one describing a single Pact test case. I choose one GET and one POST request. Every interaction entry has:

provider state - it allows the provider to set up its state when validating the contract

response - expected, returned by the provider

Everything until this moment happened on the consumer's side. Now we need to send the contract to the provider and offer them a chance to validate it.

There are many ways we can share our contracts. It’s possible to use almost any way that includes S3/FTP/SCP. But Pact offers a component for storing and sharing contracts called Pact Broker. It provides many features out-of-the-box, which we would need to implement manually in other solutions (for example, S3).

We can use several different options for implementing the Pact Broker into the CDC flow. It can be deployed into our existing infrastructure because it’s an open-source project. It can also be used like a Software-as-as-Service available on https://pactflow.io/. In this article, we’re going to use the SaaS version of Pact Broker. After the registration, we get our own Pact Broker URL and API tokens (read and read/write).

Pact Broker exposes HTTP API we can use to manage contracts. But there are also plugins that make the process of sharing contracts easy to integrate with the existing build process.

And we’re ready to generate a contract:

That concludes everything we need to do on the consumer side. Now it’s time for the provider part.

The consumer generates a contract that describes what they want rather than referring to the provider’s real capacity to fulfill it. Each contract has to be verified by the provider to be treated as applicable. In our case, the provider is a simple 

 app that exposes two HTTP endpoints. It has the logic needed to validate the contract by design because is has some domain implemented. But we need to tweak the provider a bit to be able to fetch the contract and use it as an input to a test suite.

We need to do two things. First, we have to add a dependency to the Pact provider library (in our case, it’s a version for Junit5, but there are many more options):

Then we need to prepare code capable of verifying the contract.

This is the Junit5 test that can be divided into two parts: lines 1-5 are responsible for setting up Pact. We need to define how to connect with Pact Broker. We have also to tell Pact where it can expect the provider API (lines 16-19). Finally, we have also enabled the publishing verification process result to Pact Broker.

The second part manages the test flow. We inform the JUnit of how to perform the test in lines 26-30. It’s one line delivered by the Pact library we have added.

The next three methods are responsible for setting up the system to the state expected by the contract. Each case written in a contract defines the state in which the system has to be to act in some way. For instance, the contract we have prepared started with the following lines:

We now have to define this state so that the two sensors are stored and then returned by the API.

Some states don’t require any specific set up steps because the system is able to handle some behaviors on its own - for example, validating requests and managing access to non-existing resources.

Let’s verify whether the consumer expectations and producer capabilities have met.

The Pact junit library has quite good logs. Among them, the most important one for us is the following:

Let’s briefly check the pactflow.io dashboard to check if everything we did is reflected here:

As you can see, the contract has been shared and verified by the provider.

You may have noticed that even though we used a few tools, we still several manual steps we needed to take to produce, share, and verify the contract. More importantly, we didn’t do anything with the result yet. The ultimate goal of the CDC is to integrate it with the CI/CD pipelines and make everything work automatically.

In the end, the most important outcome of this process is the decision to go/no go to production. We’re going to explore this topic further in the next article of this short series about Consumer-Driven Contracts. Stay tuned!

Our introduction and the first article of the series about Consumer-Driven Contact.

Some time ago, we had a discussion in one of our projects whether it was the right time to introduce the Consumer-Driven Contract (CDC) approach. After a short debate, we came to the conclusion that while CDC may be a very useful tool, it’s not a silver bullet. It’s extra to what we already have and makes some processes much faster and less error-prone.

Normally, as a part of the API developers prepare some documentation like a swagger or wiki page. They specify all the possible input values with information on whether a particular parameter is mandatory or not. They do the same with the returned value. All values returned by the API are described in this document. We share documentation with third parties (or with other teams inside a company), and that’s all.

There is no space for feedback from the API’s user. We don’t know how our API is used on the data level. We probably have some monitoring in place, and we know how often users are calling our service. But do they use the entire response? Or maybe just one field? As in many other places, a short feedback loop may bring value here.

CDC is nothing else than the feedback from users about how they’re going to use our API. CDC is all about reversing the most common approach I described above. With CDC, the contract is not documentation written by the provider, but information from the consumer describing how they use the API.

Contracts over communication and documentation?

Does it mean that someday instead of receiving an email from someone who is going to use our service or ask us for something custom we’ll receive just a formal contract? Or does it mean that the consumer will define the API and we will just have to implement it? Of course, in most cases, that’s not going to happen.

The CDC doesn’t remove the need for good documentation. And for sure, it doesn’t replace face-to-face communication with a formal contract. Especially the second element - communication - is crucial for implementing the CDC successfully. In most cases, the CDC is only about feedback on how people use our API. We’re still responsible for providing documentation and discussing the details if the API we’re providing is something new. Sometimes not only the contract but the entire API may be driven by the consumer, using the same tools (some kind of formal contract). It all depends on our process and relations between the consumer team and the producer team.

How risky is refactoring code without test coverage? We would have to go to each place where the refactored function is used and check whether everyone is working in some way. Sometimes, it’s too expensive and risky so developers might give up.

Here we have a similar situation but in the HTTP world. Since we may not work with all the API consumers in the same office, such verification is usually even more expensive. Even in the same location, asking all developers to check the code because we are going to fix a typo in one of many fields returned by our API (which may be never used by someone) doesn’t sound good. By keeping contracts from all consumers in one place, we can quickly check whether anyone uses a specific parameter. We can safely modify our code without thinking about backward compatibility. The CI won’t pass any element that breaks any of the existing contracts.

It may sound as if we’re looking for a quick fix instead of proper API versioning. But API versioning is very expensive and complicated. We should avoid it for as long as possible (

). The CDC may allow us postpone that moment when versioning is the only solution.

Another aspect, in some cases an even more important one, is testing distributed systems. When we’re writing integration tests for a single service, we have two options - set up all dependencies or mock them. Since a contract consists of input and expected output, it can be used for automatic stubs generation. Having stubs for all the dependent services, we can easily prepare an isolated test that doesn’t take hours to start or need as many resources as integration test setting up many services.

What if we don’t know our consumers? Of course, we can’t ask them to prepare contracts. It means the CDC isn’t going to work in our case. In such a situation, we have to treat our documentation as a contract and be very careful when modifying anything resulting from changes to API. We have to assume that all the data we return are consumed by someone. We need to keep backward compatibility on the basis of our documentation, not specific use cases.

The second case when the CDC may not be a good fit is when we’re working with a public API. Even if we know all of our consumers, it may be really difficult, or in some cases impossible, to force them to spend time on preparing a contract and keeping it updated. It’s sometimes the matter of a paper-based contract signed by a business that states the expectations of both parties. In such cases, the CDC may not be the best solution. The biggest benefit of using the CDC will be visible if both sides of the contract work at the same company and share the same development practices.

One of the most important steps in CDC is sharing the contract written by the consumer with the producer. It can be done, for instance, through a merge request containing a contract in a format supported by the producer. Contracts may be also shared using a special tool, for instance, Pact (Pact Broker). Pack is a great tool that offers much more than just sharing contracts. It supports the entire CDC flow, including testing consumers using stubs created based on contract, sharing the contract, and verifying the contract on the producer side. Another tool that may be helpful here is the Spring Cloud Contract. You may also use Pact Broker as a contract repository.

Contract-Driven Design and contract testing are two useful tools that bring stability to distributed systems. They allow getting rid of late validation of problems regarding the integration between services. It doesn’t require deploying a service and checking integration manually. The feedback loop is quite short and can stop the building process before an artifact is created. It’s also good documentation showing who uses the API we provide and how they do it.

Our introduction and the first article of the series about Consumer-Driven Contact. Click to learn more!

Consumer-Driven Contract (1) - how to increase stability in distributed systems.

Third part of our Consumer-Driven Contact series of articles.

Consumer-Driven Contract (2) - Pact-based implementation

 we described the base principles and implementation of the Consumer-Driven Contract without any automation.

In this article, we’re going to orchestrate CDC and make them part of the existing CI/CD pipelines. We will refer to the code from this post, so if you haven’t read it, some parts of this post may be unclear.

We’ve already chosen Pact as a tool that will help us introduce CDC. Naturally, we also decided to use the Pactflow platform that plays the role of a Pact Broker. This time, we will focus much more on the platform than on our code. We will prepare a simple pipeline using Gitlab CI/CD cooperating with a Pact Broker. Let’s dive into the details now!

This Picture describes the Customer CI/CD flow responsible for validating contracts.

Everything starts with a change in the Consumer codebase. It triggers the entire pipeline and, as a part of it, we’re going to check whether the Consumer assumptions of what can be delivered by the producer are valid. We described the process of preparing the contract and uploading it to the broker 

, but just to recall it: we need two commands here. One for generating the contract:

Before we move those two commands into the pipeline, let’s focus on the important part of publishing contract which is contract versioning.

The contract version is the internal Pact value that is not exposed to the users. But each contract may be identified by the consumer version and provider version. So instead of thinking in terms of contract versioning, we should switch to versioning of consumer and provider apps.

According to Pact best practices - which are similar to the best practices of versioning in general - we should be able to refer from the app version to some point in SCM easily.

The easiest way to do it is by appending the commit hash to the version. In real life, we would need to prepare some release scripts for our projects, but for the sake of simplicity, we’re going to use the feature of the pact Gradle plugin that allows specifying the consumer version explicitly.

As you can see, the parameter name is “providerVersion”. It’s a bit tricky because it's the consumer, not the provider, who publishes the contract. But maybe designers decided to name the parameter that way because the API consumer actually acts as a contract provider.

CONSUMER_VERSION will be populated by the GitLab pipeline with the current value during the pipeline run. We’re ready to prepare the first box from the diagram.

Here is the part of the GitLab pipeline responsible for generating and pushing the contract.

There are two steps we need to take here:

running a tests suite that builds the contract

and invoking the pact gradle plugin for publishing it.

Triggering the producer to validate the contract

Pact Flow offers two main types of webhooks:

A webhook has a form of an HTTP POST call. When a new contract is published, what we want to do is trigger the API Provider pipeline able to validate the contract. We’re going to prepare the Provider pipeline in the next section.

Let’s focus now on triggering the Provider pipeline. The Gitlab platform has an HTTP API allowing us to integrate the service with other tools. We can easily generate an API token to trigger project pipelines:

We have the url now, so we can configure the Pack flow webhook:

The code responsible for validating the contract has been presented in a previous 

. Let’s just go back to the command used to execute the check.

As we can see, it looks like something we can move to the CI/CD pipeline easily.

There some additional commands required by test containers used for testing, but functionally, it’s just invoking a single Gradle task. As we build the contract validation logic using the Pact library, the verification result is shared with the Pact Broker automatically. Now it’s time to push the customer’s pipeline waiting for information about the finished contract validation.

Here’s where Pact Flow webhooks come into play again. But it’s going to be a bit more complicated this time. Gitlab’s API allows us to continue a stopped pipeline, but we need to know the id of the job we want to push. Any time a job runs, a new instance with a new id of the job is created. We can’t prepare a static URL to continue the consumer’s pipeline. Since you can run two pipelines at the same time, referring to a specific pipeline for running it further is impossible. How to determine which pipeline should be unlocked?

Fortunately, there are few features and tools that make it possible.

First, there are the dynamic variables that can be used to construct a Pact Flow webhook url or body. So our endpoint may contain, for instance, consumer’s name and version, producer’s name and version, and more different variables. We decided to use the commit’s short id as a part of the customer version. So the webhook url may contain a commit identifier. But that’s not enough to trigger a blocked pipeline job. We have to find the job id.

As we already mentioned, Gitlab exposes the HTTP API. A job is a first-class entity, so we can perform a useful operation on it. For instance, we can look for all project’s jobs with a specific scope. So we can have something like this:

Note: projectId can be found in the Gitlab frontend.

We’re interested in jobs with a manual scope because that’s how we implemented the job waiting to be triggered. A manual job is a job that can be triggered by the click or API call. As a response, we get a detailed description of jobs. I removed 90% of the response to focus on the parts that are most important for us:

can_i_deploy

Good news: we have the commit short id and the job’s id! So in theory, we found a way to map the commit short id to the job id (or multiple ids). Now we have to trigger the the job. Here’s the Gitlab API method we can use:

We have the theory ready. But where to put all this code? Pact Flow allows entering a string as a webhook url. It sounds like a good candidate for AWS Lambda. And that’s how we implemented it. Lambda exposes the endpoint accepting the commit short id (acting as consumer version) and creates a really simple algorithm:

Get all jobs from a specified project with scope "manual,"

Find jobs with the commit short id matching requested one,

Trigger jobs based on job ids from step 2.

Lambda is exposed to the world using the AWS API Gateway and can be called by the Pact Flow webhook. Here is the most important part of the Lambda querying jobs and filtering the commit short id:

We’re not going to discuss the topic of security at the moment but, of course, such an endpoint needs to be secured somehow.

The code above is simple, but someone may say that it’s not the best option to maintain such a custom code. And actually, who should maintain it? But if you think about how many different CI/CD platforms companies use, there are some concepts in one platform that don’t exist in others, or have different definitions.

A successful introduction of CDC requires work on both sides, consumer and producer. In my opinion, this specific code should be maintained by the consumer. The consumer should know how to trigger its pipeline.

There’s another very important reason to keep this code on the consumer side which is security. Giving external company access to the API of our Gitlab may not be an option. Exposing one very specific method sounds much safer.

The webhook is triggered when contract is verified, no matter the result of the verification. The HTTP POST call to the registered url will be performed even if the contract doesn’t pass verification. It’s the customer’s responsibility to decide whether the new version should be deployed or not. Pact prepares a cli tool that ensures we can deploy some version of our code to the server safely. The tool has a self-descriptive name “can-I-deploy”. We have to specify a few parameters and, as a result, we get a go/no go decision. The invocation in our case looks as follows:

We can install cli manually, but Pact also delivers a Docker image containing all required dependencies. That makes wrapping this invocation in a pipeline job really straightforward:

This crowns the introduction of Customer-Driven Contracts into the Consumer’s CI/CD pipeline. Here’s the full Consumer’s pipeline diagram:

Customer-Driven Contracts/Testing is a pattern that is relatively expensive to set up. Once set up, it also requires cooperation between different teams, systems, and tools - which can be time-consuming. But in the end, we can have much more confidence when applying any changes to our API.

How to build consumer CI/CD pipelines with Gitlab and Pactflow. Click to read the whole article.

Consumer-Driven Contract (3)

API Acceptance testing with Kotlin, DSL, testcontainers, and snapshots.

Krzysztof is our Head of Products and he leads the technical development behind the suite of our ready-made solutions. Throughout the years in Solidstudio, he gained an invaluable insight into the eMobilty industry’s requirements and is now more than qualified to share his knowledge with others.

Krzysztof Ciombor

Let's be honest for a moment: Testing is hard. The promise of TDD is delusive. In the beginning, everything works great - we add new test cases, write the code, and then are happy to see green, passing tests everywhere. However, the longer a project is developed, the harder it becomes to maintain those tests. At some point, we're spending twice as much time on adding new features and refactoring the old ones because we need to update our test code as well constantly.

I think the main source of this problem is that we tend to focus on the implementation details too much.

What we could do instead is focus on the system as a whole and treat its internals as a black box. This is commonly known as integration and acceptance testing.

For the purpose of this article, we assume that we're working on a simple application for managing Electric Vehicles Charge Points. Imagine that we have a simple service capable of performing basing CRUD operations for Charge Points. We use the Spring Framework, Kotlin, and PostgreSQL as the DB.

The ideas and technologies we're showing here aren't anything new and have been around for a while now. This article describes how to use them together. Although the examples are based on Kotlin and Spring, the general principles they demonstrate are universal and can be applied to any language or tech stack.

Our goal with this architecture is providing a pleasant developer experience (DX) so that everyone is happy to write tests, the tests themselves are easy to implement and maintain, and the code is easy to follow and understand.

Before we dive into the test code, we need to explore testcontainers. For our integration tests, we have to connect to a database. Here are the available options for getting that done:

Use in-memory DB, like H2 - this method could cause trouble when using a DB-specific code in the implementation (for example, geospatial features).

Connect to a local DB instance - this option could be troublesome to set up. It requires each developer (+CI environment) to have a DB instance set up locally.

Use testcontainers - this will automatically take care of initializing the DB docker image, setting up a running DB instance, and cleaning up after the tests once they're completed.

As you can see, using testcontainers greatly simplifies the DB setup process. Please refer to the testcontainers 

 for full setup instructions. Here’s a simplified version:

1. Add testcontainers to build.gradle dependencies.

2. Override Spring properties for test, providing the correct testcontainers JDBC connection url (in application-test.properties):

The actual DB name is irrelevant. Adding TC_DAEMON=true allows us to keep the container running and be reused across multiple tests.

3. (Optional) To simplify things further, we'll create a base class for all the integration tests to inherit from. This will take care of selecting the correct test properties file (defined in the previous step) and provide a couple of useful helper methods. Such class looks as follows:

We'll see how those helpers come in handy once we start writing tests.

Here's where our custom Kotlin DSL comes into play. We want to be able to set up our DB with test data quickly. However, creating our object hierarchies is sometimes very tedious and requires a lot of boilerplate code. Creating a custom DSL helps with reducing that unnecessary noise from the test code by moving object creation responsibilities into the DSL code. Compare these two examples:

Not only the Kotlin code is less verbose, the DSL can do much more for us. For example, initializing default properties by generating random values using 

Kotlin code is also more explicit. How would you know if the geocoordinates constructor parameters order is lat/long or long/lat without looking into the code or using IDE to help you?

Once we have the objects ready we can easily persist them to the DB:

For acting step we will reproduce the usual app’s use-case, that is, handling an HTTP request. Inside our helper base class we already initialized mockMvc. Let’s take a look at a simple action of retrieving Charge Points by sending a GET request:

Notice, that by using mockMvc we’re emulating a real-world HTTP request which also goes through (and validates) elements like the Spring validation, authorization, JSON body parsing etc.

Finally, we'd like to confirm that the HTTP response is correct. We could check if the returned result includes all the correct fields manually, but that might include a lot of manual assertions for complex responses. What we could validate instead is whether the response is the same as before when we make changes inside our code. After all, the most important thing for us is not introducing some API breaking changes by accident.

This is where snapshot tests come into play. Let's see an example (based on the code above)

Create a new snapshot file when it's run for the first time.

Check if an existing snapshot is already present in the code and compare the result to see whether it is the same or not.

If the result and the saved snapshot differ, report a test failure.

An example snapshot could look like this:

As you can see, that's the exact JSON representation of the HTTP response, as expected. Now we can safely modify the internals of our app, and the test will validate that no breaking changes were made to the API.

However, sometimes we need to apply these changes - for example, when adding new fields. What then? We can easily update the snapshot executing:

Note that one should still manually validate that the update makes sense and the API should indeed change. We can do that easily during the code review process. Remember that snapshots need to be committed to the git repository!

There's one more scenario where we need to perform a different type of validation. Let's look at creating a new Charge Point. We still have to send an HTTP request via mockMvc, but the response could simply be HTTP 201, without any data returned. A snapshot would be of no use to us here.

How do we assert that the request did what it was supposed to do? We could "invert" the process a little. We can create an expected object using our DSL, retrieve the saved entity from the DB, and compare these two.

Note: Since we're clearing the DB for each test, it's fine to do findAll().first() since no other Charge Point (than the one created during this test) should exist in the DB. Using the recently introduced AssertJ functionality to compare objects recursively, we can easily validate that the saved entity matched the expected one. Skipping id fields is useful since they're only important for the persistence layer and will be regenerated by Hibernate automatically, so there's no point in comparing them.

Of course, there's no silver bullet when it comes to testing, and this approach has some trade-offs that you need to conside.

Integration tests are usually way "heavier" than unit tests, meaning that the test suite will take longer to complete (having to set up the DB, Spring context, API requests, etc.), leading to longer "feedback cycles."

It's difficult to reason about failures. Since the entire stack is tested at once, it may be tricky to pinpoint the exact place where an error occurred.

If you'd like to check out the full sample project, you can clone it from here:

https://gitlab.com/solidstudio-team/acceptance-testing-demo

We've achieved over ~80% of code coverage with a relatively small amount of test code. Adding new tests is easy and painless. The tests themselves are clear and concise. Note that I haven't said that Unit Tests are bad or unnecessary anywhere in this article. On the contrary, I believe that they're still useful and have their place. Unit Tests are always super useful for testing business logic. However, I believe that they should be applied only to "units" that don't have any dependencies. Once we start introducing mocks, are the tests really validating the correct behavior?

Developing a well-defined set of integration tests that emulates the real-world scenarios performed against our app's API is a great way to ensure that the app is behaving correctly. However, remember that what worked for us here may not work equally well for everyone.

API testing is not an easy task. Learn about making it a bit easier.