Serveless testing - tools and techniques.

Being a co-founder of Solidstudio, Paweł has got to see all the development phases. He’s a skilled professional with years of experience, both in the software and in the eMobility worlds.

Paweł Głogowski

Serverless architecture was one of the hottest topics in 2018. Just take a look at Medium articles and the number of threads and discussions on Reddit, Twitter, and other social media. It’s becoming more and more popular - and some well-known companies have started to use serverless architecture in their projects (Netflix or Coca-Cola, for example). However, popularity also serves to reveal some drawbacks and challenges of introducing serverless architecture. One of the biggest challenges is testing which is much different than in “traditional” applications. In this article, I would like to share some hints and tips about testing a 

The obvious difference between a traditional and serverless application is that it requires a lot of collaboration with external services - in the AWS world, it can be API Gateway, Dynamodb or S3, for instance. It’s often hard or even impossible to integrate with real services. Fortunately, there are some useful tools that can help with emulating some external services to help with writing tests. Let’s talk a little about possible tests that we can perform in the serverless world.

Firstly, to speed up every local development developers should be able to invoke code locally. At Solidstudio Software House we choose NodeJS to write our lambda functions because it’s lightweight and it’s easy to execute your code locally. We also find the serverless framework very handy as it that allows us to execute NodeJS code with some input parameters that can be really helpful.

Let’s imagine that our lambda subscribes to AWS S3 event for further processing. It thus has to consume the S3 event. It could be very helpful if we could use that event and invoke lambda as if it was consuming a real S3 event. We can easily capture the real AWS event and save it in a local JSON file. We can later use it for input of our lambda without actually using our infrastructure.

Of course, invoking lambda locally for several scenarios every time we want to change a single line of code isn’t very efficient. That’s why we need to write some automated tests. The most basic and easiest type of tests is, of course, unit testing. It's a must-have in all kinds of projects and the serverless environment is no different. The good thing is that serverless functions are "well-modularize” and so can be unit tested easily.

Like in any other system, we need to have decent code coverage in unit testing. We need to simply mock every interaction with external services. If we want to simulate some AWS events, we can use the captured AWS events discussed earlier. Every modern programming language has a framework to write and mock unit tests with. For NodeJS, we can use Mocha or Jasmine which offers a very easy way to simulate external events. Building and running tests must be a part of building a deployable package.

Once you have your code well-tested locally and are confident that the code is working correctly, you should deploy it to the development environment. This process need to be automated to avoid the potential risk of manual deployment. In serverless development, it’s crucial to test your function on the real environment because of some configurations like IAM permissions or AWS lambda limitations that don’t occur in the local environment like memory, disk space, timeout, payload size. Sometimes we forget about these limitations and something that works well on our local machine may fail in the real AWS world.

When our code has successfully reached the dev environment, you can check its behaviour by emitting event that particular functions are subscribed to. You can, for instance, upload a file to the target bucket and check if particular lambda behaves as we expect. Obviously, it's just a technique to debug and improve the development process - and testing cannot rely on it. We need some automated tests to make sure that all our functions work well after every change. We should run the entire test suite after applying changes to a single function.

Integration testing in serverless application

In a serverless project, we also need to have a decent integration of the test suite to make sure our application works as expected. Each lambda usually handles one event and most of the time we need several lambdas to cover our business story. Creating a user by the REST API can trigger events that will be consumed by several lambdas that will handle adding a user to DynamoDB, uploading photos to S3, or sending greeting emails. So, one integration test must make sure that all these three lambdas are invoked and done their job properly.

With a serverless framework along with Event Gateway, we can very simple hook up our functions to particular events and trigger our functions. Then we need to have code that will verify whether the lambdas have done their job well. It's good to write tests in one language rather than mix several techniques. The programming language should work well with AWS and update their SDK quite often so it works with the newest version of services.

Each test suite has to start with an emitting event and wait till lambdas react on it. That could be challenging as there’s no easy way to indicate whether a lambda is complete. One approach could be invoking some lambda and waiting for some fixed calculated time before testing - for instance, we could take lambda timeout as our wait time. However, it's not a good idea since delays may happen or some lambdas will finish before that time - its either risky or inefficient.

The best approach is checking periodically if some condition is fulfilled - if we expect an object to appear on S3 then we can check whether it’s there, as well as check for data in our storage (DynamoDB, RDS etc). We can then finish the test as soon as the lambda has finished its job, or fail if the outcome is not what we expected or the lambda hasn’t finished its work in a reasonable amount of time.

At Solidstudio Software House, we’ve started our latest project in a fully serverless architecture. We noticed that it requires a significant mindset change, but developers who have in microservice architecture should be able to move to serverless quite smoothly. As serverless architecture becomes more and more popular, they can take advantage of reliable and easy-to-use tools. We find serverless framework and event gateway as the best technology choice.

 If you’d like to stay in touch, please sign up for our newsletter: no spamming, just information about the latest content.

Read this article to get familiar with serverless testing.

Krzysztof is our Head of Products and he leads the technical development behind the suite of our ready-made solutions. Throughout the years in Solidstudio, he gained an invaluable insight into the eMobilty industry’s requirements and is now more than qualified to share his knowledge with others.

Krzysztof Ciombor

. Read it first, if you haven’t already. Today, we’re going to take a closer look at snapshot testing to show you how we improved it and took it to the next level.

The original idea of Snapshot Testing was first included in Jest - have a look 

 to learn more. Its main purpose was testing frontend component changes over time to catch any unexpected UI changes. This concept, however, is more universal and can be applied to almost all codebases. The overall process is always the same:

Execute your code (either by rendering the UI, calling a method, or issuing an HTTP request).

If you were running the test for the first time, save the result to a “snapshot” file.

Otherwise, find an existing “snapshot” file and compare it with the result. If the contents are the same, pass the test. If the contents don’t match, report the test failure.

By using snapshot testing, we get a few benefits:

The tests are easy to write and maintain.

Since they are essentially similar to integration/acceptance tests, we’ll be able to validate the flow through the entire application stack (i.e. HTTP requests, Service layer, DB integration, etc.).)

They are a “simpler” alternative to Contract Based Testing (see 

) that still give us confidence that we’re not introducing any breaking changes to our APIs.

Our initial implementation of Snapshot Tests was based on the 

 library. It worked great at the beginning, but a couple of issues started to arise once we added more and more tests. One of the most frequent ones were non-deterministic JSON outputs, for example, when generating random UUIDs. We were frequently facing failures like this:

Even though it makes sense from the String equality perspective, it doesn't really matter from business requirements side. As long as the output JSON contains the id field, we don’t really care about its content. This can be mitigated by creating deterministic helpers for UUID creation.

The next issue started happening when we compared arrays. Our output was very often serialized with arrays contents in random order.

The second biggest issue we faced was the order of arrays. Even though the serialization itself should preserve the elements order, we can’t guarantee that the collections were always sorted the same (for example, a simple .addAll() on a Collection doesn’t have any ordering guarantees). Again, this became a problem when comparing snapshots as strings, even though from a logical standpoint, as long as the array elements are the same, the ordering doesn’t really matter.

So the question we started asking ourselves: “Can we do better than this?" And the answer is… “Yes we can!”

For our purpose here, we’re using snapshots for validating API responses. Since they always come in the form of JSON, we can take advantage of that. As it turns out, there’s a library out there that is perfect for our 

. However, this also means that we had to ditch the KotlinSnapshot library in favor of a custom implementation. Fortunately, that’s not so complicated to achieve and in our PoC we were able to (almost) replicate all the functionalities of KotlinSnapshot, re-using a couple of its code snippets. One of our goals was to keep the compatibility between both to facilitate the migration of the existing tests. In the end, the only required for existing test code is as simple as:

So what exactly do we get by switching from String comparison to JSON comparison? Here are two major benefits:

Taking advantage of JSONassert’s JSONCompareMode.NON_EXTENSIBLE comparison mode, we can compare an array’s contents disregarding their order, but still guarantee that no element is missing and no element was added. This means that we no longer have to focus our efforts for making the array order deterministic and can instead concentrate on the array’s contents.

Maybe a more interesting part comes with custom validation config which can be passed to validateSnapshot method. Using JSONassert’s full potential, we can ignore properties that we don’t want to validate during the snapshot comparison. Going back to our previous example, we can now easily exclude id fields from being checked for equality by doing:

Moreover, we can use a wildcard operator for more sophisticated exclusions, namely:

 - Will match all id fields, no matter the nesting level

 - Will match ALL fields (useful for doing selected fields comparison only)

Keep in mind that ignoring the field for validation only skips the comparison part. The field still needs to be present in the response, so we don’t need to worry that we’ll introduce API-breaking changes by removing some response fields.

One should always keep in mind that there’s no such thing as “silver bullet” in software development. Each solution comes with its tradeoffs. That’s the case with snapshot testing as well. There are still a couple of pain points with using it as the main testing method, namely:

These tests are much “heavier” than unit tests, meaning that the “feedback loop” is considerably longer.

Sometimes it will be difficult to reason about failures and pinpoint the exact erroneous place in the code.

Snapshot files require maintenance, they must be updated every time a change in the API is correct and required.

Assertions are not obvious and it may not be immediately visible what is being tested just by looking at the test code.

Pull Request diffs “blow up”. A single change to add one more parameters to the response DTO may cause dozens of snapshot to be updated. Since snapshot files should always be committed to the repo, those changes will show up in the code review.

As usual, it’s best to always evaluate potential solutions and new approaches with care and adjust them to our needs in the project. What worked well for us may not work so well for others. We are quite happy with our current test stack, and we’re able to achieve ~80% code coverage easily with just snapshot tests and a couple of unit tests mixed here and there for the most crucial business logic. This allows us to be confident when making changes, adding new features, and refactoring the old code.

How to take snapshot testing to a next level? Read our article.

Snapshot testing

Learn more about the advantages of both solutions Amazon Web Services and Oracle Cloud 

After founding Solidstudio, Pawel was an originator in terms of pursuing expertise in eMobility. With the build-up of a solid knowledge-base on industry’s needs & wants, he now keeps a close eye on products’ development. He’s also hugely responsible for the business’ side and designs the business models for our products.

Paweł Małkowiak

During the last decade, cloud computing has been growing at an exponential rate. The size of the cloud computing market is today larger than ever. COVID-19 outbreak accelerated digital transformation, making cloud computing a priority of choice. In 2020, the public cloud services market reached $257.5 billion in 2020. And according to Gartner forecasts, in 2021 the market revenue will increase 18.4% to a total of $304.9 billion.

Naturally, the cloud computing market has witnessed the rise of several players who now dominate the scene. Gartner's 2020 Magic Quadrant for Public Cloud Infrastructure Professional and Managed Services, positions:

If your organization aims to become more responsive to market changes and increase its operational efficiency, moving to the cloud is a smart move. And sooner or later, you'll have to choose a cloud provider. That's why we are here to give you a helping hand.

Why compare Oracle Cloud and Amazon Web Services? We decided to compare these cloud computing solutions delivered by two different vendors – a market leader and a smaller, but promising provider – to show you their pros and cons. Present how they differ in detail and indicate which factors you need to take into account when choosing the best cloud solution for your business.

Before we jump into details, let’s clarify some significant abbreviations, to be sure we’re on the same page. While browsing tips and clues on computing services you may find IaaS, PaaS, SaaS, DaaS and serverless. Following Microsoft and techopedia definitions:

IaaS - Infrastructure as a service - infrastructure is hosted on the public or private cloud instead of in a traditional on-premises data center. The infrastructure is delivered to customers on demand while being fully managed by the service provider.

PaaS - Platform as a service - offers the same kit as IaaS, plus additional features - middleware, development tools, business intelligence (BI) services, database management systems, and more. PaaS is designed to support the complete web application lifecycle: building, testing, deploying, managing, and updating.

SaaS - Software as a service - a complete software solution that you purchase on a subscription plan from a cloud service provider.

DaaS - Desktop as a service - a cloud computing solution in which virtual desktop infrastructure is outsourced to a third-party provider. DaaS relies on the virtual desktop, which is a user-controlled session or dedicated machine that transforms on-demand cloud services for users and organizations around the world.

Serverless - allows developers to build applications faster by cutting out the need to manage infrastructure. For serverless applications, the cloud provider automatically provisions, scales and manages the infrastructure required to run the code.

Note: the term ‘serverless computing’ applies to the event when the tasks linked with infrastructure provisioning and management are invisible to the developer. Servers are still running the code.

Oracle Cloud is a relatively new solution on the market and can't be compared to AWS in terms of scale. While Oracle Cloud might not occupy top positions in global cloud service provider rankings, the solution has a lot to offer. In fact, Oracle has set its cores on becoming a cloud leader in the future.

To help you recognize what Oracle Cloud is all about, let's explore its key features and capabilities.

By combining IaaS, PaaS, SaaS and DaaS solutions, Oracle provides a broad range of cloud services and products such as:

Business analytics, including Business Intelligence (BI) solutions, Big Data, data visualization, autonomous analytics, and more.

Application development, including mobile, Internet of Things (IoT), blockchain, database, and more.

Cloud infrastructure setup and maintenance.

Cloud for customer services, Customer Relationship Management (CRM) systems.

Security solutions focused on safety and compliance.

Oracle Cloud offers many services and products as an integrated set, allowing companies to lower their spending by eliminating the need to purchase third-party solutions. This is just one of the many advantages of Oracle. Check a detailed look at the pros of Oracle Cloud:

Integrated technology stack – Oracle delivers apps, middleware, and database as an entire, well-integrated system. There's no need to create a patchwork of third-party services coming from different vendors and go through the trouble of adapting them to the solution. To make the most of Oracle Cloud, it's best to invest in the all-inclusive version.

Smooth customization – Oracle allows its clients to be flexible in setting up the cloud service to match their unique requirements.

Cutting-edge technologies – Oracle is an industry leader whose advanced technologies make its cloud platform fast and high-performing. Speed is definitely a factor that sets this solution apart from others.

Scalable architecture – Oracle offers a sophisticated infrastructure that is fully scalable, allowing organizations to expand their IT platforms as they evolve. That feature is important for large enterprises that are going through the process of digital transformation and may want to revamp their operations in one area after another.

Niche focus – when building its cloud offer, Oracle focused on creating solutions that would match the requirements of specific customer segments. Instead of trying to grow its platform in every possible way, Oracle focuses on improving these capabilities.

Great customer support – you can be sure that your issue will be addressed quickly and effectively.

No solution is perfect, and Oracle Cloud also has its cons. Here are the most important disadvantages of this cloud vendor you should be aware of:

Poor low-end offering – since Oracle is focused on the high end, the offering around the low end is limited.

Poor performance with solutions based on .NET.

A minimum product might not be enough – Oracle provides all the services and products out-of-the-box, but these are very minimal solutions that might not be sufficient for companies that need simple functionalities. Organizations in need of a comprehensive platform are even less likely to get what they need.

Amazon Web Services (AWS) is a longstanding leader in providing cloud servers and storage to organizations looking to offload their data center infrastructures. In the fourth quarter of 2020, AWS generated revenue of approximately $12.7 billion.

Here's why so many companies choose AWS as their cloud provider today.

AWS offers cloud storage that supports managed, hybrid, block, and low-cost archive storages (such as Elastic File System, Storage Gateway, Glacier, S3, EBS), as well as a wide range of data transport functionalities.

Support for various databases (Aurora, RDS, Redshift, DynamoDB). Note: Aurora, Redshift, and DynamoDB are AWS custom technologies.

Support for advanced technologies (the Internet of Things, augmented reality and virtual reality, machine learning)

Tools that enable monitoring of workloads, configuration, and more.

Tools for security and performance optimization.

Analytics and business productivity, Business Intelligence (BI).

A broad range of developer tools (AWS Code Build, CodePipeline, CodeDeploy).

Top security, identity, and compliance management.

AWS has an excellent track record of successful implementations, and its financial performance attracts even more organizations to its offer.

Here are the key advantages of AWS over competing solutions:

Wide range of capabilities – compared to other cloud providers, Amazon offers the largest number and variety of functionalities. Large organizations invest in AWS because it's a sure-fire way to take full advantage of everything cloud technologies have to offer.

Easy implementation, migration, and management – AWS is surrounded by a large ecosystem of consulting partners and independent vendors who help companies to implement AWS-pre-integrated SaaS solutions.

Security and compliance – companies migrating to AWS can be sure that the processes they perform in the cloud are fully secure and compliant.

Excellent support – Amazon's global reach is reflected in the technical support it provides to the customers of AWS. The extensive multilingual customer support with high availability makes the solution one of the most reliable options on the market. The list of supported languages is constantly growing too.

Data centers around the world – Amazon operates multiple small data centers across the entire globe. This enhances service localization and limits the risk of local incidents.

Continuous offer development – AWS is constantly evolving and enriching its platform with new offerings. The vendor also improves existing functionalities, especially their integration and management.

Pay-as-you-go fee model – AWS offers a pay-as-you-go fee model where customers get to control over how many services they pay for, adding new ones as they grow. Minimizing upfront costs is easy with AWS. Companies can lay out expenses as they grow, without investing in products or services they don't need.

Still, AWS has its limitations that might inspire some customers to choose a different provider. Here are the most relevant disadvantages of AWS you need to take into account when making your decision:

The service implementation is complex – many AWS services require a lot of expertise when it comes to their implementation and support. Moreover, companies that want to make the most of them need to have a firm grasp of service updates and innovations. However, you can be sure that managed services providers know how to implement AWS technologies properly and can seek their assistance if you need it.

High pricing – an AWS-based solution might seem to come at a reasonable price. But if you want to create a custom solution that meets the specific requirements, be prepared for larger expenses than expected.

AWS or Oracle – which one should you choose?

The payment model of AWS allows smooth adaptability to market changes and fluctuations, helping your business stay nimble and invest in cloud computing at its own pace. Oracle Cloud also enables users to leverage flexibility, depending on how often customers need to rely on its services. AWS has established a large market presence and operates multiple data centers across the world. Their experience in delivering cloud services and excellent multilingual support is a significant competitive advantage.

Oracle Cloud and Amazon Web Services (AWS) are the two notable players in the cloud market. Choosing between the two options might be tricky, but we hope this guide helps you determine which platform will suit your business needs best.

Running a small startup or an enterprise-sized organization? It doesn’t matter - your data and cloud computing needs will rapidly grow, so you need a solution that will accommodate easily. At Solidstudio we help to fit the right solutions. If you still have some concerns, feel free to drop your e-mail. Or fill the contact form and take advantage of our digital transformation experience. Check back soon!

develops a fleet of software products designed to facilitate EV charging network management. See more about our 

Learn more about how Amazon Web Services (AWS) compares to Oracle Cloud platform and what to choose to fully benefit the cloud.

Oracle Cloud vs. AWS - Which one should you choose for your business?

Take advantage over your competitors with cloud

At first glance, you may think that the e-commerce market has reached its maximum limit and there is no way to grow. However, by using new technologies companies can still increase their revenue - for example, by providing better recommendations or creating more attractive and user-friendly websites.

In this article, I will focus on describing new trends and technologies that help to improve e-commerce businesses. I won’t focus on Big Data technologies that are vital for e-commerce, but instead turn to the cloud technologies, Java, and Node.js.

There are always new trends emerging in online shopping on the e-commerce scene. First is using technologies that offer a literally one-click purchasing method that is quick but also safe. E-commerce ventures are also very interested in artificial intelligence (AI) - it includes chatbots or automatic customer service processes. AI is becoming more and more popular since it can easily interact with customers and facilitate the buying process. The last huge trend in online marketing is Machine Learning that helps profile the user experience for each individual buyer. However, it seems that e-commerce business won’t be able to achieve any of above if they don’t migrate their solution to the cloud.

One of the main problems in e-commerce business is its seasonality. There are some times during the year where the demand for online shopping rises - like Christmas or Black Friday. Companies used to lose billions of dollars because they couldn’t handle the increased volume of transactions. On the other hand, investing in architecture that will be used only for a couple of days during whole year isn’t very efficient. So is there a way to handle that situation? Yes - and the answer is the Cloud!

On AWS, it is possible to improve your architecture based on the current demand and then reduce it when it’s no longer required. This process is completely automated and adjusted to demand. It's very convenient and can help companies to save a tons of money. You can compare it to paying for electricity - you just pay for how much you consumed. You do not need to maintain and pay for servers on the virtual machine when you actually don’t need them.

Moving an e-commerce solution to cloud is also beneficial for other reasons. If your e-commerce solution is hosted on-premise, you need to deal with lot of additional problems like storing, maintaining and securing physical machines. Additionally, you should update software on a regular basis. Experience shows that it usually does not happen and sooner or later you end up with outdated software and less-than-expected availability. Also, you need more resources to maintain all servers. The cloud, on the other hand, is free from these issues. You need to set up your architecture once and then focus on developing software to meet the changing requirements. Your team won’t have to implement solutions like autoscaling or security backups since they are already provided by AWS.

At Solidstudio Software House, we have a lot of experience in moving monolith applications into the cloud, and maintaining and managing them accordingly. We are familiar with AWS solutions that can help your business grow.

Why Java is a great option for ecommerce?

Each branch of e-commerce is different and needs to face different problems. In e-commerce, there are many problems to solve. You will have problems with performance if you plan to sell a lot of cheap products relying on the scale of your project. However, if you plan to sell few but very expensive and luxury products then you need to rely on marketing and relations with customers. Each of these approaches requires different tools.

Java supports a lot of different frameworks which solve these problems. If you are an omnichannel company that needs complex tools for marketing or a CRM solution with customers, you may pick Hybris. If your company is smaller and you just need some basic support for orders or payments, you can choose one of the dozens available e-commerce solutions.

Java as a language that can also bring a lot of benefit for your company. It offers a lot of well-established tools, there are already many engineers who know how to use Java, the language itself is developed and updated accordingly to worldwide trends. That popularity makes it really easy to integrate Java tools with various solutions. The only drawback I can see from using Java as an e-commerce business is that for a very small scale it can be a little overkill. If you need to set up a small online shop, then using Java along with frameworks can end up becoming too costly. Consider using other languages to avoid following some Java conventions that are not feasible to all solutions.

Java is our main expertise at Solidstudio Software House and we already integrate our clients’ applications with dozens of third-party solutions, not only from the e-commerce field. We can help you choose the best option for your needs and integrate it with selected solutions seamlessly.

Node.js as a great tool for smaller scale.

If you do not need a complex CRM tool or support for online marketing and your business is rather small, you can start with some basic solutions based on Node.js. It uses Javascript for both the server and client side which can be beneficial since you need only experts with one programming language skills. It also offers great performance, so the costs of Node.js solutions are rather low. The existing Node.js frameworks like vue.js or koa.js can be really helpful for solving e-commerce problems. Node.js is a great tool to implement an MVP solution, however once your scale grows and you will need more complex tools and frameworks, we recommend switching to Java and one of its frameworks.

E-commerce has still a huge potential for growth and there is a lot of challenges that e-commerce ventures will have to face in the future. Technology can be a great support in achieving more, so keep a close eye on the cloud revolution. If you waste your time on outdated solutions, you will surely lose with your competitors. Start to move into cloud and use available tools wisely to be two steps ahead!

AWS and Java - a good duo for your ecommerce environment

Introduction to serverless testing

05 APRIL 2019 • 10 MIN READ