Quality Assurance automation on a serverless architecture

When encountering a new challenge, particularly on a project, the initial step is to engage in discussions with all the developers involved. This dialogue is crucial for gaining insight into the project’s status, determining the main areas of focus, ensuring optimal performance, and identifying the requirements. However, quite frequently, the Product Owner has some well-defined features, architecture, infrastructure, and UX/UI design. In these cases, it is important to be flexible, adaptable, and have a good understanding of their goals.

Such was the case with Mekanoid. Anton, Mekanoid’s CEO and one of its founders, not only had a well-defined idea of the end product but also of its architecture. With scalability in mind from the very beginning, Anton decided he wanted a serverless architecture for Mekanoid. A serverless architecture not only has the benefit of scalability, but it also comes with no server management and maintenance. The disadvantages, on the other hand, are cost and complexity.

A glimpse of the architecture

The whole application runs on AWS Lambda. Traditional lambda functions are difficult to deploy if not using a framework. In this case, the framework of choice was SST (Serverless Stack). In combination with Seed, a fully managed CI/CD (Continuous Integration/Continuous Delivery) platform by the creators of SST, the application would deploy to a staging environment every time a new PR (Pull Request) was opened on GitHub.

On the front end side of things, a React application would serve from AWS S3.

For storage, AWS Neptune and AWS DynamoDB were the chosen databases. As for user management, AWS Cognito was appointed.

Challenge

In a monolithic application, such as a Ruby on Rails application, one would normally set up the CI to create a new disposable database, start the application, and run some tasks, such as the linter and running the test suite. Normally, that test suite would be a Cypress test suite.

With this architecture, the way we normally do things doesn’t work anymore. We can’t start an SST application locally (ie. the CI) as it only runs in lambda, and we can’t create disposable database resources in the cloud that easily.

So how do we test a serverless application?

Solution

If you used Cypress in the past you’d know that its tests run in the browser and if you’re a QA engineer then you’d know that writing tests is like doing manual QA except that it’ll be automated. That’s it. So essentially, testing a serverless application should be the same as testing a monolith.

But then how do we bring up the whole distributed infrastructure to run the tests?

The answer is: with another CI/CD platform.

See, with Seed properly set up, whenever a push to GitHub happened, Seed would start and run the linter, the unit tests, and if successful it’d build the artifacts and deploy a new staging environment in AWS. With GitHub Actions, then, we could listen to the deployment_status event and, when it’s successful start the Cypress job.

If you’re curious, the .github/workflows/main.yml file looks like this:

name: CI

on: deployment_status

jobs:
  cypress:
    if: github.event_name == 'deployment_status' && github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 16
      - uses: cypress-io/github-action@v4
        env:
          CYPRESS_baseUrl: 'https://app-$.secure-url.com'
          CYPRESS_RECORD_KEY: $
        with:
          record: true
          wait-on: 'https://app-$.secure-url.com'
          browser: chrome

Note: secure-url.com is not the real URL. Hidden for security.

It’s also worth noticing that there should be an integration seed (for creating dummy test data) that runs as part of the deployment process of each branch, which is where tests will be run. The other environments in the Seed pipeline are protected against running these seeds.

Conclusion

In conclusion, when doing it right, QA automation for a serverless architecture doesn’t feel much different than QA automation for a monolithic architecture. As QA engineers, the challenge is always designing and delivering effective tests.