(Cross-post) End to end serverless testing with the Driver Pattern

(Cross-posted from https://blog.freetrade.io/end-to-end-serverless-testing-with-the-driver-pattern-bcf2d22bf73a)

In September 2019, Freetrade’s test code consisted of a large number of fast unit tests and a smaller number of end-to-end API tests. These make real HTTP requests against the platform and assert on the results.

The unit tests confirm that individual classes are behaving correctly and improve the design of interfaces, while the API tests confirm that higher-level business logic is working correctly. This is not a bad combination and at the time already gave us decent test coverage of a large proportion of our platform.

However, the API tests could be flaky and were giving us a lot of false positives about failures. That was due to the way they make real network requests and then wait for asynchronous operations to complete.

As a result, the integration tests were difficult to maintain, as a given test can’t determine if a failure is due to an asynchronous operation in the platform taking a little longer than expected, or due to a genuine bug. Running the tests also requires deployment to a real Google Cloud Platform project, which adds friction to the development iteration cycle.

While we were finding ways to improve the integration tests and make them more effective, we also decided to experiment with another testing approach: swapping out Firebase as a “driver”.

The Driver Pattern is related to the Bridge Pattern, Facade Pattern and Proxy Pattern. I’m not a fan of getting obsessed with patterns, so I don’t think a super-distinct definition is that helpful. The key idea is to access an area of functionality through a single point of entry (the driver), which allows you to easily swap the underlying implementation from a single place. Whether you call this a driver, bridge, facade, proxy, gateway or something else, it’s the seamless swapping in and out of implementations that is important.

In our testing strategies, we created an in-memory, single-process implementation of the key Firebase behaviours that we relied on in our platform. This implementation is an ongoing work-in-progress and has been open-sourced here.

In production, pre-prod and integration test environments, the driver uses the real Firebase platform. In “driver tests”, as we’ve started calling them, Firebase is swapped out for the in-memory, single-process implementation.

This has several benefits:

Some of the tradeoffs with this approach are:

Overall the driver-testing has been successful and we expect to continue using our in-process Firebase tool to test a lot of our platform.