Lessons learned on Automation Episode I: Introduction and parallelism

End to End (E2E) WebDriver Testing is a two-edged sword; it can be the hero, saving you a lot of time during regression testing and preventing faulty releases, or it can be the villain, slowing down the development and release process with flakey, hard to maintain and time-consuming suites that you don’t trust or want. I have seen both.

In these series of blog posts, I will share my experience developing AugmentedDriver, an open source framework that allowed my team to run 80,000 tests per month and more than 275 tests in parallel in less than 15 minutes.

Data Independent Tests

Selenium tests are inherently slow. If you want throughput, you need to focus on parallelism, leading us to the first lesson in this saga:

If each test creates its own data that is not shared with anyone else, then all of your tests can run in parallel.

With parallelism, your throughput will no longer be constrained to the length of the test. Instead, it will rely on how many tests you can run at the same time. I personally like using SauceLabs, but there are other commercial options to help with that.
You can always play with your own Selenium Farm, but I caution against it because it has been a nightmare for me to maintain.
Here is a simple example of how to achieve this in a rest-based application using Java + JUnit:

First I create a TestResource with an endpoint that cannot be executed in production. This creates and cleans the basic data that every test needs.

@Path("/test")
public class TestResource {
    @Post
    @Path("create")
    public Response create(String uniqueUser) {
        If(isProd()) {
            throw new NotAllowedException("NOT ALLOWED");
        }
        //... create unique user, organization, etc...
        //...
    }
    @Post
    @Path("clean")
    public Response clean(String uniqueUser) {
        If(isProd()) {
            throw new NotAllowedException("NOT ALLOWED");
        }
        //...clean unique user, organization, etc...
        //...
    }
}

Then I create a JUnit TestRule that calls those resource endpoints:

public class DataCreatorRule implements TestRule {
    public Statement apply(Statement statement, Description description) {
        return new Statement() {
            @Override
            public void evaluate() throws Throwable {
                String uniqueUser = "uniqueuser";
                client
                    .test()
                    .create(uniqueUser);
                statement.evaluate();
                client
                    .test()
                    .clean(uniqueUser);
            }
        }
    }
}

With that rule in place, each test creates its own data at the beginning. If the test does not fail, it also cleans afterward.

Achieving Data Independence is the foundation on which a Selenium framework is built. You should strive to not relying on existing data (except when that data is immutable). Following that simple rule will avoid nasty situations when data gets corrupt, or worse—when one test modifies data that another test needs (leading to one test depending on another test—yikes!). Finally, it will also open the gates for parallelism, significantly decreasing the time spent on the testing phase.

Get Split Certified

Split Arcade includes product explainer videos, clickable product tutorials, manipulatable code examples, and interactive challenges.

Switch It On With Split

The Split Feature Data Platform™ gives you the confidence to move fast without breaking things. Set up feature flags and safely deploy to production, controlling who sees which features and when. Connect every flag to contextual data, so you can know if your features are making things better or worse and act without hesitation. Effortlessly conduct feature experiments like A/B tests without slowing down. Whether you’re looking to increase your releases, to decrease your MTTR, or to ignite your dev team without burning them out–Split is both a feature management platform and partnership to revolutionize the way the work gets done. Switch on a free account today, schedule a demo, or contact us for further questions.