design patterns

Cucumber-JVM Global Hook Workarounds

Almost all BDD automation frameworks have some sort of hooks that run before and after scenarios. However, not all frameworks have global hooks that run once at the beginning or end of a suite of scenarios – and Cucumber-JVM is one of these unlucky few. Cucumber-JVM GitHub Issue #515, which seeks to add @BeforeAll and @AfterAll hooks, has been open and active since 2013, but it looks unclear if the issue will ever be resolved. Thankfully, there are some workarounds to effect the same behavior as global hooks.

Workaround #1: Don’t Do It

From a purist’s perspective, each scenario (or test) should be completely independent, meaning it should not share parts with any other tests. Independence provides the following benefits:

  • Safety between tests
  • Consistency across tests
  • The ability to run any tests individually, in any order, or in parallel
  • More sensible, understandable tests

If not handled properly, global hooks can be dangerous because they make tests interdependent. Changes or failures in one test may cascade into others. Global test data would waste memory for tests that don’t use it. Furthermore, the fact that Issue #515 has been open for years indicates the difficulty of properly implementing global hooks.

However, the main cost of independence is runtime. Independent tests often repeat similar setup and cleanup routines. Even a few extra seconds per test can add up tremendously. Google Guava, for example, has over 286,000 tests – adding one second to each test would amount to nearly 80 hours! Performance becomes especially critical for continuous integration, in which wasted time means either delivery delays or coverage gaps. Certain operations like preparing a database or fetching authentication tokens may be pragmatic candidates for global hooks.

The best strategy is to use global hooks only when necessary for time-intensive setup that can be shared safely. Any shared test data should be immutable. Always question the need for global hooks. Most tests probably won’t need them.

Workaround #2: Static Variables

A basic hack for global hooks is actually provided in Issue #515. A static Boolean flag can indicate when the @Before hook has run more than once because it isn’t “reset” when a new scenario re-instantiates the step definition classes. The runtime shutdown hook will be called once all tests are done and the program exits. (Note that a static flag cannot be used in an @After hook due to the halting problem.) The example from the issue is shamelessly copied below:

public class GlobalHooks {
    private static boolean dunit = false;

    @Before
    public void beforeAll() {
        if(!dunit) {
            Runtime.getRuntime().addShutdownHook(afterAllThread);
            // do the beforeAll stuff...
            dunit = true;
        }
    }
}

Workaround #3: Singleton Caching

The basic hack is useful for simple setup and cleanup routines, but it becomes inelegant when objects must be shared by scenarios. Rather than polluting the class with static members, a singleton can cache test data between scenarios, and global setup logic may be put into the singleton’s constructor. Furthermore, if the singleton uses lazy initialization, then @Before hooks may not be needed at all. A “lazy” singleton will not be instantiated until the first time its getInstance method is called, meaning it will be skipped if the scenarios do not need them. This is a huge advantage when selectively running scenarios by name, tag, or feature. (Please refer to the previous post, Static or Singleton, for a deeper explanation of the singleton pattern.)

Consider scenarios that must generate authentication tokens (like OAuth) for API testing. A singleton “token holder” could cache tokens for usernames, rather than doing the authorization dance for every scenario. The snippet below shows how such a singleton could be called within a @When step definition with no @Before method.

public class ExampleSteps {
    ...
    @When("^some API is called$")
    public void whenSomeApiIsCalled() {
        // Get the token from the singleton cache lazily
        String token = TokenHolder.getInstance().getToken("user", "pass");
        // Use the token to call some API (method not shown)
        callSomeApi(token);
    }
    ...
}

And the singleton class could be defined like this:

public class TokenHolder {
    private static volatile TokenHolder instance = null;
    private HashMap<String, String> tokens;

    private TokenHolder() {
        tokens = new HashMap<String, String>();
    }

    public static TokenHolder getInstance() {
        // Lazy and thread-safe
        if (instance == null) {
            synchronized(TokenHolder.class) {
                if (instance == null) {
                    instance = new TokenHolder();
                }
            }
        }

        return instance;
    }
    
    public String getToken(String username, String password) {
        // This check could be extended to handle token expiration
        if (!tokens.containsKey(username)) {
            // Request a fresh authentication token (method not shown)
            String token = requestToken(username, password);
            // Cache the token for later
            tokens.put(username, token);
        }
        
        return tokens.get(username);
    }
    
    ...
}

Workaround #4: JUnit Class Annotations

Another workaround mentioned in Issue #515 and elsewhere is to use JUnit‘s @BeforeClass and @AfterClass annotations in the runner class, like this:

@RunWith(Cucumber.class)
@Cucumber.Options(format = {
    "html:target/cucumber-html-report",
    "json-pretty:target/cucumber-json-report.json"})
public class RunCukesTest {

    @BeforeClass
    public static void setup() {
        System.out.println("Ran the before");
    }

    @AfterClass
    public static void teardown() {
        System.out.println("Ran the after");
    }
}

While @BeforeClass and @AfterClass may look like the cleanest solution at first, they are not very practical to use. They work only when Cucumber-JVM is set to use the JUnit runner. Other runners, like TestNG, the command line runner, and special IDE runners, won’t pick up these hooks. Their methods must also be are static and would need static variables or singletons to share data anyway. Therefore, I personally discourage using these annotations in Cucumber-JVM.

What About Dependency Injection?

Dependency injection is a marvelous technique. As defined by Wikipedia:

In software engineering, dependency injection is a technique whereby one object supplies the dependencies of another object. A dependency is an object that can be used (a service). An injection is the passing of a dependency to a dependent object (a client) that would use it. The service is made part of the client’s state. Passing the service to the client, rather than allowing a client to build or find the service, is the fundamental requirement of the pattern.

Dependency injection can be a powerful alternative to singletons because DI provides finer control over the scope of objects. However, Cucumber-JVM’s dependency injection cannot be applied with global hooks because dependency objects, like step definition objects, are constructed and destroyed for each scenario.

Comparison Table

Ultimately, the best approach for global hooks in Cucumber-JVM is the one that best fits the tests’ needs. Below is a table to make workaround comparisons easier.

Workaround Pros Cons
Don’t Do It Scenarios are completely independent. No complicated or risky workarounds. Repeated setup and cleanup procedures may add significant execution time.
Static Variables Simple yet effective implementation. May need many static variables to share test data.
Singleton Caching Abstracts test data and setup procedures. Easily handles lazy initialization and evaluation. May not need a @Before hook. More complicated design.
JUnit Class Annotations Clean look for basic setup and cleanup routines. May be used only with the JUnit runner. Requires static variables or singletons to share test data anyway.

Static or Singleton

Let’s face it: there are times in object-oriented programming when we need to share something somewhat globally. When I say that, many purists will scream, “But global variables are evil and should never be used!” Hence, I used the word “somewhat” – attributes (variables) and behaviors (methods) can be class members, allowing them to be used wherever their class is in scope. Things like constants and stateless utility methods are perfect candidates to be class members; Java’s Math class is a perfect example. Many engineers use the word “static” interchangeably with “class member” because languages like Java and C# use the static keyword to denote class members.

Class Membership Limitations

While going “static” is great for one-off constants and stateless methods, it is a poor solution for managing global state. Class methods require class variables to manage state. Class variables are mutable (unlike constants which are immutable) and require extra precautions for handling. And as class variables, the state is divorced from many benefits offered by the object-oriented marriage between attributes and behaviors. A much better solution for global state is the singleton pattern.

The Singleton Pattern

The singleton pattern is one of the best known, and most controversial, design patterns in object-oriented programming. A singleton class is restricted to constructing only one instance of an object so that one instance may be shared somewhat globally. Love it or hate it, the singleton pattern is quite useful when applied appropriately – factories, state machines, and even test automation often employ singletons.

For reference, below is a thread-safe singleton class written in Java, courtesy of Wikipedia. The single instance may be accessed anywhere by simply calling Singleton.getInstance(). Any attributes or behaviors are added as instance members (without “static”).

public final class Singleton {
    private static volatile Singleton instance = null;

    private Singleton() {}

    public static Singleton getInstance() {
        if (instance == null) {
            synchronized(Singleton.class) {
                if (instance == null) {
                    instance = new Singleton();
                }
            }
        }
        return instance;
    }
}

Singleton Benefits

The singleton pattern is a much better solution for managing global state for several reasons.

True Object Orientation Singleton objects may be treated as any other object (like POJOs). Classes themselves are not plain-old objects, thus limiting their usability.
Inheritance Singleton classes may employ inheritance to add or modify attributes and behaviors. Class members cannot be manipulated through inheritance.
Lazy Initialization Singleton classes can easily use lazy initialization to avoid constructing the instance until it is first used. Lazy initialization for class variables is possible but often more difficult. Many times, class variables are simply initialized when declared, which would unnecessarily bloat memory if the variables are never used.
Object Pooling The singleton pattern is essentially a special case of the object pool pattern, in which the pool size is one. Thus, a singleton class could easily be updated to handle a pool of objects instead of just a single one.
Cleaner Implementation Singleton state is nicely encapsulated. Singleton classes provide a sensible central place for shared stuff. Other classes don’t have the “static” clutter. Object references need not be passed around.

Anecdotally, engineers who use design patterns (like singleton) have a better grasp on good object-oriented principles, and they tend to make better, more thoughtful software design decisions. Adding static members to a class tends to be a hasty decision made for expediency – it will work, but it may not be the best practice.

TL;DR

  • Use class members for one-off constants and stateless methods.
  • Use singletons for managing global, mutable state.