Test Style

Currently, all of our tests follow a Behavior-Driven Development (BDD) approach. To enforce this approach, we write tests using the Cucumber-style Behave API and Gherkin syntax files. This page will describe these concepts and how they’re currently used by us.

Behavior-Driven Development (BDD)

BDD centers around documenting the expected behavior of a single feature from the end user’s perspective. This documentation is used as or related to an automated test that passes when the feature exhibits the expected behavior. So, BDD tests are closely tied to real-world examples we expect our users to go through, and they should be written in a high-level syntax to mimic plain-text documentation. The Cucumber tool helps us maintain these traits.

It may be helpful to note that the main difference between BDD and test-driven development (TDD) is that BDD focuses on the behavior of a feature from the user’s perspective, whereas TDD usually focuses on the results of a certain function or class. BDD isn’t particularly helpful for testing individual functions.

More info on BDD can be found here.

Gherkin

The high-level plain-text files, which document expected behaviors, all end with ‘.feature’ and use the Gherkin language. Some of these files can be seen in backend/projects/user_service/tests/component/features in the FoodFightV2 repo.

Example:

Feature: login
  As a user
  I want to be able to log in to the application
  So that I can access my account information

  Scenario: Successful login
    Given I am on the login page
    When I enter my username and password
    And I click the login button
    Then I should be taken to the dashboard page

Gherkin files use a few keywords, which go at the start of each line. Full documentation can be found here. Here is a brief overview of the keywords we currently use.

Feature

Must be the first keyword in each file. It is followed by a “:” and a name for the feature. It groups multiple Scenarios related to a single software feature. The first lines of a feature can be a plain-text description with any characters as long as none of the lines start with another keyword.

Scenario

It is followed by a “:” and a name for the scenario. It conceptually describes an example of a feature that a user could go through. It also may have a plain-text description in its first lines as long as those lines don’t start with a keyword.

Given, Then, When, And

All of these keywords go within a Scenario. The text that follows these keywords must correspond to a Step defined in a separate file, which will be explained next. The distinction between them is arbitrary, but here are some guidelines for when to define each. “Given” should be used for specifying context and the state the system is in before a scenario conceptually starts. “When” should describe an event that the user triggers in this scenario. “Then” describes the expected outcome of an event. “And” will repeat the keyword in the line above it, so it’s only used for stylistic purposes.

Cucumber and Behave API

Cucumber is a tool that uses Gherkin files as automated tests, so that our documentation-style files can directly and automatically test behavior. To do this, it requires we define “steps” in a separate steps file, which correspond to lines with a “Given”, “Then”, or “When” keyword. When a feature file is run, the steps that correspond to each line are run in the order the lines appear in.

For the backend code, it’s convenient to write these steps in Python to match the rest of our codebase. However, Cucumber doesn’t support Python step definitions out of the box. So, we use the Behave API, which lets us define steps in a very similar style to Cucumber. An example of this is:

@given('I\'m testing in "{env}"')
def step_set_environment(context, env):
    # Set the base_url based on env
    if env == "local":
        context.base_url = "http://user-service:8000"
    else:
        context.base_url = os.getenv("BASE_URL")

    context.aliased_data = {}

    print(f"[DEBUG] Running in '{env}' environment")
    print(f"[DEBUG] Base URL set to: {context.base_url}")

The function decorator defines the line that corresponds to this function. The other decorators are “@when” and “@then”. The string in the decorator defines the text that comes after the keyword in the line. It may optionally take additional parameters using standard Python string formatting. So the line ‘Given I’m testing in “local”’ in a Gherkin-language feature file would cause this function to run with env=”local”.

The context parameter is the first parameter in all steps. It is a dictionary that persists between steps, so it can be used to share state across a whole scenario.

Each backend service defines multiple step files with functions like these. The feature files can be run using the Makefile commands in each service.

Docs for the Behave API and step definitions can be found here