Creating Testbenches

Coding and Running Testbenches for Functional Simulation: A Caravel cocotb Guide

This article provides a comprehensive walkthrough for developing and executing functional simulation testbenches leveraging the Caravel cocotb Application Programming Interfaces (APIs). You will learn how to design a test environment that interacts with Caravel's General Purpose Input/Outputs (GPIOs), implement device firmware, and utilize a Python-based cocotb testbench to observe and validate the GPIO behavior against predefined expectations.

Foundational Setup: Creating Your Test Directory

To begin, establish a dedicated directory for your test under the project's root verilog/dv/cocotb folder. Name this directory after your test; for instance, gpio_test. Within this new directory, create two essential files: gpio_test.c for your firmware code and gpio_test.py for your Python cocotb testbench. Additionally, ensure that your main cocotb_tests.py file is updated to import your new test by adding the line from gpio_test.gpio_test import gpio_test.

Your project structure should resemble the following:

├── README.md
├── cocotb_tests.py
├── design_info.yaml
└── gpio_test
    ├── gpio_test.c
    └── gpio_test.py

Firmware Development: Orchestrating GPIO Behavior

The firmware, written in C, executes on the Caravel management System-on-Chip (SoC) and is responsible for configuring and manipulating the GPIO states. Below is an example demonstrating how to set GPIOs:

#include <common.h>

void main() {
    // Enable management GPIOs for output.
    mgmt_gpio_o_enable();
    // Signal start of configuration by setting management GPIO to 0.
    mgmt_gpio_wr(0);
    // Disable housekeeping SPI to free up GPIO 3 for general use.
    enable_hk_spi(0);
    // Configure all 38 Caravel GPIOs for standard management output.
    configure_all_gpios(GPIO_MODE_MGMT_STD_OUTPUT);
    // Apply the configured GPIO settings.
    gpio_config_load();
    // Set a specific bit pattern for the lower 32 GPIOs (0x8F sets bits 0, 1, 2, 3, and 7).
    set_gpio_l(0x8F);
    // Signal completion of configuration by setting management GPIO to 1.
    mgmt_gpio_wr(1);
    return;
}

Understanding Key Firmware Functions:

#include <common.h>: Incorporates necessary Caravel firmware utilities.
mgmt_gpio_o_enable(): Prepares the management GPIOs for data output.
mgmt_gpio_wr(0): Indicates that a GPIO configuration process is commencing.
enable_hk_spi(0): Deactivates the housekeeping SPI, allowing GPIO 3 to function as a regular I/O.
configure_all_gpios(GPIO_MODE_MGMT_STD_OUTPUT): Sets all 38 Caravel GPIO pins to operate in management output mode.
gpio_config_load(): Activates the previously defined GPIO configurations.
set_gpio_l(0x8F): Assigns the hexadecimal value 0x8F to the lower 32 GPIOs, meaning specific pins will be high.
mgmt_gpio_wr(1): Signals that the GPIO configuration procedure has concluded.

For more in-depth information on these firmware APIs, consult the official documentation.

Testbench Creation: Python-Powered Verification

The Python testbench, built with cocotb, is responsible for observing the Caravel signals and validating the GPIO states against the expected values. Here’s an illustrative testbench example:

 from cocotb_includes import *
import cocotb

@cocotb.test()
@repot_test
async def gpio_test(dut):
    # Configure the Caravel test environment.
    caravelEnv = await test_configure(dut)
    # Release the chip select for the housekeeping SPI.
    await caravelEnv.release_csb()
    # Wait for the management GPIO to signal firmware configuration completion (value 1).
    await caravelEnv.wait_mgmt_gpio(1)

    # Monitor and retrieve all GPIO values as a binary string.
    gpios_value_str = caravelEnv.monitor_gpio(37, 0).binstr
    cocotb.log.info(f"All monitored GPIOs: '{gpios_value_str}'")

    # Retrieve the GPIO values as an integer.
    gpio_value_int = caravelEnv.monitor_gpio(37, 0).integer
    expected_gpio_value = 0x8F # Define the expected value for the GPIOs.

    # Compare the actual GPIO value with the expected value and log the result.
    if gpio_value_int == expected_gpio_value:
        cocotb.log.info(f"[TEST] Pass: Observed GPIO value is '{hex(gpio_value_int)}'")
    else:
        cocotb.log.error(f"[TEST] Fail: Observed GPIO value is '{hex(gpio_value_int)}', expected '{hex(expected_gpio_value)}'")

Decoding Key Testbench Elements:

from cocotb_includes import *: Imports necessary Caravel Python APIs for test environment interaction.
import cocotb: Imports the core cocotb library.
@cocotb.test(): Decorator that identifies the function as a cocotb test.
@repot_test: Configures the reporting mechanism for the test.
async def gpio_test(dut): Defines the main test function as an asynchronous coroutine.
caravelEnv = await test_configure(dut): Initializes and sets up the Caravel test environment.
await caravelEnv.release_csb(): Releases the chip select line for housekeeping SPI operations.
await caravelEnv.wait_mgmt_gpio(1): Pauses execution until the management GPIO transitions to a high state, indicating the firmware has completed its initial setup.
caravelEnv.monitor_gpio(37, 0).binstr: Captures the state of GPIOs from pin 0 to 37 and converts it into a binary string representation.
cocotb.log.info(): A utility for logging informational messages, which are recorded in the full.log file.
caravelEnv.monitor_gpio(37, 0).integer: Retrieves the current integer value of the monitored GPIOs.

The final if statement block in the testbench performs the crucial comparison between the observed GPIO state and the anticipated state, providing a clear pass or fail indication.

Executing Your Testbench

There are two primary methods for running your newly created testbench:

Using the Makefile (Recommended): Navigate to your project's root directory and execute the following command: make cocotb-verify-gpio_test-rtl
Manual Execution: Change your current directory to verilog/dv/cocotb and then run the caravel_cocotb command with the appropriate parameters: caravel_cocotb -t gpio_test -sim RTL -tag first_test
For a comprehensive list of caravel_cocotb command options, please consult the official Caravel cocotb documentation.

Verifying Test Results