Integration Testing with GitHub Actions¶

GitHub Actions is a great CI/CD tool to automate the daily operations of your application lifecycle in many ways. It comes with a lot of features out of the box and even the ones that are missing are wholeheartedly provided by the community.

There are many great and brilliant engineers working daily to provide a fantastic experience for the rest of us.

In this blog post, you will learn how to perform your integration testing using GitHub Actions with all its dependencies and services spun up beforehand.

Stick around till the end to find out how.

Introduction¶

If you're a fan of writing tests for your software, there's a good chance that you like to run the fast and small tests on your machine, and delegate the task of long-running test execution to another host, likely the CI machine.

This allows for improved productivity as you continue with your development & enhancements for your software while the tests are running in the background.

It also allows for a high confidence and a robust delivery, knowing that all the changes are gated behind a set of tests that will run automatically for every push.

Thanks to GitHub and the huge productivity boost it has provided to the developer community, we have less to worry about these days when it comes to the test infrastructure compared to what used to be the case a few years ago.

Let's see how we can leverage this great technology in our advantage so that our typical flow is not interrupted and every push to the repository triggers and enforces the passing of our previously written tests.

Objective¶

As per our tradition in this blog post, we should set a clear goal of what we aim to achieve in here. If things work for the best, we should accomplish the following tasks:

Find an application that has a couple of dependencies (e.g. database, caching, etc.) during its runtime operation. The requirement is that the app has to be useless without these dependencies. This is a required objective as we aim to provide a solution to this very common problem.
Have a bunch of integrations tests actually talking to those dependencies and verifying that the application is working as expected.
Write a GitHub Actions workflow that spins up the dependencies before the app, and then runs the integration tests against the app with the correct set of configurations so that the app knows how to talk to its dependencies.

If that is something you have seen and dealt with before in the past, this blog post may provide you a robust yet underutilized approach.

CRUD Application¶

There are numerous apps that fall into this category¹, especially opensource products where we are able to grab the source code and tweak it to our needs.

The one picked for this blog post comes from the famous Sebastián Ramírez², the creator of FastAPI³. The app is called Full-Stack FastAPI Template⁴.

It has both frontend and backend. However, the focus of this blog post will be mainly on the backend part of the application.

The backend needs a PostgreSQL database to run and operate and the same goes for its integration tests.

The framework of the test and other aspects of writing tests is not in the scope for this blog post. We are mainly interested in:

Having integration tests
Running them in GitHub Actions

Since running the tests in the CI means having the database up and running, it's a great example to show how you can run your integration tests in GitHub Actions, having GitHub taking care of the dependencies for you.

Run Tests Locally¶

Let us first run the tests locally, track the dependencies and understand the interconnections between the app and the database.

VERSION="0.6.0"
git clone github.com:tiangolo/full-stack-fastapi-template -b ${VERSION}
cd full-stack-fastapi-template

To keep the results consistent, we are using the latest version of the app as of the time of writing this blog post.

This will complain about a detached HEAD, but it's not an issue for what we want to achieve here.

At this point, we should have the repo in the local machine.

We are not gonna do lots of crazy stuff here, so let us just run the database.

docker compose up -d db

With the database up and running, we will head over to the backend/ directory to prepare the dependencies.

cd backend
poetry install

We have our virtual environment set up with all its libraries installed.

For the app to work, we need to set a couple of environment variables.

# inside the backend/ directory
poetry self add 'poetry-dotenv-plugin<1'
# grab variables from the sample file
egrep '^\w.*' ../.env > .env

And finally, let's migrate the database and run the tests.

poetry run alembic upgrade head
poetry run pytest

Two of the tests are failing at this point, but that is not our concern really!

Running Tests in the CI¶

Now that we have a locally working example, let's move on to the CI part.

At this step, we want to have the same setup, spinning up the database and running the tests.

Note

The current version of the CI in the target repository is using a simple docker-compose up -d right before running the tests⁵.

Honestly, there is nothing wrong with this approach and if it works for you and your team, by all means, go ahead and own your decision and celebrate it proudly.

I am not here to tell you which approach is better; that is your responsibility to figure out.

I would only invite you to read this article to see if the proposed solution is something that you would like to try out.

GitHub Actions provide a way to run services before the actual job starts. These are great for running dependencies like databases, caches, etc. in the CI environment.

The idea is just the same as we had in our local environment, and the implementation and its syntax is specific to GitHub Actions.

For your reference, GitLab also provides the same functionality with the services directive⁶.

Let's see how we can achieve this in GitHub Actions.

Starting the Database¶

First, we need to start the database before the app.

.github/workflows/ci.yml

    services:
      db:
        image: postgres:16
        env:
          POSTGRES_DB: app
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: changethis
        ports:
          - 5432:5432

If you notice the syntax is very similar to what you see in a docker-compose.yml file. Example from the same target repository below⁴.

docker-compose.yml

services:
  db:
    image: postgres:12
    restart: always
    volumes:
      - app-db-data:/var/lib/postgresql/data/pgdata
    env_file:
      - .env
    environment:
      - PGDATA=/var/lib/postgresql/data/pgdata
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD?Variable not set}
      - POSTGRES_USER=${POSTGRES_USER?Variable not set}
      - POSTGRES_DB=${POSTGRES_DB?Variable not set}

These so called services in GitHub Actions are spun up before the actual job starts. That gives a good leverage for all the dependencies we need up and running before the CI starts its first step.

The services defined here will run as soon as possible during the execution of our job, as you see below.

And if you dig deeper, you will find the exact environment variables passed to the container as specified earlier.

Installing Dependencies & Database Migration¶

As before, we will require the installed libraries of our application, as well as the database migration for every new Postgres instance.

.github/workflows/ci.yml

      - name: Install dependencies
        run: |
          pip install -U pip poetry
          poetry install
      - name: Set up migration
        run: |
          poetry run alembic upgrade head

Running the Tests¶

Finally, we will run the tests.

.github/workflows/ci.yml

      - name: Run tests
        run: |
          poetry run coverage run --source=app -m pytest
          poetry run coverage report --show-missing
          poetry run coverage html

Optionally: Upload Coverage to GitHub Pages¶

The coverage results is a great measure of how well your tests are covering your codebase, whether or not you have a dead code anywhere, etc.

These are usually static HTML files that can be viewed in a browser for a good overall visual on the coverage and the places where you need to improve.

Additionally, GitHub Pages, is an excellent choice for serving such static files right inside your GitHub repository; it's even free of charge if you are using a public repository.

Let's upload the coverage from our last step into GitHub Pages.

.github/workflows/ci.yml

      - name: Upload Pages artifact
        uses: actions/upload-pages-artifact@v3
        with:
          name: coverage-html
          path: backend/htmlcov
      - name: Deploy to GitHub Pages
        uses: actions/deploy-pages@v4
        with:
          artifact_name: coverage-html

The resulting CI run will have an artifact in its summary page just as below.

And if you view the deployed GitHub Pages, you will see the coverage report as shown below, which you can sort based on your custom column.

Full Workflow¶

The final workflow is not rocket science really . It's just a typical workflow you would see elsewhere.

Here is the full workflow for your reference.

.github/workflows/ci.yml

name: ci

on:
  push:
    branches:
      - master
  pull_request:
    types:
      - opened
      - synchronize

env:
  DOMAIN: localhost
  ENVIRONMENT: local
  PROJECT_NAME: github-actions-integration-testing
  STACK_NAME: full-stack-fastapi-project
  BACKEND_CORS_ORIGINS: "http://localhost,http://localhost:5173,https://localhost,https://localhost:5173,http://localhost.tiangolo.com"
  SECRET_KEY: changethis
  FIRST_SUPERUSER: [email protected]
  FIRST_SUPERUSER_PASSWORD: changethis
  USERS_OPEN_REGISTRATION: "False"
  SMTP_HOST: ""
  SMTP_USER: ""
  SMTP_PASSWORD: ""
  EMAILS_FROM_EMAIL: [email protected]
  SMTP_TLS: "True"
  SMTP_SSL: "False"
  SMTP_PORT: 587
  POSTGRES_SERVER: localhost
  POSTGRES_PORT: 5432
  POSTGRES_DB: app
  POSTGRES_USER: postgres
  POSTGRES_PASSWORD: changethis
  SENTRY_DSN: ""
  DOCKER_IMAGE_BACKEND: backend
  DOCKER_IMAGE_FRONTEND: frontend

jobs:
  test:
    runs-on: ubuntu-latest
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    permissions:
      contents: read
      id-token: write
      pages: write
    services:
      db:
        image: postgres:16
        env:
          POSTGRES_DB: ${{ env.POSTGRES_DB }}
          POSTGRES_USER: ${{ env.POSTGRES_USER }}
          POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
        ports:
          - 5432:5432
        options: --health-cmd "pg_isready -h localhost" --health-interval 10s --health-timeout 5s --health-retries 5
    defaults:
      run:
        working-directory: ./backend
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.x"
      - name: Install dependencies
        run: |
          pip install -U pip poetry
          poetry install
      - name: Set up migration
        run: |
          poetry run alembic upgrade head
      - name: Run tests
        run: |
          poetry run coverage run --source=app -m pytest
          poetry run coverage report --show-missing
          poetry run coverage html
      - name: Upload Pages artifact
        uses: actions/upload-pages-artifact@v3
        with:
          name: coverage-html
          path: backend/htmlcov
      - if: github.event_name == 'push' && github.ref == 'refs/heads/master'
        name: Deploy to GitHub Pages
        uses: actions/deploy-pages@v4
        with:
          artifact_name: coverage-html

Conclusion¶

I use GitHub for all my projects, and close to second to that is GitHub Actions for all my automation tasks. There is rarely anything that can't be done in a GitHub CI these days, especially if you look long and hard enough.

I spend most of my day to day operational tasks preparing a working example locally before pushing it all into the yard of GitHub Actions.

It is such a life saver, and it has gotten even stronger in the recent years after the acquisition by Microsoft. I'm glad to say that this is one of the few moments in the history where conglomerates' acquisition have actually improved the product for the better.

As for you, I hope you have gained some insights and ideas of your own after seeing the pattern used here. It can go beyond this once you realize that many of our applications these days aren't just the app itself; it's all the tooling and dependencies around it as well.

For any of your current and/or upcoming projects, I seriously recommend you to take a close look at what GitHub and GitHub Actions can bring to your table. Most of the time, they are a one-size-fits-all, with all the off-the-shelf tooling you need to get started.

I hope you have enjoyed this blog post and I look forward to seeing you in the next one . Until then, take care and happy hacking!

If you enjoyed this blog post, consider sharing it with these buttons . Please leave a comment for us at the end, we read & love 'em all.

Share on Share on Share on Share on

Integration Testing with GitHub Actions¶

Introduction¶

Subscribe to the Newsletter

Objective¶

CRUD Application¶

Run Tests Locally¶

Running Tests in the CI¶

Starting the Database¶

Installing Dependencies & Database Migration¶

Running the Tests¶

Optionally: Upload Coverage to GitHub Pages¶

Full Workflow¶

Conclusion¶

Comments