How to schedule run all tests in Salesforce with GitHub Actions for unlimited orgs, nothing to install

A common requirement among Salesforce teams is being able to schedule Run All Tests daily in their Salesforce orgs.

There are free and commercial tools to do this, but I wasn't satisfied with any of them, so I built my own free and open-source using GitHub Actions, the sf CLI and the Slack API.

This is available to you right now and it takes 5 min to get it working!

With this app, you'll get a slack notification every morning (or when you desire) letting you know that the tests runs for any number of orgs have finished, and whether the tests passed or failed, like this:

Before I show you how to set it up, let's talk about why anyone would want to do this in the first place. If you don't care about this and just want to go straight to the implementation, read this section.

Nightly build

In traditional software development, there's the concept of a nightly build, which ChatGPT explains as follows:

Translating this to Salesforce terms means running all tests in the org every night so that when we come to work in the morning, we can tell if any changes made the day before broke our tests (or "broke the build").

If the build is broken (or tests are failing), we know the first thing to do is to fix that. Ideally, all development would stop until the build is fixed (i.e. tests are passing again).

This is one of the core principles of DevOps: Nothing is more important than to keep the build stable.

Why on a schedule and not on demand

Ideally, Salesforce developers would run all tests when they commit changes to an integration branch as part of a Continuous Integration workflow.

Design Patterns for Salesforce Git Branching Strategies

One branch per org? A single branch? GitFlow or GitHub Flow? There are many ways to organize your Salesforce Git branches. I explain how to go about it in this in-depth article.

Pablo GonzalezPablo Gonzalez

Salesforce Continuous Integration—Concepts, components, and challenges

Continuous Integration in Salesforce is not straight forward. Let me show you the way…

Pablo GonzalezPablo Gonzalez

However, in large Salesforce orgs, running all tests can easily take 4 hours or more! There are many things teams can do to reduce the test run times, but that's a whole different topic.

If you run all tests on a schedule, you can then run specified tests in your deployments. The idea is your deployments only run specific tests, while every night, all tests are run (assuming people aren't working, it's ok if it takes 4 hours).

Then, as I explained earlier, you'd have to check in the morning that all tests indeed pass before you continue developing new features.

Long story short, you get the best of both worlds: run all tests and keep the build stable, quick deployments with specified tests.

Why I built my own instead of using existing tools

First of all, I don't think this functionality alone is worth paying for. If this comes as part of an existing DevOps platform, then yes, it's definitely a nice addition!

However, most free and commercial tools in this domain are reinventing the wheel. They display code coverage results, lines of code that are covered/uncovered, percentage of coverage per class, etc.

Believe it or not, all of this is available for free in the developer console. I'm not going to cover this in detail, but here are some screenshots of how the developer console already solves these problems:

See? Why reinvent all this? I wanted to focus on the core problem, which I define as follows:

• I want the tests to run automatically
• I don't want to install anything on any org!
• I want this to scale to any number of orgs
• I want to be notified when the tests are done running, and I want to know if there are any failures
• I don't want to pay for this :)

I realised none of the solutions out there met the above criteria for me, so I had no choice but to build my own (poor me...).

Using sfdx and GitHub Actions

Let's break down the solution bit by bit

GitHub Actions

I'm going to assume you have a basic understanding of GitHub Actions. For this article, all you need to know is that I'm using GitHub actions as a scheduler, and as a run time to run sf CLI commands against my orgs.

The sf CLI

First, to run the tests, I wanted to use the sf CLI, as opposed to using the Tooling API objects that expose test coverage information.

The reason is the sf CLI already uses these objects behind the scenes, so following the "don't reinvent the wheel" pattern, I wanted to reuse the existing functionality.

To run tests with the CLI, you can use the apex run test command

sfdx apex run test --test-level RunLocalTests --result-format human

Because I built this with large orgs in mind where tests can take hours to run, I'm using the async version of this command, which returns a test run ID that I can use later to check the results. If I used the sync version of this and the tests take 4 hours to run, that's 4 hours of GitHub Actions processing...not something you want to do!

Two GitHub Actions workflows

One challenge is that if the run takes 4 hours, how and when do you get the results? The first command returns a test run ID, but I need to run a second command to get the results.

This brings certain challenges:

• Where do I store the test run ID?
• How do I run a scheduled action that gets the results?
• What if the test run isn't finished? do I fail, or retry?
• If this is meant to work for multiple orgs, how I know which test run ID belongs to which org?

I don't want to explain in detail how I solved this because that could be a whole article on its own, so I'll only give a few points:

• I'm using GitHub Actions Matrices to be able to scale this to many different orgs without having to create new actions.
• One job schedules the test run, and another checks the ID and sends the slack message. You have full control on how far the 2nd job should run after the first one
• I'm using GitHub Actions artifacts to upload a file with the test run ID. Then, the 2nd workflow queries this file and checks the test run.
• I'm using a GitHub Action to send slack messages.

There's a bit more too it. If you want to see the implementation, you can read the workflow files here.

Setting it up

Ok, after that wall of text, let's see how to actually set it up.

I admit it's not the easiest setup, but it's really a one time thing and then it'll work forever and scale to any number of orgs. So either you pay for a product, or spend a few min setting it up 😄

1- Fork the GitHub repo

First, fork the git repo https://github.com/pgonzaleznetwork/salesforce-daily-unit-tests

Then, clone this to your local machine. Replace pgonzaleznetwork with your GitHub username

git clone https://github.com/pgonzaleznetwork/salesforce-daily-unit-tests
cd salesforce-daily-unit-tests

2- Get the authentication URL for your orgs

Then, use the sf CLI to get an sfdx authentication URL for one of your orgs, let's say production

sfdx force:org:display -u <OrgUsername> --verbose --json

You will get a result like this

{
  "status": 0,
  "result": {
    "id": "00D2o000000hotIEAQ",
    "apiVersion": "59.0",
    "accessToken": "00D2o000000hotI!ARwAQCMy.ZviySvjXh8Vl73IlnnaXbv7Bz9NRBgfyWRw3Q.p7MUJpr98hCOsmRB9hQZ9EmFUp6iOuuavrzBuqmaJmwmZCAgd",
    "instanceUrl": "https://salto-c2-dev-ed.develop.my.salesforce.com",
    "username": "cpqwebinar3@salto.io",
    "clientId": "PlatformCLI",
    "connectedStatus": "Connected",
    "sfdxAuthUrl": "force://PlatformCLI::feafeakfeafea.7mKaGXeIogFw4xzNsybhDuoLHjtUISq@salto-c2-dev-ed.develop.my.salesforce.com",
    "alias": "cpqwebinar3"
  },

Copy the value of sfdxAuthUrl

3- Create GitHub Secrets for each org

Now, enter this URL as a GitHub secret, with a name for your org, for example SFDX_PROD_URL

Repeat the process for all the orgs you want to run tests for. My short names are INTEGRATION and UAT

4- Edit the workflow files with the orgs' shortnames

Open the project in vscode, and open the .github/workflows/matrix-schedule-test-runs.yml file

There, replace the following section with the short names you used in your secrets

The must match exactly the name you gave it when creating the secret and they should be upper case, like the secret as well.

Now, open the other file .github/workflows/matrix-check-test-runs.yml and do exactly the same

5- Specify when the tests should run

Now, we need to specify 2 things: a) when the job that requests the tests is run, and b) when the second job, which queries the test results, runs.

You have full flexibility here. In my sample scenario, the first job runs at 12 AM GMT, and the 2nd one, at 4 AM GMT. This is because I'm assuming it'll take at least 4 hours for the test run to complete.

In your case, you can specify one hour, two, or whatever makes sense for you.Let's do it.

Open the first file .github/workflows/matrix-schedule-test-runs.yml and edit the cron expression to be your desired time (ask ChatGPT to give you a cron expression for any time+timezone)

Now, edit the 2nd file .github/workflows/matrix-check-test-runs.yml and edit the cron expression to be the time you want the tests to be collected.

6- Create a Slack App and get the API token

To send the slack notification, I'm using this Slack action.

Follow the steps here to create a simple Slack app, which will provide you with an access token

Copy the access token and add it as a GitHub secret called SLACK_BOT_TOKEN

Then, figure out which channel you want to send messages to, and copy the channel ID

Paste the channel ID in the 2nd file, here

7- Commit changes and push to the repo

Commit all the changes and use git push to push them to your remote repo.

You are done! See? It was that bad!

Seeing it in action

Now you have 2 options. Either you wait for the scheduled run, or you can run the jobs manually to see if they are working. To run them manually:

1- go to your repo > Actions and there you will see the 2 workflows

2- Click on Schedule Test Runs and click Run Workflow

3- You will now see the job in progress

If you click on the job, you'll see 2 jobs running, one per org. This is possible thanks to GitHub Actions Matrixs

The job is done when Salesforce returns a Test Run ID, not when the tests actually finish running. You will see the test run Id for each org as an artifact (just reload the page)

4- Now, assuming enough time has passed, you can run the 2nd job to collect the results and send the slack notification.

Remember, all of this is meant to happen automatically based on the schedules you configured; I'm only showing you how to run it manually so that you can confirm it works.

Go to the Check Test Runs and follow the same steps to run the job manually. If the tests runs have completed, and everything is working, you'll get a slack notification!

And that's it! Enjoy free Salesforce unit test automation!