DevOps

How to Retry Failed Steps in GitHub Action Workflows

Allan M. Jeremy
3 min read
How to Retry Failed Steps in GitHub Action Workflows
This article was written over 18 months ago and may contain information that is out of date. Some content may still be relevant, but please refer to official documentation for the latest information.

Sometimes things can go wrong in your GitHub Action workflow step(s), and you may want to retry them. In this article, we'll cover two methods for doing this!

Pre-requisites

  • Git This should be installed in your path.
  • GitHub account: We'll need this to use GitHub Actions.

Initial setup

In order to follow along, here are the steps you can take to setup your GitHub Actions workflow:

Initialize your git repository

In your terminal, run git init to create an empty git repository or skip this step if you already have an existing git repository.

Create a workflow file

GitHub workflow files are usually .yaml/.yml files that contain a series of jobs and steps to be executed by GitHub Actions. These files often reside in .github/workflows. If the directories do not exist, go ahead and create them. Create a file retry.yml in .github/workflows. For now, the file can contain the following:

name: "Retry action using retry step"
on:
	# This action is called when this is pushed to github
	push:
	# This action can be manually triggered
	workflow_call:
jobs:
    # This name is up to you
	retry-job:
		runs-on: "ubuntu-latest"
		name: My Job
		steps:
			- name: Checkout repository
			  uses: actions/checkout@v3

			- name: Print
			  run: |
				echo 'Hello'

Testing your workflow

You can test your GitHub Action workflow by pushing your changes to GitHub and going to the actions tab of the repository. You can also choose to test locally using Act.

Retrying failed steps

Approach 1: Using the retry-step action

By using the retry-step action, we can retry any failed shell commands. If our step or series of steps are shell commands, we can use the retry-step action to retry them.

If, however, you'd like to try retry a step that is using another action, then the retry-step action will NOT work for you. In that case, you may want to try the alternative steps mentioned below.

Modify your action file to contain the following:

name: "Retry action using retry step"
on:
	# This action is called when this is pushed to github
	push:
	# This action can be manually triggered
	workflow_call:
jobs:
    # This name is up to you
	retry-job:
		runs-on: "ubuntu-latest"
		name: My Job
		steps:
			- name: Checkout repository
			  uses: actions/checkout@v3

			- name: Use the reusable workflow
			  # Use the retry action
			  uses: nick-fields/retry@v2
			  with:
				max_attempts: 3
				retry_on: error
				timeout_seconds: 5
				# You can specify the shell commands you want to retry here
				command: |
					echo 'some command that would potentially fail'				

Approach 2: Duplicate steps

If you are trying to retry steps that use other actions, the retry-step action may not get the job done. In this case, you can still retry steps by retrying steps conditionally, depending on whether or not a step failed.

GitHub provides us with two main additional attributes in our steps:

  • continue-on-error - Setting this to true means that the even if the current step fails, the job will continue on to the next one (by default failure stops a job's running).
  • steps.{id}.outcome - where {id} is an id you add to the steps you want to retry. This can be used to tell whether a step failed or not, potential values include 'failure' and 'success'.
  • if - allows us to conditionally run a step
name: "Retry action using retry step"
on:
	# This action is called when this is pushed to GitHub
	push:
	# This action can be manually triggered
	workflow_call:
jobs:
    # This name is up to you
	retry-job:
		runs-on: "ubuntu-latest"
		name: My Job
		steps:
			- name: Checkout repository
			  uses: actions/checkout@v3

			- name: Some action that can fail
			  # You need to specify an id to be able to tell what the status of this action was
			  id: myStepId1
			  # This needs to be true to proceed to the next step of failure
			  continue-on-error: true
			  uses: actions/someaction

			# Duplicate of the step that might fail ~ manual retry
			- name: Some action that can fail
			  id: myStepId2
			  # Only run this step if step 1 fails. It knows that step one failed because we specified an `id` for the first step
			  if: steps.myStepId1.outcome == 'failure'
			  # This needs to be true to proceed to the next step of failure
			  continue-on-error: true
			  uses: actions/someaction

Bonus: Retrying multiple steps

If you want to retry multiple steps at once, then you can use composite actions to group the steps you want to retry, and then use the duplicate steps approach mentioned above.

Conclusion

How do you decide which approach to use?

  • If you are retrying a step that is only shell commands, then you can use the retry step action.

  • If you are retrying a step that needs to use another action, then you can use duplication of steps with conditional running to manually retry the steps.

About the author

Allan M. Jeremy

Allan M. Jeremy

Senior Software Engineer

Allan Jeremy is a self-proclaimed polymath, entrepreneur and Senior Software Engineer at ThisDot Labs. He is an AI and crypto enthusiast that loves to learn most things tech and business as well as sharing what he learns.

Keep reading

View all posts →

Deploying Multiple Apps From a Monorepo to GitHub Pages

Explore deploying multiple front-end applications on GitHub Pages with our guide. Learn how to navigate the challenges of client-side routing and efficiently manage multiple apps in one repository....

Jan Kaiser4 mins
GitHubGitHub ActionsNxReact

How to Create a Bot That Sends Slack Messages Using Block Kit and GitHub Actions

Have you ever wanted to get custom notifications in Slack about new interactions in your GitHub repository? If so, then you're in luck. With the help of GitHub actions and Slack's Block Kit, it is super easy to set up automated workflows that will send custom messages to your Slack channel of choice. In this article, I will guide you on how to set up the Slack bot and send automatic messages using GH actions. Create a Slack app Firstly, we need to create a new Slack application. Go to Slack's app page. If you haven't created an app before you should see: otherwise you might see a list of your existing apps: Let's click the Create an App button. Frpm a modal that shows up, choose From scratch option: In the next step, we can choose the app's name (eg. My Awesome Slack App) and pick a workspace that you want to use for testing the app. After the app is created successfully, we need to configure a couple of additional options. Firstly we need to configure the OAuth & Permissions section: In the Scopes section, we need to add a proper scope for our bot. Let's click Add an OAuth Scope in the Bot Token Scopes section, and select an incoming webhook scope: Next, in OAuth Tokens for Your Workspace section, click Install to Workspace and choose a channel that you want messages to be posted to. Finally, let's go to Incoming Webhooks page, and activate the incoming hooks toggle (if it wasn't already activated). Copy the webhook URL (we will need it for our GitHub action). Create a Github Action Workflow In this section, we will focus on setting up the GitHub action workflow that will post messages on behalf of the app we've just created. You can use any of your existing repositories, or create a new one. Setting Up Secrets In your repository, go to Settings Secrets and variables Actions section and create a New Repository Secret. We will call the secret SLACK WEBHOOK URL and paste the url we've previously copied as a value. Create a workflow To actually send a message we can use slackapi/slack github action GitHub action. To get started, we need to create a workflow file in .github/workflows directory. Let's create .github/workflows/slack message.yml file to your repository with the following content and commit the changes to main branch. In this workflow, we've created a job that uses slackapi/slack github action action and sends a basic message with an action run id. The important thing is that we need to set our webhook url as an env variable. This was the action can use it to send a message to the correct endpoint. We've configured the action so that it can be triggered manually. Let's trigger it by going to Actions Send Slack notification We can run the workflow manually in the top right corner. After running the workflow, we should see our first message in the Slack channel that we've configured earlier. Manually triggering the workflow to send a message is not very useful. However, we now have the basics to create more useful actions. Automatic message on pull request merge Let's create an action that will send a notification to Slack about a new contribution to our repository. We will use Slack's Block Kit to construct our message. Firstly, we need to modify our workflow so that instead of being manually triggered, it runs automatically when a pull requests to main branch is merged. This can be configured in the on section of the workflow file: Secondly, let's make sure that we only run the workflow when a pull request is merged and not eg. closed without merging. We can configure that by using if condition on the job: We've used a repository name (github.repository) as well as the user login that created a pull request (github.event.pull request.user.login), but we could customize the message with as many information as we can find in the pull request event. If you want to quickly edit and preview the message template, you can use the Slack's Block Kit Builder. Now we can create any PR, eg. add some changes to README.md, and after the PR is merged, we will get a Slack message like this. Summary As I have shown in this article, sending Slack messages automatically using GitHub actions is quite easy. If you want to check the real life example, visit the starter.dev project where we are using the slackapi/slack github action to get notifications about new contributions (send slack notification.yml) If you have any questions, you can always Tweet or DM me at @ktrz. I'm always happy to help!...

Chris Trześniewski7 mins
GitHubGitHub Actions

Tag and Release Your Project with GitHub Actions Workflows

Tag and Release your project with GitHub Actions Workflows GitHub Actions is a powerful automation tool that enables developers to automate various workflows in their repositories. One common use case is to automate the process of tagging and releasing new versions of a project. This ensures that your project's releases are properly versioned, documented, and published in a streamlined manner. In this blog post, we will walk you through two GitHub Actions workflows that can help you achieve this. Understanding GitHub Tags and Releases GitHub tags and releases are essential features that help manage and communicate the progress and milestones of a project. Let's take a closer look at what they are, why they are useful, and how they can be used effectively. GitHub Tags A GitHub tag is a specific reference point in a repository's history that marks a significant point of development, such as a release or a specific commit. Tags are typically used to identify specific versions of a project. They are lightweight and do not contain any additional metadata by default. Tags are useful for several reasons: 1. Versioning : Tags allow you to assign meaningful version numbers to your project, making it easier to track and reference specific releases. 2. Stability : By tagging stable versions of your project, you can provide users with a reliable and tested codebase. 3. Collaboration : Tags enable contributors to work on specific versions of the project, ensuring that everyone is on the same page. GitHub Releases GitHub releases are a way to package and distribute specific versions of your project to users. A release typically includes the source code, compiled binaries, documentation, and release notes. Releases provide a convenient way for users to access and download specific versions of your project. Releases offer several benefits: 1. Communication : Releases allow you to communicate important information about the changes, improvements, and bug fixes included in a specific version. 2. Distribution : By packaging your project into a release, you make it easier for users to download and use your software. 3. Documentation : Including release notes in a release helps users understand the changes made in each version and any potential compatibility issues. Effective Use of Tags and Releases To make the most of GitHub tags and releases, consider the following tips: 1. Semantic Versioning : Follow a consistent versioning scheme, such as semantic versioning (e.g., MAJOR.MINOR.PATCH), to clearly communicate the nature of changes in each release. 2. Release Notes : Provide detailed and concise release notes that highlight the key changes, bug fixes, and new features introduced in each version. This helps users understand the impact of the changes and make informed decisions. 3. Release Automation : Automate the release process using workflows, like the one described in this blog post, to streamline the creation of tags and releases. This saves time and reduces the chances of human error. By leveraging GitHub tags and releases effectively, you can enhance collaboration, improve communication, and provide a better experience for users of your project. The Goal The idea is to have a GitHub action that, once triggered, updates our project's version, creates a new tag for our repository, and pushes the updates to the main branch. Unfortunately, the main branch is a protected branch, and it's not possible to directly push changes to a protected branch through a GitHub action. Therefore, we need to go through a pull request on the main branch, which, once merged, will apply the changes due to the version update to the main branch. We had to split the workflow into two different GitHub actions: one that creates a pull request towards the main branch with the necessary code changes to update the repository's version, and another one that creates a new tag and releases the updated main branch. This way, we have one additional click to perform (the one required to merge the PR), but we also have an intermediate step where we can verify that the version update has been carried out correctly. Let’s dive into these two workflows. Update version and create Release's PR Workflow Walkthrough: Step 1: Define the Workflow The workflow starts with specifying the workflow name and the event that triggers it using the on keyword. In this case, the workflow is triggered manually using the "workflow dispatch" event, which means it can be run on demand by a user. Additionally, the workflow accepts an input parameter called "version," which allows the user to specify the type of version bump (major, minor, or patch). The workflow dispatch event allows the user to set the "version" input when running the workflow. Step 2: Prepare the Environment The workflow will run on an Ubuntu environment (ubuntu latest) using a series of steps under the jobs section. The first job is named "version." Step 3: Checkout the Code The workflow starts by checking out the code of the repository using the actions/checkout@v3 action. This step ensures that the workflow has access to the latest codebase before making any modifications. Step 4: Set up Node.js Next, the workflow sets up the Node.js environment using the actions/setup node@v3 action and specifying the Node.js version 16.x. It's essential to use the appropriate Node.js version required by your project to avoid compatibility issues. Step 5: Install Dependencies To ensure the project's dependencies are up to date, the workflow runs npm install to install the necessary packages as defined in the package.json file. Step 6: Configure Git To perform version bump and create a pull request, the workflow configures Git with a user name and email. This allows Git to identify the author when making changes in the repository. Step 7: Update the Version The workflow now performs the actual version bump using the npm version command. The new version is determined based on the "version" input provided when running the workflow. The updated version number is stored in an output variable named update version, which can be referenced later in the workflow. Step 8: Update the Changelog After bumping the version, the workflow updates the CHANGELOG.md file to reflect the new release version. It replaces the placeholder "Unreleased" with the updated version using the sed command. [ We will return to this step later ] Step 9: Create a Pull Request Finally, the workflow creates a pull request using the peter evans/create pull request@v5 action. This action automatically creates a pull request with the changes made in the workflow. The pull request will have a branch name following the pattern "release/ ", where corresponds to the updated version number. The outcome of this workflow will be a new open PR in the project with package.json and CHANGELOG.md file changed. [ we will speak about the changelog file later ] Now we can check if the changes are good, approve the PR and merge it into main. Merge a PR with a title that starts with "Release:" automatically triggers the second workflow Tag & Release Workflow Walkthrough: As you can see we added a check for the PR title before starting the job once the PR is merged and closed. Only the PRs with a title that starts with "Release:" will trigger the workflow. The first three steps are the same as the one described in the previous workflow: we check out the code from the repository, we set up node and we install dependencies. Let's start with: Step 4: Check formatting To maintain code quality, we run the npm run format:check command to check if the code adheres to the specified formatting rules. This step helps catch any formatting issues before proceeding further. Step 5: Build The npm run build command is executed in this step to build the project. This step is particularly useful for projects that require compilation or bundling before deployment. Step 6: Set up Git To perform Git operations, such as tagging and pushing changes, we need to configure the Git user's name and email. This step ensures that the correct user information is associated with the Git actions performed later in the workflow. Step 7: Get tag In this step, we retrieve the current version of the project from the package.json file. The version is then stored in an output variable called get tag.outputs.version for later use. Step 8: Tag the commit Using the version obtained in the previous step, we create a Git tag for the commit. The tag is annotated with a message indicating the version number. Finally, we push the tag and associated changes to the repository. Step 9: Create changelog diff To generate release notes, we extract the relevant changelog entries from the CHANGELOG.md file. This step helps summarize the changes made since the previous release. (We will return to this step later) Step 10: Create release Using the actions/create release action, we create a new release on GitHub. The release is associated with the tag created in the previous step, and the release notes are provided in the body of the release. Step 11: Delete release notes file Finally, we delete the temporary release notes.md file created in Step 9. This step helps keep the repository clean and organized. Once also the second workflow is finished our project is tagged and the new release has been created. The "Changelog Steps" As you can see the release notes are automatically filled, with a detailed description of what has been added, fixed, or updated in the project. This was made possible thanks to the "Changelog steps" in our workflows, but to use them correctly, we need to pay attention to a couple of things while developing our project. Firstly, to the format of the CHANGELOG.md file. This will be our generic template: But the most important aspect, in addition to keeping the file up to date during developments by adding the news or improvements we are making to the code under their respective sections, is that every time we start working on a new project release, we begin the paragraph with ## [Unreleased]. This is because, in the first workflow, the step related to the changelog will replace the word "Unreleased" with the newly created project version. In the subsequent workflow, we will create a temporary file (which will then be deleted in the latest step of the workflow), where we will extract the part of the changelog file related to the new version and populate the release notes with it. Conclusion Following these Tag and Release Workflows, you can automate the process of creating releases for your GitHub projects. This workflow saves time, ensures consistency, and improves collaboration among team members. Remember to customize the workflow to fit your project's specific requirements and enjoy the benefits of streamlined release management....

Mattia Magi9 mins
GitHubGitHub Actions

How to Add Continuous Benchmarking to Your Projects Using GitHub Actions

Over the lifetime of a project performance, issues may arise from time to time. Lots of the time, these issues don't get detected until they get into production. Adding continuous benchmarking to your project and build pipeline can help you catch these issues before that happens. What is Continuous Benchmarking Benchmarking is the process of measuring the performance of an application. Continuous benchmarking builds on top of this by doing so either on a regular basis, or whenever new code is pushed so that performance regressions can be identified and found as soon as they are introduced. Adding continuous benchmarking to your build pipeline can help you effectively catch performance issues before they ever make it to production. Much like with tests, you are still responsible for writing benchmark logic. But once that’s done, integrating it with your build pipeline can be done easily using the continuous benchmark GitHub Action. github action benchmark github action benchmark allows you to easily integrate your existing benchmarks written with your benchmark framework of choice with your build pipeline, with a wide range of configuration options. This action allows you to track the performance of benchmarks against branches in your repository over the history of your project. You can also set thresholds on workflows in PRs, so performance regressions automatically prevent PRs from merging. Benchmark results can vary from framework to framework. This action supports a few different frameworks out of the box, and if yours is not supported, then it can be extended. For your benchmark results to be consumed, they must be kept in a file named output.txt, and formatted in a way that the action will understand. Each benchmark framework will have a different format. This action supports a few of the most popular ones. Example Benchmark in Rust Firstly, we need a benchmark to test with, and we’re going to use Rust. I am not going to detail everything to setup Rust projects in general, but a full example can be found here. In this case, there is just a simple fibonacci number generator. Then, a benchmark for this function can be written like so: In this case, we have two benchmarks that use the fib function with a different amount of iterations. The more iterations that you execute, the more accurate your results will be. Finally, if your project is setup to compile with cargo already, running the benchmarks should be as simple as running cargo bench. Now that the benchmark itself is setup, it’s time to move to the action. GitHub Action Setup The most basic use case of this action is setting it up against your main branch so it can collect performance data from every merge moving forward. GitHub actions are configured using yaml files. Let’s go over an example configuration that will run benchmarks on a rust project every time code gets pushed to main, starting with the event trigger. If you aren’t familiar with GitHub Actions already, the ‘on’ key allows us to specify the circumstances that this workflow will run. In our case, we want it to trigger when pushes happen against the main branch. If we want to, we can add additional triggers and branches as well. But for this example,, we’re only focusing on push for now. The jobs portion is relatively standard. The code gets checked out from source control, the tooling needed to build the Rust project is installed, the benchmarks are run, and then the results get pushed. For the results storing step, a GitHub API token is required. This is automatically generated when the workflow runs, and is not something that you need to add yourself. The results are then pushed to a special 'gh pages' branch where the performance data is stored. This branch does need to exist already for this step to work. Considerations There are some performance considerations to be aware of when utilizing GitHub Actions to execute benchmarks. Although the specifications of machines used for different action executions are similar, the runtime performance may vary. GitHub Actions are executed in virtual machines that are hosted on servers. The workloads of other actions on the same servers can affect the runtime performance of your benchmarks. Usually, this is not an issue at all, and results in minimal deviations. This is just something to keep in mind if you expect the results of each of your runs to be extremely accurate. Running benchmarks with more iterations does help, but isn’t a magic bullet solution. Here are the hardware specifications currently being used by GitHub Actions at the time of writing this article. This information comes from the GitHub Actions Documentation. Hardware specification for Windows and Linux virtual machines: 2 core CPU (x86 64) 7 GB of RAM 14 GB of SSD space Hardware specification for macOS virtual machines: 3 core CPU (x86 64) 14 GB of RAM 14 GB of SSD space If you need more consistent performance out of your runners, then you should use self hosted runners. Setting these up is outside the scope of this article, and is deserving of its own. Conclusion Continuous benchmarking can help detect performance issues before they cause issues in production, and with GitHub Actions, it is easier than ever to implement it. If you want to learn more about GitHub Qctions and even implementing your own, check out this JS Marathon video by Chris Trzesniewski....

Jamie Kuppens4 mins
GitHub Actions