General

A Guide to Keeping Secrets out of Git Repositories

May 31, 2022

6 min read

If you’ve been a developer for a while, then you hopefully know it is wise to keep secret information such as passwords and encryption keys outside of source control. If you didn’t know that, then surprise! Now you know.

Sometimes slip-ups do happen and a password ends up in a default config file or a new config file was not added to “.gitignore” and that same someone ran “git add .” and didn’t even notice it got committed. There should be protections in place no matter how diligent your programmers are since nobody is infallible, and the peace of mind is well worth it.

How can software know something is a secret?

It can’t know for sure, but it can make an educated guess. Secrets typically fit certain known patterns, or have higher entropy than other strings in your code and configuration files. A good scanner should check for strings that fit these patterns throughout your entire repository’s history, and raise anything suspicious to you.

Checking for Secrets in CI and CD

When it comes to automatically checking for secrets in your code, you have quite an array of options. To keep this article brief, I am just going to cover a few tools, and which tools you use may depend on your repository host.

GitHub

If you’re using GitHub for your project and your repository is either public or you use GitHub Enterprise Cloud, then GitHub will automatically scan the code you upload for secrets. GitHub’s solution is special because they have partnered with several different companies to allow for automatic revocation of secrets pushed to the repo. See the following excerpt from GitHub’s secret scanner documentation:

When you make a repository public, or push changes to a public repository, GitHub always scans the code for secrets that match partner patterns. If secret scanning detects a potential secret, we notify the service provider who issued the secret. The service provider validates the string and then decides whether they should revoke the secret, issue a new secret, or contact you directly. Their action will depend on the associated risks to you or them.

Well, that’s nice now isn’t it? If you’re curious if the services you use are partnered with GitHub so that their secrets can be scanned for, you can view the full list here. Just keep in mind that this functionality is only available for public repositories and private repositories using GitHub Enterprise Cloud with a “GitHub Advanced Security” license.

GitLab

GitLab, like GitHub, has secret detection as well. GitLab uses Gitleaks for their secret detection. This is a well documented tool whose source code is freely available. The capabilities of secret detection in GitLab does vary based on your tier, though.

You will have to use GitLab Ultimate to view detected secrets in the pipeline, and merge request sections for example. You can still use the scanner in free and premium versions, but it isn’t nearly as integrated as it is in the ultimate version.

Gitleaks

We mentioned that GitLab uses Gitleaks, but you aren’t just limited to using it with GitLab! Since Gitleaks is open source, that means you can use it with other providers such as GitHub, and even run it locally on your own system. It is also very easy to set up either as a CI job, or if you need to run it locally.

Scanning for Secrets using a CI Job

For GitHub you can simply use this action made by the author of Gitleaks. In this case Gitleaks is helpful if you’re using private repositories on GitHub without GitHub Enterprise Cloud. It is fully configurable with the action as it allows you to specify a custom .gitleaks.toml file. This is optional of course, and the default might work fine for you.

Checking for Secrets in a Pre-Commit Hook

There are a couple of ways to accomplish setting up the hook. A pre-commit script is available on the Gitleaks GitHub that will run Gitleaks on your staged files before you commit. Your commit will be stopped if any secrets are detected. This script can simply be copied into your .git/hooks/ directory. It does require that Gitleaks is installed and in your $PATH, however.

The other method involves using the pre-commit utility. It will assist with installing Gitleaks automatically for any developers that clone the repository and it can also assist with installing the hooks for the first time as well. Using the pre-commit tool might make more sense if you want to ensure other linters and checkers run, and you don’t want to have developers juggle installing everything themselves.

A Good Code Review Process Goes a Long Way

Although automated tooling for identifying secrets in code works well, it’s still good to keep an eye out for them when reviewing code. Automated scanning tools, as I mentioned earlier in the article, work great and you should definitely use them. However, they aren’t perfect. These tools look for sets of patterns and strings with high entropy, but not all secrets fit these criteria.

Knowing that even with the best scanning tools it’s still possible secrets could sneak through, it’s easier to understand why it is important to also have a good code review process to catch these issues. Also remember that committing secrets isn’t the only thing you should be worried about. You should have others review your work to help mitigate the chances that your changes could introduce new security vulnerabilities in the code as well!

Oh no, there’s already a secret in my Git history!

If you already have secrets in your repository, and they’re pushed to a main branch, not all hope is lost. Before we get into methods of removing the secrets, I have a massive disclaimer I should get out of the way. The only way to truly remove secrets from your repository is to rewrite your git history. This is a destructive operation, and will require developers to re-pull branches and cherry-pick changes from their local branches if applicable.

Did you read the disclaimer? Good, we can discuss methods then. What method we do depends on how and when the secret made it into the repository.

The Secret is in a Single Branch

If a secret made it into a feature branch by mistake, then you could simply initiate an interactive rebase to remove it and force push that branch to the remote. It should be noted that this is only effective if no other branches are based off of your branch, and if it isn’t tagged.

Let’s say, for example, you push your code to the remote and a CI job identifies a secret after your push. At this point there would be nobody else using your branch and it shouldn’t be tagged, so this is the perfect opportunity to just rewrite the commit that triggered the CI failure. If your commit hash is let’s say 09fac8dbfd27bd9b4d23a00eb648aa751789536d, then these are the first command you would have to execute to begin cleaning up your branch’s history:

$ git rebase --interactive 09fac8dbfd27bd9b4d23a00eb648aa751789536d^

Note the caret at the end of the SHA1 commit hash. This is vitally important to include as we need to rebase to the commit prior to the commit introducing the secret. The gist is that we’re going to return back to that point in time, and prevent the secret from ever being added in the first place.

Git will now open your default command-line text editor and ask you how you want to execute your rebase. Find the line referencing the problematic commit and replace pick with edit on that line. If you save the file and quit you then should now find yourself at the commit where the secret was introduced. From here you can remove the secret, stage the affected files, and then execute the following commands:

$ git commit --all --amend --no-edit

And if that’s successful:

$ git rebase --continue

Then after that the history should be successfully rewritten to not include the secret locally. If you pushed your branch already, then you’ll need to get these changes pushed to the remote as well. You can do this with a force push using the -f flag like so:

$ git push origin <your_branch_name> -f

Now you should be set! At this point, you can get to fixing up your code to pull the secret some other way that doesn’t include config files or strings hard-coded inside of your codebase.

The Secrets are in Main Already…

If your secrets are present in many branches, like tagged versions or your main branch, then things get a little more complicated. There’s more than one way to handle this situation, but I am going to cover only one. Just revoke the secret.

In this scenario, you should revoke the secret and issue a new one. If you do this, then it doesn’t matter that the old secret is still in the repository history because it will be entirely useless! How this is done of course depends on what service the secret was issued from.

There is also the added benefit in that anyone that has cloned your repo with the secret will also be unable to use it. Simply rewriting history doesn’t matter if an adversary has already downloaded it before you deleted it.

It’s important to note that this isn’t always great if, for example, the secret is used in multiple projects. You will need to ensure your revoked secret is replaced everywhere before you actually revoke it or else you may experience downtime.

Conclusion

With scanning tools becoming more accessible, there are fewer and fewer reasons to not use them. Secret scanning is especially important for public repositories, but it is also useful for private repositories where a compromised developer account can access secrets and wreak havoc.

This Dot Labs is a development consultancy that is trusted by top industry companies, including Stripe, Xero, Wikimedia, Docusign, and Twilio. This Dot takes a hands-on approach by providing tailored development strategies to help you approach your most pressing challenges with clarity and confidence. Whether it's bridging the gap between business and technology or modernizing legacy systems, you’ll find a breadth of experience and knowledge you need. Check out how This Dot Labs can empower your tech journey.

Jamie Kuppens

I’m a software engineer with an interest in web development and some more esoteric things like emulator development.

@Reshurum @Reshurum

A Deep Dive into SvelteKit Routing with Our Starter.dev GitHub Showcase Example

Introduction SvelteKit is an excellent framework for building web applications of all sizes, with a beautiful development experience and flexible filesystem-based routing. At the heart of SvelteKit is a filesystem-based router. The routes of your app — i.e. the URL paths that users can access — are defined by the directories in your codebase. In this tutorial, we are going to discuss SvelteKit routing with an awesome SvelteKit GitHub showcase built by This Dot Labs. The showcase is built with the SvelteKit starter kit on starter.dev. We are going to tackle: - Filesystem-based router - +page.svelte - +page.server - +layout.svelte - +layout.server - +error.svelte - Advanced Routing - Rest Parameters - (group) layouts - Matching Below is the current routes folder. Prerequisites You will need a development environment running Node.js; this tutorial was tested on Node.js version 16.18.0, and npm version 8.19.2. Filesystem-based router The src/routes` is the root route. You can change `src/routes` to a different directory by editing the project config. `js // svelte.config.js / @type {import('@sveltejs/kit').Config} */ const config = { kit: { routes: "src/routes", // 👈 you can change it here to anything you want }, }; ` Each route directory contains one or more route files, which can be identified by their + prefix. +page.svelte A +page.svelte` component defines a page of your app. By default, pages are rendered both on the server (SSR) for the initial request, and in the browser (CSR) for subsequent navigation. In the below example, we see how to render a simple login page component: ` // src/routes/signin/(auth)/+page.svelte import Auth from '$lib/components/auth/Auth.svelte'; ` +page.ts Often, a page will need to load some data before it can be rendered. For this, we add a +page.js` (or `+page.ts`, if you're TypeScript-inclined) module that exports a load function. +page.server.ts` If your load function can only run on the server— ie, if it needs to fetch data from a database or you need to access private environment variables like API key— then you can rename +page.js` to `+page.server.js`, and change the `PageLoad` type to `PageServerLoad`. To pass top user repository data, and user’s gists to the client-rendered page, we do the following: `ts // src/routes/(authenticated)/(home)/+page.server.ts import type { PageServerLoad } from "./$types"; import { mapUserReposToTopRepos, mapGistsToHomeGists } from "$lib/helpers"; import type { UserGistsApiResponse, UserReposApiResponse, } from "$lib/interfaces"; import { ENV } from "$lib/constants/env"; export const load: PageServerLoad = async ({ fetch, parent }) => { const repoURL = new URL("/user/repos", ENV.GITHUBURL); repoURL.searchParams.append("sort", "updated"); repoURL.searchParams.append("perpage", "20"); const { userInfo } = await parent(); const gistsURL = new URL( /users/${userInfo?.username}/gists`, ENV.GITHUBURL ); try { const reposPromise = await fetch(repoURL); const gistsPromise = await fetch(gistsURL); const [repoRes, gistsRes] = await Promise.all([reposPromise, gistsPromise]); const gists = (await gistsRes.json()) as UserGistsApiResponse; const repos = (await repoRes.json()) as UserReposApiResponse; return { topRepos: mapUserReposToTopRepos(repos), gists: mapGistsToHomeGists(gists), username: userInfo?.username, }; } catch (err) { console.log(err); } }; ` The page.svelte` gets access to the data by using the data variable which is of type `PageServerData`. `html import TopRepositories from '$lib/components/TopRepositories/TopRepositories.svelte'; import Gists from '$lib/components/Gists/Gists.svelte'; import type { PageServerData } from './$types'; export let data: PageServerData; {#if data?.gists} {/if} {#if data?.topRepos} {/if} @use 'src/lib/styles/variables.scss'; .page-container { display: grid; grid-template-columns: 1fr; background: variables.$gray100; @media (min-width: variables.$md) { grid-template-columns: 24rem 1fr; } } aside { background: variables.$white; padding: 2rem; @media (max-width: variables.$md) { order: 2; } } ` +layout.svelte As there are elements that should be visible on every page, such as top-level navigation or a footer. Instead of repeating them in every +page.svelte, we can put them in layouts. The only requirement is that the component includes a ` for the page content. For example, let's add a nav bar: `html import NavBar from '$lib/components/NavBar/NavBar.svelte'; import type { LayoutServerData } from './$types'; export let data: LayoutServerData; // 👈 child routes of the layout page ` +layout.server.ts Just like +page.server.ts`, your `+layout.svelte` component can get data from a load function in `+layout.server.js`, and change the type from `PageServerLoad` type to LayoutServerLoad. `ts // src/routes/(authenticated)/+layout.server.ts import { ENV } from "$lib/constants/env"; import { remapContextUserAsync } from "$lib/helpers/context-user"; import type { LayoutServerLoad } from "./$types"; import { mapUserInfoResponseToUserInfo } from "$lib/helpers/user"; export const load: LayoutServerLoad = async ({ locals, fetch }) => { const getContextUserUrl = new URL("/user", ENV.GITHUBURL); const response = await fetch(getContextUserUrl.toString()); const contextUser = await remapContextUserAsync(response); locals.user = contextUser; return { userInfo: mapUserInfoResponseToUserInfo(locals.user), }; }; ` +error.svelte If an error occurs during load, SvelteKit will render a default error page. You can customize this error page on a per-route basis by adding an +error.svelte file. In the showcase, an error.svelte` page has been added for authenticated view in case of an error. `html import { page } from '$app/stores'; import ErrorMain from '$lib/components/ErrorPage/ErrorMain.svelte'; import ErrorFlash from '$lib/components/ErrorPage/ErrorFlash.svelte'; ` Advanced Routing Rest Parameters If the number of route segments is unknown, you can use spread operator syntax. This is done to implement Github’s file viewer. ` /[org]/[repo]/tree/[branch]/[...file] ` svelte-kit-scss.starter.dev/thisdot/starter.dev/blob/main/starters/svelte-kit-scss/README.md` would result in the following parameters being available to the page: `js { org: ‘thisdot’, repo: 'starter.dev', branch: 'main', file: ‘/starters/svelte-kit-scss/README.md' } ` (group) layouts By default, the layout hierarchy mirrors the route hierarchy. In some cases, that might not be what you want. In the GitHub showcase, we would like an authenticated user to be able to have access to the navigation bar, error page, and user information. This is done by grouping all the relevant pages which an authenticated user can access. Grouping can also be used to tidy your file tree and ‘group’ similar pages together for easy navigation, and understanding of the project. Matching In the Github showcase, we needed to have a page to show issues and pull requests for a single repo. The route src/routes/(authenticated)/[username]/[repo]/[issues]` would match `/thisdot/starter.dev-github-showcases/issues` or `/thisdot/starter.dev-github-showcases/pull-requests` but also `/thisdot/starter.dev-github-showcases/anything` and we don't want that. You can ensure that route parameters are well-formed by adding a matcher— which takes only `issues` or `pull-requests`, and returns true if it is valid– to your params directory. `ts // src/params/issuesearch_type.ts import { IssueSearchPageTypeFiltersMap } from "$lib/constants/matchers"; import type { ParamMatcher } from "@sveltejs/kit"; export const match: ParamMatcher = (param: string): boolean => { return Object.keys(IssueSearchPageTypeFiltersMap).includes( param?.toLowerCase() ); }; ` `ts // src/lib/constants/matchers.ts import { IssueSearchQueryType } from './issues-search-query-filters'; export const IssueSearchPageTypeFiltersMap = { issues: ‘issues’, pulls: ’pull-requests’, }; export type IssueSearchTypePage = keyof typeof IssueSearchPageTypeFiltersMap; ` ...and augmenting your routes: ` [issueSearchType=issuesearch_type] ` If the pathname doesn't match, SvelteKit will try to match other routes (using the sort order specified below), before eventually returning a 404. Note: Matchers run both on the server and in the browser.` Conclusion In this article, we learned about basic and advanced routing in SvelteKit by using the SvelteKit showcase example. We looked at how to work with SvelteKit's Filesystem-based router, rest parameters, and (group) layouts. If you want to learn more about SvelteKit, please check out the SvelteKit and SCSS starter kit and the SvelteKit and SCSS GitHub showcase. All the code for our showcase project is open source. If you want to collaborate with us or have suggestions, we're always welcome to new contributions. Thanks for reading! If you have any questions, or run into any trouble, feel free to reach out on Twitter....

Mar 22, 2023

5 mins

SvelteGit

Git Strategies For Teams, Another Take

Recently we published an article about a pragmatic approach to using Git in teams. It outlines a strategy which is easy to implement while safeguarding against common issues when using git. It is, however, in my opinion, a compromise. Allowing some issues with edge cases as a trade-off for ease of use. Status Quo Before jumping into another strategy, let us establish the pros and cons of what we’re up against. This will give us a reference for determining whether we did better, or not. Pro: Squash Merging One of the stronger arguments is the squash merge. It offers freedom to all developers within the team to develop as they see fit. One might prefer to develop all at the same time, and push a big commit in the end. Others might like to play it safe and simply commit every 5 minutes, allowing them to roll back or backup changes. The only rule is that at the end of the work, all changes get squashed into a single commit that has to adhere to the team's standards. Con: You Get A Single Commit Each piece of work gets a single commit. To circumvent this, one could create multiple PRs and break-up the work into bite-size chunks. This is, undeniably, a good practice. But it can be cumbersome and distributive when trying to keep pace. In addition, you put the team at risk of getting git conflicts. Say you’re in the middle of building your feature and uncover a bug which requires fixing. As a good scout, you implement the fix and create a separate PR to deliver the fix to the team. At the same time, you keep the fix in your branch as you need it for your feature. At the point of squashing, your newly squashed commit conflicts with the stand-alone fix you’ve delivered to your team. This is nothing that git can’t fix, but it is a nuisance nonetheless. Con: Review Potential A good PR shouldn’t have too many changes, making it easy to review. In the real world however, things get messy. Commits can give us some insight into how the complete changeset came to be. This requires the team to write well-curated commits though. This conflicts with the strength we’re getting from allowing freedom to commit as one sees fit. The History Rewriting Controversy It is good to know that what I’m about to suggest is considered blasphemy by many. Rewriting history is not without its dangers. Changes can go missing, and others who have based their work on now-changed history need to deal with conflicts. However, when applied prudently, rewriting history can yield benefits as well. In this context, some advanced git knowledge is required. The Alternative There was a soft hint towards using conventional commits in Dustin’s article. Let's go ahead and fully endorse adopting it. The convention is simple enough, and the documentation is exhaustive. Now I hear you think, “but we just concluded that allowing us to commit as we like was a good thing”. And you are not wrong. This is where history rewriting comes into play. As you’re working, commit as you like. Then, when it’s time to put your changes up for review, start editing your branch to ensure each change is nicely wrapped and documented in a proper commit. Finally, after getting a thumbs-up on the PR, rebase your changes on top of the branch you’re merging into, and finally do a normal merge. Most git hosting services offer this workflow for you. While I endorse rewriting history in your own branch, restrain from altering shared branches like “main” or “develop”. By sticking to this small rule, you’ve already negated most disadvantages of rewriting history. Shared Changes If we look back at our scenario where both the main and our feature branch include a fix, we get to the same point where we want to merge. However in this case, given that you’ve made the same commit in both branches, git is clever enough to fix the flow of history and remove the fix from your new changes. The following flow... ... will look like this after merging: Fixes On Your Own Features Although this is part of the conventional commit strategy, I feel it deserves some special attention. If you have introduced a new feature in your branch, and committed the changes. It can happen that you introduced a bug. Your first intuition might be to create a “fix” commit. Instead, consider going back to your feature commit and amending the fix to it. This has two advantages. First of all, the history will be less cluttered. Looking back at what changed, it's easy to see which features got introduced and what bugs we found along the way. On top of that, it will prevent confusion for your reviewers. Now, the code presented to others is fixed code. At no time in its history does it ever contain the bug. Your co-workers are not going to have any comments on it. How To Rewrite Your Branch Now we know why to clean-up, let's look at strategies to actually do the orchestration. The most obvious route is to keep the changeset you want to present in mind. Doing so, one prevents having to go back and rewrite everything from scratch. As an added bonus, I’ve found that it helps me better separate concerns. Complete Wipe If you like making periodic commits (or some other strategy that results in you creating arbitrary commits) chances are you are going to completely wipe all commits (not the changes) in your branch. The simplest way to accomplish this is by doing a soft reset to where you forked from the main branch. This can be achieved by rebasing and resetting to main (given main is where you want to merge into). This is a good approach as you also prepare your branch for being merged back. `bash git rebase main git reset main ` This can also be accomplished by counting the amount of commits and making that amount of steps back from HEAD. For example, if you have made 4 commits in your branch. `bash git reset HEAD4 ` And lastly, you can do this by knowing where you started off. One can find this by looking at the logs: `bash git reset 6a5c8e8f2b ` Using either of these methods will leave you with no commits in your branch, and all your changes in your workspace. From this point, you can start cherry-picking your changes, and making well-curated commits. Interactive Rebase If you already have somewhat of a structure, interactive rebasing might be a better solution for you. This will allow you to go over each commit, and decide on how to alter them. The most interesting options being: s, squash__ - this will add the changes from this commit to its parent, followed by allowing you to change the commit message, and thus appending the message with the squashed changes. e, edit__ - using this option, the rebase will stop right before the commit gets added to the branch as if you went back in time and just did the development work. From this point, you can add files, split the commit in multiple different commits, change the commit message, or do whatever you’d like to do. d, drop__ - in the rare occasion you simply don’t want this commit anymore. r, reword__ - like edit, but you’re only offered the option to change the commit message. To start an interactive rebase, simply run `bash git rebase -i main ` Conclusion By embracing history rewriting and dropping squash merging. A team could produce an even cleaner git history. This option might not be for everyone, as it requires a little work and git knowledge. But if done well, it will circumvent some of the drawbacks of our pragmatic approach....

Oct 17, 2022

6 mins

Git

How to Add Continuous Benchmarking to Your Projects Using GitHub Actions

Over the lifetime of a project performance, issues may arise from time to time. Lots of the time, these issues don't get detected until they get into production. Adding continuous benchmarking to your project and build pipeline can help you catch these issues before that happens. What is Continuous Benchmarking Benchmarking is the process of measuring the performance of an application. Continuous benchmarking builds on top of this by doing so either on a regular basis, or whenever new code is pushed so that performance regressions can be identified and found as soon as they are introduced. Adding continuous benchmarking to your build pipeline can help you effectively catch performance issues before they ever make it to production. Much like with tests, you are still responsible for writing benchmark logic. But once that’s done, integrating it with your build pipeline can be done easily using the continuous-benchmark GitHub Action. github-action-benchmark github-action-benchmark allows you to easily integrate your existing benchmarks written with your benchmark framework of choice with your build pipeline, with a wide range of configuration options. This action allows you to track the performance of benchmarks against branches in your repository over the history of your project. You can also set thresholds on workflows in PRs, so performance regressions automatically prevent PRs from merging. Benchmark results can vary from framework to framework. This action supports a few different frameworks out of the box, and if yours is not supported, then it can be extended. For your benchmark results to be consumed, they must be kept in a file named output.txt`, and formatted in a way that the action will understand. Each benchmark framework will have a different format. This action supports a few of the most popular ones. Example Benchmark in Rust Firstly, we need a benchmark to test with, and we’re going to use Rust. I am not going to detail everything to setup Rust projects in general, but a full example can be found here. In this case, there is just a simple fibonacci number generator. ` src/lib.rs pub fn fib(u: u32) -> u32 { if u GitHub Action Setup The most basic use-case of this action is setting it up against your main branch so it can collect performance data from every merge moving forward. GitHub actions are configured using yaml files. Let’s go over an example configuration that will run benchmarks on a rust project every time code gets pushed to main, starting with the event trigger. ` name: Benchmarks on: push: branches: - main ` If you aren’t familiar with GitHub Actions already, the ‘on’ key allows us to specify the circumstances that this workflow will run. In our case, we want it to trigger when pushes happen against the main branch. If we want to, we can add additional triggers and branches as well. But for this example,, we’re only focusing on push for now. ` jobs: benchmark: name: Run Rust Benchmark runs-on: ubuntu-latest steps: # Checkout the code and run the benchmarks using cargo. - uses: actions/checkout@v2 - run: rustup toolchain update nightly && rustup default nightly - name: Run benchmark run: cargo bench | tee output.txt # Push the results using the benchmark action. - name: Store Benchmark Result uses: benchmark-action/github-action-benchmark@v1 with: name: Rust Benchmark tool: 'cargo' output-file-path: output.txt github-token: ${{ secrets.GITHUBTOKEN }} auto-push: true alert-threshold: '200%' fail-on-alert: true ` The jobs portion is relatively standard. The code gets checked out from source control, the tooling needed to build the Rust project is installed, the benchmarks are run, and then the results get pushed. For the results storing step, a GitHub API token is required. This is automatically generated when the workflow runs, and is not something that you need to add yourself. The results are then pushed to a special 'gh-pages' branch where the performance data is stored. This branch does need to exist already for this step to work. Considerations There are some performance considerations to be aware of when utilizing GitHub Actions to execute benchmarks. Although the specifications of machines used for different action executions are similar, the runtime performance may vary. GitHub Actions are executed in virtual machines that are hosted on servers. The workloads of other actions on the same servers can affect the runtime performance of your benchmarks. Usually, this is not an issue at all, and results in minimal deviations. This is just something to keep in mind if you expect the results of each of your runs to be extremely accurate. Running benchmarks with more iterations does help, but isn’t a magic bullet solution. Here are the hardware specifications currently being used by GitHub Actions at the time of writing this article. This information comes from the GitHub Actions Documentation. Hardware specification for Windows and Linux virtual machines: - 2-core CPU (x8664) - 7 GB of RAM - 14 GB of SSD space Hardware specification for macOS virtual machines: - 3-core CPU (x8664) - 14 GB of RAM - 14 GB of SSD space If you need more consistent performance out of your runners, then you should use self-hosted runners. Setting these up is outside the scope of this article, and is deserving of its own. Conclusion Continuous benchmarking can help detect performance issues before they cause issues in production, and with GitHub Actions, it is easier than ever to implement it. If you want to learn more about GitHub Qctions and even implementing your own, check out this JS Marathon video by Chris Trzesniewski....

Apr 24, 2023

4 mins

GitHub Actions

Testing a Fastify app with the NodeJS test runner

Introduction Node.js has shipped a built-in test runner for a couple of major versions. Since its release I haven’t heard much about it so I decided to try it out on a simple Fastify API server application that I was working on. It turns out, it’s pretty good! It’s also really nice to start testing a node application without dealing with the hassle of installing some additional dependencies and managing more configurations. Since it’s got my stamp of approval, why not write a post about it? In this post, we will hit the highlights of the testing API and write some basic but real-life tests for an API server. This server will be built with Fastify, a plugin-centric API framework. They have some good documentation on testing that should make this pretty easy. We’ll also add a SQL driver for the plugin we will test. Setup Let's set up our simple API server by creating a new project, adding our dependencies, and creating some files. Ensure you’re running node v20 or greater (Test runner is a stable API as of the 20 major releases) Overview `index.js` - node entry that initializes our Fastify app and listens for incoming http requests on port 3001 `app.js` - this file exports a function that creates and returns our Fastify application instance `sql-plugin.js` - a Fastify plugin that sets up and connects to a SQL driver and makes it available on our app instance Application Code A simple first test For our first test we will just test our servers index route. If you recall from the app.js` code above, our index route returns a 501 response for “not implemented”. In this test, we're using the createApp` function to create a new instance of our Fastify app, and then using the `inject` method from the Fastify API to make a request to the `/` route. We import our test utilities directly from the node. Notice we can pass async functions to our test to use async/await. Node’s assert API has been around for a long time, this is what we are using to make our test assertions. To run this test, we can use the following command: By default the Node.js test runner uses the TAP reporter. You can configure it using other reporters or even create your own custom reporters for it to use. Testing our SQL plugin Next, let's take a look at how to test our Fastify Postgres plugin. This one is a bit more involved and gives us an opportunity to use more of the test runner features. In this example, we are using a feature called Subtests. This simply means when nested tests inside of a top-level test. In our top-level test call, we get a test parameter t` that we call methods on in our nested test structure. In this example, we use `t.beforeEach` to create a new Fastify app instance for each test, and call the `test` method to register our nested tests. Along with `beforeEach` the other methods you might expect are also available: `afterEach`, `before`, `after`. Since we don’t want to connect to our Postgres database in our tests, we are using the available Mocking API to mock out the client. This was the API that I was most excited to see included in the Node Test Runner. After the basics, you almost always need to mock some functions, methods, or libraries in your tests. After trying this feature, it works easily and as expected, I was confident that I could get pretty far testing with the new Node.js core API’s. Since my plugin only uses the end method of the Postgres driver, it’s the only method I provide a mock function for. Our second test confirms that it gets called when our Fastify server is shutting down. Additional features A lot of other features that are common in other popular testing frameworks are also available. Test styles and methods Along with our basic test` based tests we used for our Fastify plugins - `test` also includes `skip`, `todo`, and `only` methods. They are for what you would expect based on the names, skipping or only running certain tests, and work-in-progress tests. If you prefer, you also have the option of using the describe` → `it` test syntax. They both come with the same methods as `test` and I think it really comes down to a matter of personal preference. Test coverage This might be the deal breaker for some since this feature is still experimental. As popular as test coverage reporting is, I expect this API to be finalized and become stable in an upcoming version. Since this isn’t something that’s being shipped for the end user though, I say go for it. What’s the worst that could happen really? Other CLI flags —watch` - https://nodejs.org/dist/latest-v20.x/docs/api/cli.html#--watch —test-name-pattern` - https://nodejs.org/dist/latest-v20.x/docs/api/cli.html#--test-name-pattern TypeScript support You can use a loader like you would for a regular node application to execute TypeScript files. Some popular examples are tsx` and `ts-node`. In practice, I found that this currently doesn’t work well since the test runner only looks for JS file types. After digging in I found that they added support to locate your test files via a glob string but it won’t be available until the next major version release. Conclusion The built-in test runner is a lot more comprehensive than I expected it to be. I was able to easily write some real-world tests for my application. If you don’t mind some of the features like coverage reporting being experimental, you can get pretty far without installing any additional dependencies. The biggest deal breaker on many projects at this point, in my opinion, is the lack of straightforward TypeScript support. This is the test command that I ended up with in my application: I’ll be honest, I stole this from a GitHub issue thread and I don’t know exactly how it works (but it does). If TypeScript is a requirement, maybe stick with Jest or Vitest for now 🙂...

Nov 29, 2023

5 mins

NodeJSTypeScriptTesting