Javascript

BullMQ with ExpressJS

Published Feb 24, 2023

Updated Mar 2, 2023

4 min read

This article was written over 18 months ago and may contain information that is out of date. Some content may be relevant but please refer to the relevant official documentation or available resources for the latest information.

Node.js uses an event loop to process asynchronous tasks. The event loop is responsible for handling the execution of asynchronous tasks, but it does it in a single thread. If there is some CPU-intensive or long-running (blocking) logic that needs to be executed, it should not be run on the main thread. This is where BullMQ can help us.

In this blog post, we are going to set up a basic queue, using BullMQ. If you'd like to jump right into developing with a queue, check out our starter.dev kit for ExpressJS, where a fully functioning queue is already provided for you.

Why and when to use a queue?

BullMQ is a library that can be used to implement message queues in Node.js applications. A message queue allows different parts of an application, or different applications, to communicate with each other asynchronously by sending and receiving messages. This can be useful in a variety of situations, such as when one part of the application needs to perform a task that could take a long time, or when different parts of the application need to be decoupled from each other for flexibility and scalability. If you need to process large numbers of messages in a distributed environment, it can help you solve your problems.

One such scenario to use a queue is when you need to deal with webhooks. Webhook handler endpoints usually need to respond with 2xx quickly, therefore, you cannot put long running tasks inside that handler, but you also need to process the incoming data. A good way of mitigating this is to put the incoming data into the queue, and respond quickly. The processing gets taken care of with the BullMQ worker, and if it is set up correctly, it will run inside a child process not blocking the main thread.

Prerequisites

BullMQ utilizes Redis to handle its message queue. In development mode, we are using docker-compose to start up a redis instance. We expose the redis docker container's 6379 port to be reachable on the host machine. We also mount the /misc/data and the misc/conf folders to preserve data for our local development environments.

version: '3'
services:
  ## Other docker containers ...
  redis:
    image: 'redis:alpine'
    command: redis-server /usr/local/etc/redis/redis.conf
    ports:
      - '6379:6379'
    volumes:
      - ./misc/data:/var/lib/redis
      - ./misc/conf:/usr/local/etc/redis/
    environment:
      - REDIS_REPLICATION_MODE=master

We can start up our infrastructure with the docker-compose up -d command and we can stop it with the docker-compose stop command. We also need to set up connection information for BullMQ. We make it configurable by using environment variables.

// config.constants.ts
export const REDIS_QUEUE_HOST = process.env.REDIS_QUEUE_HOST || 'localhost';
export const REDIS_QUEUE_PORT = process.env.REDIS_QUEUE_PORT
	? parseInt(process.env.REDIS_QUEUE_PORT)
	: 6479;

To start working with BullMQ, we also need to install it to our node project:

npm install bullmq

Setting up a queue

Creating a queue is pretty straightforward, we need to pass the queue name as a string and the connection information.

// queue.ts
import { Queue } from 'bullmq';
import { 	
	REDIS_QUEUE_HOST,
	REDIS_QUEUE_PORT,
} from './config.constants';

export const myQueue = new Queue('my-queue', {
	connection: {
		host: REDIS_QUEUE_HOST,
		port: REDIS_QUEUE_PORT,
	},
});

We also create a function that can be used to add jobs to the queue from an endpoint handler. I suggest setting up a rule that removes completed and failed jobs in a timely manner. In this example we remove completed jobs after an hour from redis, and we leave failed jobs for a day. These values depend on what you want to achieve, It can very well happen that in a production app, you would keep the jobs for weeks.

// queue.ts
import { Queue, Job } from 'bullmq';
// ...

const DEFAULT_REMOVE_CONFIG = {
	removeOnComplete: {
		age: 3600,
	},
	removeOnFail: {
		age: 24 * 3600,
	},
};

export async function addJobToQueue<T>(data: T): Promise<Job<T>> {
	return myQueue.add('job', data, DEFAULT_REMOVE_CONFIG);
}

It would be called when a specific endpoint gets called.

// main.ts

app.post('/', async (req: Request, res: Response, next: NextFunction) => {
	const job = await addJobToQueue(req.body);
	res.json({ jobId: job.id });
	return next();
});

The queue now can store jobs, but in order for us to be able to process those jobs, we need to set up a worker.

Put the processing into a thread

We set up a worker with an async function at the beginning. The worker needs the same name as the queue to start consuming jobs in that queue. It also needs the same connection information as the queue we set up before.

// worker.ts
import { Job, Worker } from 'bullmq';
import {
	REDIS_QUEUE_HOST,
	REDIS_QUEUE_PORT,
} from './config.constants';

let worker: Worker

export function setUpWorker(): void {
	worker = new Worker('my-queue', async () => {/** ... */}, {
		connection: {
			host: REDIS_QUEUE_HOST,
			port: REDIS_QUEUE_PORT,
		},
		autorun: true,
	});

	defaultWorker.on('completed', (job: Job, returnvalue: 'DONE') => {
		console.debug(`Completed job with id ${job.id}`, returnvalue);
	});

	defaultWorker.on('active', (job: Job<unknown>) => {
		console.debug(`Completed job with id ${job.id}`);
	});
	defaultWorker.on('error', (failedReason: Error) => {
		console.error(`Job encountered an error`, failedReason);
	});	
}

// we call the method after we set up the queue in queue.ts
import { Queue } from 'bullmq';
import { setUpWorker } from './worker';
// ...

setUpWorker();

After we create the worker and set up event listeners, we call the setUpWorker() method in the queue.ts file after the Queue gets created. Let's set up the job processor function.

// job-processor.ts
import { Job } from 'bullmq';

module.exports = async function jobProcessor(job: Job): Promise<'DONE'> {
	await job.log(`Started processing job with id ${job.id}`);
	console.log(`Job with id ${job.id}`, job.data);

	// TODO: do your CPU intense logic here

	await job.updateProgress(100);
	return 'DONE';
};

This example processor function doesn't do much, but if we had a long running job, like complex database update operations, or sending data towards a third-party API, we would do it here. Let's make sure our worker will run these jobs on a separate thread.

// worker.ts
// ...

let worker: Worker

const processorPath = path.join(__dirname, 'job-processor.js');

export function setUpWorker(): void {
	worker = new Worker('my-queue', processorPath, {
		connection: {
			host: REDIS_QUEUE_HOST,
			port: REDIS_QUEUE_PORT,
		},
		autorun: true,
	});

	// ...
}

If you provide a file path to the worker as the second parameter, BullMQ will run the function exported from the file in a separate thread. That way, the main thread is not used for the CPU intense work the processor does.

The above example works if you run the TypeScript compiler on your back-end code (tsc), but if you prefer keeping your code in TypeScript and run the logic with ts-node, then you should use the TypeScript file as your processor path.

const processorPath = path.join(__dirname, 'job-processor.ts');

The problem with bundlers

Sometimes, your back-end code gets bundled with webpack. For example, if you use NX to keep your front-end and back-end code together in one repository, you will notice that your back-end code gets bundled with webpack. In order to be able to run your processors in a separate thread, you need to tweak your project.json configuration:

{
  "targets": {
    "build": {
      "options": {
        "outputPath": "dist/apps/api",
        "main": "apps/api/src/main.ts",
        "tsConfig": "apps/api/tsconfig.app.json",
        "assets": [],
        "additionalEntryPoints": [
          {
            "entryName": "sync.processor",
            "entryPath": "apps/api/src/app/queue/job-processor.ts"
          }
        ]
      }
    }
  }
}

Conclusion

In an ExpressJS application, running CPU intense tasks on the main thread could cause your endpoints to turn unresponsive and/or slow. Moving these tasks into a different thread can help alleviate performance issues, and using BullMQ can help you out greatly.

If you want to learn more about NodeJS, check out node.framework.dev for a curated list of libraries and resources. If you are looking to start a new ExpressJS project, check out our starter kit resources at starter.dev

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

Balázs Tápai
He is a Software Engineer with a passion for automated testing. He loves Angular on the Front-End and NestJS on the Back-End. He also uses Cypress to reduce developer anxiety before demo meetings.
@TapaiBalazs @TapaiBalazs

Building a Multi-Response Streaming API with Node.js, Express, and React

Introduction As web applications become increasingly complex and data-driven, efficient and effective data transfer methods become critically important. A streaming API that can send multiple responses to a single request can be a powerful tool for handling large amounts of data or for delivering real-time updates. In this article, we will guide you through the process of creating such an API. We will use video streaming as an illustrative example. With their large file sizes and the need for flexible, on-demand delivery, videos present a fitting scenario for showcasing the power of multi-response streaming APIs. The backend will be built with Node.js and Express, utilizing HTTP range requests to facilitate efficient data delivery in chunks. Next, we'll build a React front-end to interact with our streaming API. This front-end will handle both the display of the streamed video content and its download, offering users real-time progress updates. By the end of this walkthrough, you will have a working example of a multi-response streaming API, and you will be able to apply the principles learned to a wide array of use cases beyond video streaming. Let's jump right into it! Hands-On Implementing the Streaming API in Express In this section, we will dive into the server-side implementation, specifically our Node.js and Express application. We'll be implementing an API endpoint to deliver video content in a streaming fashion. Assuming you have already set up your Express server with TypeScript, we first need to define our video-serving route. We'll create a GET endpoint that, when hit, will stream a video file back to the client. Please make sure to install cors for handling cross-origin requests, dotenv for loading environment variables, and throttle for controlling the rate of data transfer. You can install these with the following command: ` ` In the code snippet above, we are implementing a basic video streaming server that responds to HTTP range requests. Here's a brief overview of the key parts: 1. File and Range Setup: We start by determining the path to the video file and getting the file size. We also grab the range header from the request, which contains the range of bytes the client is requesting. 2. Range Requests Handling: If a range is provided, we extract the start and end bytes from the range header, then create a read stream for that specific range. This allows us to stream a portion of the file rather than the entire thing. 3. Response Headers: We then set up our response headers. In the case of a range request, we send back a '206 Partial Content' status along with information about the byte range and total file size. For non-range requests, we simply send back the total file size and the file type. 4. Data Streaming: Finally, we pipe the read stream directly to the response. This step is where the video data actually gets sent back to the client. The use of pipe() here automatically handles backpressure, ensuring that data isn't read faster than it can be sent to the client. With this setup in place, our streaming server is capable of efficiently delivering large video files to the client in small chunks, providing a smoother user experience. Implementing the Download API in Express Now, let's add another endpoint to our Express application, which will provide more granular control over the data transfer process. We'll set up a GET endpoint for '/download', and within this endpoint, we'll handle streaming the video file to the client for download. ` This endpoint has a similar setup to the video streaming endpoint, but it comes with a few key differences: 1. Response Headers: Here, we include a 'Content-Disposition' header with an 'attachment' directive. This header tells the browser to present the file as a downloadable file named 'video.mp4'. 2. Throttling: We use the 'throttle' package to limit the data transfer rate. Throttling can be useful for simulating lower-speed connections during testing, or for preventing your server from getting overwhelmed by data transfer operations. 3. Data Writing: Instead of directly piping the read stream to the response, we attach 'data' and 'end' event listeners to the throttled stream. On the 'data' event, we manually write each chunk of data to the response, and on the 'end' event, we close the response. This implementation provides a more hands-on way to control the data transfer process. It allows for the addition of custom logic to handle events like pausing and resuming the data transfer, adding custom transformations to the data stream, or handling errors during transfer. Utilizing the APIs: A React Application Now that we have a server-side setup for video streaming and downloading, let's put these APIs into action within a client-side React application. Note that we'll be using Tailwind CSS for quick, utility-based styling in our components. Our React application will consist of a video player that uses the video streaming API, a download button to trigger the download API, and a progress bar to show the real-time download progress. First, let's define the Video Player component that will play the streamed video: ` In the above VideoPlayer component, we're using an HTML5 video tag to handle video playback. The src attribute of the source tag is set to the video endpoint of our Express server. When this component is rendered, it sends a request to our video API and starts streaming the video in response to the range requests that the browser automatically makes. Next, let's create the DownloadButton component that will handle the video download and display the download progress: ` In this DownloadButton component, when the download button is clicked, it sends a fetch request to our download API. It then uses a while loop to continually read chunks of data from the response as they arrive, updating the download progress until the download is complete. This is an example of more controlled handling of multi-response APIs where we are not just directly piping the data, but instead, processing it and manually sending it as a downloadable file. Bringing It All Together Let's now integrate these components into our main application component. ` In this simple App component, we've included our VideoPlayer and DownloadButton components. It places the video player and download button on the screen in a neat, centered layout thanks to Tailwind CSS. Here is a summary of how our system operates: - The video player makes a request to our Express server as soon as it is rendered in the React application. Our server handles this request, reading the video file and sending back the appropriate chunks as per the range requested by the browser. This results in the video being streamed in our player. - When the download button is clicked, a fetch request is sent to our server's download API. This time, the server reads the file, but instead of just piping the data to the response, it controls the data sending process. It sends chunks of data and also logs the sent chunks for monitoring purposes. The React application collects these chunks and concatenates them, displaying the download progress in real-time. When all chunks are received, it compiles them into a Blob and triggers a download in the browser. This setup allows us to build a full-featured video streaming and downloading application with fine control over the data transmission process. To see this system in action, you can check out this video demo. Conclusion While the focus of this article was on video streaming and downloading, the principles we discussed here extend beyond just media files. The pattern of responding to HTTP range requests is common in various data-heavy applications, and understanding it can be a useful tool in your web development arsenal. Finally, remember that the code shown in this article is just a simple example to demonstrate the concepts. In a real-world application, you would want to add proper error handling, validation, and possibly some form of access control depending on your use case. I hope this article helps you in your journey as a developer. Building something yourself is the best way to learn, so don't hesitate to get your hands dirty and start coding!...

Aug 9, 2023

6 mins

NodeJSExpress.jsReact

Introducing the express-typeorm-postgres Starter Kit

Here at This Dot, we've been working with ExpressJS APIs for a while, and we've created a starter.dev kit for ExpressJS that you can use to scaffold your next backend project. The starter kit uses many well-known npm packages, such as TypeORM or BullMQ and integrates with databases such as PostgreSQL and Redis. Kit contents The express-typeorm-postgres starter kit provides you with infrastructure for development, and integrations with these infrastructures. It comes with a working Redis instance for caching and a second Redis instance for queues. It also starts up a Postgres instance for you, which you can seed with TypeORM. The infrastructure runs on docker using docker-compose. The generated project comes with prettier and eslint set-up, so you only need to spend time on configuration if you want to tweak or change the existing rules. Unit testing is set up using Jest, and there are some example tests provided with the example controllers. How to initialise API development usually requires more infrastructure than front-end development. Before you start, please make sure you have docker and docker-compose installed on your machine. To initialize a project with the express-typeorm-postgres kit, run the following: 1. Run npx @this-dot/create-starter to run the scaffolding tool 2. Select the Express.js, TypeORM, and PostgreSQL kit from the CLI library options 3. Name your project 4. cd into your project directory, and install dependencies using the tool of your choice (npm, yarn or pnpm) 5. copy the contents of the .env.example file into a .env file With this setup, you have a working starter kit that you can modify to your needs. TypeORM and Database When we started developing the kit, we decided to use PostgreSQL as the database, because it is a powerful, open-source object-relational database system that is widely used for storing and manipulating data. It has a strong reputation for reliability, performance, and feature richness, making it a great choice for a wide range of applications. It can also handle high levels of concurrency and large amounts of data, and supports complex queries and data types. Postgres is also highly extensible because it allows developers to add custom functions and data types to the database. It has a large and active community of developers and users who contribute to its ongoing development and support. The kit uses TypeORM to connect to the database instance. We chose TypeORM because it makes it easy to manage database connections and perform common database operations, such as querying, inserting, updating and deleting data. It supports TypeScript and a wide range of databases, such as PostgreSQL, MySQL, SQLite and MongoDB, therefore if you want to be able to switch between databases, it makes it easier. TypeORM also includes features such as database migrations, which help manage changes to database schema over time, and an entity model that allows you to define your database schema using classes and decorators. Overall, TypeORM is a useful tool for improving the efficiency and reliability of database-related code, and it can be a valuable addition to any TypeScript or JavaScript project that needs to interact with a database. To seed an initial set of data into your database, run the following commands: 1. npm run infrastructure:start - this starts up the database instance 2. npm run db:seed - this leverages TypeORM to seed the database. The seed command runs the src/db/run-seeders.ts file, where you can introduce your seeders for your own needs. The kit uses TypeORM-extension for seeding. Please refer to the src/db/seeding/technology-seeder.ts file for an example. Caching Storing response data in caches allows subsequent requests for the same data to be served more quickly. This can improve the performance and user experience of an API by reducing the amount of time it takes to retrieve data from the server. It can reduce the load on the database or mitigate rate limiting on third-party APIs called from your back-end. It also improves the reliability of an application by providing a fallback mechanism in case the database or the server is unavailable or slow to respond. There is a Redis instance set up in the kit to be used for caching data. Under the hood, we use the cachified library to store cached data in Redis. The kit has a useCache method exported from src/cache/cache.ts, which requires a key and a callback function to be called to fetch the data. ` When you need to invalidate cache entries, you can use the clearCacheEntry method by supplying a key string to it. It will remove the cached data from Redis, and the next request that fetches from the database will cache the new values. ` Under the src/modules/technology folder, you can see a complete example of a basic CRUD REST endpoint with caching enabled. Feel free to use those handlers as examples for your development needs. Queue A message queue allows different parts of an application, or different applications, to communicate with each other asynchronously by sending and receiving messages. This can be useful in a variety of situations, such as when one part of the application needs to perform a task that could take a long time, or when different parts of the application need to be decoupled from each other for flexibility and scalability. We chose BullMQ because it is a fast, reliable, and feature-rich message queue system. It is built on top of the popular Redis in-memory data store, which makes it very performant and scalable. It has support for parallel processing, rate limiting, retries, and a variety of other features that make it well-suited for a wide range of use cases. BullMQ has a straightforward API and good documentation. The kit has a second Redis instance set up to be used with BullMQ, and there is a queue set up out of the box, so resource-intensive tasks can be offloaded to a background process. The src/queue/ folder contains all the configuration and setup steps for the queue. Both the queue and its worker is set up in the queue.ts file. The job-processor.ts file contains the function that will process the data. To run int in a separate thread, we must pass the path to this file into the worker: ` When to use this kit This kit is most optimal when you: - want to build back-end services that can be consumed by other applications and services using ExpressJS - need a flexible and scalable way to build server-side applications - need to deal with CPU-intense operations on the server and you need a messaging queue - need to build an API with relational data - would like to just jump right into API development with ExpressJS using TypeORM and Postgres Conclusion The express-typeorm-postgres starter kit can help you kickstart your development by providing you with a working preset. It has testing configured, and it comes with a complete infrastructure orchestrated by docker-compose....

Jan 13, 2023

5 mins

NodeJSExpress.jsTypeScriptPostgreSQL

How to automatically deploy your full-stack JavaScript app with AWS CodePipeline

How to automatically deploy your full-stack JavaScript app from an NX monorepo with AWS CodePipeline In our previous blog post (How to host a full-stack JavaScript app with AWS CloudFront and Elastic Beanstalk) we set up a horizontally scalable deployment for our full-stack javascript app. In this article, we would like to show you how to set up AWS CodePipeline to automatically deploy changes to the application. APP Structure Our application is a simple front-end with an API back-end set up in an NX monorepo. The production built API code is hosted in Elastic Beanstalk, while the front-end is stored in S3 and hosted through CloudFront. Whenever we are ready to make a new release, we want to be able to deploy the new API and front-end versions to the existing distribution. In this article, we will set up a CodePipeline to deploy changes to the main branch of our connected repository. CodePipeline CodeBuild and the buildspec file First and foremost, we should set up the build job that will run the deploy logic. For this, we are going to need to use CodeBuild. Let's go into our repository and set up a build-and-deploy.buildspec.yml file. We put this file under the tools/aws/ folder. ` This buildspec file does not do much so far, we are going to extend it. In the installation phase, it will run npm ci to install the dependencies and in the build phase, we are going to run the build command using the ENVIRONMENT_TARGET variable. This is useful, because if you have more environments, like development and staging you can have different configurations and builds for those and still use the same buildspec file. Let's go to the Codebuild page in our AWS console and create a build project. Add a descriptive name, such as your-appp-build-and-deploy. Please provide a meaningful description for your future self. For this example, we are going to restrict the number of concurrent builds to 1. The next step is to set up the source for this job, so we can keep the buildspec file in the repository and make sure this job uses the steps declared in the yaml file. We use an access token that allows us to connect to GitHub. Here you can read more on setting up a GitHub connection with an access token. You can also connect with Oauth, or use an entirely different Git provider. We set our provider to GitHub and provided the repository URL. We also set the Git clone depth to 1, because that makes checking out the repo faster. In the Environment section, we recommend using an AWS CodeBuild managed image. We use the Ubuntu Standard runtime with the aws/codebuild/standard:7.0 version. This version uses Node 18. We want to always use the latest image version for this runtime and as the Environment type we are good with Linux EC2. We don't need elevated privileges, because we won't build docker images, but we do want to create a new service role. In the Buildspec section select Use a buildspec file and give the path from your repository root as the Buildspec name. For our example, it is tools/aws/build-and-deploy.buildspec.yml. We leave the Batch configuration and the Artifacts sections as they are and in the Logs section we select how we want the logs to work. For this example, to reduce cost, we are going to use S3 logs and save the build logs in the aws-codebuild-build-logs bucket that we created for this purpose. We are finished, so let's create the build project. CodePipeline setup To set up automated deployment, we need to create a CodePipeline. Click on Create pipeline and give it a name. We also want a new service role to be created for this pipeline. Next, we should set up the source stage. As the source provider, we need to use GitHub (version2) and set up a connection. You can read about how to do it here. After the connection is set up, select your repository and the branch you want to deploy from. We also want to start the pipeline if the source code changes. For the sake of simplicity, we want to have the Output artefact format as CodePipeline default. At the Build stage, we select AWS CodeBuild as the build provider and let's select the build that we created above. Remember that we have the ENVIRONMENT_TARGET as a variable used in our build, so let's add it to this stage with the Plaintext value prod. This way the build will run the build:prod command from our package.json. As the Build type we want Single build. We can skip the deployment stage because we are going to set up deployment in our build job. Review our build pipeline and create it. After it is created, it will run for the first time. At this time it will not deploy anything but it should run successfully. Deployment prerequisites To be able to deploy to S3 and Elastic Beanstalk, we need our CodeBuild job to be able to interact with those services. When we created the build, we created a service role for it. In this example, the service role is codebuild-aws-test-build-and-deploy-service-role. Let's go to the IAM page in the console and open the Roles page. Search for our codebuild role and let's add permissions to it. Click the Add permissions button and select Attach policies. We need two AWS-managed policies to be added to this service role. The AdministratorAccess-AWSElasticBeanstalk will allow us to deploy the API and the AmazonS3FullAccess will allow us to deploy the front-end. The CloudFrontFullAccess will allow us to invalidate the caches so CloudFront will send the new front-end files after the deployment is ready. Deployment Upload the front-end to S3 Uploading the front-end should be pretty straightforward. We use an AWS CodeBuild managed image in our pipeline, therefore, we have access to the aws command. Let's update our buildspec file with the following changes: ` First, we upload the fresh front-end build to the S3 bucket, and then we invalidate the caches for the index.html file, so CloudFront will immediately serve the changes. If you have more static files in your app, you might need to invalidate caches for those as well. Before we push the above changes up, we need to update the environment variables in our CodePipeline. To do this open the pipeline and click on the Edit button. This will then enable us to edit the Build stage. Edit the build step by clicking on the edit button. On this screen, we add the new environment variables. For this example, it is aws-hosting-prod as Plaintext for the FRONT_END_BUCKET and E3FV1Q1P98H4EZ as Plaintext for the CLOUDFRONT_DISTRIBUTION_ID Now if we add changes to our index.html file, for example, change the button to HELLO 2, commit it and push it. It gets deployed. Deploying the API to Elastic Beanstalk We are going to need some environment variables passed down to the build pipeline to be able to deploy to different environments, like staging or prod. We gathered these below: - COMMIT_ID: #{SourceVariables.CommitId} - This will have the commit id from the checkout step. We include this, so we can always check what commit is deployed. - ELASTIC_BEANSTALK_APPLICATION_NAME: Test AWS App - This is the Elastic Beanstalk app which has your environment associated. - ELASTIC_BEANSTALK_ENVIRONMENT_NAME: TestAWSApp-prod - This is the Elastic Beanstalk environment you want to deploy to - API_VERSION_BUCKET: elasticbeanstalk-us-east-1-474671518642 - This is the S3 bucket that was created by Elastic Beanstalk With the above variables, we can make some new variables during the build time, so we can make sure that every API version is unique and gets deployed. We set this up in the install phase. ` The APP_VERSION variable is the version property from the package.json file. In a release process, the application's version is stored here. The API_VERSION variable will contain the APP_VERSION and as a suffix, we include the build number. We want to upload this API version by indicating the commit ID, so the API_ZIP_KEY will have this information. The APP_VERSION_DESCRIPTION will be the description of the deployed version in Elastic Beanstalk. Finally, we are going to update the buildspec file with the actual Elastic Beanstalk deployment steps. ` Let's make a change in the API, for example, the message sent back by the /api/hello endpoint and push up the changes. --- Now every time a change is merged to the main branch, it gets pushed to our production deployment. Using these guides, you can set up multiple environments, and you can configure separate CodePipeline instances to deploy from different branches. I hope this guide proved to be helpful to you....

Sep 15, 2023

9 mins

AWSNxNodeJSJavaScript

“Music and code have a lot in common,” freeCodeCamp’s Jessica Wilkins on what the tech community is doing right to onboard new software engineers

Before she was a software developer at freeCodeCamp, Jessica Wilkins was a classically trained clarinetist performing across the country. Her days were filled with rehearsals, concerts, and teaching, and she hadn’t considered a tech career until the world changed in 2020. > “When the pandemic hit, most of my gigs were canceled,” she says. “I suddenly had time on my hands and an idea for a site I wanted to build.” That site, a tribute to Black musicians in classical and jazz music, turned into much more than a personal project. It opened the door to a whole new career where her creative instincts and curiosity could thrive just as much as they had in music. Now at freeCodeCamp, Jessica maintains and develops the very JavaScript curriculum that has helped her and millions of developers around the world. We spoke with Jessica about her advice for JavaScript learners, why musicians make great developers, and how inclusive communities are helping more women thrive in tech. Jessica’s Top 3 JavaScript Skill Picks for 2025 If you ask Jessica what it takes to succeed as a JavaScript developer in 2025, she won’t point you straight to the newest library or trend. Instead, she lists three skills that sound simple, but take real time to build: > “Learning how to ask questions and research when you get stuck. Learning how to read error messages. And having a strong foundation in the fundamentals” She says those skills don’t come from shortcuts or shiny tools. They come from building. > “Start with small projects and keep building,” she says. “Books like You Don’t Know JS help you understand the theory, but experience comes from writing and shipping code. You learn a lot by doing.” And don’t forget the people around you. > “Meetups and conferences are amazing,” she adds. “You’ll pick up things faster, get feedback, and make friends who are learning alongside you.” Why So Many Musicians End Up in Tech A musical past like Jessica’s isn’t unheard of in the JavaScript industry. In fact, she’s noticed a surprising number of musicians making the leap into software. > “I think it’s because music and code have a lot in common,” she says. “They both require creativity, pattern recognition, problem-solving… and you can really get into flow when you’re deep in either one.” That crossover between artistry and logic feels like home to people who’ve lived in both worlds. What the Tech Community Is Getting Right Jessica has seen both the challenges and the wins when it comes to supporting women in tech. > “There’s still a lot of toxicity in some corners,” she says. “But the communities that are doing it right—like Women Who Code, Women in Tech, and Virtual Coffee—create safe, supportive spaces to grow and share experiences.” She believes those spaces aren’t just helpful, but they’re essential. > “Having a network makes a huge difference, especially early in your career.” What’s Next for Jessica Wilkins? With a catalog of published articles, open-source projects under her belt, and a growing audience of devs following her journey, Jessica is just getting started. She’s still writing. Still mentoring. Still building. And still proving that creativity doesn’t stop at the orchestra pit—it just finds a new stage. Follow Jessica Wilkins on X and Linkedin to keep up with her work in tech, her musical roots, and whatever she’s building next. Sticker illustration by Jacob Ashley....

Apr 25, 2025

3 mins

JavaScriptSoftware Engineering

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

BullMQ with ExpressJS

Why and when to use a queue?

Prerequisites

Setting up a queue

Put the processing into a thread

The problem with bundlers

Conclusion

Balázs Tápai

You might also like

Building a Multi-Response Streaming API with Node.js, Express, and React

Introducing the express-typeorm-postgres Starter Kit

How to automatically deploy your full-stack JavaScript app with AWS CodePipeline

“Music and code have a lot in common,” freeCodeCamp’s Jessica Wilkins on what the tech community is doing right to onboard new software engineers

Let's innovate together!

You might also like

Building a Multi-Response Streaming API with Node.js, Express, and React

Introducing the express-typeorm-postgres Starter Kit

How to automatically deploy your full-stack JavaScript app with AWS CodePipeline

“Music and code have a lot in common,” freeCodeCamp’s Jessica Wilkins on what the tech community is doing right to onboard new software engineers