Skip to content

Building a Multi-Response Streaming API with Node.js, Express, and React


As web applications become increasingly complex and data-driven, efficient and effective data transfer methods become critically important. A streaming API that can send multiple responses to a single request can be a powerful tool for handling large amounts of data or for delivering real-time updates. In this article, we will guide you through the process of creating such an API.

We will use video streaming as an illustrative example. With their large file sizes and the need for flexible, on-demand delivery, videos present a fitting scenario for showcasing the power of multi-response streaming APIs. The backend will be built with Node.js and Express, utilizing HTTP range requests to facilitate efficient data delivery in chunks.

Next, we'll build a React front-end to interact with our streaming API. This front-end will handle both the display of the streamed video content and its download, offering users real-time progress updates.

By the end of this walkthrough, you will have a working example of a multi-response streaming API, and you will be able to apply the principles learned to a wide array of use cases beyond video streaming.

Let's jump right into it!


Implementing the Streaming API in Express

In this section, we will dive into the server-side implementation, specifically our Node.js and Express application. We'll be implementing an API endpoint to deliver video content in a streaming fashion.

Assuming you have already set up your Express server with TypeScript, we first need to define our video-serving route. We'll create a GET endpoint that, when hit, will stream a video file back to the client.

Please make sure to install cors for handling cross-origin requests, dotenv for loading environment variables, and throttle for controlling the rate of data transfer. You can install these with the following command:

yarn add cors dotenv throttle @types/cors @types/dotenv @types/throttle
import cors from 'cors';
import 'dotenv/config';
import express, { Request, Response } from 'express';
import fs from 'fs';
import Throttle from 'throttle';

const app = express();
const port = 8000;


app.get('/video', (req: Request, res: Response) => {
  // Video by Zlatin Georgiev from Pexels:
  // For testing purposes - add the video in you `static` folder
  const path = 'src/static/pexels-zlatin-georgiev-15708449 (2160p).mp4';
  const stat = fs.statSync(path);
  const fileSize = stat.size;
  const range = req.headers.range;
  if (range) {
    const parts = range.replace(/bytes=/, '').split('-');
    const start = parseInt(parts[0], 10);
    const end = parts[1] ? parseInt(parts[1], 10) : fileSize - 1;

    const chunksize = end - start + 1;
    const file = fs.createReadStream(path, { start, end });
    const head = {
      'Content-Range': `bytes ${start}-${end}/${fileSize}`,
      'Accept-Ranges': 'bytes',
      'Content-Length': chunksize,
      'Content-Type': 'video/mp4',

    res.writeHead(206, head);
  } else {
    const head = {
      'Content-Length': fileSize,
      'Content-Type': 'video/mp4',

    res.writeHead(200, head);

app.listen(port, () => {
  console.log(`Server listening at ${process.env.SERVER_URL}:${port}`);

In the code snippet above, we are implementing a basic video streaming server that responds to HTTP range requests. Here's a brief overview of the key parts:

  1. File and Range Setup: We start by determining the path to the video file and getting the file size. We also grab the range header from the request, which contains the range of bytes the client is requesting.
  2. Range Requests Handling: If a range is provided, we extract the start and end bytes from the range header, then create a read stream for that specific range. This allows us to stream a portion of the file rather than the entire thing.
  3. Response Headers: We then set up our response headers. In the case of a range request, we send back a '206 Partial Content' status along with information about the byte range and total file size. For non-range requests, we simply send back the total file size and the file type.
  4. Data Streaming: Finally, we pipe the read stream directly to the response. This step is where the video data actually gets sent back to the client. The use of pipe() here automatically handles backpressure, ensuring that data isn't read faster than it can be sent to the client.

With this setup in place, our streaming server is capable of efficiently delivering large video files to the client in small chunks, providing a smoother user experience.

Implementing the Download API in Express

Now, let's add another endpoint to our Express application, which will provide more granular control over the data transfer process. We'll set up a GET endpoint for '/download', and within this endpoint, we'll handle streaming the video file to the client for download.

app.get('/download', (req: Request, res: Response) => {
  // Again, for testing purposes - add the video in you `static` folder
  const path = 'src/static/pexels-zlatin-georgiev-15708449 (2160p).mp4';
  const stat = fs.statSync(path);
  const fileSize = stat.size;

  res.writeHead(200, {
    'Content-Type': 'video/mp4',
    'Content-Disposition': 'attachment; filename=video.mp4',
    'Content-Length': fileSize,

  const readStream = fs.createReadStream(path);
  const throttle = new Throttle(1024 * 1024 * 5); // throttle to 5MB/sec - simulate lower speed


  throttle.on('data', (chunk) => {
    Console.log(`Sent ${chunk.length} bytes to client.`);

  throttle.on('end', () => {
    console.log('File fully sent to client.');

This endpoint has a similar setup to the video streaming endpoint, but it comes with a few key differences:

  1. Response Headers: Here, we include a 'Content-Disposition' header with an 'attachment' directive. This header tells the browser to present the file as a downloadable file named 'video.mp4'.
  2. Throttling: We use the 'throttle' package to limit the data transfer rate. Throttling can be useful for simulating lower-speed connections during testing, or for preventing your server from getting overwhelmed by data transfer operations.
  3. Data Writing: Instead of directly piping the read stream to the response, we attach 'data' and 'end' event listeners to the throttled stream. On the 'data' event, we manually write each chunk of data to the response, and on the 'end' event, we close the response.

This implementation provides a more hands-on way to control the data transfer process. It allows for the addition of custom logic to handle events like pausing and resuming the data transfer, adding custom transformations to the data stream, or handling errors during transfer.

Utilizing the APIs: A React Application

Now that we have a server-side setup for video streaming and downloading, let's put these APIs into action within a client-side React application. Note that we'll be using Tailwind CSS for quick, utility-based styling in our components.

Our React application will consist of a video player that uses the video streaming API, a download button to trigger the download API, and a progress bar to show the real-time download progress.

First, let's define the Video Player component that will play the streamed video:

import React from 'react';

const VideoPlayer: React.FC = () => {
  return (
    <div className="mb-3">
      <video controls width="860">
        <source src="http://localhost:8000/video" type="video/mp4" />
        Your browser does not support the video tag.

export default VideoPlayer;

In the above VideoPlayer component, we're using an HTML5 video tag to handle video playback. The src attribute of the source tag is set to the video endpoint of our Express server. When this component is rendered, it sends a request to our video API and starts streaming the video in response to the range requests that the browser automatically makes.

Next, let's create the DownloadButton component that will handle the video download and display the download progress:

import React, { useState } from 'react';

const DownloadButton: React.FC = () => {
  const [downloadProgress, setDownloadProgress] = useState(0);

  const handleDownload = async () => {
    try {
      const response = await fetch('http://localhost:8000/download');
      const reader = response.body?.getReader();

      if (!reader) {

      const contentLength = +(response.headers?.get('Content-Length') || 0);
      let receivedLength = 0;
      let chunks = [];

      while (true) {
        const { done, value } = await;

        if (done) {
          console.log('Download complete.');

          const blob = new Blob(chunks, { type: 'video/mp4' });
          const url = window.URL.createObjectURL(blob);
          const a = document.createElement('a');
 = 'none';
          a.href = url;
 = 'video.mp4';

        receivedLength += value.length;
        const progress = (receivedLength / contentLength) * 100;
    } catch (err) {

  return (
    <div className="flex gap-3 items-center">
        className="py-2 px-3 bg-indigo-500 text-white text-sm font-semibold rounded-md shadow focus:outline-none"
        Download Video
      {downloadProgress > 0 && downloadProgress < 100 && (
      <p className="flex gap-3 items-center">
        Download progress: <progress value={downloadProgress} max={100} />
      {downloadProgress === 100 && <p>Download complete!</p>}

export default DownloadButton;

In this DownloadButton component, when the download button is clicked, it sends a fetch request to our download API. It then uses a while loop to continually read chunks of data from the response as they arrive, updating the download progress until the download is complete. This is an example of more controlled handling of multi-response APIs where we are not just directly piping the data, but instead, processing it and manually sending it as a downloadable file.

Bringing It All Together

Let's now integrate these components into our main application component.

import React from 'react';
import VideoPlayer from './components/VideoPlayer';
import DownloadButton from './components/DownloadButton';

function App() {
  return (
    <div className="flex flex-col items-center justify-center min-h-screen bg-gray-100 py-2">
      <h1 className="text-3xl font-bold mb-5">My Video Player</h1>
      <VideoPlayer />
      <DownloadButton />

export default App;

In this simple App component, we've included our VideoPlayer and DownloadButton components. It places the video player and download button on the screen in a neat, centered layout thanks to Tailwind CSS.

Here is a summary of how our system operates:

  • The video player makes a request to our Express server as soon as it is rendered in the React application. Our server handles this request, reading the video file and sending back the appropriate chunks as per the range requested by the browser. This results in the video being streamed in our player.
  • When the download button is clicked, a fetch request is sent to our server's download API. This time, the server reads the file, but instead of just piping the data to the response, it controls the data sending process. It sends chunks of data and also logs the sent chunks for monitoring purposes. The React application collects these chunks and concatenates them, displaying the download progress in real-time. When all chunks are received, it compiles them into a Blob and triggers a download in the browser.

This setup allows us to build a full-featured video streaming and downloading application with fine control over the data transmission process. To see this system in action, you can check out this video demo.


While the focus of this article was on video streaming and downloading, the principles we discussed here extend beyond just media files. The pattern of responding to HTTP range requests is common in various data-heavy applications, and understanding it can be a useful tool in your web development arsenal.

Finally, remember that the code shown in this article is just a simple example to demonstrate the concepts. In a real-world application, you would want to add proper error handling, validation, and possibly some form of access control depending on your use case.

I hope this article helps you in your journey as a developer. Building something yourself is the best way to learn, so don't hesitate to get your hands dirty and start coding!

This Dot Labs is a development consultancy that is trusted by top industry companies, including Stripe, Xero, Wikimedia, Docusign, and Twilio. This Dot takes a hands-on approach by providing tailored development strategies to help you approach your most pressing challenges with clarity and confidence. Whether it's bridging the gap between business and technology or modernizing legacy systems, you’ll find a breadth of experience and knowledge you need. Check out how This Dot Labs can empower your tech journey.

You might also like

State Management with React: Client State and Beyond cover image

State Management with React: Client State and Beyond

Introduction State management has long been a hotly debated subject in the React world. Best practices have continued to evolve, and there’s still a lot of confusion around the subject. In this article, we are going to dive into what tools we might want to use for solving common problems we face in modern web application development. There are different types of state and different types of ways to store and manage your application state. For example, you might have some local client state in a component that controls a dropdown, and you might also have a global store that contains authenticated user state. Aside from those typical examples, on some pages of your website, you might store some state in the URL. For quite a while, we’ve been leaning into global client state in single-page app land. This is what state management tools like Redux and other similar libraries help us out with. In this post, we will cover these different strategies in more detail so that we can try to make the best decisions going forward regarding state management in our applications. Local and global client state In this section, we'll discuss the differences between local and global client state, when to use each, and some popular libraries that can help you manage them effectively. When to use local client state Local client state is best suited for managing state that is specific to a single component or a small group of related components. Examples of local state include form inputs, component visibility, and UI element states like button toggles. Using local state keeps the component self-contained, making it easier to understand and maintain. In general, it's a good idea to start with local state and only use global state when you have a clear need for it. This can help keep your code simple and easy to understand. When to use global client state Global client state is useful when you have state that needs to be accessed by multiple unrelated components, or when the state is complex and would benefit from a more carefully designed API. Common examples of global state include user authentication, theme preferences, and application-wide settings. By centralizing this state, you can easily share it across the entire application, making it more efficient and consistent. If you’re building out a feature that has some very complex state requirements, this could also be a good case for using a “global” state management library. Truth is, you can still use one of these libraries to manage state that is actually localized to your feature. Most of these libraries support creating multiple stores. For example, if I was building a video chat client like Google Meets with a lot of different state values that are constantly changing, it might be a good idea to create a store to manage the state for a video call. Most state management libraries support more features than what you get out of the box with React, which can help design a clean and easy to reason about modules and APIs for scenarios where the state is complex. Popular libraries for global state management There are a lot of great libraries for managing state in React out there these days. I think deciding which might be best for you or your project is mostly a matter of preference. Redux and MobX are a couple of options that have been around for quite a long time. Redux has gained a reputation for being overly complex and requiring a lot of boilerplate code. The experience is actually much improved these days thanks to the Redux Toolkit library. MobX provides an easy-to-use API that revolves around reactive data/variables. These are both mature and battle-tested options that are always worth considering. Meta also has a state management named Recoil that provides a pretty easy-to-use API that revolves around the concept of atoms and selectors. I don’t see this library being used a ton in the wild, but I think it’s worth mentioning. A couple of the more popular new players on the block are named jotai and zustand. I think after the Redux hangover, these libraries showed up as a refreshing oasis of simplicity. Both of these libraries have grown a ton in popularity due to their small byte footprints and simple, straightforward APIs. Context is not evil The React Context API, like Redux, has also been stigmatized over the years to the point where many developers have their pitchforks out, declaring that you should never use it. We leaned on it for state management a bit too much for a while, and now it is a forbidden fruit. I really dislike these hard all-or-nothing stances though. We just need to be a little bit more considerate about when and where we choose to use it. Typically, React Context is best for storing, and making available global state that doesn’t change much. Some of the most common use cases are things like themes, authentication, localization, and user preferences. Contrary to popular belief, only components (and their children) that use that context (const context = useContext(someContext);) are re-rendered in the event of a state change, not all of the children below the context provider. Storing state in the URL The most underused and underrated tool in the web app state management tool belt is using the URL to store state. Storing state in the URL can be beneficial for several reasons, such as enabling users to bookmark and share application state, improving SEO, and simplifying navigation. The classic example for this is filters on an e-commerce website. A good user experience would be that the user can select some filters to show only the products that they are looking for and then be able to share that URL with a friend and them see the same exact results. Before you add some state to a page, I think it’s always worth considering the question: “Should I be able to set this state from the URL?”. Tools for managing URL state We typically have a couple of different tools available to use for managing URL state. Built-in browser APIs like the URL class and URLSearchParams. Both of these APIs allow you to easily parse out parts of a URL. Most often, you will store URL state in the parameters. In most React applications, you will typically have a routing library available to help with URL and route state management as well. React Router has multiple hooks and other APIs for managing URL state like useLocation that returns a parsed object of the current URL state. Keeping URL and application state in sync The tricky part of storing state in the URL is when you need to keep local application state in sync with the URL values. Let’s look at an example component with a simple name component that stores a piece of state called name. ` import React, { useState, useEffect } from 'react'; import { useLocation, useHistory } from 'react-router-dom'; function MyComponent() { const location = useLocation(); const history = useHistory(); const [name, setName] = useState(''); useEffect(() => { // Update the name state when the URL changes const searchParams = new URLSearchParams(; setName(searchParams.get('name') || ''); }, [location]); function handleNameChange(event) { setName(; // Update the URL when the name changes history.push(/my-component?name=${}`); } return ( ); } ` The general idea is to pull the initial value off of the URL when the component mounts and set the state value. After that, in our event handler, we make sure to update the URL state as well as our local React state. Moving state to the server Moving web application state to the server can be beneficial in several scenarios. For example, when you have a complex state that is difficult to manage on the client-side. By moving the state to the server, you can simplify the client-side code and reduce the amount of data that needs to be transferred between the client and server. This can be useful for applications that have a lot of business logic or complex data structures. In most cases if there’s some logic or other work that you can move off of your client web application and onto the server that is a win. Conclusion State management is a crucial aspect of building modern web applications with React. By understanding the different types of state and the tools available for managing them, you can make informed decisions about the best approach for your specific use case. Remember to consider local and global client state, URL-based state, and server-side state when designing your application's state management strategy....

How to Integrate Mailchimp Forms in a React Project cover image

How to Integrate Mailchimp Forms in a React Project

Intro Today we will cover how to set up an email signup form using React and Mailchimp. This blog will be using the cra-rxjs-styled-components template to expedite the process. This article assumes you have a basic understanding of React, and have set up a Mailchimp account. Here is the code repo if you want to review it while reading, or just skip ahead. We will start with setting up our React project using for simplicity, and then finish it up by integrating the two for our signup form. To start, we will be using the command yarn create @this-dot/starter --kit cra-rxjs-styled-components, which can be found here. We’ll go ahead, and give the project a name. I will be calling mine react-mailchimp. Now we will navigate into the project and do a yarn install. Then we can run yarn run dev to get it up and running locally on localhost:3000. This should have us load up on the React App, RxJS, and styled-components Starter kit page. With that all set, we’ll also need to install jsonp by using yarn add jsonp`. We’ll be using jsonp instead of fetch to avoid any CORS issues we may run into. This also makes for an easy and quick process by not relying on their API, which can’t be utilized by the client. Now that we have our project set up, we will go ahead and go and grab our form action URL from MailChimp. This can be found by going to your Audience > Signup Forms > Embedded Forms > Continue and then grabbing the form action URL found in the Embedded Form Code. We need to make a small change to the URL and swap /post? with /post-json?. We can now start setting up our form input, and our submit function. I will add a simple form input and follow it up, and a submit function. Inside the submit function, we will use our imported jsonp to invoke our action URL. ` import { useState } from 'react'; import jsonp from 'jsonp'; export const MailChimp = () => { const [email, setEmail] = useState(''); const onSubmit = (e: any) => { e.preventDefault(); const url = 'insert-mailchimp-action-url-here'; jsonp(${url}&EMAIL=${email}`, { param: 'c' }, (_: any, data: any) => { console.log('data', data); const { msg } = data; alert(msg); }); }; return ( Email setEmail(} > Submit ); }; ` We’ll also add a quick alert to let the user know that it was successful and that’s it! We’ve now successfully added the email to our MailChimp account. Conclusion Today, we covered how to integrate Mailchimp with a react app using the cra-rxjs-styled-components template from I highly recommend using to get your project up and running quickly. Here is the code repo again for you to check out....

Follow These Best Practices for Using Git cover image

Follow These Best Practices for Using Git

As a developer, working in a version control system is a basic necessity as soon as you start any project. The 2021 Technology Survey by Stack Overflow shows Git as the most popular technology tool, reinforcing how important it is to understand, and use it correctly, in your projects. This article compiles some tips and hopefully good advice when working with Git. Never push to the main branch. > "The Git feature that really makes it stand apart from nearly every other SCM out there is its branching model." - from git-scm Pushing directly to the main branch doesn't make much sense in Git as it doesn't promote collaboration. Instead, make use of merges/rebases from pull requests** that most providers have (GitHub, AWS CodeCommit, etc.). Define patterns & standards within your team Every team has its patterns and standards for naming conventions, tags, and commit messages. Those should be followed and pointed out in all pull-request reviews. If you don't have any starting pattern, here are a couple of suggestions: - branch name**: `user_name/ticket_number,-short_description_dashed` - example: mimurawil/1,-my-first-commit` - commit title**: `type(scope): short_description` - type: feat, fix, revert, etc - scope: ticket number (can be prefixed by "#") - example: fix(#1): fix broken build` Write useful commit messages Try to focus more on "why" and "what" instead of "how". You are spending a bit more time to complete your commit, but it will pay back when your future self or teammates revisit that piece of code. ` fix(#42): use proper url prefix on social share images During build time, our pages don't know or don't have access to the location` props, making the prefix for our image URL's to be `undefined`. Fixing this by using the proper SITE_URL` from our env variables. ref: ticket #42 link ` Rebase your branch frequently Rebasing your branch frequently ensures you are always working with the latest version from the main branch. Your teammates will review the correct changes, and you will usually encounter no conflicts when merging to the main branch. ` -i for interactive rebase git rebase -i main ` (make sure you have your local main branch updated from remote)_ Make use of Git commands A few Git commands that I found incredibly useful in my journey: - git revert`: creates a new commit that reverts changes from a specific commit - git stash`: stores the changes and goes back to a clean working directory - git cherry-pick`: picks one specific commit and adds it on top of your current branch - git bisect`: help you find a specific commit in the history that added a bug in your codebase...

Testing a Fastify app with the NodeJS test runner cover image

Testing a Fastify app with the NodeJS test runner

Introduction Node.js has shipped a built-in test runner for a couple of major versions. Since its release I haven’t heard much about it so I decided to try it out on a simple Fastify API server application that I was working on. It turns out, it’s pretty good! It’s also really nice to start testing a node application without dealing with the hassle of installing some additional dependencies and managing more configurations. Since it’s got my stamp of approval, why not write a post about it? In this post, we will hit the highlights of the testing API and write some basic but real-life tests for an API server. This server will be built with Fastify, a plugin-centric API framework. They have some good documentation on testing that should make this pretty easy. We’ll also add a SQL driver for the plugin we will test. Setup Let's set up our simple API server by creating a new project, adding our dependencies, and creating some files. Ensure you’re running node v20 or greater (Test runner is a stable API as of the 20 major releases) Overview `index.js` - node entry that initializes our Fastify app and listens for incoming http requests on port 3001 `app.js` - this file exports a function that creates and returns our Fastify application instance `sql-plugin.js` - a Fastify plugin that sets up and connects to a SQL driver and makes it available on our app instance Application Code A simple first test For our first test we will just test our servers index route. If you recall from the app.js` code above, our index route returns a 501 response for “not implemented”. In this test, we're using the createApp` function to create a new instance of our Fastify app, and then using the `inject` method from the Fastify API to make a request to the `/` route. We import our test utilities directly from the node. Notice we can pass async functions to our test to use async/await. Node’s assert API has been around for a long time, this is what we are using to make our test assertions. To run this test, we can use the following command: By default the Node.js test runner uses the TAP reporter. You can configure it using other reporters or even create your own custom reporters for it to use. Testing our SQL plugin Next, let's take a look at how to test our Fastify Postgres plugin. This one is a bit more involved and gives us an opportunity to use more of the test runner features. In this example, we are using a feature called Subtests. This simply means when nested tests inside of a top-level test. In our top-level test call, we get a test parameter t` that we call methods on in our nested test structure. In this example, we use `t.beforeEach` to create a new Fastify app instance for each test, and call the `test` method to register our nested tests. Along with `beforeEach` the other methods you might expect are also available: `afterEach`, `before`, `after`. Since we don’t want to connect to our Postgres database in our tests, we are using the available Mocking API to mock out the client. This was the API that I was most excited to see included in the Node Test Runner. After the basics, you almost always need to mock some functions, methods, or libraries in your tests. After trying this feature, it works easily and as expected, I was confident that I could get pretty far testing with the new Node.js core API’s. Since my plugin only uses the end method of the Postgres driver, it’s the only method I provide a mock function for. Our second test confirms that it gets called when our Fastify server is shutting down. Additional features A lot of other features that are common in other popular testing frameworks are also available. Test styles and methods Along with our basic test` based tests we used for our Fastify plugins - `test` also includes `skip`, `todo`, and `only` methods. They are for what you would expect based on the names, skipping or only running certain tests, and work-in-progress tests. If you prefer, you also have the option of using the describe` → `it` test syntax. They both come with the same methods as `test` and I think it really comes down to a matter of personal preference. Test coverage This might be the deal breaker for some since this feature is still experimental. As popular as test coverage reporting is, I expect this API to be finalized and become stable in an upcoming version. Since this isn’t something that’s being shipped for the end user though, I say go for it. What’s the worst that could happen really? Other CLI flags —watch` - —test-name-pattern` - TypeScript support You can use a loader like you would for a regular node application to execute TypeScript files. Some popular examples are tsx` and `ts-node`. In practice, I found that this currently doesn’t work well since the test runner only looks for JS file types. After digging in I found that they added support to locate your test files via a glob string but it won’t be available until the next major version release. Conclusion The built-in test runner is a lot more comprehensive than I expected it to be. I was able to easily write some real-world tests for my application. If you don’t mind some of the features like coverage reporting being experimental, you can get pretty far without installing any additional dependencies. The biggest deal breaker on many projects at this point, in my opinion, is the lack of straightforward TypeScript support. This is the test command that I ended up with in my application: I’ll be honest, I stole this from a GitHub issue thread and I don’t know exactly how it works (but it does). If TypeScript is a requirement, maybe stick with Jest or Vitest for now 🙂...