ReactJS

Demystifying React Server Components

Feb 2, 2024

9 min read

React Server Components (RSCs) are the latest addition to the React ecosystem, and they've caused a bit of a disruption to how we think about React. Dan Abramov recently wrote an article titled "The Two Reacts" that explores the paradigms of client component and server component mental models and leaves us with the thought on how we can cohesively combine these concepts in a meaningful way. It took me a while to finally give RSCs the proper exploration to truly understand the "new" model and grasp where React is heading. First off, the new model isn't really new so much as it introduces a few new concepts for us to consider when architecting our applications. Once I understood the pattern, I found an appreciation for the model and what it's trying to help us accomplish. In this post, I hope I can show you the progression of the React architecture in applications as I've experienced them and how I think RSCs help us improve this model and our apps.

A "Brief" History of React Rendering and Data-Fetching Patterns

One of the biggest challenges in React since its early days is "how do we server render pages?" Server-side rendering (SSR) is one of the techniques we can use to ensure users see data on initial load and helps with our site's SEO. Without SSR, users would see blank screens or loading spinners, and then a bunch of content would appear shortly after. An excellent graphic on Remix's website demonstrates this behavior from the end user's perspective and it's a problem we generally try to avoid as developers.

This problem is so vast and difficult that we've been trying to solve it since 2015. Rick Hanlon from the React Core Team reminded us just how complicated this problem was recently.

If you think react is complicated now, go back to 2015 when you had to configure babel and webpack yourself, you had to do SSR and routing yourself, you had to decide between flow and typescript, you had to learn flux and immutable.js, and you had to choose 1 of 100 boilerplates
— Ricky (@rickhanlonii) January 28, 2024

But SSR has its issues too. Sometimes SSR is slow because we need to do a lot of data fetching for our page. Because of these large payloads, we'll defer their rendering using lazy loading patterns. All of a sudden we have spinners again!

How we've managed these components has changed too. Over the years, we've seen a variety of patterns emerge for managing these problems. We had server-side pre-fetching where we had to hydrate our frontends application state. Then we tried controller-view patterned components for our lazily loaded client-side components. With the evolution of React, we were able to simplify the controller-view patterns to leverage hooks. Now, we're in a new era of multi-server entry points on a page with RSCs.

The Benefits of React Server Components

RSCs give us this new paradigm that allows us to have multiple entries into our server on a single page. Leveraging features like Next.js' streaming mode and caching, we can limit what our pages block on for SSR and optimize their performance. To illustrate this, let's look at this small block of PHP for something we might have done in the 2000s:

index.php

<?php
include "PostDAO.php"
$name = $_SESSION['name'];
$posts = PostDAO::getAll()
?>
<html>
    <body>
        <h1>Welcome back, <?= name ?></h1>
        <ul>
            <?php
                for ($i = 0; $i <= len($posts); $i++) {
            ?>
                <li><?= $posts[$i]['title'] ?></li>
            <?php
                }
            ?>
        </ul>
    </body>
</html>

For simplicity, the <?php … ?> blocks indicate server boundaries where we can execute PHP functionality. Once we exit that block, we can no longer leverage or communicate with the server. In this example, we're displaying a welcome message to a user and a list of posts. If we look at Next.js' page router leveraging getServerSideProps, we would maybe write this same page as follows:

pages/posts.jsx

import { getSession } from 'utils/session';
import { getPosts } from 'lib/posts';

export async function getServerSideProps() {
    const { name } = getSession();
    const posts = await getPosts();
    return {
        props: {
            name,
            posts,
        }
    }
}

export default function Page({ name, todos }) {
    return (
        <>
            <h1>Welcome back, {name}</h1>
            <ul>
                {posts.map((post) => (
                    <li key={post.id}>{post.title}</li>
                ))}
            </ul>
        </>
    )
}

In this case, we're having our server do a lot to fetch the data we need to render this page and all its components. This also makes the line between server and client much clearer as getServerSideProps runs before our client renders, and we're unable to go back to that function without an API route and client-side fetch.

Now, let's look at Next.js' app router with server components. This same component could be rendered as follows:

app/posts/page.tsx

import { getSession } from 'utils/session';
import { getPosts } from 'lib/todos';

export default async function Page() {
    const { name } = getSession();
    const posts = await getPosts();

    return (
        <>
            <h1>Welcome back, {name}</h1>
            <ul>
                {posts.map((post) => (
                    <li key={post.id}>{post.title}</li>
                ))}
            </ul>
        </>
    )
}

This moves us a bit closer back to our PHP version as we don't have to split our server and client functionality as explicitly. This example is relatively simple and only renders a few components that could be easily rendered as static content. This page also probably requires all the content to be rendered upfront. But, from this, we can see that we're able to simplify how server data fetching is done. If nothing else, we could use this pattern to make our SSR patterns better and client render the rest of our apps, but then we'd lose out on some of the additional benefits that we can get when we combine this with streaming. Let's look at a post page that probably has the content and a comments section. We don't need to render the comments immediately on page load or block on it because it's a secondary feature on the page. This is where we can pull in Suspense and make our page shine. Our code might look as follows:

app/posts/[slug]/page.jsx

import { Suspense } from "react";
import { notFound } from "next/navigation";
import ReactMarkdown from "react-markdown";
import { getPostBySlug, getPostCommentsBySlug } from '@/utils/posts';

export default async function Page({ params }) {
  const { slug } = params;
  const post = await getPostBySlug(slug);

  if (!post) {
    notFound();
  }

  return (
    <article>
      <h2>{post.title}</h2>
      <p>{post.author}</p>
      <ReactMarkdown>
        {post.content}
      </ReactMarkdown>
      <Suspense>
        <PostComments slug={slug} />
      </Suspense>
    </article>
  )
}

async function PostComments({ slug }) {
  const comments = await getPostCommentsBySlug(slug);
  return (
    <div>
      <h3>Comments</h3>
      {comments.map((comment) => (
        <div key={comment.id}>
          <p>{comment.content}</p>
          <p>{comment.author}</p>
        </div>
      ))}
    </div>
  )
}

Our PostComments are a server component that renders static content again, but we don't want its render to block our page from being served to the client. What we've done here is moved the fetch operation into the comments component and wrapped it in a Suspense boundary in our page to defer its render until its content has been fetched and then stream it onto the page into its render location. We could also add a fallback to our suspense boundary if we wanted to put a skeleton loader or other indication UI. With these changes, we're writing components similarly to how we've written them on the client historically but simplified our data fetch. What happens when we start to progressively enhance our features with client-side JavaScript?

Client Components with RSCs

All our examples focused on staying in the server context which is great for simple applications, but what happens when we want to add interactivity? If you were to put a useState, useEffect, onClick, or other client-side React feature into a server component, you'd find some error messages because you can't run client code from a server context. If we think back to our PHP example, that makes a lot of sense why this is the case, but how do we work around this? For me, this is where my first really mental challenge with RSCs started. Let's use our PostComments as an example to enhance by in-place sorting the section from the server.

// `app/posts/[slug]/PostComments.server.jsx`
import { getPostCommentsBySlug } from "@/utils/posts";

export async function PostCommentsServer({ slug }) {
  const comments = await getPostCommentsBySlug(slug);
  return <PostCommentsClient slug={slug} initialComments={comments} />;
}

// `app/posts/[slug]/PostComments.client.jsx`
'use client';
import { useState } from "react";

export function PostCommentsClient({ slug, initialComments }) {
  const [comments, setComments] = useState(initialComments);

  function getComments(slug, { sortOrder }) {
    fetch(`/api/posts/${slug}/comments?sort=${sortOrder}`)
      .then((resp) => resp.json())
      .then((data) => setComments(data));
  }

  return (
    <div>
      <div>
        <h3>Comments</h3>
        <ul>
          <li>
            <button onClick={() => getComments(slug, { sortOrder: "newest" })}>
              Newest
            </button>
          </li>
          <li>
            <button onClick={() => getComments(slug, { sortOrder: "oldest" })}>
              Oldest
            </button>
          </li>
          <li>
            <button
              onClick={() => getComments(slug, { sortOrder: "most-liked" })}
            >
              Most Liked
            </button>
          </li>
        </ul>
      </div>
      {comments.map((comment) => (
        <div key={comment.id}>
          <p>{comment.content}</p>
          <p>{comment.author}</p>
        </div>
      ))}
    </div>
  );
}

In this example, we're using a server component the same way as our previous example to do the initial data fetch and stream when ready. However, we're immediately passing the results to a client component that renders the elements and enhances our code with a list of sort options. When we select the sort we want, our code makes a request to the server at a predefined route and gets the new data that we re-render in the new order on screen. This is how we might have done things without RSCs before but without a useEffect for our initial rendering. If you're familiar with the old controller-view pattern, this is relatively similar to that pattern, but we have to relegate client re-fetching to the view (client) component, where we might have had all fetch and re-fetch patterns in the controller (server) component

Another way to solve this same problem is leveraging server actions, but that would cause the page to re-render. Given the caching mechanisms in Next.js, this is probably fine, but it's not the user experience people are expecting. Server actions are a topic of their own, so I won't cover them in this post, but they're important for the holistic ecosystem experience in the new mindset.

Interleaving Nested Server & Client Components

These examples have shown a clean approach where we keep our server components at the top of our rendering tree and have a clear line where we move from server mode to client mode. But one of the advantages of RSCs is that we can interleave where our server component entry points may exist. Let's think about a product carousel on a storefront page for example.

We may have built this as follows before:

export default async function Page() {
  return (
    <>
      <HomepageHero />
      <ProductCarousel title="Featured Products" collectionSlug="featured" />
      <ProductCarousel title="Best Sellers" collectionSlug="best-sellers" />
      <ProductCarousel title="Recommended Products" collectionSlug="recommended" />
    <>
  )
}

// app/ProductCarousel.jsx
'use client';

export default function ProductCarousel({ title, collectionSlug }) {
  // logic for carousel goes here
  return (
    <section>
      <h2>{title}</h2>
      <div>
        <ProductCards collectionSlug={collectionSlug} />
      </div>
    </section>
  )
}

// app/ProductCards.jsx
export default async function ProductCards({ collectionSlug }) {
  const products = await getProductsFromCollection(collectionSlug);
  return (
    <>
      {products.map((product) => (
        <div key={product.id}>
          <img src={product.img} />
          <h3>{product.title}</h3>
          <p>{product.description}</p>
        </div>
      ))}
    </>
  )
}

Here we're rendering carousels that render their cards and our tree goes server -> client -> server. This is not allowed in the new paradigm because the React compiler cannot detect that we moved back into a server component when we called ProductCards from a client component. Instead, we would need to refactor this to be:

// app/page.jsx
export default async function Page() {
  return (
    <>
      <HomepageHero />
      <ProductCarousel title="Featured Products">
        <ProductCards collectionSlug="featured" />
      </ProductCarousel>
      <ProductCarousel title="Best Sellers" >
        <ProductCards collectionSlug="best-sellers" />
      </ProductCarousel>
      <ProductCarousel title="Recommended Products" >
        <ProductCards collectionSlug="recommended" />
      </ProductCarousel>
    </>
  )
}

// app/ProductCarousel.jsx
'use client';

export default function ProductCarousel({ title, children }) {
  // logic for carousel goes here
  return (
    <section>
      <h2>{title}</h2>
      <div>
        {children}
      </div>
    </section>
  )
}

// app/ProductCards.jsx
export default async function ProductCards({ collectionSlug }) {
  const products = await getProductsFromCollection(collectionSlug);
  return (
    <>
      {products.map((product) => (
        <div key={product.id}>
          <img src={product.img} />
          <h3>{product.title}</h3>
          <p>{product.description}</p>
        </div>
      ))}
    </>
  )
}

Here, we've changed ProductCarousel to accept children that are our ProductCards server component. This allows the compiler to detect the boundary and render it appropriately. I'd also recommend adding Suspense boundaries to these carousels but I omitted it for the sake of brevity in this example.

Some Suggested Best Practices

Our team has been using these new patterns for some time and have developed some patterns we've identified as best practices for us to help keep some of these boundaries clear that I thought worth sharing.

1. Be explicit with component types

We've found that being explicit with component types is essential to expressing intent. Some components are strictly server, some are strictly client, and others can be used in both contexts. Our team likes using the server-only package to express this intent but we found we needed more. We've opted for the following naming conventions: Server only components: component-name.server.(jsx|tsx) Client only components: component-name.client.(jsx|tsx) Universal components: component-name.(jsx|tsx)

This does not apply to special framework file names but we've found this helps us delineate where we are and to help with interleaving.

2. Defer Fetch when Cache is Available

If we're trying to render a component that is a collection of other components, we've found deferring the fetch to the child allows our cache to do more for us in rendering in cases where the same element may exist in different locations. This is similar to how GraphQL's data loaders works and can ideally boost the performance of cached server components. So, in our product example above, we may only fetch the IDs of the products we need for the ProductCards and then fetch all the data per card. Yes, this is an extra fetch and causes an N+1, but if our cache is in play, end users will feel a performance gain.

3. Colocate loaders and queries

If you're using GraphQL or need loading states for components for Suspense fallback, we recommend colocating those elements in the same file or directory. This makes it easier to modify and manage related elements for your components in a more meaningful way.

4. Use Suspense to defer non-essential content

This will depend on your website and needs. Still, we recommend deferring as many non-essential elements to Suspense as possible. We define non-essential to be anything you could consider secondary to the page. On a blog post, this could be recommended posts or comments. On a product page, this could be reviews and related products. So long as the primary focus of your page is outside a Suspense boundary, your usage is probably acceptable.

Conclusion

RSCs represent a significant evolution in the React ecosystem, offering new paradigms and opportunities for improving the architecture of web applications. They enable multiple server entry points on a single page, optimizing server-side rendering, and simplifying data fetching. RSCs allow for a clearer separation between server and client functionality, enhancing both performance and maintainability.

To make the most of RSCs, developers should consider best practices such as being explicit with component types, deferring fetch when caching is available, colocating loaders and queries, and using Suspense to defer non-essential content. Embracing these practices can help harness the potential of React Server Components and pave the way for more efficient and interactive web applications.

We're excited about this development in the React ecosystem and the developments happening across teams and frameworks that are opting into using them. While Next.js offers a great solution, we're excited to see similar enhancements to Remix and RedwoodJS.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Dustin Goodman

Engineering Manager with a passion for web and application development. Speaker and writer on work experiences and software development. Dog dad and musician fueled by coffee.

@dustinsgoodman @dustinsgoodman

How to test React custom hooks and components with Vitest

Introduction In this guide, we'll navigate through the process of testing React hooks and components using Vitest—a powerful JavaScript unit testing framework. Discover how Vitest simplifies testing setups, providing an optimal solution for both Vite-powered projects and beyond. Vitest is a javascript unit testing framework that aims to position itself as the Test Runner of choice for Vite projects and as a solid alternative even for projects not using Vite. Vitest was built primarily for Vite-powered projects, to help reduce the complexity of setting up testing with other testing frameworks like Jest. Vitest uses the same configuration of your App (through vite.config.js), sharing a common transformation pipeline during dev, build, and test time. Prerequisites This article assumes a solid understanding of React and frontend unit testing. Familiarity with tools like React Testing Library and JSDOM will enhance your grasp of the testing process with Vitest. Installation and configuration Let’s see how we can use Vitest for testing React custom hooks and components. But first, we will need to create a new project with Vite! If you already have an existing project, you can skip this step. ` Follow the prompts to create a new React project successfully. For testing, we need the following dependencies installed: Vitest as the unit testing framework JSDOM as the DOM environment for running our tests React Testing Library as the React testing utilities. To do so, we run the following command: ` Once we have those packages installed, we need to configure the vite.config.js file to run tests. By default, some of the extra configs we need to set up Vitest are not available in the Vite config types, so we will need the vite.config.ts file to reference Vitest types by adding /// reference types=”vitest” /> at the top of the file. Add the following code to the vite.config.ts ` We set globals to true because, by default, Vitest does not provide global APIs for explicitness. So with this set to true, we can use keywords like describe, test and it without needing to import them. To get TypeScript working with the global APIs, add vitest/globals to the types field in your tsconfig.json. ` The environment property tells Vitest which environment to run the test. We are using jsdom as the environment. The root property tells Vitest the root folder from where it should start looking for test files. We should add a script for running the test in package.json ` With all that configured, we can now start writing unit tests for customs hooks and React components. Writing test for custom hooks Let’s write a test for a simple useCounter hook that takes an initial value and returns the value, an increment function and a decrement function. ` We can write a test to check the default return values of the hook for value as below: ` To test if the hook works when we increment the value, we can use the act() method from @testing-library/react to simulate the increment function, as shown in the below test case: ` Kindly Note that you can't destructure the reactive properties of the result.current instance, or they will lose their reactivity. Testing hooks with asynchronous logic Now let’s test a more complex logic that contains asynchronous logic. Let’s write a useProducts hook that fetches data from an external api and return that value ` Now, let’s see what the test looks like: ` In the above example, we had to spy on the global fetch API, so that we can mock its return value. We wrapped that inside a beforeAll so that this runs before any test in this file. Then we added an afterAll method and called the mockRestore() to run after all test cases have been completed and return all mock implementations to their original function. We can also use the mockClear() method to clear all the mock's information, such as the number of calls and the mock's results. This method is handy when mocking the same function with different return values for different tests. We usually use mockClear() in beforeEach() or afterEach() methods to ensure our test is isolated completely. Then in our test case, we used a waitFor(), to wait for the return value to be resolved. Writing test for components Like Jest, Vitest provides assertion methods (matchers) to use with the expect methods for asserting values, but to test DOM elements easily, we will need to make use of custom matchers such as toBeInTheDocument() or toHaveTextContent(). Luckily the Vitest API is mostly compatible with the Jest API, making it possible to reuse many tools originally built for Jest. For such methods, we can install the @testing-library/jest-dom package and extend the expect method from Vitest to include the assertion methods in matchers from this package. ` After installing the jest-dom testing library package, create a file named vitest-setup.ts on the root of the project and import the following into the project to extend js-dom custom matchers: ` Since we are using typescript, we also need to include our setup file in our tsconfig.json: ` In vite.config.ts, we need to add the vitest-setup.ts file to the test.setupFiles field: ` Now let’s test the Products.tsx component: ` We start by spying and mocking the useProducts hook with vi.spyOn() method from Vitest: ` Now, we render the Products component using the render method from @testing-library/react and assert that the component renders the list of products as expected and also the product has the title as follows: ` In the above code, we use the render method from @testing-library/react to render the component and this returns some useful methods we can use to extract information from the component like getByTestId and getByText. The getByTestId method will retrieve the element whose data-testid attribute value equals product-list, and we can then assert its children to equal the length of our mocked items array. Using data-testid attribute values is a good practice for identifying a DOM element for testing purposes and avoiding affecting the component's implementation in production and tests. We also used the getByText method to find a text in the rendered component. We were able to call the toBeInTheDocument() because we extended the matchers to work with Vitest earlier. Here is what the full test looks like: ` Conclusion In this article, we delved into the world of testing React hooks and components using Vitest, a versatile JavaScript unit testing framework. We walked through the installation and configuration process, ensuring compatibility with React, JSDOM, and React Testing Library. The comprehensive guide covered writing tests for custom hooks, including handling asynchronous logic, and testing React components, leveraging custom matchers for DOM assertions. By adopting Vitest, developers can streamline the testing process for their React applications, whether powered by Vite or not. The framework's seamless integration with Vite projects simplifies the setup, reducing the complexities associated with other testing tools like Jest....

Mar 15, 2024

5 mins

ViteReactTesting

CSS Hooks: A new way to style your React apps

In the world of web development, the management of styles has always been a crucial aspect of building modern and responsive user interfaces. With the rise of CSS in JS libraries like Material UI and Chakra, developers have started creating dynamic and reusable styles using JavaScript; however, the performance implications of these libraries have led to the exploration of alternative solutions. One such solution is CSS Hooks, a library that offers a different approach to managing styles in web applications. The Problem with CSS in React and CSS in JS Libraries In React applications, you typically write styles directly with the style prop by applying classes with className, or if you’re using a framework like Material UI or Chakra then you may use the sx attribute, which is more powerful than the baseline attributes. There are some performance concerns in the case of the sx attribute specifically. The framework has to dynamically generate classes with your styles and apply them to the document whenever it is used. This can become an issue with large and complex applications as your styles will be regularly regenerated on the fly as the user navigates through the application. This has led developers to seek alternative solutions that offer similar flexibility but with better performance. Introduction to CSS Hooks CSS Hooks improve upon the aforementioned issues by pre-generating your CSS using configuration hooks. The styles that these hooks generate remain static during the lifetime of the application and are reused wherever possible. CSS variables are utilized under the hood whenever dynamic behavior is needed. By dynamic behavior, I’m referring to advanced CSS features such as pseudo-class selectors and media queries, which are both traditionally unavailable for use with inline styles. CSS Hooks utilize an interesting trick under the hood to accomplish this, which we will now explore. The Power of CSS Variables and the Fallback Trick As mentioned before, CSS variables are utilized in order to support using advanced dynamic behavior such as pseudo-selectors in our inline styles. CSS variables are core to something known as the “fallback trick”. The gist of how the fallback trick works is to have the pseudo-selector or media query toggle the state of a variable between initial and an empty invalid value, and then we put the value that we actually want to toggle as the “fallback” value in the reference to the CSS variable inside of the inline style itself. Changing this fallback value inside of an inline style essentially allows us to control the values used by the pseudo-selector without having to regenerate linked stylesheets as we only have to change the inline style attribute. This is best explained with an example: ` This plain HTML and CSS code effectively allows us to utilize the focus pseudo-selector on any element that we want using inline styles only. This is what CSS Hooks effectively does in the background when you use dynamic styles. No nasty re-renders and mucking with stylesheets needed! How to use CSS Hooks Now that we know how CSS Hooks fundamentally work, let’s try using them. The library itself is much easier to use than the aforementioned example. Installation and usage of CSS Hooks is easy. Firstly, we must install the @css-hooks/react package into a React project with the package manager of your choice (npm, pnpm yarn, etc). Once that’s done, all that’s left is a little bit of configuration. You have to first use a utility function called createHooks that is exported from the package we just installed. The result of that function returns a couple of variables in an array that can be deconstructed, the first being a stylesheet in the form of a string, and the second being a function. ` I recommend putting this in a separate file that can be imported from multiple other places in the project as this is where we will be configuring hooks across the project. We’re exporting both hooks and css, and both are important. We need to include hooks into the DOM as it contains the styles of the hooks that we’ll be referencing in our components. In your root component you need to add the following tag so that the generated CSS can be used. ` How this is done may vary based on the libraries that you are using. After that’s done, you can integrate CSS Hooks into your components while writing inline styles mostly just like you would do normally, with the only change being the addition of an additional call to the css function that we exported earlier: ` Now that the core boilerplate is set up, your components can now utilize CSS Hooks in theory; however we need to actually add some hooks to start with before we can use them, or else we’ll just be limited to basic styles in the same way that React inline styles are. Make your own hooks We can define our hooks inside of the css-hooks file that we created earlier, the same one that exports the css helper function that gets CSS Hooks working with our components. Hooks are defined in the options parameters that are currently empty. Defining a simple hook that allows us to define inline hover styles for a component is very easy and can be done as follows: ` Although both the key and the value in this example are the same, they both serve different purposes. The key is the name of the hook that we reference when we want to use it, while the value is the spec. The spec is the actual CSS selector that we’re writing. The name can be anything that you want it to be so long as it doesn’t clash with any existing style properties or hooks. Recommended hooks Although making your own hooks may be necessary at times, there is another repository provided by the CSS Hooks authors that adds a way to generate many useful hooks called @css-hooks/recommended. I highly recommend using this library to generate your hooks if at all possible, and the usage is pretty simple. Here’s an example showing how you can use it to generate media queries, color scheme specific styles and pseudo-selectors: ` Then with that done, these can be utilized as follows: ` We can also modify the hooks configuration to add our own hooks in addition to the recommended hooks by using the spread operator: ` Then that custom hook can be used by simply using its key name: ` Summary CSS Hooks offers a performant alternative to traditional CSS in JS libraries by leveraging CSS variables and providing a unique approach to managing styles in web applications. By pre-generating the stylesheet based on configured hooks and utilizing CSS Variables for dynamic behavior, developers can create reusable styles without the performance overhead associated with dynamic stylesheet generation. If you're interested in exploring CSS Hooks further, be sure to check out @CSSHooks for more information. You can also find some of the examples demonstrated here in a working app in our demos repository. Thank you for reading!...

Feb 28, 2024

5 mins

CSSReact

Configure your project with Drizzle for Local & Deployed Databases

Updated April 8, 2024: Thanks to Jassi Bacha for pointing out that the migrate function is not exported from the drizzle-orm/postgres-js/migrator module for Vercel. I've updated the article to reflect the correct import path. It was a fun Friday, and Jason Lengstorf and I both decided to try and use Drizzle on our respective projects. Jason went the SQLite route and wrote an amazing article about how he got his setup working. My approach was a bit different. I started with Vercel's Postgres + Drizzle Next.js Starter and wanted to use PostgreSQL. If you don't know what Drizzle is, it's a type-safe ORM similar to Prisma. My colleague, Dane Grant, wrote a great intro post on it, so go check his article out if you want to learn more about Drizzle. Getting my project off the ground took longer than I expected, especially coming from a starter kit, but I figured it out. This is the article I wish I had at the time to help get this project set up with less friction. I will focus on using local and Vercel PostgreSQL, but this same setup should work with other databases and adapters. I'll make some notes about where those places are. While I did use Next.js here, these setup instructions work on other projects, too. Configuring Drizzle Every project that leverages Drizzle requires a drizzle.config in the root. Because I'm leveraging TypeScript, I named mine drizzle.config.ts, and to secure secrets, I also installed dotenv. My final file appeared as follows: ` The schema field is used to identify where your project's database schema is defined. Mine is in a file called schema.ts but you can split your schema into multiple files and use glob patterns to detect all of them. The out field determines where your migration outputs will be stored. I recommend putting them in a folder in the same directory as your schema to keep all your database-related information together. Additionally, the config requires a driver and dbCredentials.connectionString to be specified so Drizzle knows what APIs to leverage and where your database lives. For the connectionString, I'm using dotenv to store the value in a secret and protect it. The connectionString should be in a valid connection format for your database. For PostgreSQL, this format is postgresql://:@:/. Getting your connection string Now, you may be wondering how to get that connection string. If you're hosting on Vercel using their Postgres offering, you need to go to your Vercel dashboard and select their Postgres option and attach it to your app. This will set environment variables for you that you can pull into your local development environment. This is all covered in their "Getting Started with Vercel Postgres" guide. If you use a different database hosting solution, they'll offer similar instructions for fetching your connection string. However, I wanted a local database to modify and blow away as needed for this project. Out of the box, Drizzle does not offer a database initialization command and I needed something that could be easily and quickly replicated across different machines. For this, I pulled in Docker Compose and set up my docker-compose.yaml as follows: ` The 3 most important values to note here are the values in the environment key and the ports. These are what allowed me to determine my connection key. For this example, it would be: postgresql://postgres:postgres@localhost:5432/my-local-db. With the compose file set, I ran docker-compose up -d to get the container running, which also initializes the database. Now, we can connect and operate on the database as needed. Creating the database connection To make operations in our app, we need to get a database connection instance. I put mine in db/drizzle.ts to keep all my related database files together. My file looks like: ` This is a bit more complicated because we're using 2 different Drizzle adapters depending on our environment. For local development, we're using the generic PostgreSQL adapter, but for production, we're using the Vercel adapter. While these have different initializers, they have the same output interface which is why this works. The same wouldn't be true if you used MySQL locally and PostgreSQL in production. If we chose a RDS or similar PostgreSQL solution, we could use the same postgres adapter in both cases but change the connection string. That's all this file does at the end - detects which environment and uses the chosen adapter. If we go to use this exported instance, it won't be able to find our tables or provide type safety. This is because we haven't created our database tables yet. Creating database tables To get our database tables created, we're going to leverage Drizzle's Migrations. This allows us to create atomic changes to our database as our schema evolves. To accomplish this, we define the schema changes in our schema files as specified in our config. Then we can run npm run drizzle-kit generate:pg (or whatever script runner you use) to generate the migration SQL file that will be located where we specified in our config. You want to check this file into source! By default, Drizzle doesn't allow you to override migration names _yet_ (they're working on it!) so if you want to make your migration file more descriptive, you need to take both of these steps: 1. Rename the migration file. Take note of the old name. 2. Locate _journal.json. It should be in your migration folder in a folder called meta. From here, find the old file name and replace it with the new file name. Now, we need to run the migrations. I had some issues with top-level awaits and tsx like the Drizzle docs recommend, so I had to go a slightly different route and I'm not thrilled about it still. I made a file called migrate.mts that I stored next to my drizzle.ts. In theory, I should have been able to import my drizzle connection instance here and use that, but I ran out of time to figure it out and ended up repeating myself across files. Here's the file: ` Here, I'm connecting to the correct database pending environment then running the drizzle migrate command. For local development, I set my connection pool to max at 1. This probably isn't necessary for this use case, but when connecting to a cluster, this is a recommended best practice from the Drizzle team. For the local case, I also had to close the connection to the db when I was done. For both cases, though, I had to specify the migrations folder location. I could probably DRY this up a bit, but hopefully, the Drizzle team will eliminate this need and use the config to set this value in the future. With the above file set and our schema generated, we can now run npm run tsx db/migrate.mts and our database will have our latest schema. We can now use the db client to fetch and store data in our database. Note: Jason uses the push command here. This is fine for an initial database creation, but it will override tables in the future. The migration path is the recommended pattern for non-destructive database updates. Conclusion Congratulations! We can connect to our database and perform CRUD operations against our tables. We can also use Drizzle Studio to modify and inspect our data. To review, we had to: 1. Setup a local PostgreSQL server via a tool like Docker Compose 2. Configure the database adapter to work in local mode 3. Generate a schema 4. Create a script to execute migrations so our database is aligned with our schema This was my first experience with Drizzle, and I enjoyed its SQL-like interfaces which made it easy to quickly prototype my project. I noticed in their Discord that they're about to have a full-time maintainer so I'm excited to see what their future looks like. I hope you enjoy it too!...

Mar 8, 2024

6 mins

VercelJavaScript

Intro to EdgeDB - The 10x ORM

Intro to EdgeDB - The 10x ORM I’ve written a couple of posts recently covering different TypeScript ORMs. One about Prisma, and another about Drizzle. ORM’s are a controversial topic in their own right - some people think they are evil, and others think they are great. I enjoy them quite a bit. They make it easy to interact with your databases. What is more important and magical for an application than data? SQL without an ORM is amazing as well, but there are some pain points with that approach. Today I’m excited to write about EdgeDB, which isn’t _exactly_ an ORM or a database from my perspective (although they call themselves one). It is, however an incredibly impressive piece of technology that solves these common pain points in a pretty novel way. So if it’s not an ORM or a database, what exactly is it? I don’t think I can answer that in one or two sentences, so we will explore the various pieces that make up EdgeDB in this article. From a high-level standpoint, though, an interface/query language that sits in front of PostgreSQL. This may seem like some less important implementation detail, but in my eyes, it’s a feature and one of the most compelling selling points. Data Modeling EdgeDB advertises itself as a “graph-relational database”. The core components of EdgeDB data modeling are the schema, type system, and relationship definitions. A schema will consist of objects that contain various typed attributes and links that connect the objects. In SQL, a table is analogous to an Object, and a foreign-key is associated with a link. Here’s what a simple schema in EdgeDB looks like ` There’s a few things to highlight here * We defined two different objects (tables) User and Post * Each object contains properties with their types defined * str is one of several scalar types (bool, int, float, json, datetime, etc) * the author property is a required link to the User object Defining relations / associations In our example above we defined a one-to-many relationship between a user and posts. All the relation types that you can define in traditional SQL are available. One interesting feature though is called backward links. These can be defined in your schema and it allows you to access related data from both sides of a relationship. ` likes are a many-to-many relationship between Tweet and User. With a backlink defined multi link likers := Tweet.<likes[is User]; - we can access likes from a User and likers from a Tweet. ` That's how we can access these relations in our queries. You might be looking at these queries and thinking it looks a lot like GraphQL. This is why they call it a ‘Graph-relational’ database. We’ve only scratched the surface of EdgeDB schema’s. Hopefully, I’ve at least managed to pique your interest. Computed properties Computed properties are a super powerful feature that can be added to your schema or queries. This example user schema creates a computed discriminator and username property. The discriminator uses an EdgeDB standard library function to generate a discriminator number and the username property is a combination between the name and discriminator properties. ` Globals and Access Policies EdgeDB allows you to define global variables as part of your schema. The most common use case I’ve seen for this is to power the access policy feature. You can define a global variable as part of your schema: global current_user: uuid; With a global variable defined, you can provide the value as a sort of context from your application by providing it into your EdgeDB driver/client. ` You can then add access policies directly to your schema, for example, to provide fine-grained access control to blog posts in your blogging application. ` Aside from access policies, you can use your global variables in your queries as well. For example, if you wanted a query to select the current user. ` Types and Sets EdgeDB is very type-centric. All of your data is strongly typed. We’ve touched on some of the types already. * Scalars - There are a lot of scalar types available out of the box * Custom scalars - Custom scalar types are user-defined extensions of existing types * Enums - Are supported out of the box - enum<Admin, Moderator, Member> * Arrays - Defined by passing the singular value type - array<str>; * Tuples - In EdgeD, tuples can contain more than 2 elements and come in named and unnamed varieties - <str, number>; tuple<name: str, jersey_number: float64, active: bool>; All queries return a Set which is a collection of values of a given type. In the query language, all values are Sets. Sets are a collection of values of a given type. A comma-separated list of values inside a set of {curly braces}. A query with no results will return an empty or singleton set. If we have no User values stored yet - select User returns {}. Paired with the query language types and set provides an incredibly powerful and expressive system for interacting with your data. If you thought TypeScript was cool, wait until you start writing EdgeQL! 🙂 EdgeQL Now to the fun stuff: the query language. We’ll use a schema from the docs and start by looking at some of those queries and build on those. The example schema has an abstract type Person with two sub-types based on it Hero and Villian. This is known as a polymorphic type in EdgeDB. The Movie type includes a 1:m association with Person ` Selecting properties / data Before we dig into some real queries we should just touch on how we select actual data from a query. It’s pretty obvious and GraphQL-like but worth mentioning. To specify properties to select, you attach a shape. This works for getting nested link / association data as well. Based on our schema, here’s how we could select fields from Movie, including data from the collection of related characters. ` There is also a feature called a splats that allows you to select all fields and/or all linked fields without specifying them individually. ` If you don’t specify any properties or splats, only id’s get returned select Movie; . Adding some objects with insert To get started, we can use insert to add objects to our database. We’ll start big by looking at the nested insert example. This example is interesting because it shows the creation of two objects in a single query. You’ll notice the simplicity of the syntax. Even though this is the first EdgeQL query we’re looking at, in my experience, it’s like this across the board. I’ve found EdgeQL queries to be simple and intuitive to the point where I’ve been able to intuit how to accomplish things in my head without having to reference the docs or ask the AI. This example adds a new Villian and a new Hero which gets assigned as a link or association to the nemesis field on our Villian. To accomplish this we see that we can nest queries by wrapping them in (). ` The next example is pretty similar, but instead of creating the linked property, we are select ing and adding several potential objects to the characters list of Movie since it is a multi link. This is a pretty complex query that is doing a lot of different things. It’s deceivingly succinct. To accomplish the same thing with SQL this would probably be about 3 different queries. This query finds the objects to add to the characters multi link by filtering on a collection of different strings to match against the name property. ` The last thing we’ll cover for insert is bulk inserts. This is particularly useful for things like seed scripts. In this example, you can just imagine that you have a JSON array of objects with hero names that gets passed in as an argument to your query ` Querying data with select We’ve already seen subqueries and a select in the last section where we found a collection of Person records with a filter. We’ll build on that and see what tools are available to us when it comes to querying data. This one covers a lot of ground. Very similar to SQL we have order and limit, and offset operators to support sorting and pagination. Also there is a whole standard library of functions and operators like count that can be used in our queries. This example returns a collection of villian names, excluding the first and last result. ` Most commonly, you will want to filter by an id ` Here’s another common example filtering by datetime. Since we’re using a string value here we need to cast it to the EdgeDB datetime type. ` You get a pretty similar toolbox to SQL when it comes to filtering with your common operators and things. Combined with all the tools in the standard library, you can get pretty creative with it. Changing values and links with update The update..filter..set statement is how we can update existing data with EdgeQL. set is followed by a shape with assignments of properties to be updated. ` You can replace links for an object ` or add additional ones ` An even more interesting example is removing links matched on a type. Since Villian is a sub-type of Person , this query will remove all characters linked of the Villian type. ` Deleting objects with delete Deleting is pretty straight forward. Using the delete command you can just filter for the objects that you would like to remove. ` When the EdgeQL pieces fall into place As you become more familiar with the EdgeQL query language chances are you’ll start writing very complex queries fluently because everything just makes sense once you’ve learned the building blocks. Domain and business concerns I don’t think they explicitly mention this as a goal anywhere but it’s something that I picked up on pretty quickly. EdgeDB nudges you to move more of what might have traditionally been application logic into your database layer. This is a topic that can bring a lot of division since even things like foreign keys and constraints in SQL are frowned upon in some circles. EdgeDB goes as far as providing constraints, global variables, contexts, and authorization support built into the database. I think that the ability to bake some of these concerns into your EdgeDB Schema is great. The way you model your schema and database in EdgeDB map to your domain in a much more intuitive way where domain concerns don’t really feel out of place there. Database Clients and Query Builders and Generators We’ve covered a lot so far to highlight what EdgeDB is and how to handle common use cases with the query language. To use it in your project though, you will need a client/driver library. There are clients available in several different languages. The one that they clearly have put the most investment into is the TypeScript query builder. We’ll briefly look at both options: simple driver/client and query builder. Whichever you end up choosing you will need to instantiate a driver and make sure you have a connection to your database instance configured. Basic client Although the TS query builder is very popular and pretty amazing, I couldn’t get away from just writing EdgeQL queries. In my application, I composed queries using template strings, and it worked great. The clients all have a set of methods available for passing in EdgeQL queries and parameters. querySingle is a method for queries where you are only expecting a single result. If your query will have multiple results you would use query instead. There is also a queryRequiredSingle which will throw an error if no results are found. There are some other methods available as well including one for running queries in a transaction ` The first argument is the query, and the second is a map of parameters. In this example we include the title parameter and it is accessed in our query via $title. TypeScript query builder If you have a TypeScript app and type-safety is important, you might prefer using the query builder. It is a pretty incredible feat of TypeScript magic initially developed by the same developer behind the popular library Zod. We can’t cover it in very much depth here but we’ll look at an example just to have an idea of what the query builder looks like in an application. ` The query builder is able to infer the result type automatically. It knows which fields you’ve selected, it knows that the result will be a single item. Query generator There are generators for queries and types. So even if you opt out of using the query builder you can still have queries that are strongly typed. It’s nice to have this option if you want to just write your queries as EdgeQL in .edgeql files. ` We end up with an exported function named getUser that is strongly typed. ` Tools and Utilities The team at EdgeDB puts a big emphasis on developer experience. It shows up all over the place. We’ve already seen some utilities with the generators that are available. There are some other tools available as well that help complete the entire experience. EdgeDB CLI The first and most important tool to mention is the CLI. If you’ve started using EdgeDB then you’ve most likely already installed and used it. The CLI is pretty extensive. It includes commands for things like migrations, managing EdgeDB versions and installations, managing projects and local/cloud database instances, dumps and restores, a repl, and more. The CLI makes managing EdgeDB a breeze. Admin UI The CLI includes a command to launch an admin UI for any project or database. The Admin UI includes a awesome interactive diagram of your database schema, a repl for running queries, and a table to inspect and make changes to the data stored in your database. Summary Adopting newer database technology is a tough sales pitch. Replacing your application’s database technology at any point in its lifecycle is not a problem that anyone wants to have. This is one of the reasons why EdgeDB being a superset of PostgreSQL is a huge feature in my opinion. The underlying database technology is tried and true, and EdgeDB is open-source. Based on this, I would feel confident using EdgeDB if it aligned well from a technical and business perspective. We’ve covered a lot of ground in this post. EdgeDB is feature-packed and powerful. Databases is a tough nut to crack, and I commend the team for all their hard work to help continue pushing forward one of the most important components of almost any application. I’m typically pretty conservative when it comes to databases, but EdgeDB took a great approach, in my opinion. I recommend at least giving it a try. You might catch the EdgeDB bug like I did!...

Apr 26, 2024

11 mins

JavaScriptEdgeDB

Demystifying React Server Components

A "Brief" History of React Rendering and Data-Fetching Patterns

The Benefits of React Server Components

Client Components with RSCs

Interleaving Nested Server & Client Components

Some Suggested Best Practices

Conclusion

Dustin Goodman

You might also like

How to test React custom hooks and components with Vitest

CSS Hooks: A new way to style your React apps

Configure your project with Drizzle for Local & Deployed Databases

Intro to EdgeDB - The 10x ORM

You might also like

How to test React custom hooks and components with Vitest

CSS Hooks: A new way to style your React apps

Configure your project with Drizzle for Local & Deployed Databases

Intro to EdgeDB - The 10x ORM