General

What is a Monorepo and What Are the Advantages for Using It in Your Project?

Sep 23, 2022

6 min read

Monorepos are very popular in the tech industry, and many large companies like Microsoft and Google use them. But what exactly is a monorepo, and would it work for your project?

In this article, we are going to explore what the monorepo architecture is, and learn about some of the commonly used tools for monorepos.

What is a monorepo?
A look into the starter.dev monorepo
Conclusion

What is a monorepo?

A monorepo is a single version controlled repository that contains multiple independent projects. Some monorepos might contain just 2 or 3 projects while others will contain hundreds of projects using different technologies. This differs from the polyrepo structure where the projects are spread out amongst multiple repositories. It is also important to note that a monorepo is not the same as a monolith. A monolith has multiple sub-projects inside one giant project while monorepos have multiple independent projects inside one repository.

Unified build process and CI/CD process

In a monorepo structure, all of CI/CD(Continuous integration/Continuous delivery) lives in the same repository. This provides teams with more flexibility, and allows you to deploy your services together. You can also choose to deploy your projects separately.

Increased cross team communication

In a polyrepo structure, you will have multiple teams working on different projects spread across many repositories. Often times these teams will not be aware of the status of other repositories outside of their main one. With a monorepo structure, teams are more aware of the changes being made to the projects used in the repo and might spot issues in another projects.

When all of your projects are under a single repository, it makes it easier to establish a consistent code style and set of guidelines across all projects. Some of the guidelines can include naming conventions, best practices for code reviews, branching policies, etc.

In situations where there are breaking changes in the main branch, a monorepo can help identify where the breaking change came from and that team would be responsible for fixing it. A monorepo can also help teams discuss different versioning strategies for the projects.

Steeper learning curve and issues of inclusivity for newer developers

Inclusivity of all developers on a team is really important for the success and outcome of a software project. When you have many diverse perspectives and levels it will lead to a stronger finished product. But a monorepo structure could be seen as intimidating to novice developers. It might be their first time seeing a list of independent projects within the same repository and they might not know how to best contribute in a meaningful way especially if it is an open source situation. It can also be difficult for newer developers to understand the git history under a monorepo structure. If you are using a tool like git blame,it will have to sift through a lot of unrelated commits just to generate that information for you. For these reasons, it is important for the team to help newer developers through the monorepo structure and create issues where all levels can meaningfully contribute.

A look into the starter.dev monorepo

Let's take a look at a couple of This Dot Labs open source monorepos and understand why the decision was made to use a monorepo.

Please note: We will not be exploring all of the possible tools used for monorepos. If you are interested in exploring more monorepo tools, then please checkout this curated list.

What is starter.dev?

starter.dev makes it easy to setup a new project with zero configuration involved. Once you install the starter.dev package(npm i @this-dot/create-starter), you can run the npx @this-dot/create-starter command and it will provide you with a list of starter kits to choose from. Each starter kit includes testing, linting, code examples and a basic configuration setup.

starter.dev is an open source project that consists of a documentation website repository and a GitHub showcases repository dedicated to all of the different starter kits.

Why did the team choose a monorepo structure?

When it came time to plan out the project, the team had to decide if they wanted to maintain each kit in isolation or create different folders for each kit under the same repository. The reason for using a monorepo structure is to provide the team the ability to build the CLI(command-line interface) with greater ease and allow the team to still build the kits in isolation.

What are Yarn Workspaces?

The documentation website repository uses Yarn Workspaces which allows you to setup multiple packages and use the yarn install command to install them all at once.

In the root directory, we have starters and packages directories. If we look at the starters directory, we will see all of the starter kits. Each starter kit will have its own package.json, basic configuration, README and code examples setup.

Inside the root package.json file, you will notice this "workspaces": [ "packages/*"] key.

{
  "name": "starter.dev",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "lint": "yarn workspace website lint && yarn workspace @this-dot/create-starter lint",
    "build:cli": "yarn workspace @this-dot/create-starter build",
    "build:website": "yarn workspace website build",
    "start:cli": "yarn workspace @this-dot/create-starter start",
    "start:website": "yarn workspace website preview",
    "dev:website": "yarn workspace website dev",
    "watch:cli": "yarn workspace @this-dot/create-starter watch"
  },
  "workspaces": [
    "packages/*"
  ]
}

That will tell Yarn to import all of the packages that are listed inside of the packages folder.

Unified CI/CD pipeline with Amplify

The starter.dev GitHub showcases repository has a unified CI/CD and uses Amplify. Inside the root directory, there is an Amplify yaml file where each application has their own set of build commands. Each project listed in the monorepo has its own deployment process setup.

version: 1
applications:
  - appRoot: next-react-query-tailwind
    frontend:
      phases:
        preBuild:
          commands:
            - nvm install --lts=gallium
            - yarn install
        build:
          commands:
            - yarn run build
      artifacts:
        baseDirectory: .next
        files:
          - "**/*"
      cache:
        paths:
          - node_modules/**/*

  - appRoot: angular-apollo-tailwind
    frontend:
      phases:
        preBuild:
          commands:
            - nvm install --lts=gallium
            - yarn install
            - yarn generate
        build:
          commands:
            - node -v
            - yarn run build
      artifacts:
        baseDirectory: dist/starter-dev-angular
        files:
          - "**/*"
      cache:
        paths:
          - node_modules/**/*

How does the team manage issues?

If you look at the issues tab, you will notice that all of the developers are following this same naming convention for issues: [starter kit] title for issue.

This works well for a variety of reasons:

Any developer interested in contributing to the project, can see at a quick glance which issues fit their skill and interest level.
It will be easier to track the status of the issues on a project board and see which areas need more attention.

Conclusion

We have explored the monorepo structure and talked about a few features. We have also compared it to other architectures like monoliths and polyrepos to see how if differs.

Then we took an in depth look into starter.dev and how it uses Yarn Workspaces to help organize the project.

I hope your enjoyed this article and best of luck on your programming journey.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Jessica Wilkins

Jessica Wilkins is a classical musician turned Software Engineer. Prior to joining the tech industry, she spent her time running her own sheet music company (JDW Sheet Music) as well as performing and teaching in Los Angeles, CA. She enjoys working with React and TypeScript. She is also a prolific technical writer for freeCodeCamp.

@codergirl1991 @jdwilkin4

It's Impossible For This Code to Fail - with Loren Sands-Ramshaw

Loren Sands-Ramshaw, Developer Relations Engineer at Temporal joins Rob Ocel to talk about reliable application development. They introduce the topic of durable execution and talk about reliability in systems, unraveling common issues developers face and showcase the benefits that durable execution can bring to software development. They also talk about the challenges of traditional programming and the complexities of event-driven architecture. Listen to the full podcast here: https://modernweb.podbean.com/e/modern-web-podcast-s11e19-its-impossible-for-this-code-to-fail/...

Jan 24, 2024

1 min

ArchitectureJavaScript

Demystifying React Server Components

React Server Components (RSCs) are the latest addition to the React ecosystem, and they've caused a bit of a disruption to how we think about React. Dan Abramov recently wrote an article titled "The Two Reacts" that explores the paradigms of client component and server component mental models and leaves us with the thought on how we can cohesively combine these concepts in a meaningful way. It took me a while to finally give RSCs the proper exploration to truly understand the "new" model and grasp where React is heading. First off, the new model isn't really new so much as it introduces a few new concepts for us to consider when architecting our applications. Once I understood the pattern, I found an appreciation for the model and what it's trying to help us accomplish. In this post, I hope I can show you the progression of the React architecture in applications as I've experienced them and how I think RSCs help us improve this model and our apps. A "Brief" History of React Rendering and Data-Fetching Patterns One of the biggest challenges in React since its early days is "how do we server render pages?" Server-side rendering (SSR) is one of the techniques we can use to ensure users see data on initial load and helps with our site's SEO. Without SSR, users would see blank screens or loading spinners, and then a bunch of content would appear shortly after. An excellent graphic on Remix's website demonstrates this behavior from the end user's perspective and it's a problem we generally try to avoid as developers. This problem is so vast and difficult that we've been trying to solve it since 2015. Rick Hanlon from the React Core Team reminded us just how complicated this problem was recently. If you think react is complicated now, go back to 2015 when you had to configure babel and webpack yourself, you had to do SSR and routing yourself, you had to decide between flow and typescript, you had to learn flux and immutable.js, and you had to choose 1 of 100 boilerplates— Ricky (@rickhanlonii) January 28, 2024 But SSR has its issues too. Sometimes SSR is slow because we need to do a lot of data fetching for our page. Because of these large payloads, we'll defer their rendering using lazy loading patterns. All of a sudden we have spinners again! How we've managed these components has changed too. Over the years, we've seen a variety of patterns emerge for managing these problems. We had server-side pre-fetching where we had to hydrate our frontends application state. Then we tried controller-view patterned components for our lazily loaded client-side components. With the evolution of React, we were able to simplify the controller-view patterns to leverage hooks. Now, we're in a new era of multi-server entry points on a page with RSCs. The Benefits of React Server Components RSCs give us this new paradigm that allows us to have multiple entries into our server on a single page. Leveraging features like Next.js' streaming mode and caching, we can limit what our pages block on for SSR and optimize their performance. To illustrate this, let's look at this small block of PHP for something we might have done in the 2000s: index.php ` For simplicity, the blocks indicate server boundaries where we can execute PHP functionality. Once we exit that block, we can no longer leverage or communicate with the server. In this example, we're displaying a welcome message to a user and a list of posts. If we look at Next.js' page router leveraging getServerSideProps, we would maybe write this same page as follows: pages/posts.jsx ` In this case, we're having our server do a lot to fetch the data we need to render this page and all its components. This also makes the line between server and client much clearer as getServerSideProps runs before our client renders, and we're unable to go back to that function without an API route and client-side fetch. Now, let's look at Next.js' app router with server components. This same component could be rendered as follows: app/posts/page.tsx ` This moves us a bit closer back to our PHP version as we don't have to split our server and client functionality as explicitly. This example is relatively simple and only renders a few components that could be easily rendered as static content. This page also probably requires all the content to be rendered upfront. But, from this, we can see that we're able to simplify how server data fetching is done. If nothing else, we could use this pattern to make our SSR patterns better and client render the rest of our apps, but then we'd lose out on some of the additional benefits that we can get when we combine this with streaming. Let's look at a post page that probably has the content and a comments section. We don't need to render the comments immediately on page load or block on it because it's a secondary feature on the page. This is where we can pull in Suspense and make our page shine. Our code might look as follows: app/posts/[slug]/page.jsx ` Our PostComments are a server component that renders static content again, but we don't want its render to block our page from being served to the client. What we've done here is moved the fetch operation into the comments component and wrapped it in a Suspense boundary in our page to defer its render until its content has been fetched and then stream it onto the page into its render location. We could also add a fallback to our suspense boundary if we wanted to put a skeleton loader or other indication UI. With these changes, we're writing components similarly to how we've written them on the client historically but simplified our data fetch. What happens when we start to progressively enhance our features with client-side JavaScript? Client Components with RSCs All our examples focused on staying in the server context which is great for simple applications, but what happens when we want to add interactivity? If you were to put a useState, useEffect, onClick, or other client-side React feature into a server component, you'd find some error messages because you can't run client code from a server context. If we think back to our PHP example, that makes a lot of sense why this is the case, but how do we work around this? For me, this is where my first really mental challenge with RSCs started. Let's use our PostComments as an example to enhance by in-place sorting the section from the server. ` In this example, we're using a server component the same way as our previous example to do the initial data fetch and stream when ready. However, we're immediately passing the results to a client component that renders the elements and enhances our code with a list of sort options. When we select the sort we want, our code makes a request to the server at a predefined route and gets the new data that we re-render in the new order on screen. This is how we might have done things without RSCs before but without a useEffect for our initial rendering. If you're familiar with the old controller-view pattern, this is relatively similar to that pattern, but we have to relegate client re-fetching to the view (client) component, where we might have had all fetch and re-fetch patterns in the controller (server) component Another way to solve this same problem is leveraging server actions, but that would cause the page to re-render. Given the caching mechanisms in Next.js, this is probably fine, but it's not the user experience people are expecting. Server actions are a topic of their own, so I won't cover them in this post, but they're important for the holistic ecosystem experience in the new mindset. Interleaving Nested Server & Client Components These examples have shown a clean approach where we keep our server components at the top of our rendering tree and have a clear line where we move from server mode to client mode. But one of the advantages of RSCs is that we can interleave where our server component entry points may exist. Let's think about a product carousel on a storefront page for example. We may have built this as follows before: ` Here we're rendering carousels that render their cards and our tree goes server -> client -> server. This is not allowed in the new paradigm because the React compiler cannot detect that we moved back into a server component when we called ProductCards from a client component. Instead, we would need to refactor this to be: ` Here, we've changed ProductCarousel to accept children that are our ProductCards server component. This allows the compiler to detect the boundary and render it appropriately. I'd also recommend adding Suspense boundaries to these carousels but I omitted it for the sake of brevity in this example. Some Suggested Best Practices Our team has been using these new patterns for some time and have developed some patterns we've identified as best practices for us to help keep some of these boundaries clear that I thought worth sharing. 1. Be explicit with component types We've found that being explicit with component types is essential to expressing intent. Some components are strictly server, some are strictly client, and others can be used in both contexts. Our team likes using the server-only package to express this intent but we found we needed more. We've opted for the following naming conventions: Server only components: component-name.server.(jsx|tsx) Client only components: component-name.client.(jsx|tsx) Universal components: component-name.(jsx|tsx) This does not apply to special framework file names but we've found this helps us delineate where we are and to help with interleaving. 2. Defer Fetch when Cache is Available If we're trying to render a component that is a collection of other components, we've found deferring the fetch to the child allows our cache to do more for us in rendering in cases where the same element may exist in different locations. This is similar to how GraphQL's data loaders works and can ideally boost the performance of cached server components. So, in our product example above, we may only fetch the IDs of the products we need for the ProductCards and then fetch all the data per card. Yes, this is an extra fetch and causes an N+1, but if our cache is in play, end users will feel a performance gain. 3. Colocate loaders and queries If you're using GraphQL or need loading states for components for Suspense fallback, we recommend colocating those elements in the same file or directory. This makes it easier to modify and manage related elements for your components in a more meaningful way. 4. Use Suspense to defer non-essential content This will depend on your website and needs. Still, we recommend deferring as many non-essential elements to Suspense as possible. We define non-essential to be anything you could consider secondary to the page. On a blog post, this could be recommended posts or comments. On a product page, this could be reviews and related products. So long as the primary focus of your page is outside a Suspense boundary, your usage is probably acceptable. Conclusion RSCs represent a significant evolution in the React ecosystem, offering new paradigms and opportunities for improving the architecture of web applications. They enable multiple server entry points on a single page, optimizing server-side rendering, and simplifying data fetching. RSCs allow for a clearer separation between server and client functionality, enhancing both performance and maintainability. To make the most of RSCs, developers should consider best practices such as being explicit with component types, deferring fetch when caching is available, colocating loaders and queries, and using Suspense to defer non-essential content. Embracing these practices can help harness the potential of React Server Components and pave the way for more efficient and interactive web applications. We're excited about this development in the React ecosystem and the developments happening across teams and frameworks that are opting into using them. While Next.js offers a great solution, we're excited to see similar enhancements to Remix and RedwoodJS....

Feb 2, 2024

9 mins

ReactNextJSArchitecture

How to Handle Uploaded Images and Avoid Image Distortion

When you are working with images in your application, you might run into issues where the image's aspect ratio is different from the container's specified width and height. This could lead to images looking stretched and distorted. In this article, we will take a look at how to solve this problem by using the object-fit CSS property. A Look Into the Issue Using the "Let's Chat With" App Let's Chat With is an open source application that facilitates networking between attendees for virtual and in-person conferences. When users sign up for the app, they can join a conference and create a new profile with their name, image, and bio. When the team at This Dot Labs was testing the application, they noticed that some of the profile images were coming out distorted. The original uploaded source image did not have an aspect ratio of 1:1. A 1:1 aspect ratio refers to an image's width and height being the same. Since the image was not a square, it was not fitting well within the dimensions below. ` In order to fix this problem, the team decided to use the object-fit CSS property. What is the object-fit CSS property? The object-fit property is used to determine how an image or video should resize in order to fit inside its container. There are 5 main values you can use with the object-fit property. - object-fit: contain; - resizes the content to fit inside the container without cropping it - object-fit: cover; - ensures the all of the content covers the container and will crop if necessary - object-fit: fill; - fills the container with the content by stretching it and ignoring the aspect ratio. This could lead to image distortion. - object-fit: none; - does not resize the content which could lead to the content spilling out of the container - object-fit: scale-down; - scales larger content down to fit inside the container When the object-fit: cover; property was applied to the profile image in Let's Chat With, the image was no longer distorted. ` When Should You Consider Using the object-fit Property? There will be times where you will not be able to upload different sized images to fit different containers. You might be in a situation like Let's Chat With, where the user is uploading images to your application. In that case, you will need to apply a solution to ensure that the content appropriately resizes within the container without becoming distorted. Conclusion In this article, we learned about how to fix distorted uploaded images using the object-fit property. We examined the bug inside the Let's Chat With application and how that bug was solved using object-fit: cover;. We also talked about when you should consider using the object-fit property. If you want to check out the Let's Chat with app, you can signup here. If you are interested in contributing to the app, you can check out the GitHub repository....

Jul 28, 2023

3 mins

JavaScript

How to Go from Software Engineer to AI Engineer with Shawn Wang

Shawn Wang discusses AI engineering’s potential impact on the software development labor market and developer experience. With the rise of artificial intelligence technologies, there is an increasing demand for professionals who can bridge the gap between AI and software engineering, and engineers are looking around for guidance on how to best learn AI. Shawn emphasizes the importance of acquiring core knowledge and skills in AI engineering, highlighting the need for a systematic approach to education and development. He envisions a future where AI engineering becomes a recognized profession, with specialized training programs and certifications. He further talks about mental models for approaching AI literacy, and skill building. Shawn also discusses the use of advanced AI technology in software development. He showcases tools like GPT Vision, which can automate tasks and generate code efficiently. While acknowledging the potential of AI in enhancing productivity, Shawn emphasizes the need to leverage AI tools to augment human capabilities rather than replace them. Download this episode here....

Apr 17, 2024

1 min

Artificial IntelligenceEngineering Leadership

What is a Monorepo and What Are the Advantages for Using It in Your Project?

Table of contents

What is a monorepo?

Unified build process and CI/CD process

Increased cross team communication

Steeper learning curve and issues of inclusivity for newer developers

A look into the starter.dev monorepo

What is starter.dev?

Why did the team choose a monorepo structure?

What are Yarn Workspaces?

Unified CI/CD pipeline with Amplify

How does the team manage issues?

Conclusion

Jessica Wilkins

You might also like

It's Impossible For This Code to Fail - with Loren Sands-Ramshaw

Demystifying React Server Components

How to Handle Uploaded Images and Avoid Image Distortion

How to Go from Software Engineer to AI Engineer with Shawn Wang

You might also like

It's Impossible For This Code to Fail - with Loren Sands-Ramshaw

Demystifying React Server Components

How to Handle Uploaded Images and Avoid Image Distortion

How to Go from Software Engineer to AI Engineer with Shawn Wang