This Dot Blog
This Dot provides teams with technical leaders who bring deep knowledge of the web platform. We help teams set new standards, and deliver results predictably.
Build a Next-Gen Chat App with AI and WebSockets in Just 3 Hours with Ben Lesh and Tracy Lee
In this comprehensive 3-hour training session, Tracy Lee and Ben Lesh guided participants through the process of building a chat application using various cutting-edge technologies and methodologies. The training began with an introduction to using v0.dev, followed by leveraging AI tools like GitHub Copilot and ChatGPT to scaffold the initial version of the application. This segment highlighted the practical use of AI in real-time coding, demonstrating how these tools can significantly streamline development workflows. Next, the session delved into creating a monorepo using Nx, showcasing how to efficiently manage multiple projects and libraries within a single repository. Participants then learned how to connect their application to the OpenAI API, incorporating AI functionalities. The training continued with a focus on refactoring the application to enable real-time streaming using WebSockets. This segment was particularly insightful for developers, as it covered essential techniques for implementing WebSocket functionality in real-world scenarios. Throughout the training, attendees had the opportunity to observe Ben Lesh's debugging process, gaining practical knowledge on troubleshooting and optimizing code. The session concluded with integrating the application with AstraDB and Cohere to implement the retrieval part of Retrieval-Augmented Generation (RAG). This final step equipped the chat app with the capability to understand and utilize RxJS, bringing the project to fruition....
May 30, 2024
1 min
Deploying Multiple Apps From a Monorepo to GitHub Pages
Explore deploying multiple front-end applications on GitHub Pages with our guide. Learn how to navigate the challenges of client-side routing and efficiently manage multiple apps in one repository....
Jan 24, 2024
4 mins
How to automatically deploy your full-stack JavaScript app with AWS CodePipeline
In our previous blog post we set up a horizontally scalable deployment for our full-stack javascript app. In this article we would like to show you how to set up AWS CodePipeline to automatically deploy changes to the application....
Sep 15, 2023
9 mins
Linting, Formatting, and Type Checking Commits in an Nx Monorepo with Husky and lint-staged
One way to keep your codebase clean is to enforce linting, formatting, and type checking on every commit. This is made very easy with pre-commit hooks. Using Husky, you can run arbitrary commands before a commit is made. This can be combined with lint-staged, which allows you to run commands on only the files that have been staged for commit. This is useful because you don't want to run linting, formatting, and type checking on every file in your project, but only on the ones that have been changed. But if you're using an Nx monorepo for your project, things can get a little more complicated. Rather than have you use eslint or prettier directly, Nx has its own scripts for linting and formatting. And type checking is complicated by the use of specific tsconfig.json files for each app or library. Setting up pre-commit hooks with Nx isn't as straightforward as in a simpler repository. This guide will show you how to set up pre-commit hooks to run linting, formatting, and type checking in an Nx monorepo. Configure Formatting Nx comes with a command, nx format:write for applying formatting to affected files which we can give directly to lint-staged. This command uses Prettier under the hood, so it will abide by whatever rules you have in your root-level .prettierrc file. Just install Prettier, and add your preferred configuration. ` Then add a .prettierrc file to the root of your project with your preferred configuration. For example, if you want to use single quotes and trailing commas, you can add the following: ` Configure Linting Nx has its own plugin that uses ESLint to lint projects in your monorepo. It also has a plugin with sensible ESLint defaults for your linter commands to use, including ones specific to Nx. To install them, run the following command: ` Then, we can create a default .eslintrc.json file in the root of our project: ` The above ESLint configuration will, by default, apply Nx's module boundary rules to any TypeScript or JavaScript files in your project. It also applies its recommended rules for JavaScript and TypeScript respectively, and gives you room to add your own. You can also have ESLint configurations specific to your apps and libraries. For example, if you have a React app, you can add a .eslintrc.json file to the root of your app directory with the following contents: ` Set Up Type Checking Type checking with tsc is normally a very straightforward process. You can just run tsc --noEmit to check your code for type errors. But things are more complicated in Nx with lint-staged. There are a two tricky things about type checking with lint-staged in an Nx monorepo. First, different apps and libraries can have their own tsconfig.json files. When type checking each app or library, we need to make sure we're using that specific configuration. The second wrinkle comes from the fact that lint-staged passes a list of staged files to commands it runs by default. And tsc will only accept either a specific tsconfig file, or a list of files to check. We do want to use the specific tsconfig.json files, and we also only want to run type checking against apps and libraries with changes. To do this, we're going to create some Nx run commands within our apps and libraries and run those instead of calling tsc directly. Within each app or library you want type checked, open the project.json file, and add a new run command like this one: ` Inside commands is our type-checking command, using the local tsconfig.json file for that specific Nx app. The cwd option tells Nx where to run the command from. The forwardAllArgs option tells Nx to ignore any arguments passed to the command. This is important because tsc will fail if you pass both a tsconfig.json and a list of files from lint-staged. Now if we ran nx affected --target=typecheck from the command line, we would be able to type check all affected apps and libraries that have a typecheck target in their project.json. Next we'll have lint-staged handle this for us. Installing Husky and lint-staged Finally, we'll install and configure Husky and lint-staged. These are the two packages that will allow us to run commands on staged files before a commit is made. ` In your package.json file, add the prepare script to run Husky's install command: ` Then, run your prepare script to set up git hooks in your repository. This will create a .husky directory in your project root with the necessary file system permissions. ` The next step is to create our pre-commit hook. We can do this from the command line: ` It's important to use Husky's CLI to create our hooks, because it handles file system permissions for us. Creating files manually could cause problems when we actually want to use the git hooks. After running the command, we will now have a file at .husky/pre-commit that looks like this: ` Now whenever we try to commit, Husky will run the lint-staged command. We've given it some extra options. First, --concurrent false to make sure attempts to write fixes with formatting and linting don't conflict with simultaneous attempts at type checking. Second is --relative, because our Nx commands for formatting and linting expect a list of file paths relative to the repo root, but lint-staged would otherwise pass the full path by default. We've got our pre-commit command ready, but we haven't actually configured lint-staged yet. Let's do that next. Configuring lint-staged In a simpler repository, it would be easy to add some lint-staged configuration to our package.json file. But because we're trying to check a complex monorepo in Nx, we need to add a separate configuration file. We'll call it lint-staged.config.js and put it in the root of our project. Here is what our configuration file will look like: ` Within our module.exports object, we've defined two globs: one that will match any TypeScript files in our apps, libraries, and tools directories, and another that also matches JavaScript and JSON files in those directories. We only need to run type checking for the TypeScript files, which is why that one is broken out and narrowed down to only those files. These globs defining our directories can be passed a single command, or an array of commands. It's common with lint-staged to just pass a string like tsc --noEmit or eslint --fix. But we're going to pass a function instead to combine the list of files provided by lint-staged with the desired Nx commands. The nx affected and nx format:write commands both accept a --files option. And remember that lint-staged always passes in a list of staged files. That array of file paths becomes the argument to our functions, and we concatenate our list of files from lint-staged into a comma-delimitted string and interpolate that into the desired Nx command's --files option. This will override Nx's normal behavior to explicitly tell it to only run the commands on the files that have changed and any other files affected by those changes. Testing It Out Now that we've got everything set up, let's try it out. Make a change to a TypeScript file in one of your apps or libraries. Then try to commit that change. You should see the following in your terminal as lint-staged runs: ` Now, whenever you try to commit changes to files that match the globs defined in lint-staged.config.js, the defined commands will run first, and verify that the files contain no type errors, linting errors, or formatting errors. If any of those commands fail, the commit will be aborted, and you'll have to fix the errors before you can commit. Conclusion We've now set up a monorepo with Nx and configured it to run type checking, linting, and formatting on staged files before a commit is made. This will help us catch errors before they make it into our codebase, and it will also help us keep our codebase consistent and readable. To see an example Nx monorepo with these configurations, check out this repo....
May 10, 2023
6 mins
Setting Up TypeORM Migrations in an Nx/NestJS Project
TypeORM is a powerful Object-Relational Mapping (ORM) library for TypeScript and JavaScript that serves as an easy-to-use interface between an application's business logic and a database, providing an abstraction layer that is not tied to a particular database vendor. TypeORM is the recommended ORM for NestJS as both are written in TypeScript, and TypeORM is one of the most mature ORM frameworks available for TypeScript and JavaScript. One of the key features of any ORM is handling database migrations, and TypeORM is no exception. A database migration is a way to keep the database schema in sync with the application's codebase. Whenever you update your codebase's persistence layer, perhaps you'll want the database schema to be updated as well, and you want a reliable way for all developers in your team to do the same with their local development databases. In this blog post, we'll take a look at how you could implement database migrations in your development workflow if you use a NestJS project. Furthermore, we'll give you some ideas of how nx can help you as well, if you use NestJS in an nx-powered monorepo. Migrations Overview In a nutshell, migrations in TypeORM are TypeScript classes that implement the MigrationInterface interface. This interface has two methods: up and down, where up is used to execute the migration, and down is used to rollback the migration. Assuming that you have an entity (class representing the table) as below: ` If you generate a migration from this entity, it could look as follows: ` As can be seen by the SQL commands, the up method will create the post table, while the down method will drop it. How do we generate the migration file, though? The recommended way is through the TypeORM CLI. TypeORM CLI and TypeScript The CLI can be installed globally, by using npm i -g typeorm. It can also be used without installation by utilizing the npx command: npx typeorm . The TypeORM CLI comes with several scripts that you can use, depending on the project you have, and whether the entities are in JavaScript or TypeScript, with ESM or CommonJS modules: - typeorm: for JavaScript entities - typeorm-ts-node-commonjs: for TypeScript entities using CommonJS - typeorm-ts-node-esm: for TypeScript entities using ESM Many of the TypeORM CLI commands accept a data source file as a mandatory parameter. This file provides configuration for connecting to the database as well as other properties, such as the list of entities to process. The data source file should export an instance of DataSource, as shown in the below example: ` To use this data source, you would need to provide its path through the -d argument to the TypeORM CLI. In a NestJS project using ESM, this would be: ` If the DataSource did not import the Post entity from another file, this would most likely succeed. However, in our case, we would get an error saying that we "cannot use import statement outside a module". The typeorm-ts-node-esm script expects our project to be a module -- and any importing files need to be modules as well. To turn the Post entity file into a module, it would need to be named post.entity.mts to be treated as a module. This kind of approach is not always preferable in NestJS projects, so one alternative is to transform our DataSource configuration to JavaScript - just like NestJS is transpiled to JavaScript through Webpack. The first step is the transpilation step: ` Once transpiled, you can then use the regular typeorm CLI to generate a migration: ` Both commands can be combined together in a package.json script: ` After the migrations are generated, you can use the migration:run command to run the generated migrations. Let's upgrade our package.json with that command: ` Using Tasks in Nx If your NestJS project is part of an nx monorepo, then you can utilize nx project tasks. The benefit of this is that nx will detect your tsconfig.json as well as inject any environment variables defined in the project. Assuming that your NestJS project is located in an app called api, the above npm scripts can be written as nx tasks as follows: ` The typeorm-generate-migration and typeorm-run-migrations tasks depend on the build-migration-config task, meaning that they will always transpile the data source config first, before invoking the typeorm CLI. For example, the previous CreatePost migration could be generated through the following command: ` Conclusion TypeORM is an amazing ORM framework, but there are a few things you should be aware of when running migrations within a big TypeScript project like NestJS. We hope we managed to give you some tips on how to best incorporate migrations in an NestJS project, with and without nx....
Feb 22, 2023
4 mins
GitHub Actions for Serverless Framework Deployments
Background Our team was building a Serverless Framework API for a client that wanted to use the Serverless Dashboard) for deployment and monitoring. Based on some challenges from last year, we agreed with the client that using a monorepo tool like Nx) would be beneficial moving forward as we were potentially shipping multiple Serverless APIs and frontend applications. Unfortunately, we discovered several challenges integrating with the Serverless Dashboard, and eventually opted into custom CI/CD with GitHub Actions. We’ll cover the challenges we faced, and the solution we created to mitigate our problems and generate a solution. Serverless Configuration Restrictions By default, the Serverless Framework does all its configuration via a serverless.yml file. However, the framework officially supports alternative formats) including .json, .js, and .ts. Our team opted into the TypeScript format as we wanted to setup some validation for our engineers that were newer to the framework through type checks. When we eventually went to configure our CI/CD via the Serverless Dashboard UI, the dashboard itself restricted the file format to just the YAML format. This was unfortunate, but we were able to quickly revert back to YAML as configuration was relatively simple, and we were able to bypass this hurdle. Prohibitive Project Structures With our configuration now working, we were able to select the project, and launch our first attempt at deploying the app through the dashboard. Immediately, we ran into a build issue: ` What we found was having our package.json in a parent directory of our serverless app prevented the dashboard CI/CD from being able to appropriately detect and resolve dependencies prior to deployment. We had been deploying using an Nx command: npx nx run api:deploy --stage=dev which was able to resolve our dependency tree which looked like: To resolve, we thought maybe we could customize the build commands utilized by the dashboard. Unfortunately, the only way to customize these commands is via the package.json of our project. Nx allows for package.json per app in their structure, but it defeated the purpose of us opting into Nx and made leveraging the tool nearly obsolete. Moving to GitHub Actions with the Serverless Dashboard We thought to move all of our CI/CD to GitHub Actions while still proxying the dashboard for deployment credentials and monitoring. In the dashboard docs), we found that you could set a SERVERLESS_ACCESS_KEY and still deploy through the dashboard. It took us a few attempts to understand exactly how to specify this key in our action code, but eventually, we discovered that it had to be set explicitly in the .env file due to the usage of the Nx build system to deploy. Thus the following actions were born: api-ci.yml ` api-clean.yml ` These actions ran smoothly and allowed us to leverage the dashboard appropriately. All in all this seemed like a success. Local Development Problems The above is a great solution if your team is willing to pay for everyone to have a seat on the dashboard. Unfortunately, our client wanted to avoid the cost of additional seats because the pricing was too high. Why is this a problem? Our configuration looks similar to this (I’ve highlighted the important lines with a comment): serverless.ts ` The app and org variables make it so it is required to have a valid dashboard login. This meant our developers working on the API problems couldn’t do local development because the client was not paying for the dashboard logins. They would get the following error: Resulting Configuration At this point, we had to opt to bypass the dashboard entirely via CI/CD. We had to make the following changes to our actions and configuration to get everything 100% working: serverless.ts - Remove app and org fields - Remove accessing environment secrets via the param option ` api-ci.yml - Add all our secrets to GitHub and include them in the scripts - Add serverless confg ` api-cleanup.yml - Add serverless config - Remove secrets ` Conclusions The Serverless Dashboard is a great product for monitoring and seamless deployment in simple applications, but it still has a ways to go to support different architectures and setups while being scalable for teams. I hope to see them make the following changes: - Add support for different configuration file types - Add better support custom deployment commands - Update the framework to not fail on login so local development works regardless of dashboard credentials The Nx + GitHub actions setup was a bit unnatural as well with the reliance on the .env file existing, so we hope the above action code will help someone in the future. That being said, we’ve been working with this on the team and it’s been a very seamless and positive change as our developers can quickly reference their deploys and know how to interact with Lambda directly for debugging issues already....
Sep 26, 2022
5 mins