Javascript

Combining Validators and Transformers in NestJS

Oct 27, 2022

3 min read

When building a new API, it is imperative to validate that requests towards the API conform to a predefined specification or a contract.

For example, the specification may state that an input field must be a valid e-mail string. Or, the specification may state that one field is optional, while another field is mandatory. Although such validation can also be performed on the client side, we should never rely on it alone. There should always be a validation mechanism on the server side as well.

After all, you never know who's acting on behalf of the client. Therefore, you can never fully trust the data you receive. Popular backend frameworks usually have a very good support for validation out of the box, and NestJS, which we will cover in this blog post, is no exception.

In this blog post, we will be focusing on NestJS's validation using ValidationPipe- specifically on one lesser known feature- which is the ability to not only validate input, but transform it beforehand as well, thereby combining transformation and validation of data in one go.

Using ValidationPipe

To test this out, let's build a UsersController that supports getting a list of users, and with the option to filter by several conditions. After scaffolding our project using nest new [project-name], let's define a class that will represent this collection of filters, and name it GetUsersQuery:

class GetUsersQuery {
  userIds: string[];

  nameContains: string;

  pageSize: number;
}

Now, let's use it in the controller:

class User {
  id: string;
  name: string;
  active = false;
}

@Controller('users')
export class UsersController {
  @Get()
  getUsers(@Query() query: GetUsersQuery): User[] {
    console.log(JSON.stringify(query));
    return [
      { id: '1', name: 'Zeus Carver', active: true },
      { id: '2', name: 'Holly Gennero', active: true },
    ];
  }
}

The problem with this approach is that there is no validation performed whatsoever. Although we've defined userIds as an array of strings, and pageSize as a number, this is just compile-time verification - there is no runtime validation. In fact, if you execute a GET request on http://localhost:3000/users?userIds=1,2,3&pageSize=3, the query object will actually contain only string fields:

{
  "userIds": "1,2,3",
  "pageSize": "3"
}

There's a way to fix this in NestJS. First, let's install the dependencies needed for using data transformation and validation in NestJS:

npm i --save class-validator class-transformer

As their names would suggest, the class-validator package brings support for validating data, while the class-transformer package brings support for transforming data.

Each package adds some decorators of their own to aid you in this. For example, the class-validator package has the @IsNumber() decorator to perform runtime validation that a field is a valid number, while the class-transformer package has the @Type() decorator to perform runtime transformation from one type to another.

Having that in mind, let's decorate our GetUsersQuery a bit:

class GetUsersQuery {
  @IsArray()
  @IsOptional()
  @Transform(({ value }) => value.split(','))
  userIds: string[];

  @IsOptional()
  nameContains: string;

  @IsOptional()
  @IsNumber()
  @Type(() => Number)
  pageSize: number;
}

This is not enough, though. To utilize the class-validator decorators, we need to use the ValidationPipe. Additionally, to utilize the class-transformer decorators, we need to use ValidationPipe with its transform: true flag:

class User {
  id: string;
  name: string;
  active = false;
}

@Controller('users')
export class UsersController {
  @Get()
  @UsePipes(new ValidationPipe({ transform: true }))
  getUsers(@Query() query: GetUsersQuery): User[] {
    console.log(JSON.stringify(query));
    return [
      { id: '1', name: 'Zeus Carver', active: true },
      { id: '2', name: 'Holly Gennero', active: true },
    ];
  }
}

Here's what happens in the background. As said earlier, by default, every path parameter and query parameter comes over the network as a string. We could convert these values to their JavaScript primitives in the controller (array of strings and a number, respectively), or we can use the transform: true property of the ValidationPipe to do this automatically.

NestJS does need some guidance on how to do it, though. That's where class-transformer decorators come in. Internally, NestJS will use Class Transformer's plainToClass method to convert the above object to an instance of the GetUsersQuery class, using the Class Transformer decorators to transform the data along the way. After this, our object becomes:

{
  "userIds": ["1", "2", "3"],
  "pageSize": 3
}

Now, Class Validator comes in, using its annotations to validate that the data comes in as expected. Why is Class Validator needed if we already transformed the data beforehand? Well, Class Transformer will not throw any errors if it failed to transform the data. This means that, if you provided a string like "testPageSize" to the pageSize query parameter, our query object will actually come in as:

{
  "userIds": ["1", "2", "3"],
  "pageSize": null
}

And this is where Class Validator will kick in and raise an error that pageSize is not a proper number:

{
  "statusCode": 400,
  "message": [
    "pageSize must be a number conforming to the specified constraints"
  ],
  "error": "Bad Request"
}

Other transformation options

The @Type and @Transform decorators give us all kinds of options for transforming data. For example, strings can be converted to dates and then validated using the following combination of decorators:

@IsOptional()
@Type(() => Date)
@IsDate()
registeredSince: Date;

We can do the same for booleans:

@IsOptional()
@Type(() => Boolean)
@IsBoolean()
isActive: boolean;

If we wanted to define advanced transformation rules, we can do so through an anonymous function passed to the @Transform decorator. With the following transformation, we can also accept isActive=1 in addition to isActive=true, and it will properly get converted to a boolean value:

@IsOptional()
@Transform(({ value }) => value === '1' || value === 'true')
@IsBoolean()
isActive: boolean;

Conclusion

This was an overview of the various options you have at your disposal when validating and transforming data. As you can see, NestJS gives you many options to declaratively define your validation and transformation rules, which will be enforced by ValidationPipe. This allows you to focus on your business logic in controllers and services, while being assured that the controller inputs have been properly validated.

You'll find the source code for this blog post's project on our GitHub.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Dario Djuric

Dario is a full-stack engineer who has spent most of his career doing enterprise Java projects. He has always had a hidden passion for frontend, though -- and he is now able to pursue that passion at This Dot. He spends most of his free time with his two sons, but occasionally, he manages to squeeze in some casual sports activities such as jogging, cycling, and soccer.

@dario_djuric @dariodjuric

Introduction to RESTful APIs with NestJS

Welcome to an introduction to RESTful APIs with NestJS. Understanding JavaScript and TypeScript will make it easier to follow the directions in this article, but you don't necessarily need to be proficient. NestJS is one of the most rapidly growing frameworks for building Node.js server-side applications. Companies such as *Roche*, *Adidas*, and *Autodesk*, all trust NestJS when it comes to building efficient and scalable server-side applications. NestJS is based heavily on *Angular*, and uses Angular-like modules, services, controllers, pipes, and decorators. This allows NestJS to help developers create scalable, testable, loosely coupled, and easily maintainable applications. NestJS was built with TypeScript, but it can also support pure JavaScript development. NestJS offers a level of abstraction above two very popular Node.js frameworks, either *Express.js* or *Fastify*. This means that all of the great middleware that are available for Express.js and Fastify can also be used with NestJS. The best way to get familiar with NestJS is to build a basic RESTful API with CRUD (Create, Read, Update, and Delete) functionality. This is exactly what we'll be doing together in this article. We'll be building a simple RESTful API for a Blog, with endpoints to handle CRUD operations on blog posts. Getting Started Code editor Using Visual Studio Code as a code editor can help speed up NestJS development because of its smart IntelliSense and great TypeScript support. In Visual Studio Code, make sure that you have the following user setting by going to File / Preferences / Settings, and searching for the user setting named typescript.preferences.importModuleSpecifier. Make sure to set it to relative as seen below. ` This will allow Visual Studio Code to use relative paths rather than absolute paths when auto-importing. Using absolute path imports in our application can lead to problems if and when our code ends up in a different directory. Insomnia Insomnia is a useful API testing tool that we will use to test the NestJS API that we will be building. The NestJS CLI To get started with NestJS, let's install the Nest CLI. The Nest CLI is a command-line interface tool that makes it easy to develop, and maintain NestJS applications. It allows us to run our application in development mode, and to build and bundle it for a production-ready release. ` Creating a new NestJS project With the Nest CLI now installed, we can use it to create a new project. ` This command will create a new project directory called rest-api. It will create a base structure for our project, and add in the following core Nest files: - app.controller.ts: A controller with a single route. - app.controller.spec.ts: The controller's unit tests. - app.module.ts: The root module of our application. - app.service.ts: A service for the AppModule's business logic. - main.ts: The entry file of our application. The initial project structure created by the Nest CLI encourages us to follow the common convention of keeping each module in its own directory. Testing the sample endpoint The installation of NestJS comes with a sample API endpoint that we can test by making a request to it. If we open app.controller.ts, we can see that there is a GET endpoint that was created for us with the @Get() decorator. It returns a 'Hello World!' string. ` Let's run npm run start:dev from our project folder. This will run our NestJS app in watch mode, which provides live-reload support when application files are changed. Once NestJS is running, let's open http://localhost:3000 in our web browser. We should see a blank page with the Hello World! greeting. We can also use API testing tools such as Insomnia to make a GET request to http://localhost:3000. We should get the same Hello World! greeting as our result. Let's remove this endpoint since it was only added by the Nest CLI for demo purposes. Go ahead and delete app.controller.ts, app.service.ts, and app.controller.spec.ts. Also, delete all references to AppController and AppService in app.module.ts. Creating a feature module The architectural design of NestJS encourages feature modules. This feature-based design groups the functionality of a single feature in one folder, registered in one module. This design simplifies the codebase and makes code-splitting very easy. Module We create modules in NestJS by decorating a class with the @Module decorator. Modules are needed to register controllers, services, and any other imported sub-modules. Imported sub-modules can have their own controllers, and services, registered. Let's use the Nest CLI to create the module for our blog posts. ` This gives us an empty PostsModule class in the posts.module.ts file. Interface We will use a TypeScript interface to define the structure of our JSON object that will represent a blog post. An interface is a virtual or abstract structure that only exists in TypeScript. Interfaces are used only for type-checking purposes by the TypeScript compiler. TypeScript interfaces do not produce any JavaScript code during the transpilation of TypeScript to JavaScript. Let's use the Nest CLI to create our interface. ` These commands allow us to create a posts.interface.ts file in the feature-based folder for our blog posts, which is /src/posts. The TypeScript interface keyword is used to define our interface. Let's make sure to prefix it with export to allow this interface to be used throughout our application ` From the command-line prompt, let's reset our current working directory back to the root folder of our project by using the following command. ` Service Services are classes that handle business logic. The PostsService that we will be creating will handle the business logic needed to manage our blog posts. Let's use the Nest CLI to create the service for our blog posts. ` This will give us an empty PostsService class in the posts.service.ts file. The @Injectable() decorator marks the PostsService class as a provider that we can register in the providers array of our PostsModule, and then inject into our controller class. More on this later. Controller Controllers are classes that handle incoming requests and return responses to the client. A controller can have more than one route or endpoint. Each route or endpoint can implement its own set of actions. The NestJS routing mechanism handles the routing of requests to the right controller. Let's use the Nest CLI to create the controller for our blog posts. ` This will give us an empty PostsController class in the posts.controller.ts file. Let's inject our PostsService into the constructor of the PostsController class. ` NestJS uses dependency injection to set up a reference to PostsService from within our controller. We can now use the methods provided by PostsService by referencing this.postsService from within our controller class. Registering our controller and service Let's make sure that our PostsModule registers our PostsController and PostsService. The nest generate service post and nest generate controller post commands that we ran earlier have automatically registered the PostsService and PostsController classes in the PostsModule for us. ` NestJS uses the term providers to refer to service classes, middleware, guards, and more. If the PostsService class needs to be made available to other modules within our application, we can export it from PostsModule by using the exports array. This way, any module importing the PostsModule will be able to use the PostsService. ` Importing PostsModule into AppModule We now have one cohesive and well-organized module for all the functionality related to our blog posts. However, all the functionality that will be provided by our PostsModule is not available to our application unless the AppModule imports it. If we revisit the AppModule, we can see that the PostsModule has been automatically added to the imports array by the nest generate module post command that we ran earlier. ` Adding Service and Controller logic Our PostsService and PostsController have no functionality implemented at the moment. Let's now implement our *CRUD* endpoints and their corresponding logic while adhering to RESTful standards. NestJS makes it easy to use MongoDB (using the @nestjs/mongoose package), or PostgreSQL (using *Prisma* or *TypeORM*) to persist and manage our application's data. However, for the sake of simplicity, let's create a local array in our service to mock a database. ` A service, or provider, as NestJS refers to them, can have its scope configured. By default, a service has a *singleton* scope. This means that a single instance of a service is shared across our entire application. The initialization of our service will occur only once during the startup of our application. The default singleton scope of services means that any class that injects the PostsService will share the same posts array data in memory. Get all posts Let's add a method to our PostsService that will return all of our blog posts. ` Let's add a method to our PostsController that will make the logic of the service's findAll() method available to client requests. ` The @Get decorator is used to create a GET /posts endpoint. The /posts path of the request comes from the @Controller('posts') decorator that was used in order to define our controller. Get one post Let's add a method to our PostsService that will return a specific blog post that a client may be looking for. If the id of the requested post is not found in our list of posts, let's return an appropriate 404 NOT FOUND HTTP error. ` Let's add a method to our PostsController that will make the logic of the service's findOne() method available to client requests. ` The @Get decorator is used with a parameter as @Get(':id') here to create a GET /post/[id] endpoint, where [id] is an identification number for a blog post. @Param, from the @nestjs/common package, is a decorator that makes route parameters available to us as properties in our method. @Param values are always of type string. Since we defined id to be of type number in TypeScript, we need to do a string to number conversion. NestJS provides a number of pipes that allow us to transform request parameters. Let's use the NestJS ParseIntPipe to convert the id to a number. Create a post Let's add a method to our PostsService that will create a new blog post, assign the next sequential id to it, and return it. If the title is already being used by an existing blog post, we will throw a 422 UNPROCESSABLE ENTITY HTTP error. ` Let's add a method to our PostsController that will make the logic of the service's create method available to client requests. ` The @Post decorator is used to create a POST /post endpoint. When we use the NestJS decorators for the POST, PUT, and PATCH HTTP verbs, the *HTTP body* is used to transfer data to the API, typically in JSON format. We can use the @Body decorator to parse the *HTTP body*. When this decorator is used, NestJS will run JSON.parse() on the *HTTP body* and provide us with a JSON object for our controller method. Within this decorator, we declare post to be of type Post because this is the data structure that we are expecting the client to provide for this request. Delete a post Let's add a method to our PostsService that will delete a blog post from our in-memory list of posts using the JavaScript splice() function. A 404 NOT FOUND HTTP error will be returned if the id of the requested post is not found in our list of posts. ` Let's add a method to our PostsController that will make the logic of the service's delete method available to client requests. ` Update a post Let's add a method to our PostsService that will find a blog post by id, update it with newly submitted data, and return the updated post. A 404 NOT FOUND HTTP error will be returned if the id of the requested post is not found in our list of posts. A 422 UNPROCESSABLE ENTITY HTTP error will be returned if the title is being used for another blog post. ` Let's add a method to our PostsController that will make the logic of the service's update method available to client requests. ` We use the @Put decorator to make use of the HTTP PUT request method. PUT can be used to both create and update the state of a resource on the server. If we know that a resource already exists, PUT will replace the state of that resource on the server. Testing our feature module Let's start our development server using npm run start:dev. Then, let's open the Insomnia application to test the API endpoints that we created for the PostsModule. Get all posts Let's make a GET request to http://localhost:3000/posts. The result should be a 200 OK success code with an empty array. Create a post Let's make a POST request to http://localhost:3000/posts using the following as our JSON body. ` We should get a successful 201 Created response code, along with a JSON representation of the post that was created, including the id field that has been automatically generated. Get a post Let's make a GET request to http://localhost:3000/posts/1. The result should be a successful 200 OK response code, along with the data for the post with an id of 1. Update a post Let's make a PUT request to http://localhost:3000/posts/1 using the following as our JSON body. ` The result should be a successful 200 OK response code, along with a JSON representation of the post that was updated. Delete a post Let's make a DELETE request to http://localhost:3000/posts/1. The result should be a successful 200 OK response code with no JSON object returned. Logging NestJS makes it easy to add logging to our application. Rather than using console.log() statements, we should use the Logger class provided by NestJS. This will provide us with nicely formatted log messages in the Terminal. Let's add logging to our API. The first step is to define the logger in our service class. ` With the logger now defined, we can add log statements in our service class. Here is an example of a log statement that we can add as the first line of the findAll() method in the PostsService class. ` Such statements provide us with a convenient log message in the Terminal every time a service method is called during a client's request to our API. When a GET /posts request is sent, we should see the following message in the Terminal. ` Swagger NestJS makes it easy to add the *OpenAPI* specification to our API using the NestJS *Swagger* package. The OpenAPI specification is used to describe RESTful APIs for documentation and reference purposes. Swagger setup Let's install Swagger for NestJS. ` Swagger configuration Let's update the main.ts file that bootstraps our NestJS application by adding Swagger configuration to it. ` With our NestJS app running, we can now go to http://localhost:3000/api to view the Swagger documentation for our API. Notice that default is displayed above the routes for our posts. Let's change that by adding @ApiTags('posts') right below @Controller('posts') in our PostsController class. This will replace default with posts to indicate that this set of endpoints belongs to the posts feature model. ` ApiProperty Let's make the properties of our PostModel interface visible to Swagger. We do so by annotating fields with the ApiProperty() decorator, or the ApiPropertyOptional() decorator for optional fields. To use these decorators, we must first change our interface to a class in the posts.interface.ts file. ` Within the ApiProperty() decorator applied to each field, we describe the field type. The id field is optional since we don't know of a new blog post's id when creating it. A date-time string format is used for the date field. These changes allow the PostModel structure to be documented within Swagger. With our NestJS app running, we can now go to http://localhost:3000/api to view the documentation of the PostModel. ApiResponse Let's use the @ApiResponse() decorator for Swagger to document all the possible responses from our API's endpoints. This will be helpful for informing users of our API what responses they can expect to receive by calling a given endpoint. We will be making these changes in the PostsController class. For the findAll method, let's use the @ApiOkResponse() decorator to document a 200 OK success response. ` For the findOne method, let's use the @ApiOkResponse() decorator to document a 200 OK response when the post is found. Let's use the @ApiNotFoundResponse() decorator to document a 404 NOT FOUND HTTP error when a post is not found. ` For the create method, let's use the @ApiCreatedResponse() decorator to document a 201 CREATED response when a post is created. Let's use the @ApiUnprocessableEntityResponse() decorator to document a 422 UNPROCESSABLE ENTITY HTTP error when a duplicate post title is found. ` For the delete method, let's use the @ApiOkResponse() decorator to document a 200 OK response when a post is deleted. Let's use the @ApiNotFoundResponse() decorator to document a 404 NOT FOUND HTTP error when the post to be deleted is not found. ` For the update method, let's use the @ApiOkResponse() decorator to document a 200 OK response when a post is updated. Let's use the @ApiNotFoundResponse() decorator to document a 404 NOT FOUND HTTP error when the post to be updated is not found. Let's use the @ApiUnprocessableEntityResponse() decorator to document a 422 UNPROCESSABLE ENTITY HTTP error when a duplicate post title is found. ` After saving these changes, all response codes and their related descriptions should now be listed for each endpoint on the Swagger web page at http://localhost:3000/api. We could also use Swagger instead of Insomnia to test our API. By clicking on the "Try it out" button that appears under each endpoint on the Swagger web page, we can see if they are working as expected. Exception filter Exception filters give us full control over the exceptions layer of NestJS. We can use an exception filter to add custom fields to a HTTP exception response body or to print out logs of every HTTP exception that occurs to the Terminal. Let's create a new filters folder in the /src folder, as well as a new file called http-exception.filter.ts within it. Let's add the following class in this file. ` This class makes use of the NestJS logger to print a warning message to the Terminal whenever a HTTP exception occurs. It also returns two custom fields within the response body whenever a HTTP exception occurs. The timestamp field reports when the exception occurred, and the endpoint field reports which route triggered the exception. In order to apply this filter, we need to decorate the PostsController with @UseFilters(new HttpExceptionFilter()). ` After saving these changes, NestJS will reload our application for us. If we use Insomnia to send a PUT /posts/1 request to our API, it should trigger a 404 NOT FOUND HTTP error, since no blog posts exist for updating in our application when it starts. The HTTP exception response body that is returned to Insomnia should now contain the timestamp and endpoint fields. ` We should also see the following line printed out to the Terminal. ` Summary In this article, we saw how NestJS makes back-end API development fast, simple, and effective. The NestJS application structure has helped us build a very well-organized project. We covered a lot, so let's recap what we learned: - How to use the NestJS CLI. - How to build feature modules in NestJS. - How to use services and controllers. - How to test an API with Insomnia. - How to add logging to NestJS apps. - How to use Swagger for documenting and previewing NestJS APIs. - How to get full control of HTTP exceptions in NestJS. Happy developing with NestJS! Code Visit the following GitHub repository in order to access the code featured in this article....

Sep 15, 2021

13 mins

NestJSJavaScriptNodeJS

Setting Up TypeORM Migrations in an Nx/NestJS Project

TypeORM is a powerful Object-Relational Mapping (ORM) library for TypeScript and JavaScript that serves as an easy-to-use interface between an application's business logic and a database, providing an abstraction layer that is not tied to a particular database vendor. TypeORM is the recommended ORM for NestJS as both are written in TypeScript, and TypeORM is one of the most mature ORM frameworks available for TypeScript and JavaScript. One of the key features of any ORM is handling database migrations, and TypeORM is no exception. A database migration is a way to keep the database schema in sync with the application's codebase. Whenever you update your codebase's persistence layer, perhaps you'll want the database schema to be updated as well, and you want a reliable way for all developers in your team to do the same with their local development databases. In this blog post, we'll take a look at how you could implement database migrations in your development workflow if you use a NestJS project. Furthermore, we'll give you some ideas of how nx can help you as well, if you use NestJS in an nx-powered monorepo. Migrations Overview In a nutshell, migrations in TypeORM are TypeScript classes that implement the MigrationInterface interface. This interface has two methods: up and down, where up is used to execute the migration, and down is used to rollback the migration. Assuming that you have an entity (class representing the table) as below: ` If you generate a migration from this entity, it could look as follows: ` As can be seen by the SQL commands, the up method will create the post table, while the down method will drop it. How do we generate the migration file, though? The recommended way is through the TypeORM CLI. TypeORM CLI and TypeScript The CLI can be installed globally, by using npm i -g typeorm. It can also be used without installation by utilizing the npx command: npx typeorm . The TypeORM CLI comes with several scripts that you can use, depending on the project you have, and whether the entities are in JavaScript or TypeScript, with ESM or CommonJS modules: - typeorm: for JavaScript entities - typeorm-ts-node-commonjs: for TypeScript entities using CommonJS - typeorm-ts-node-esm: for TypeScript entities using ESM Many of the TypeORM CLI commands accept a data source file as a mandatory parameter. This file provides configuration for connecting to the database as well as other properties, such as the list of entities to process. The data source file should export an instance of DataSource, as shown in the below example: ` To use this data source, you would need to provide its path through the -d argument to the TypeORM CLI. In a NestJS project using ESM, this would be: ` If the DataSource did not import the Post entity from another file, this would most likely succeed. However, in our case, we would get an error saying that we "cannot use import statement outside a module". The typeorm-ts-node-esm script expects our project to be a module -- and any importing files need to be modules as well. To turn the Post entity file into a module, it would need to be named post.entity.mts to be treated as a module. This kind of approach is not always preferable in NestJS projects, so one alternative is to transform our DataSource configuration to JavaScript - just like NestJS is transpiled to JavaScript through Webpack. The first step is the transpilation step: ` Once transpiled, you can then use the regular typeorm CLI to generate a migration: ` Both commands can be combined together in a package.json script: ` After the migrations are generated, you can use the migration:run command to run the generated migrations. Let's upgrade our package.json with that command: ` Using Tasks in Nx If your NestJS project is part of an nx monorepo, then you can utilize nx project tasks. The benefit of this is that nx will detect your tsconfig.json as well as inject any environment variables defined in the project. Assuming that your NestJS project is located in an app called api, the above npm scripts can be written as nx tasks as follows: ` The typeorm-generate-migration and typeorm-run-migrations tasks depend on the build-migration-config task, meaning that they will always transpile the data source config first, before invoking the typeorm CLI. For example, the previous CreatePost migration could be generated through the following command: ` Conclusion TypeORM is an amazing ORM framework, but there are a few things you should be aware of when running migrations within a big TypeScript project like NestJS. We hope we managed to give you some tips on how to best incorporate migrations in an NestJS project, with and without nx....

Feb 22, 2023

4 mins

NxNestJS

OAuth2 for JavaScript Developers

OAuth2 for JavaScript Developers OAuth is a staple of modern web development. It's the magic behind the "Log in with Facebook" or "Connect to Google" buttons you see everywhere. But what exactly is it, and how does it work, especially for a JavaScript developer? Let's dive in, using GitHub as our primary example. What is OAuth? At its core, OAuth (Open Authorization) is a protocol for authorization. It allows third-party applications to access user data without needing the user's password. Instead of users sharing their password with an app, they are granted a token that the app can use to act on their behalf. This offers several benefits; the most significant is that the app doesn't need to store user passwords, which greatly reduces potential attack vectors. Although OAuth is often used interchangeably with OAuth2, they are distinct. OAuth 1.0, introduced in 2007, is the first version of the protocol. While it provided a solid method for consumers to access protected resources without user credentials, it had complexities, such as the need for cryptographic libraries for request signing. Recognizing these challenges, the community introduced OAuth2 in 2012. OAuth2 simplified many aspects, eliminating the need for cryptographic signatures and emphasizing bearer tokens, which are easier to manage. OAuth2 provides flexibility with various methods of obtaining access tokens suitable for different application types. OAuth2 Steps Explained Using GitHub To explain how OAuth2 works, it's best to describe the entire process step-by-step. It's challenging to do this without code, so we'll reference our starter.dev backend showcase, which illustrates integrating a serverless backend with GitHub as an OAuth2 provider. Although the steps below are tailored to GitHub's OAuth2 implementation, it's important to note that the core flow is consistent across most OAuth2 services. For most of these services, you'll only need to adjust elements like URLs, query parameter names, and so on. Also, please note that our starter.dev backend showcase is designed to be deployed on Netlify. As such, all the API endpoints run as Netlify functions. When running the app locally, you must prepend the URL path with /.netlify/functions/server to access these endpoints. This step is unnecessary for the deployed version; API endpoints can be accessed directly (e.g., /api/auth/signin instead of /.netlify/functions/server/api/auth/signin) due to the redirects configured in the netlify.toml file. Step 0: Creating the App on GitHub The first step is to create your app with the provider. For GitHub, this is done on the Settings / Developer Settings / OAuth Apps page. This step is essential so that the provider can identify your app and possibly manage service-level parameters, like rate limits. In the screenshot above, when registering our app on GitHub, the authorization callback URL is a local URL. This is suitable for local development. However, in production, it should be a public API endpoint accessible to GitHub, like an API endpoint on Netlify. After registering the app, you'll receive a public client ID for your app and will need to generate a client secret (which should stay private). The client secret, used alongside the client ID, helps authenticate your application when trading an authorization code for an access token. Essentially, the client secret assures GitHub that the request for an access token genuinely comes from your app and not a malicious actor. Step 1: App Sends User to GitHub for Authorization The first step in the OAuth2 flow typically involves the user clicking a "Log in with GitHub" button on our web app or something similar. This action redirects the user from the web app to the OAuth2 provider's page (in this case, GitHub). For GitHub, the base URL for such a page is https://github.com/login/oauth/authorize, followed by query parameters that provide more details about the app. These parameters usually include the client ID — a mandatory identifier for your app — and several optional parameters, such as the redirect URI (where GitHub will send the user back) and scope (defining the permissions you're requesting). In our starter.dev example, clicking the "Log in with GitHub" button should lead the user to the /api/auth/signin route in our app. This route generates the GitHub authorization URL, populating it with all the necessary query parameters, and then redirects the user to that URL using a standard HTTP 303 redirect. ` Step 2: User Grants Permission In this step, the user arrives at the provider's authorization page and sees a prompt asking if they allow your app to access the specified data. The user can either approve or decline this request. For our scenario, this page would appear as follows: If either the cancel or authorize button is clicked, GitHub will invoke the callback URL specified when registering the app on GitHub. However, if the authorize button is clicked, GitHub will also send a one-time-use code query parameter that can be used to get the access token. Step 3: App Receives an Authorization Code After the user grants permission, GitHub redirects them to the specified callback URL. Attached to this URL is a one-time-use code query parameter. ` Step 3: App Exchanges Authorization Code for Access Token With this code, the app needs to make a POST request to the GitHub API to exchange the code for an access token. As shown below, the app sends the client ID, client secret, and the one-time code to GitHub and receives the access token in return. The access token should then be persisted in a store (such as Redis) or sent to the frontend as a cookie, as shown below: ` Step 4: App Accesses User Data with the Access Token Once you have the access token, you can use it to make authorized API requests on behalf of the user. All the app needs to do is include the access token in the Authorization header. ` And that is all. As long as the access token is valid, you can access any of the API services that were granted by the user in the authorization step. Conclusion OAuth2, though appearing complex initially, provides a strong method for authentication and authorization. For JavaScript developers, knowing its process helps in easy third-party integrations, giving users a safe way to access different services. Whether using GitHub or another platform, the main concepts are the same, allowing for better web applications. Check out our starter.dev backend showcase for the full code! There's also an associated StackBlitz project if you want to play with it....

Jan 19, 2024

5 mins

GitHubJavaScriptOAuth 2

Transforming Platform Engineering Through Chargeback Programs with Shuchi Mittal

Shuchi Mittal, the Head of Cloud enablement at Honeywell, discusses how she as a leader in platform engineering has been able to transform internal platform engineering teams at other organizations by providing teams with value-added services, and how they effectively managed costs. She talks about how to treat these teams as customers, and delivering services that meet the customer needs, but also charging them for their usage. By providing policy-compliant infrastructure and unique services tailored to their requirements, platform engineering teams can showcase their commitment to supporting the success of other teams within an organization. This approach not only fosters trust but also positions platform engineering as a strategic partner rather than just a service provider. By going beyond the basic infrastructure provisioning, the platform engineering team can offer services that help development and product teams streamline their processes and improve efficiency. Shuchi shares her work at Fiserv where she implemented a chargeback system to track usage and costs effectively, incentivizing better development practices. This not only helped in managing costs but also encouraged teams to optimize their resource utilization, leading to improved overall efficiency and more easily scalable systems. Her platform engineering team recognized the importance of effectively managing financial aspects to secure upfront investment for product development. By implementing a billing system based on gigabyte hours, they aligned costs with usage, enabling teams to have a clear understanding of their resource consumption. This approach not only provided transparency but also incentivized teams to adopt cost-effective practices. By strategically managing financial aspects, the platform engineering team gained the trust of stakeholders and secured the necessary resources to drive innovation and deliver value to the organization. Shuchi’s journey of platform engineering serves as a valuable example of how a team can transition from being a basic service provider to becoming an innovation partner within an organization. By building trust, offering value-added services, and strategically managing financial aspects, the platform engineering team successfully elevated their role and became a strategic enabler of innovation. Download this episode here....

Apr 16, 2024

2 mins

Engineering LeadershipSoftware Engineering

Combining Validators and Transformers in NestJS

Using ValidationPipe

Other transformation options

Conclusion

Dario Djuric

You might also like

Introduction to RESTful APIs with NestJS

Setting Up TypeORM Migrations in an Nx/NestJS Project

OAuth2 for JavaScript Developers

Transforming Platform Engineering Through Chargeback Programs with Shuchi Mittal

You might also like

Introduction to RESTful APIs with NestJS

Setting Up TypeORM Migrations in an Nx/NestJS Project

OAuth2 for JavaScript Developers

Transforming Platform Engineering Through Chargeback Programs with Shuchi Mittal