How to automatically deploy your full-stack JavaScript app with AWS CodePipeline
In our previous blog post we set up a horizontally scalable deployment for our full-stack javascript app. In this article we would like to show you how to set up AWS CodePipeline to automatically deploy changes to the application....
Sep 15, 2023
9 mins
How to host a full-stack app with AWS CloudFront and Elastic Beanstalk
You have an SPA with a NestJS back-end. What if your app is a hit? You need to be prepared to serve thousands of users? You might need to scale your API horizontally, which means you need to have more instances running behind a load balancer....
Sep 11, 2023
12 mins
BullMQ with ExpressJS
Node.js uses an event loop to process asynchronous tasks. The event loop is responsible for handling the execution of asynchronous tasks, but it does it in a single thread. If there is some CPU-intensive or long-running (blocking) logic that needs to be executed, it should not be run on the main thread. This is where BullMQ can help us. In this blog post, we are going to set up a basic queue, using BullMQ. If you'd like to jump right into developing with a queue, check out our starter.dev kit for ExpressJS, where a fully functioning queue is already provided for you. Why and when to use a queue? BullMQ is a library that can be used to implement message queues in Node.js applications. A message queue allows different parts of an application, or different applications, to communicate with each other asynchronously by sending and receiving messages. This can be useful in a variety of situations, such as when one part of the application needs to perform a task that could take a long time, or when different parts of the application need to be decoupled from each other for flexibility and scalability. If you need to process large numbers of messages in a distributed environment, it can help you solve your problems. One such scenario to use a queue is when you need to deal with webhooks. Webhook handler endpoints usually need to respond with 2xx quickly, therefore, you cannot put long running tasks inside that handler, but you also need to process the incoming data. A good way of mitigating this is to put the incoming data into the queue, and respond quickly. The processing gets taken care of with the BullMQ worker, and if it is set up correctly, it will run inside a child process not blocking the main thread. Prerequisites BullMQ utilizes Redis to handle its message queue. In development mode, we are using docker-compose to start up a redis instance. We expose the redis docker container's 6379 port to be reachable on the host machine. We also mount the /misc/data and the misc/conf folders to preserve data for our local development environments. ` We can start up our infrastructure with the docker-compose up -d command and we can stop it with the docker-compose stop command. We also need to set up connection information for BullMQ. We make it configurable by using environment variables. ` To start working with BullMQ, we also need to install it to our node project: ` Setting up a queue Creating a queue is pretty straightforward, we need to pass the queue name as a string and the connection information. ` We also create a function that can be used to add jobs to the queue from an endpoint handler. I suggest setting up a rule that removes completed and failed jobs in a timely manner. In this example we remove completed jobs after an hour from redis, and we leave failed jobs for a day. These values depend on what you want to achieve, It can very well happen that in a production app, you would keep the jobs for weeks. ` It would be called when a specific endpoint gets called. ` The queue now can store jobs, but in order for us to be able to process those jobs, we need to set up a worker. Put the processing into a thread We set up a worker with an async function at the beginning. The worker needs the same name as the queue to start consuming jobs in that queue. It also needs the same connection information as the queue we set up before. ` After we create the worker and set up event listeners, we call the setUpWorker() method in the queue.ts file after the Queue gets created. Let's set up the job processor function. ` This example processor function doesn't do much, but if we had a long running job, like complex database update operations, or sending data towards a third-party API, we would do it here. Let's make sure our worker will run these jobs on a separate thread. ` If you provide a file path to the worker as the second parameter, BullMQ will run the function exported from the file in a separate thread. That way, the main thread is not used for the CPU intense work the processor does. The above example works if you run the TypeScript compiler on your back-end code (tsc), but if you prefer keeping your code in TypeScript and run the logic with ts-node, then you should use the TypeScript file as your processor path. ` The problem with bundlers Sometimes, your back-end code gets bundled with webpack. For example, if you use NX to keep your front-end and back-end code together in one repository, you will notice that your back-end code gets bundled with webpack. In order to be able to run your processors in a separate thread, you need to tweak your project.json configuration: ` Conclusion In an ExpressJS application, running CPU intense tasks on the main thread could cause your endpoints to turn unresponsive and/or slow. Moving these tasks into a different thread can help alleviate performance issues, and using BullMQ can help you out greatly. If you want to learn more about NodeJS, check out node.framework.dev for a curated list of libraries and resources. If you are looking to start a new ExpressJS project, check out our starter kit resources at starter.dev...
Feb 24, 2023
4 mins
Introducing the express-typeorm-postgres Starter Kit
At This Dot, we've been working with ExpressJS APIs for a while, and we've created a starter.dev kit for ExpressJS that you can use to scaffold your next backend project....
Jan 13, 2023
5 mins
Performance Analysis with Chrome DevTools
When it comes to performance, developers often use Lighthouse or similar performance analysis tools. But when the target site has protection against bots, getting information is not that simple. Chrome Devtools can help you with your performance analysis....
Jan 9, 2023
6 mins
Introducing @this-dot/rxidb
When we are working on PWAs, sometimes we need to implement features that require us to store data on our user's machine. One way to do that is to use IndexedDb. Our team at This Dot has developed @this-dot/rxidb to create an RxJS wrapper around it....
Oct 13, 2022
6 mins
How to Set Up OAuth with a Stripe App
Stripe Apps are a great way to extend Stripe dashboard functionality using third-party integrations. But when using these integrations, developers must prioritize security. The best way to do this is by using OAuth with the third-party product with which you would like to integrate. However, there are some constraints, and we cannot use cookies to set up a cookie based authentication front-end only. In this article, we would like to show you how to do it with a NestJS back-end. Stripe signatures The best way to secure your API is by making sure that every request comes from a verified Stripe App instance. The @stripe/ui-extension-sdk package provides a way to generate a signature on the front-end side. This signature is valid for 5 minutes, and you can send it as a header for every request you make. For this to work, you need to have @stripe/ui-extension-sdk installed in your repository. ` In order to properly validate this signature on your API, you will need some additional information to be sent in the request headers as well. That information is the Stripe user's ID, and the Stripe account's ID. We found that the best way is to implement a global context with this information. ` The above context stores the ExtensionContextValue that gets passed from the Stripe dashboard to the app when it opens in the view. For example, if you are on a payment detail page, the userContext will contain information about your Stripe user, while the environment will provide you access to the object that you are viewing. In the above example, that would be the payment's ID as the objectContext.id property. Let's set up the view with this global context. ` Now, we can set up a hook to provide a proper fetch method that always appends a Stripe signature, and the other required fields to the headers. useFetchWithCredentials hook In order to make our future job easier, we need to set up a hook that creates a proper wrapper around fetch. That wrapper will handle setting the headers for us. It needs to have access to our GlobalContext, so we can get the Stripe user's, and their account's, IDs. ` Let's set up a very basic component for demonstrating the use of the useFetchWithCredentials hook. This component will be the default route for our app's navigation wrapper. It is going to handle more later. But for now, let's just implement a basic use for our hook. The AUTH_INIT_URL constant will point at our back-end's /api/oauth/userinfo endpoint. Please note that, for this to work, you are going to need to install react-router-dom. ` As we can see from the above implementation, this component will be the initial component that gets rendered inside of the application. It will send out a request to determine if the user is logged in. If they are logged in, we are going to send them to a route that is the first page of our application. If they are not signed in, we are going to redirect them to our login page. This initial call, just as every other API call, must be verified and always have a Stripe signature. Let's visualise how routing looks like right now: ` Stripe secrets and the Stripe API In order to be able to use the Stripe NodeJS Api, you will need two secrets from Stripe. One is your Stripe account's API key, and the other one is your Stripe-app's secret. You need to set up your .env file as the following. ` Stripe API key You can find your Stripe API key at https://dashboard.stripe.com/apikeys, under the Standard keys section. The key you are looking for is called Secret key, and you need to reveal it by clicking the button that hides it. Stripe App Secret For this key, you are going to need to upload your stripe-app using the stripe apps upload command. Make sure that you set a development app ID in your app manifest (stripe-app.json). After you uploaded your app, visit https://dashboard.stripe.com/apps. Under My Apps, you should see your uploaded application. Open it and search for the Signing secret. Reveal it and copy it into your .env file. Stripe NodeJS API Please make sure you have installed the stripe nmp package for your server code. In this example series, we use NestJS as our framework for our API. We need the above two secret keys to be able to start up our Stripe API. ` NestJS VerifySignatureInterceptor implementation In NestJS, we can use interceptors to abstract away repetitive logic that needs to be done on multiple requests. In our case, we need to verify almost every API for a valid Stripe signature. We have access to the proper secret keys, and we have a Stripe NodeJS API set up. Let's create our VerifySignatureInterceptor. ` Every interceptor must implement the intercept() method. We extract the Request object from the execution context, and we get the headers that we previously set in our useFetchWithCredentials hook. We call our verifySignature function which will throw errors if the signature is invalid. We also pass the Logger instance, so we can determine when an error comes from this interceptor in our logs. Please be aware that there are several reasons signature verification can go wrong, like if we provide the wrong Stripe account keys or app secrets. In order for you to be able to easily debug these issues, proper logging is a must. That is why we set up a Logger instance in our interceptor. ` If the user_id, account_id, or the signature are missing, that could mean that the request came from outside a stripe application, or the useFetchWithCredentials hook was not used. We throw a BadRequestException that will result in the request sending back a status: 400 HTTP response. If the signature verification fails, that could mean that a not valid signature was used in the request, or that the API environment variables might have the wrong keys. Set up the userinfo endpoint Let's quickly set up our /api/oauth/userinfo endpoint. For that, we are going to create the OauthModule and the OauthController. ` In our controller, we decorate our getUserInfo() method, with the @Get() decorator, so we set up the route. We also decorate the method with the @UseInterceptors() decorator, where we pass our VerifySignatureInterceptor. ` This setup will enable us to call the /api/oauth/userinfo endpoint which will, in-turn, check if we have a valid signature present in the headers. If the request is invalid, it will throw a 400 Bad Request exception. If the signature is valid, for now, we will throw a 401 Unauthorized exception just to make our front-end navigate to the login page. The Login flow Just to keep this example simple, our login page will only have a button in the center that will start our login flow with our API. ` We need to create a state key, that can be validated before we fetch the token. This state key will first be sent to our third-party oauth client, and it will be returned to us when the authentication is finished. This key is passed securely and over https. Therefore, it can be a stringified object. While the key is not set, we disable the button. ` Pressing the Sign in button will call our API that will redirect us to our third-party login screen. When the login happens, it will redirect us to our API, where we can fetch a valid token and redirect again to the Stripe dashboard. Let's extend our environment variables. ` Now that we have every environment variable set up, let's implement our api/oauth/login and api/oauth/authorise endpoints in our OauthController. ` The login endpoint, if everything is correct, redirects us to the login page where the user should be able to log in. Make sure that if you oauth client needs to have configured redirect urls, you configure them. For example, for development, the http://localhost:3333/api/oauth/authorise endpoint should be in the allowed redirect url list. ` We validate everything to be sure that this endpoint was called from our third-party OAuth page. With the information available to us, we can fetch the access token and store it in the Stripe Secret Storage. In this example, we use axios in our bakc-end to send requests to our third-party API. ` We exchange our code returned from our OAuth client to a valid access token, and then store it in the Stripe Secret Store. That logic got extracted into a SecretService class, because the logic implemented in it can be reused later for other API calls. Please make sure you set up a NestJS module that exports this service. Stripe Secret Store Stripe's Secret Store API enables your app to securely store and retrieve strings that can be authentication credentials, tokens, etc. This API enables users to stay logged in to third party services even when they log out of their Stripe dashboard. Let's set up a service that handles access to the Secret Store on our back-end. ` Adding secrets As we can see above, the Secret Storage needs some preliminary setup, which we do in our SecretService. The StripeResource sets up the find, set, and delete methods on the Stripe Api, and interacts with the Secret Store. Let's implement the addSecret method, so we can actually store our returned token. ` With the above, we can finally store our token with which we can make authenticated requests. Getting secrets Let's implement the getSecret so we can retrieve secrets. The principles are the same. We will need the accountId, the userId, and the secret's name for it. ` Let's close the login flow, and implement the final version of the api/oauth/userinfo endpoint. ` Deleting secrets We want our users to have ability to log out from our third-party API as well. That can be achieved by deleting their access_token from the Secret store. ` The /api/oauth/logout endpoint is going to be a GET request, that will delete the token from the Secret Store. ` We can create a SignOutLink that will send the request to our back-end and navigates to the /login page. You can put this component into the footerContent property of your ContextView. ` And now we are ready with our authentication setup. When the user opens our app, it will call the /api/oauth/userinfo endpoint. Initially, it will return with a 401 error, and our front-end will navigate to the /login route. When the user presses the Sign in button, it will redirect them to the third-party OAuth page. After they log in, our back-end also redirects them back to their Stripe dashboars where the application will open. The app will call the /api/oauth/userinfo endpoint again. But this time, it will return an actual user information and it routes to the protected route. To help visualize the whole flow, you can also use the following sequence diagram for reference: Conclusion As you can see, there are many steps involved in setting up a proper OAuth flow. However, it's necessary to make it right, since this is the most critical part of the app. We hope blog post article will help you to set up some good foundations when implementing your own Stripe app....
May 24, 2022
8 mins
Cypress testing your IndexedDb contents with @this-dot/cypress-indexeddb
There are several use-cases where an application uses indexedDb to store data locally. Accessing indexedDb during tests is not that straightforward, that's why we created this cypress helper library. Introducing @this-dot/cypress-indexeddb...
Jan 6, 2022
4 mins
Getting Authenticated Images in Angular
Sometimes some applications need to display images on a page, which is only accessible with authentication. But using simply the src attribute of an tag will only accept images that are not authenticated. How can you circumvent this issue?...
Sep 9, 2021
5 mins
Introducing our first open-source library: @this-dot/route-config
While we were working on an internal project, we found we were able to solve an issue that is common during application development. We love to give back to the community. Therefore, we've decided to open-source it....
Aug 9, 2021
4 mins
How to set up screenshot comparison testing with cypress inside an NX workspace
Sometimes we need to refactor an application which can break the layout. Screenshot comparison tests can provide you with a secure safety net. In this blog post, I'd like to show you how to set up such tests in an NX monorepo....
Aug 6, 2021
6 mins
NX e2e testing with AWS Amplify
CI/CD helps you automate most of your testing and build processes. AWS Amplify can help you with that, but sometimes it is not that straightforward. In this guide, we show you how to set up e2e testing with an NX workspace using Amplify build tools....
May 5, 2021
4 mins