AWS

Performing a Migration in AWS Amplify

Jan 7, 2022

4 min read

Back-end migrations can be a tricky task if you're going in blind, and it may take a few attempts before you know the ins and outs of a specific technology being used. Those with experience in AWS will understand that sometimes things don't always go according to plan when performing even the most common tasks in Amplify, and migration of an app from one app ID to another can seem like a huge undertaking.

In this guide, we'll look at how to:

Reference a new app
Create a new back-end environment for the migration
And verify our deployment works

This guide assumes you have an existing application created.

Before Attempting a Migration

Migration can be seen as a refactor of sorts, except it's the entire project that needs to move and work. In any refactoring effort, taking into account all the moving parts is key. But with Amplify, these parts are all tied to policies while creating the environment. What this means for Amplify is that with an existing app, we have the option to use already generated policies or perform a manual shifting of resources.

With the first option, policies and references are already mapped for how the BE would rebuild. However, in option two, there may not be an easy transition of resources especially if the application is large and/or is being upgraded as part of the migration processes. In the case of an older application, it may be easier to manually migrate because the upgrade process might not take into account some older pattern.

We'll use the first option of a simpler migration by referencing the new app id.

Initial Setup

Amplify behaves similarly to git. To get started, first follow a git-like workflow by fetching the latest:

$ amplify pull

It's recommended to commit the latest stable code to your source control of choice in case the migration goes wrong.

In the new AWS account, where this app will live, we need to create a space for it to live. Thankfully, Amplify docs give a step-by-step guide on configuring a new app space.

Referencing the New AWS App

At this point, if you've created a new app in a new account, then there should be a default staging environment. We won't use that one because we want at least one working, clean environment in case we need to start over (by deleting the environment).

Clear old environments

Take a look at the contents of team-provider-info.json and look for AmplifyAppId under any existing environments. Depending on the current environment selected, Amplify Actions will be performed against that environment at that app id.

This isn't useful to use because generating a new develop environment will have:

A name conflict in this file
New environments created in the same app

If you used the wrong credentials creating the env, a new app under the old account will be created at the new app id.

Generate an env at the new app id

Typically we'd perform amplify env add to add an environment to an existing app. However, we need to reference the new app. So we'll initialize a new app instead:

$ amplify init --appId <Amplify-Service-Project-AppId>

Found in the Amplify Docs for commands amplify-cli here

The Amplify-Service-Project-AppId can be found by:

Navigating to AWS Amplify
Selecting the app
Selecting App settings > General
Selecting the end part of the App ARN

Because there won't be environments to read from team-provider-info.json, it won't prompt for using an existing environment. Follow the interactive prompts for your configuration needs.

Verify the new environment and push

To verify that everything went well, look again AmplifyAppId in team-provider-info.json under the new environment. This should match with the Amplify Console.

Now we push:

$ amplify push

Verify Successful Deployment

All's that's left now is to wait while Amplify recreates the policies for all your back-end parts. If it all went well, you should have no failures in the terminal, and see the deployment success in Amplify Console with references to your API, functions, etc. like:

If you have a front-end, re-attach the source control of choice.

Cleanup

The final step after everything is working is to remove references to the old application. We did this partially by deleting old environments in team-provider-info.json. Now we have to delete the old app from the old account, and we're done!

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Morgan Worrell

Senior Software Engineer and Affiliate Professor empowering developers to achieve their career goals. When he’s not creating online course content, you can find him playing volleyball or music and working on 3D print projects. Specialized tech: Angular

@morgnism_@morgnism

How to automatically deploy your full-stack JavaScript app with AWS CodePipeline

How to automatically deploy your full-stack JavaScript app from an NX monorepo with AWS CodePipeline In our previous blog post (How to host a full-stack JavaScript app with AWS CloudFront and Elastic Beanstalk) we set up a horizontally scalable deployment for our full-stack javascript app. In this article, we would like to show you how to set up AWS CodePipeline to automatically deploy changes to the application. APP Structure Our application is a simple front-end with an API back-end set up in an NX monorepo. The production built API code is hosted in Elastic Beanstalk, while the front-end is stored in S3 and hosted through CloudFront. Whenever we are ready to make a new release, we want to be able to deploy the new API and front-end versions to the existing distribution. In this article, we will set up a CodePipeline to deploy changes to the main branch of our connected repository. CodePipeline CodeBuild and the buildspec file First and foremost, we should set up the build job that will run the deploy logic. For this, we are going to need to use CodeBuild. Let's go into our repository and set up a build-and-deploy.buildspec.yml file. We put this file under the tools/aws/ folder. ` This buildspec file does not do much so far, we are going to extend it. In the installation phase, it will run npm ci to install the dependencies and in the build phase, we are going to run the build command using the ENVIRONMENT_TARGET variable. This is useful, because if you have more environments, like development and staging you can have different configurations and builds for those and still use the same buildspec file. Let's go to the Codebuild page in our AWS console and create a build project. Add a descriptive name, such as your-appp-build-and-deploy. Please provide a meaningful description for your future self. For this example, we are going to restrict the number of concurrent builds to 1. The next step is to set up the source for this job, so we can keep the buildspec file in the repository and make sure this job uses the steps declared in the yaml file. We use an access token that allows us to connect to GitHub. Here you can read more on setting up a GitHub connection with an access token. You can also connect with Oauth, or use an entirely different Git provider. We set our provider to GitHub and provided the repository URL. We also set the Git clone depth to 1, because that makes checking out the repo faster. In the Environment section, we recommend using an AWS CodeBuild managed image. We use the Ubuntu Standard runtime with the aws/codebuild/standard:7.0 version. This version uses Node 18. We want to always use the latest image version for this runtime and as the Environment type we are good with Linux EC2. We don't need elevated privileges, because we won't build docker images, but we do want to create a new service role. In the Buildspec section select Use a buildspec file and give the path from your repository root as the Buildspec name. For our example, it is tools/aws/build-and-deploy.buildspec.yml. We leave the Batch configuration and the Artifacts sections as they are and in the Logs section we select how we want the logs to work. For this example, to reduce cost, we are going to use S3 logs and save the build logs in the aws-codebuild-build-logs bucket that we created for this purpose. We are finished, so let's create the build project. CodePipeline setup To set up automated deployment, we need to create a CodePipeline. Click on Create pipeline and give it a name. We also want a new service role to be created for this pipeline. Next, we should set up the source stage. As the source provider, we need to use GitHub (version2) and set up a connection. You can read about how to do it here. After the connection is set up, select your repository and the branch you want to deploy from. We also want to start the pipeline if the source code changes. For the sake of simplicity, we want to have the Output artefact format as CodePipeline default. At the Build stage, we select AWS CodeBuild as the build provider and let's select the build that we created above. Remember that we have the ENVIRONMENT_TARGET as a variable used in our build, so let's add it to this stage with the Plaintext value prod. This way the build will run the build:prod command from our package.json. As the Build type we want Single build. We can skip the deployment stage because we are going to set up deployment in our build job. Review our build pipeline and create it. After it is created, it will run for the first time. At this time it will not deploy anything but it should run successfully. Deployment prerequisites To be able to deploy to S3 and Elastic Beanstalk, we need our CodeBuild job to be able to interact with those services. When we created the build, we created a service role for it. In this example, the service role is codebuild-aws-test-build-and-deploy-service-role. Let's go to the IAM page in the console and open the Roles page. Search for our codebuild role and let's add permissions to it. Click the Add permissions button and select Attach policies. We need two AWS-managed policies to be added to this service role. The AdministratorAccess-AWSElasticBeanstalk will allow us to deploy the API and the AmazonS3FullAccess will allow us to deploy the front-end. The CloudFrontFullAccess will allow us to invalidate the caches so CloudFront will send the new front-end files after the deployment is ready. Deployment Upload the front-end to S3 Uploading the front-end should be pretty straightforward. We use an AWS CodeBuild managed image in our pipeline, therefore, we have access to the aws command. Let's update our buildspec file with the following changes: ` First, we upload the fresh front-end build to the S3 bucket, and then we invalidate the caches for the index.html file, so CloudFront will immediately serve the changes. If you have more static files in your app, you might need to invalidate caches for those as well. Before we push the above changes up, we need to update the environment variables in our CodePipeline. To do this open the pipeline and click on the Edit button. This will then enable us to edit the Build stage. Edit the build step by clicking on the edit button. On this screen, we add the new environment variables. For this example, it is aws-hosting-prod as Plaintext for the FRONT_END_BUCKET and E3FV1Q1P98H4EZ as Plaintext for the CLOUDFRONT_DISTRIBUTION_ID Now if we add changes to our index.html file, for example, change the button to HELLO 2, commit it and push it. It gets deployed. Deploying the API to Elastic Beanstalk We are going to need some environment variables passed down to the build pipeline to be able to deploy to different environments, like staging or prod. We gathered these below: - COMMIT_ID: #{SourceVariables.CommitId} - This will have the commit id from the checkout step. We include this, so we can always check what commit is deployed. - ELASTIC_BEANSTALK_APPLICATION_NAME: Test AWS App - This is the Elastic Beanstalk app which has your environment associated. - ELASTIC_BEANSTALK_ENVIRONMENT_NAME: TestAWSApp-prod - This is the Elastic Beanstalk environment you want to deploy to - API_VERSION_BUCKET: elasticbeanstalk-us-east-1-474671518642 - This is the S3 bucket that was created by Elastic Beanstalk With the above variables, we can make some new variables during the build time, so we can make sure that every API version is unique and gets deployed. We set this up in the install phase. ` The APP_VERSION variable is the version property from the package.json file. In a release process, the application's version is stored here. The API_VERSION variable will contain the APP_VERSION and as a suffix, we include the build number. We want to upload this API version by indicating the commit ID, so the API_ZIP_KEY will have this information. The APP_VERSION_DESCRIPTION will be the description of the deployed version in Elastic Beanstalk. Finally, we are going to update the buildspec file with the actual Elastic Beanstalk deployment steps. ` Let's make a change in the API, for example, the message sent back by the /api/hello endpoint and push up the changes. --- Now every time a change is merged to the main branch, it gets pushed to our production deployment. Using these guides, you can set up multiple environments, and you can configure separate CodePipeline instances to deploy from different branches. I hope this guide proved to be helpful to you....

Sep 15, 2023

9 mins

AWSNxNodeJSJavaScript

Avoiding Null and Undefined with NonNullable<T> in TypeScript

Use Cases Use Case 1: Adding Two Numbers Scenario: A function that adds two numbers and returns their sum. But if one of the numbers is undefined or null, it returns the other number as-is. ` In this code snippet, the addNumbers() function takes two parameters, a and b. a is a required parameter of type number, while b is an optional parameter of type number or null. The function uses the NonNullable<T> conditional type to ensure that the return value is not null or undefined. If b is null or undefined, the function returns the value of a. Otherwise, it adds a and b together and returns the sum. To handle the case where b is null or undefined, the code uses the nullish coalescing operator, ??, which returns the value on its left-hand side if it is not null or undefined, and the value on its right-hand side otherwise. Use Case 2: Checking Contact Information Scenario: A class representing a person, but with optional email and phone properties. The contact() function logs the email and phone numbers if they are defined and not null. Otherwise, it logs a message saying that no contact information is available. ` Explanation: In this code snippet, the Person class has a name property and optional email and phone properties. The contact() function checks if both email and phone are not undefined and not null before logging the details. Otherwise, it logs a message saying that no contact information is available. To initialize the properties with null, the constructor uses the nullish coalescing operator, ??. When creating a new Person, you can pass null or undefined as arguments, and the class will interpret them as null. Conclusion Understanding and correctly implementing conditional types like NonNullable<T> in TypeScript is crucial to reduce potential code pitfalls. By reviewing examples of numerical operations and contact information handling, we've seen how this conditional type helps reinforce our code against null or undefined values. This highlights the utility of TypeScript's conditional types not only for enhancing code stability, but also for easing our coding journey. So keep experimenting and finding the best ways to deploy these tools for creating robust, secure, and efficient programs!...

Jun 9, 2023

2 mins

NextJSAWS

Using Notion as a CMS

Unlike most note-taking applications, Notion features a full suite of notation-like tools, from tools to track tasks to ones that help with journaling, all in one place. If you're an experienced user, you may have used some of their advanced concepts like tables as databases, custom templates, and table functions. As of May 13, 2021, Notion released a beta version of their API client, opening up another range of possibilities for the app. Users of the client can query databases, create pages, update blocks, perform searches, and much more. We'll look at how easy it is to get started with the client today. _Note: For this simple demonstration, we won't use authentication, but it's highly recommended no matter what type of app you decide to create._ Project Setup We have a few options for building a scalable blog site. For this stack, we'll be using NextJS because it's relatively lightweight compared to most frameworks, and has several readily available features for a standard React app. But Remix works just as well. In a folder of your choice, create the NextJS app with: ` or ` _Note: add --typescript at the end to generate a TypeScript project._ I'm using the TypeScript project for this demo. If you're unfamiliar with NextJS, I recommend their Getting Started Guide. In this new project folder, we have all the necessary things to run the app. Install the necessary dependencies: ` Create the Notion Database For this next step and the next, you'll need a Notion account to create databases, and an integration key. Now, create a table for your blog posts in Notion. This will become the database for referencing posts for the site. I'm making this generic developer blog database, but feel free to make your database specific to a target audience or subjects. Notion Setup and Integration Before using Notion's API to query any data, we need access via an integration key. Head over to the Notion's authorization guide and follow the instructions to create your key. I'm using the most basic setup for reading from my database. Continue with the integration key guide up to Step 2 which references how to grant the intergration key rights to the database. With the integration key and database ID handy, let's configure the app to use them. Querying the database In your favourite code editor, create an .env.local file with the following: ` _Note: it won't matter if you wrap your values in quotations._ NextJS comes with the ability to read local environment variables. It also ignores versions of this file in its .gitignore so we don't have to. _If publishing an app to production, it's recommended to use environment variables._ Next, create a src/api/client.ts file at the root of the project. This will contain an easily referenced version of our client for querying Notion. ` Follow that up with a src/api/query-database.ts file: ` Parse the Query Because the returned query object is so complex, we need to parse it. There are better ways to do this using more advanced techniques, but basic mapping will suffice. In a new src/utils/parse-properties file: ` Now we can leverage NextJS server rendering feature via getStaticProps to prefetch and render our Home page. In index.tsx: ` We should now see our post titles loaded on the Home page when we run npm run dev. Finishing Up With a few setup pieces, it's easy to setup a basic blog using Notion's API, but there are more possibilities and use cases for Notion as a CMS. Keep in mind that this may not be the best database to use in a production environment, but playing around with one of my favourite tools creates some new possibilies for non-Notion-tailored experiences. Full Code Example Here...

Apr 27, 2022

4 mins

JavaScript

End-to-end type-safety with JSON Schema

End-to-end type-safety with JSON Schema I recently wrote an introduction to JSON Schema post. If you’re unfamiliar with it, check out the post, but TLDR: It’s a schema specification that can be used to define the input and output data for your JSON API. In my post, I highlight many fantastic benefits you can reap from defining schemas for your JSON API. One of the more interesting things you can achieve with your schemas is end-to-end type safety from your backend API to your client application(s). In this post, we will explore how this can be accomplished slightly deeper. Overview The basic idea of what we want to achieve is: * a JSON API server that validates input and output data using JSON Schema * The JSON Schema definitions that our API uses transformed into TypeScript types With those pieces in place, we can achieve type safety on our API server and the consuming client application. The server side is pretty straightforward if you’re using a server like Fastify with already enabled JSON Schema support. This post will focus on the concepts more than the actual implementation details though. Here’s a simple diagram illustrating the high-level concept: We can share the schema and type declaration between the client and server. In that case, we can make a request to an endpoint where we know its type and schema, and assuming the server validates the data against the schema before sending it back to the client, our client can be confident about the type of the response data. Marrying JSON Schema and TypeScript There are a couple of different ways to accomplish this: * Generating types from schema definitions using code generation tools * Creating TypeBox definitions that can infer TypeScript types and be compiled to JSON Schema I recommend considering both and figuring out which would better fit your application and workflows. Like anything else, each has its own set of trade-offs. In my experience, I’ve found TypeBox to be the most compelling if you want to go deep with this pattern. Code generation A couple of different packages are available for generating TS types from JSON Schema definitions. * https://github.com/bcherny/json-schema-to-typescript * https://github.com/vega/ts-json-schema-generator They are CLI tools that you can provide a glob path to where your schema files are located and will generate TS declaration files to a specified output path. You can set up an npm hook or a similar type of script that will generate types for your development environment. TypeBox TypeBox is a JSON Schema type builder. With this approach, instead of json files, we define schemas in code using the TypeBox API. The TypeBox definitions infer to TypeScript types directly, which eliminates the code generation step described above. Here’s a simple example from the documentation of a JSON Schema definition declared with TypeBox: ` This can then be inferred as a TypeScript type: ` Aside from schemas and types, TypeBox can do a lot more to help us on our type-safety journey. We will explore it a bit more in upcoming sections. Sharing schemas between client and server applications Sharing our JSON Schema between our server and client app is the main requirement for end-to-end type-safety. There are a couple of different ways to accomplish this, but the simplest would be to set up our codebase as a monorepo that contains both the server and client app. Some popular options for TypeScript monorepos are: PNPM, Turborepo, and NX. If a monorepo is not an option, you can publish your schema and types as a package that can be installed in both projects. However, this setup would require a lot more maintenance work. Ultimately, as long as you can import your schemas and types from the client and server app, you are in good shape. Server-to-client validation and type-safety For the sake of simplicity, let's focus on data flowing from the server to the client for now. Generally speaking, the concepts also apply in reverse, as long as your JSON API server validates your inputs and outputs. We’ll look at the most basic version of having strongly typed data on the client from a request to our server. Type-safe client requests In our server application, if we validate the /users endpoint with a shared schema - on the client side, when we make the request to the endpoint, we know that the response data is validated using the user schema. As long as we are confident of this fact, we can use the generated type from that schema as the return type on our client fetch call. Here’s some pseudocode: ` Our server endpoint would look something like this: ` You could get creative and build out a map that defines all of your endpoints, their metadata, and schemas, and use the map to define your server endpoints and create an API client. Transforming data over the wire Everything looks stellar, but we can still take our efforts a bit further. To this point, we are still limited to serialized JSON data. If we have a created_at field (number or ISO string) tied to our user, and we want it to be a Date object when we get a hold of it on the client side - additional work and consideration are required. There are some different strategies out there for deserializing JSON data. The great thing about having shared schemas between our client and server is that we can encode our type information in the schema without sending additional metadata from our server to the client. Using format to declare type data In my initial JSON Schema blog post, I touched on the format field of the specification. In our schema, if the actual type of our date is a string in ISO8601 format, we can declare our format to be "date-time". We can use this information on the client to transform the field into a proper Date object. ` Transforming serialized JSON Data This can be a little bit tricky; again, there are many ways to accomplish it. To demonstrate the concept, we’ll use TypeBox to define our schemas as discussed above. TypeBox provides a Transform type that you can use to declare, encode, and decode methods for your schema definition. ` It even provides helpers to statically generate the decoded and encoded types for your schema ` If you declare your decode and encode functions for your schemas, you can then use the TypeBox API to handle decoding the serialized values returned from your JSON API. Here’s what the concept looks like in practice fetching a user from our API: ` Nice. You could use a validation library like Zod to achieve a similar result but here we aren’t actually doing any validation on our client side. That happened on the server. We just know the types based on the schema since both ends share them. On the client, we are just transforming our serialized JSON into what we want it to be in our client application. Summary There are a lot of pieces in play to accomplish end-to-end type safety. With the help of JSON Schema and TypeBox though, it feels like light work for a semi-roll-your-own type of solution. Another great thing about it is that it’s flexible and based on pretty core concepts like a JSON API paired with a TypeScript-based client application. The number of benefits that you can reap from defining JSON Schemas for your APIs is really great. If you’re like me and wanna keep it simple by avoiding GraphQL or other similar tools, this is a great approach....

Apr 17, 2024

6 mins

JSONTypeScript

Performing a Migration in AWS Amplify

Before Attempting a Migration

Initial Setup

Referencing the New AWS App

Clear old environments

Generate an env at the new app id

Verify the new environment and push

Verify Successful Deployment

Cleanup

Morgan Worrell

You might also like

How to automatically deploy your full-stack JavaScript app with AWS CodePipeline

Avoiding Null and Undefined with NonNullable<T> in TypeScript

Using Notion as a CMS

End-to-end type-safety with JSON Schema

You might also like

How to automatically deploy your full-stack JavaScript app with AWS CodePipeline

Avoiding Null and Undefined with NonNullable<T> in TypeScript

Using Notion as a CMS

End-to-end type-safety with JSON Schema