Reducing Mental Fatigue: NestJS + ObjectionJS

▶️ Introduction

Most of the article you’ll find me ranting on what helps me enjoy my job.

But you’ll also figure out how to start using Objection with Nest and what they are all about. I won’t explain in detail how Nest or Objection work. I think those two have wonderful documentation which is fun and worth being explored, BUT I’ll show you how one can start using them together 😌

☝️ Prerequisites

You need to have PostgreSQL available locally. This can be achieved in various ways but I would suggest two approaches:

1. You can install docker [Windows, macOS, Ubuntu] and execute a package.json script described later in the article . Which essentially runs PostgreSQL in a docker container and destroys it once you hit ctrl+c in your terminal

2. You can install it directly on your machine (though I’d recommend option #1 if you don’t have it already installed)

💆 Mental fatigue

Lot's of programmers nowadays are dealing with mental fatigue in software development. And it's not only because of new tools, approaches, and paradigms that pop up every minute.

With every new software development project (let's assume back-end app, cause I'm more of a back-end person) a programmer or a team should decide on a variety of things:

• Web framework
• Project structure
• Linting rules, code formatting (based on Google, Airbnb, Microsoft or homegrown conventions)
• Data storage (SQL, NoSQL)
• Deployment platform (Amazon, Google Cloud, Azure, Netlify, etc.)
• CI/CD
• Testing tools and strategies
• Documentation
• And so on and so forth ...

As you can see this list can grow on and on. A whole lot of high-level decisions involved in this process, not talking about millions of small ones.

Every decision made depletes our mental energy even if it's trivial. After lots of minor decisions, we are less capable to make a good major one.

A tools landscape that we have right now in a JavaScript world is a blessing and a curse at the same time. On one side, it causes us to make more decisions, on the other side there are gems that come with lot's of good decisions made for us, decisions trusted by thousands of developers.

So why can't we use this opportunity to reduce our mental fatigue, reduce the number of decisions we make by adopting well-proven opinions?

By choosing the right tools we have a chance to develop a project which is easy to reason about and onboard new people on.

Almost any back-end project gets built on top of a web framework and ORM of some kind that heavily influence future project architecture.

We’ll have a look at the way they can simplify mental models we built in mind and reduce the number of decisions we usually have to make. We are going to develop a simple (but enough to demonstrate the powers of Nest and Objection) back-end for a note-taking app.

🕸️ Web framework

There are plenty of web frameworks available in the Node.js land: Express, Hapi, Koa, Fastify, Restify, etc. They are flexible and time-tested folks that allow you to structure a project in many different ways.

So you need to decide on how you want to organize routes, handlers, views, authentication, services, repositories, etc. This gives you a lot of freedom but comes with a cost. You need to make plenty of decisions to properly architect the app, and the way project is organized will be different in any other project that gets built using the same framework, because developers of a new project made their decisions in a slightly different way.

You have to start over again and grasp the way this framework is used in that particular project. You’re loosing that feeling of familiarity and awareness you developed on the previous project, or in other words, the level of framework knowledge conversion is not that high.

For me personally, those frameworks are missing one important thing (though I think they are very powerful) - a shared conceptual base, on top of which you can start growing an actual business logic, this base would repeat from project to project and allow to quickly familiarize new developers with the codebase.

Such conceptual base allows the increase of the framework’s knowledge conversion and reduce the amount of mental effort needed to start using it.

What do I mean by conceptual base? It is a minimal set of concepts or building blocks that framework gives you. And if those building blocks get aligned well with what you need to develop, it becomes easy to reason about the project and to communicate its different parts to other team members (to both seasoned developers and newcomers).

For me, such a framework is Nest! It’s written in Typescript and has good, concise documentation. So for those who don’t like to read lengthy manuals (I don’t), this documentation gives just enough information and examples to do the job - no more, no less.

Nest has modules systems heavily inspired by Angular, so Angular developers should be quite comfortable reading Nest code. Angular and Nest is usually a good combination because their conceptual bases have a high intersection, and you can transfer some of your Angular knowledge to Nest.

I don’t want to repeat the docs, and encourage you to have a look on your own.

Though I’ll describe Nest’s main building blocks:

• Guard - protects system from unauthenticated/unauthorized access
• Interceptor - intercepts incoming requests or outgoing responses
• Controller - processes the requests
• Provider - this is basically a service that is dedicated to some set of tasks and can be injected into any other thing from this list thanks to Nest built-in dependency injection capabilities
• Pipe - transforms/validates incoming request body
• Middleware - the purpose of the middleware is to intercept the request, execute some logic and pass the control flow to the next middleware
• Module - this thing helps to organize your application structure and it has the same purpose as Angular modules do

Also, I suggest reading on this series of articles on Nest.js Step by Step.

You can ask me “Why do I need other stuff listed above if I have middlewares?”.

Middlewares are too generic whereas Guards, Intercepts, etc. are dedicated to one particular task. So by hearing the word “Guard”, you already know what it is responsible for.

You might need middlewares if you want to implement something beyond those concepts listed here.

💥 Big Bang! For the sake of brevity, I won’t be describing every file of our future project, but rather will be highlighting key concepts along the way.

We’re starting with the Nest application, which has all the plumbings but no database. This way it’ll be easy to talk about Nest stuff and then gradually move to Objection. As I already said we are going to develop a toy notes API. Our notes can have theme and tags.

This is what our app structure looks like initially:

Go ahead and investigate the code we have so far (the codebase is on the initial commit at the moment). We're gonna start building on top of it.

Just by looking at names we have in the codebase it becomes immediately clear the purpose and responsibilities of different classes.

Let’s have a look at notes folder in more details (cause tags and themes works exactly in the same vein).

The first thing is NotesModule

In NotesModule we’ve registered NotesService. Here how it looks

NotesService is used by NotesController and is injected by Nest once it discovers that the latter is dependent on the former.

You might have noticed that NotesService (as well as other services) is just a stub at the moment and does nothing. We’re going to fix that soon, after a small conversation about ORMs.

🦖 ORM

Historically, the purpose of ORMs was to remove object-relational impedance mismatch. They do this by abstracting out RDBMS and relational concepts as much as possible, they are especially good at hiding SQL from you and forcing you to use their DSL, which still sucks because it is a prominent example of a leaky abstraction.

I remember lot’s of situations when I was struggling with such DSL for hours, trying to mimic a query which I’d already written in SQL (and spent minutes on this). Even if we’ve managed to write a proper DSL it still might be converted into monstrous (not always performant) SQL you have no control over.

The true power of relational databases comes with SQL and its declarative expressiveness. In reality, the majority of my colleagues are quite good with the RDBMS concepts. It’s comfortable for them to think in terms of SQL queries and more often than not developers have an intuitive understanding of how DB record should be represented as an object (dictionary, map, you name it) in their language of choice.

It simply makes no sense to hide SQL from developers in ORMs because you still have to know it for fetching at least something from the DB, but apart from that you need to enable a compiler in your head that converts DSL to SQL in order to understand what kind of query will be generated eventually and whether it’ll give you what you want.

It’s double work that puts extra pressure on your brain which is trying to keep and reconcile a million of other little things about the project you are working on.

If you are already proficient with SQL, why do you need to learn another language (DSL) to fetch/update stuff from/in the database? Wouldn’t it be better for ORMs to implement an API that is as close to SQL as possible, allowing to transfer existing developer’s SQL knowledge to that API and flattening learning curve? Such API would take advantage over the language features like auto-completion and static code analysis while still being close to generated SQL.

The solutions like Hibernate, TypeORM and similar ones are overloaded, heavyweight and over-complicated in my opinion.

And here is where Objection comes in. Comparing to other ORMs it doesn’t try to put SQL and relational model behind the curtains. Here is how Objection developers describe their product:

Objection.js is an ORM for Node.js that aims to stay out of your way and make it as easy as possible to use the full power of SQL and the underlying database engine while still making the common stuff easy and enjoyable.

🍽️ Integrating Objection with Nest

TLDR; If you just need to know what should be done to have Objection support in Nest, here is the diff which shows changes that should be applied on top of our initial commit.

1️⃣ Installing required dependencies npm i @types/dotenv dotenv objection knex pg

• dotenv populates process.env with environment variables defined in the .env file
• objection - the ORM
• knex is a SQL query builder that Objection uses under the hood. It also provides migrations and data-seeding support (we’ll talk about this a bit later)
• pg is a client for PostgreSQL database.

2️⃣ Relational model Next step is to define our relation model (for now just get comfortable with the tables we are about to build)

• Notes might have a theme
• Notes can have multiple tags
• One tag can belong to multiple notes

knex_migrations and knex_migrations_lock are tables created and managed by Knex. They are not relevant for our data model.

3️⃣ Extending package.json with helper scripts Before we start creating the migrations, let’s add a couple of commands to our package.json No worries, their purpose will become clear in later sections.

4️⃣ Knexfile In the package.json excerpt above you might be noticed --knexfile knexfile.ts. This is an argument that points to Knex configuration file, so let’s create it at the root of the project.

knexSnakeCaseMappers converts camelCase names in code to snake_case names in the database. So in our database model, we have a themes table with font_family column. In order to update this column from the code, you can refer to it using fontFamily and mappers will do the job by transforming font_family → fontFamily and vice versa automatically.

The purpose of migrations is to create a database schema and subsequent changes to that schema that might come up over time. This allows versioning your database and rollback schema to its previous state when needed.

Seeds are useful in the development environment when you need to populate your database with some data.

migration.stub and seed.stub are template files which Knex uses to generate our migrations and seeds. Put those under the database folder as specified in the config

5️⃣ Migrations Now when we have knexfile.ts created we can start using Knex commands we’ve added previously to the package.json.

• npm run migrate:make CreateTags
• npm run migrate:make CreateThemes
• npm run migrate:make CreateNotes
• npm run migrate:make CreateNoteTags

Those will generate migration files under the database/migrations folder using our migration.stub.

It’s time to define our tables. Let’s do it together for CreateNotes migration for others please have a look at the final solution.

6️⃣ Connect our models with Objection In order to reflect relational tables in our code, we need to create a bunch of appropriate classes called models. For now, they are just plain Typescript classes located under database/models directory, so let’s sprinkle some Objection on them.

BaseModel ( base.model.ts )

TagModel ( tag.model.ts )

NoteTagModel ( note-tag.model.ts )

ThemeModel ( theme.model.ts )

7️⃣ Mapping relations Especially interesting for us is how Objection handles relations between tables and the way we can express them in code.

8️⃣ Connecting models to database and database.module.ts

Each model class can be used to perform various SQL queries, but for that, we need to wire those classes with a Knex database connection.

Once they are wired we can expose those classes as injectable service to other modules.

DatabaseModule needs to be registered under the main ApplicationModule, so all its exported services are available to other modules.

9️⃣ Implementing.service.ts files To start manipulating data we have in the database, we need to implement methods defined in the .service.ts files.

Each service relies on the model class(-es) we’ve exposed through the module’s exports above. Here is NotesService implementation (notes.service.ts):

As you can see the .query() method is a gateway for building rich queries. In the example above we also have a transaction example, so any error thrown in the transaction callback will cause database changes triggered inside of that callback to roll back.

🔟 Loading Note relations Let’s have a look at findOne method in NotesController:

The notable change is $loadRelated invocation. Here we’re asking Objection to load relations for this particular note:

tags and theme are names of the relations defined in the NoteModel class. This is how Objection knows how to fetch them.

All fetched relations get transformed into appropriate model instances. Once fetched, Objection will create tags and theme fields for this particular note instance.

So all the relations get loaded only on demand by default.

In case you want to fetch lots of objects with loaded relations there is another way you can use:

In here, once all notes are loaded - Objection will loads tags relation for all of them.

1️⃣ 1️⃣ Seeds Now we’re ready to generate seed files:

• npm run seed:make 01-Tags
• npm run seed:make 02-Themes
• npm run seed:make 03-Notes
• npm run seed:make 04-NoteTags

Seeds get generated under the database/seeds folder using our seed.stub.

Seed files get executed by Knex in order, so we have to ensure that it is correct. This is the reason we’ve prefixed seed files with numbers: we want to have tags created before note-tags because the latter depends on the former.

Let’s have a look at 02-Themes.ts seed implementation

1️⃣ 2️⃣ dotenv dotenv is a library that loads environment variables from .env file into process.env We’re going to utilize it for defining DATABASE_URL env var, which then will be used throughout the app including migrations and seed scripts.

All you need is to

1. create a .env file at the root of the app and put there this single line DATABASE_URL=postgres://postgres:docker@localhost:5432/postgres

This connection string gets constructed based on the command we have in package.json. Postgres uses postgres as a name for a default user and database.

2. Add dotenv import at the very top of knexfile.ts and main.ts

1️⃣ 3️⃣ Running PostgreSQL At this point we need to start our PostgreSQL instance:

`npm run run:pg-docker`

Then create the schema (by executing migrations) and populate it with data (by executing seeds):

`npm run migrate && npm run seed`

🚀 Playing with the app

Now you should have a fully working Nest application with the Objection support.

You can run it using

`npm run start`

README.md contains example http requests (using curl) which you can modify and execute against the server.

And we’re done 🎉

✍️ Summary

In the article I shared my thoughts on mental fatigue and that with the right tools it can be reduced by utilizing clear and intuitive concepts that help to communicate and share knowledge with others.

Nest does this by providing a conceptual base, which is great not only for reasoning about the project but also for communicating the way it works to other developers.

Objection gives you a framework that allows thinking in SQL terms, and avoid wasting time debugging esoteric DSLs. I would call it the “ORM without a pain”.

I hope you’ve enjoyed the article and get some understanding of how to start using Objection with Nest.

You can find the full project on my github.

Working with AWS AppSync