Skip to content

Hey Deno! Where is my package.json?

This article was written over 18 months ago and may contain information that is out of date. Some content may be relevant but please refer to the relevant official documentation or available resources for the latest information.

Disclaimer: This blog was written for Deno versions prior to 1.3.1. After 1.3.1, while Deno can handle projects that contain a package.json (to help facilitate migration), it is still recommended that you handle your dependencies as discussed in this article.

Introduction

Where is my package.json?

That was one of my first questions when I started learning Deno. Coming from a NodeJS background, I am used to having a package manager (such as NPM) for managing dependencies, scripts and other configurations, and I’m used to using the package.json file to declare my dependencies and their versions.

Deno has no concept of a package manager, as external modules are imported directly into local modules (e.g import { bold } from ‘https://deno.land/std@v0.32.0/fmt/colors.ts’). At first, this seems very convenient, but it got me wondering how I would be able to manage and update dependencies when they are imported into several different modules across a large project. And how about running scripts? Node allows you to define arbitrary scripts in the package.json that can be executed using npm run. How do we define and manage scripts in Deno?

In this article, we will discuss the various ways to manage dependencies in Deno, and also how to manage all scripts needed for our project.

Managing Dependencies

Using deps.ts

The standard practice for managing dependencies in Deno is to create a deps.ts file that imports, and then immediately re-exports all third-party code.

/**
 * deps.ts
 * Exports all project dependencies from the file.
 */
export * as log from 'https://deno.land/std@0.167.0/log/mod.ts';
export { Application, Router, Context } from 'https://deno.land/x/oak@v11.1.0/mod.ts';
export type { Middleware } from 'https://deno.land/x/oak@v11.1.0/mod.ts';
export { DataTypes, Database, Model, PostgresConnector } from 'https://deno.land/x/denodb@v1.1.0/mod.ts';
export { oakCors } from 'https://deno.land/x/cors@v1.2.2/mod.ts';
export { applyGraphQL, gql, GQLError } from 'https://deno.land/x/oak_graphql@0.6.4/mod.ts';

In your local module, these methods, and classes can be referenced from the deps.ts file.

import { Application, applyGraphQL, oakCors, Router } from '../deps.ts';

You may be familiar with the concept of dev dependencies in NPM. You can define dev dependencies in Deno using a separate dev_deps.ts file, allowing for a clean separation between dev-only and production dependencies.

With this approach, managing dependencies in Deno becomes much simpler. For example, to upgrade a dependency version, you make the change in the depts.ts file, and it propagates automatically to all modules in which it is referenced.

When using this approach, one should consider integrity checking & lock files. This is basically Deno’s solution for avoiding production issues if the content in the remote url (e.g https://some.url/a.ts) is changed. This could lead to the production module running with different dependency code than your local module.

Just like package-lock.json in NodeJS, Deno can store and check subresource integrity for modules using a small JSON file. To autogenerate a lock file, create a deno.json file at the root of your project and a deno.lock file will be autogenerated.

You can also choose a different file name by updating the deno.json file like so:

{
  "lock": "./lock.json"
}

You can also disable automatically creating, and validating a lock file by specifying:

{
  "lock": false
}

We can manually create or update the lock file using the Deno cache command, and --lock and --lock-write flags like so:

deno cache --lock=deno.lock --lock-write deps.ts

Then a new collaborator can clone the project on their machine and run:

deno cache --reload --lock=deno.lock deps.ts

Using import_map.json

Another way to manage dependencies in Deno is by using the import_map.json file.

This method is useful if you want to use "bare-specifiers" (specifiers without an absolute or relative path to them, e.g import react from ‘react’).

This file allows you to alias a specific import URL or file path, which can be useful if you want to use a custom alias for a dependency.

To use the import_map.json file, you first need to create it in the root directory of your project. The file should contain a JSON object with a single key, "imports", which maps import aliases to fully-qualified module URLs or file paths. You can use the import_map.json file to map import paths to remote dependencies and even NPM specifiers.

You can also use the import_map.json file to map aliases to local file paths. For example, if you have a local module in your project at ./src/lib/my_module.ts, you can map the import path "my_module" to this file.

Here's an example of an import_map.json file:

{
  "imports": {
    "lodash": "npm:lodash@^4.17",
    "react": "https://cdn.skypack.dev/react",
    "my_module": "./src/lib/my_module.ts"
  }
}

With this import_map.json file in place, you can now import the libraries using their aliases:

import lodash from 'lodash';
import react from 'react';
import { myFunction } from 'my_module';

console.log(lodash.defaults({ 'a': 1 }, { 'a': 3, 'b': 2 }))

Using the import_map.json file can be a convenient way to manage dependencies in your Deno projects, especially if you want to use custom aliases for your imports. Just be sure to include the --import-map flag when running your Deno application, like so:

deno run --import-map=./import_map.json main.ts

This will ensure that Deno uses the import map specified in the import_map.json file when resolving dependencies.

Managing Command Scripts

Like npm run, the Deno CLI also has a run command that is used to run scripts in files. Depending on the permission needed or the type of operation, there are certain flags that need to be passed to the run command. For example, if you want to run a web server that uses an env file and reads from it, your command will look like this:

deno run --allow-net --allow-env --allow-read main.ts

If we are reading and writing to a file, we need to add the --allow-write, or if we have an API that needs information about the operating system of the user, then we will also need to add --allow-sys. This can go on and on. We could decide to use --allow-all to accept all permissions, but this is not advisable.

The good news is that we can manage all our command scripts without having to retype them every time. We can add these scripts to the deno.json file.

The deno.json file is a configuration file for customizing basic deno CLI commands like fmt, lint, test etc. In addition, the file can also be used to define tasks using the "tasks" field. To define tasks, you can specify a mapping of task names to “tasks”. For example:

{
 "tasks": {
    "start-web": "deno run --watch --allow-net --allow-env --allow-read ./src/main.ts",
    "generate-type-definition": "deno run --allow-net --allow-env --allow-read --allow-write --allow-sys ./tools/generate_type_definition.ts"
 }
}

You can then run these tasks using the Deno task command, and specifying the task name, like this:

deno task start-web
deno task generate-type-definition

Conclusion

In this article, we saw how to manage dependencies and command scripts in a Deno project. If you’re coming from a Node background, and were confused about where the package.json file was, we hope this clarified how to accomplish some of the same things using the depts.ts, dev_depts.ts, import_map.json, and deno.json files.

We hope this article was helpful, and you were able to learn and be more comfortable with using Deno for your projects. If you want to learn more about Deno, check out deno.framework.dev for a list of libraries and resources. If you are looking to start a new Deno project, check out our starter kit resources at starter.dev.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

You might also like

State of Deno: A Look at the Deno CLI, Node.js Compatibility and the Fresh Framework cover image

State of Deno: A Look at the Deno CLI, Node.js Compatibility and the Fresh Framework

In this State of Deno event, our panelists discussed the Deno CLI, Node.js compatibility for the npm ecosystem, and the Fresh framework. In this wrap-up, we will take a deeper look into these latest developments and explore what is on the horizon for Deno. You can watch the full State of Deno event on the This Dot Media YouTube Channel. Here is a complete list of the host and panelists that participated in this online event. Hosts: Tracy Lee, CEO, This Dot Labs, @ladyleet Panelists: Colin Ihrig, Software Engineer at Deno, @cjihrig Luca Casonato, Software Engineer at Deno, @lcasdev Bartek Iwańczuk, Software Engineer at Deno, @biwanczuk David Sherret, Software Engineer at Deno, @DavidSherret Table of Contents - Exploring the Deno CLI and its features - What is Deno? - Built in support for TypeScript - Built in toolchain - Deno install and upgrade commands - Deno permissions - Upcoming features - Deno products - Deno Deploy - Deno and Node.js compatibility - Future support for npm packages - The Deno to Node Transform library tool - Fresh framework - Conclusion - We look forward to seeing you at our next State of Deno! Exploring the Deno CLI and Its Features What is Deno? Deno is server-side runtime for JavaScript that also behaves similarly to a browser because it supports all of the same browser APIs on the server. This support provides access to existing knowledge, resources, and documentation for these browser APIs. The team at Deno works closely with browser vendors to make sure that new web APIs work well for both server-side runtime and browsers. Built In Support for TypeScript One of the advantages of Deno is that it ships with TypeScript by default. This removes the setup and configuration time, and reduces the barrier of entry for getting started with TypeScript. Deno also type checks your TypeScript code so you no longer have to use tsc. Built in Toolchain The Deno CLI comes with an entire toolchain which includes a formatter, Linter, package manager, vendor remote dependencies, editor integrations, and more. One of those tools is the Documentation Generator, which annotates library function comments, types, or interfaces with JSDocs comments to easily generate documentation. For a complete list of the Deno CLI tools, please visit their documentation page. Deno install and upgrade commands Deno install is another feature that allows you to take a script and install it using a global command. ` If there is a new version of Deno, you can just run the upgrade command and it will upgrade itself, which makes it a version manager for itself. ` Deno permissions Deno by default will not have file, network or environment access unless you grant those permissions by running a script with command line flags. ` Even with permissions granted, you can scope it to certain directories to allow it to only read and write in the directory of your choosing. If you run the program without permissions granted, the program will still prompt you for permission access before accessing a file. To learn more about Deno's permissions, please read through the documentation. Upcoming features Deno is currently working on improving performance in the areas of HTTP, FFI (Foreign Function Interface) and Node compatibility. There are also improvements being made on the Documentation Generator, to make sure that the docs provided are good and removing the need to maintain two separate docs. Deno Products Deno Deploy Deno deploy is a hosted offering that makes it easy to go from local development to production ready. This service integrates well with GitHub, and provides an option to pay only when users make requests to your services. Deno deploy has a dashboard that allows you to automate most of the processes and show you metrics, deployment statistics, CPU utilization, and network utilization. It also allows you to set up custom domains and provision certificates. Deno and Node.js compatibility Deno v1.15 will introduce a Node.js compatibility mode which will make it possible to run some of Node's programs in Deno. Node APIs like the http server will work in Deno as they would in Node.js. When it comes to the NPM ecosystem compatibility, the goal is to support the large number of packages built on Node.js. The Deno team is working on these compatibility issues, because it uses web APIs for most of their operations. All of these Web APIs were created after Node.js was conceived, meaning that Node implements a whole different set of APIs to do certain operations like performing network requests. This new Node.js API compatibility layer will be able to translate API calls to modern underlying APIs which is what the actual runtime implements. Future support for npm packages When it comes to supporting npm packages on Deno, the goal is to have a transpiler server that takes common.js code and translates that into ESM (ECMAScript module) code. The reason for this is because, just like browsers, Deno only supports ESM code. Deno uses the concept of npm specifiers to provide access to the npm package. Deno downloads the npm package and runs it from a global cache. common.js is also supported ,and it runs the code as it is. ` For npm packages, Deno will create a single copy of the downloaded package instead of multiple directories or sub-directories of modules. It is one global hash file, with no node_modules directory by default, and no need for a package.json by default. If a package requires a node_modules directory, then that directory can be created using a specifier flag. The following command will create a node_modules directory in the project directory, using symlink. ` The Deno to Node Transform library tool The Deno team built a tool to allow library authors to transform Deno packages to Node.js packages. Deno to Node Transform (DNT) takes the Deno code then builds it for Node and distributes it as an npm package. This allows library authors to take advantage of the Deno tool chain. This package can also be shipped on npm for Node.js users. Fresh framework Fresh is a new web framework for Deno, that makes use of the Deno toolchain ecosystem. Fresh uses JSX for templating, and it is similar to Next.js or Remix. A key difference between Fresh and Next.js or Remix, is that Fresh is built to be server-side rendered on the edge rather than server-side in a few locations. Another difference is that with Fresh, no JavaScript is shipped to the client by default, which makes it faster. Fresh handles the Server-side rendering, generates the HTML, sends the file to the client, and hydrates only the part of the page that requires JavaScript on the client by default. Here are some products that already use the Fresh framework: - Deno - merch.deno.com - Deno Deploy To learn more about how to build apps with the Fresh framework, please read through this helpful blog post. Conclusion The team at Deno is making great progress to bring more exciting features to the community to make the runtime engine easy to use for building or migrating existing libraries. If you have any questions about the State of Deno, be sure to ask here. What is it you find exciting about Deno? We will be happy to hear about it on Twitter! We look forward to seeing you at our next State of Deno!...

Web Scraping with Deno cover image

Web Scraping with Deno

Maybe you've had this problem before: you need data from a website, but there isn't a good way to download a CSV or hit an API to get it. In these situations, you may be forced to write a web scraper. A web scraper is a script that downloads the HTML from a website as though it were a normal user, and then it parses that HTML to extract information from it. JavaScript is a natural choice for writing your web scraper, as it's the language of the web and natively understands the DOM. And with TypeScript, it is even better, as the type system helps you navigate the trickiness of HTMLCollections and Element types. You can write your TypeScript scraper as a Node script, but this has some limitations. TypeScript support isn't native to Node. Node also doesn't support web APIs very well, only recently implementing fetch for example. Making your scraper in Node is, to be frank, a bit of a pain. But there's an alternative way to write your scraper in TypeScript: Deno! Deno is a newer JavaScript/TypeScript runtime, co-created by one of Node's creators, Ryan Dahl. Deno has native support for TypeScript, and it supports web APIs out of the box. With Deno, we can write a web scraper in TypeScript with far fewer dependencies and boilerplate than what you'd need in order to write the same thing in Node. In this blog, we’re going to build a web scraper using Deno. Along the way, we’ll also explore some of the key advantages of Deno, as well as some differences to be aware of if you’re coming from writing code exclusively for Node. Getting Started First, you'll need to install Deno. Depending on your operating system, there are many methods to install it locally. After installation, we'll set up our project. We'll keep things simple and start with an empty directory. ` Now, let's add some configuration. I like to keep my code consistent with linters (e.g. ESLint) and formatters (e.g. Prettier). But Deno doesn't need ESLint or Prettier. It handles linting and formatting itself. You can set up your preferences for how to lint and format your code in a deno.json file. Check out this example: ` You can then lint and format your Deno project with deno lint and deno fmt, respectively. If you're like me, you like your editor to handle linting and formatting on save. So once Deno is installed, and your linting/formatting options are configured, you'll want to set up your development environment too. I use Visual Studio Code with the Deno extension. Once installed, I let VS Code know that I'm in a Deno project and that I'd like to auto-format by adding a .vscode/settings.json file with the following contents: ` Writing a Web Scraper With Deno installed and a project directory ready to go, let's write our web scraper! And what shall we scrape? Let's scrape the list of Deno 3rd party modules from the official documentation site. Our script will pull the top 60 Deno modules listed, and output them to a JSON file. For this example, we'll only need one script. I'll name it index.ts for simplicity's sake. Inside our script, we'll create three functions: sleep, getModules and getModule. The first will give us a way to easily wait between fetches. This will prevent issues with our script, and the owners of the website we’ll contact, because it's unkind to flood a website with many successive requests. This type of automation could appear like a Denial of Service (DoS) attack, and could cause the API owners to ban your IP address from contacting it. The sleep function is fully implemented in the code block below. The second function (getModules) will scrape the first three pages of the Deno 3rd party modules list, and the third (getModule) will scrape the details for each individual module. ` Scraping the Modules List First, let's write our getModules function. This function will scrape the first three pages of the Deno 3rd party modules list. Deno supports the fetch API, so we can use that to make our requests. We'll also use the deno_dom module to parse the HTML response into a DOM tree that we can traverse. Two things we'll want to do upfront: let's import the deno_dom parsing module, and create a type for the data we're trying to scrape. ` Next, we'll set up our initial fetch and data parsing: ` Now that we're able to grab the contents of the first three pages of Deno modules, we need to parse the document to get the list of modules we want to collect information for. Then, once we've got the URL of the modules, we'll want to scrape their individual module pages with getModule. Passing the text of the page to the DOMParser from deno_dom, you can extract information using the same APIs you'd use in the browser like querySelector and getElementsByTagName. To figure out what to select for, you can use your browser's developer tools to inspect the page, and find selectors for elements you want to select. For example, in Chrome DevTools, you can right-click on an element and select "Copy > Copy selector" to get the selector for that element. ` Scraping a Single Module Next we'll write getModule. This function will scrape a single Deno module page, and give us information about it. If you're following along so far, you might've noticed that the paths we got from the directory of modules look like this: ` But if you navigate to that page in the browser, the URL looks like this: ` Deno uses redirects to send you to the latest version of a module. We'll need to follow those redirects to get the correct URL for the module page. We can do that with the redirect: 'follow' option in the fetch call. We'll also need to set the Accept header to text/html, or else we'll get a 404. ` Now we'll parse the module data, just like we did with the directory of modules. ` Writing Our Data to a File Finally, we'll write our data to a file. We'll use the Deno.writeTextFile API to write our data to a file called output.json. ` Lastly, we just need to invoke our getModules function to start the process. ` Running Our Script Deno has some security features built into it that prevent it from doing things like accessing the file system or the network without granting it explicit permission. We give it these permissions by passing the --allow-net and --allow-write flags to our script when we run the script. ` After we let our script run (which, if you've wisely set a small delay with each request, will take some time), we'll have a new output.json file with data like this: ` Putting It All Together Tada! A quick and easy way to get data from websites using Deno and our favorite JavaScript browser APIs. You can view the entire script in one piece on GitHub. In this blog, you've gotten a basic intro to how to use Deno in lieu of Node for simple scripts, and have seen a few of the key differences. If you want to dive deeper into Deno, start with the official documentation. And when you're ready for more, check out the resources available from This Dot Labs at deno.framework.dev, and our Deno backend starter app at starter.dev!...

Internationalization in Next.js with next-intl cover image

Internationalization in Next.js with next-intl

Internationalization in Next.js with next-intl Internationalization (i18n) is essential for providing a multi-language experience for global applications. next-intl integrates well with Next.js’ App Router, handling i18n routing, locale detection, and dynamic configuration. This guide will walk you through setting up i18n in Next.js using next-intl for URL-based routing, user-specific settings, and domain-based locale routing. Getting Started First, create a Next.js app with the App Router and install next-intl: ` Next, configure next-intl in the next.config.ts file to provide a request-specific i18n configuration for Server Components: ` Without i18n Routing Setting up an app without i18n routing integration can be advantageous in scenarios where you want to provide a locale to next-intl based on user-specific settings or when your app supports only a single language. This approach offers the simplest way to begin using next-intl, as it requires no changes to your app’s structure, making it an ideal choice for straightforward implementations. ` Here’s a quick explanation of each file's role: * translations/: Stores different translations per language (e.g., en.json for English, es.json for Spanish). Organize this as needed, e.g., translations/en/common.json. * request.ts: Manages locale-based configuration scoped to each request. Setup request.ts for Request-Specific Configuration Since we will be using features from next-intl in Server Components, we need to add the following configuration in i18n/request.ts: ` Here, we define a static locale and use that to determine which translation file to import. The imported JSON data is stored in the message variable, and is returned together with the locale so that we can access them from various components in the application. Using Translation in RootLayout Inside RootLayout, we use getLocale() to retrieve the static locale and set the document language for SEO and pass translations to NextIntlClientProvider: ` Note that NextIntlClientProvider automatically inherits configuration from i18n/request.ts here, but messages must be explicitly passed. Now you can use translations and other functionality from next-intl in your components: ` In case of async components, you can use the awaitable getTranslations function instead: ` And with that, you have i18n configured and working on your application! \ Now, let’s take it a step further by introducing routing. \ With i18n Routing To set up i18n routing, we need a file structure that separates each language configuration and translation file. Below is the recommended structure: ` We updated the earlier structure to include some files that we require for routing: * routing.ts: Sets up locales, default language, and routing, shared between middleware and navigation. * middleware.ts: Handles URL rewrites and locale negotiation. * app/[locale]/: Creates dynamic routes for each locale like /en/about and /es/about. Define Routing Configuration in i18n/routing.ts The routing.ts file configures supported locales and the default locale, which is referenced by middleware.ts and other navigation functions: ` This configuration lets Next.js handle URL paths like /about, with locale management managed by next-intl. Update request.ts for Request-Specific Configuration We need to update the getRequestConfig function from the above implementation in i18n/request.ts. ` Here, request.ts ensures that each request loads the correct translation files based on the user’s locale or falls back to the default. Setup Middleware for Locale Matching The middleware.ts file matches the locale based on the request: ` Middleware handles locale matches and redirects to localized paths like /en or /es. Updating the RootLayout file Inside RootLayout, we use the locale from params (matched by middleware) instead of calling getLocale() ` The locale we get from the params was matched in the middleware.ts file and we use that here to set the document language for SEO purposes. Additionally, we used this file to pass configuration from i18n/request.ts to Client Components through NextIntlClientProvider. Note: When using the above setup with i18n routing, next-intl will currently opt into dynamic rendering when APIs like useTranslations are used in Server Components. next-intl provides a temporary API that can be used to enable static rendering. Static Rendering for i18n Routes For apps with dynamic routes, use generateStaticParams to pass all possible locale values, allowing Next.js to render at build time: ` next-intl provides an API setRequestLocale that can be used to distribute the locale that is received via params in layouts and pages for usage in all Server Components that are rendered as part of the request. You need to call this function in every layout/page that you intend to enable static rendering for since Next.js can render layouts and pages independently. ` Note: Call setRequestLocale before invoking useTranslations or getMessages or any next-intl functions. Domain Routing For domain-specific locale support, use the domains setting to map domains to locales, such as us.example.com/en or ca.example.com/fr. ` This setup allows you to serve localized content based on domains. Read more on domain routing here. Conclusion Setting up internationalization in Next.js with next-intl provides a modular way to handle URL-based routing, user-defined locales, and domain-specific configurations. Whether you need URL-based routing or a straightforward single-locale setup, next-intl adapts to fit diverse i18n needs. With these tools, your app will be ready to deliver a seamless multi-language experience to users worldwide....

Next.js + MongoDB Connection Storming cover image

Next.js + MongoDB Connection Storming

Building a Next.js application connected to MongoDB can feel like a match made in heaven. MongoDB stores all of its data as JSON objects, which don’t require transformation into JavaScript objects like relational SQL data does. However, when deploying your application to a serverless production environment such as Vercel, it is crucial to manage your database connections properly. If you encounter errors like these, you may be experiencing Connection Storming: * MongoServerSelectionError: connect ECONNREFUSED <IP_ADDRESS>:<PORT> * MongoNetworkError: failed to connect to server [<hostname>:<port>] on first connect * MongoTimeoutError: Server selection timed out after <x> ms * MongoTopologyClosedError: Topology is closed, please connect * Mongo Atlas: Connections % of configured limit has gone above 80 Connection storming occurs when your application has to mount a connection to Mongo for every serverless function or API endpoint call. Vercel executes your application’s code in a highly concurrent and isolated fashion. So, if you create new database connections on each request, your app might quickly exceed the connection limit of your database. We can leverage Vercel’s fluid compute model to keep our database connection objects warm across function invocations. Traditional serverless architecture was designed for quick, stateless web app transactions. Now, especially with the rise of LLM-oriented applications built with Next.js, interactions with applications are becoming more sequential. We just need to ensure that we assign our MongoDB connection to a global variable. Protip: Use global variables Vercel’s fluid compute model means all memory, including global constants like a MongoDB client, stays initialized between requests as long as the instance remains active. By assigning your MongoDB client to a global constant, you avoid redundant setup work and reduce the overhead of cold starts. This enables a more efficient approach to reusing connections for your application’s MongoDB client. The example below demonstrates how to retrieve an array of users from the users collection in MongoDB and either return them through an API request to /api/users or render them as an HTML list at the /users route. To support this, we initialize a global clientPromise variable that maintains the MongoDB connection across warm serverless executions, avoiding re-initialization on every request. ` Using this database connection in your API route code is easy: ` You can also use this database connection in your server-side rendered React components. ` In serverless environments like Vercel, managing database connections efficiently is key to avoiding connection storming. By reusing global variables and understanding the serverless execution model, you can ensure your Next.js app remains stable and performant....

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Prefer email? hi@thisdot.co