Building a scalable Node.js API using repository service pattern

You might be reasonably annoyed that most Node.js API designs do not consider scalability and test coverage as one of the most important considerations in API design. People waste a considerable amount of time building Node.js APIs utilising a mixture of module / middleware patterns. Because that is what the courses, articles and tutorials teach you for 99.9% of the time. You probably have seen this before:

 
import express from 'express';

const app = express();
// logger.ts
const logger = (req, res, next) => {
  console.log('Logged');
  next();
}
//authenticate.ts
const authenticate = (req, res, next) => {
  // authenticate user 
  next(); 
}

app.use(logger); 
app.use(authenticate);

app.get('/', (req, res) => {
  res.send('Hello World');
});

app.listen(3000);

Here is the repository of the latest Node.js course that I did. It has a great structure, the course is well written and was a joy to do. But as all of these all the courses that went through for my consideration: it is very basic and does not teach you how to handle scale, testing, and complexity. It does the job and does it well - but as the code grows it makes scaling and separating the logic incredibly complicated.

The repository service pattern

One of the battle-tested and popular approaches to this kind of problem is utilising a repository service pattern. This pattern came from Java land: take a read of this article about repository service pattern in Java. This pattern works really well in Node.js applications too. In this blog post we will be using examples from some take-home that I received some time ago. It displays and sorts data showing currency rates for a selected date. You can find it available here. In essence: all you do is create classes for possible layers in API:

Router

Router handles requests and validates data types

 
import express from "express";

import { ConsoleLogger } from "./utils/logger";
import { ExchangeRateRepo } from "./repositories/exchangeRateRepo";
import { ExchangeRateService } from "./services/exchangeRateService";
import { ExchangeRateHandler } from "./handlers/exchangeRateHandler";

const apiRouter = express.Router();

const consoleLogger = new ConsoleLogger();

const exchangeRatesRepo = new ExchangeRateRepo("./currencies.json"); // <-- data
const service = new ExchangeRateService(exchangeRatesRepo); // <-- business logic
const handlers = new ExchangeRateHandler(service, consoleLogger); // <--- handles http requests

//get EX dates
apiRouter.get("/exchange-rates/dates", handlers.getExchangeDates);
//get EX data
apiRouter.get("/exchange-rates/:date", handlers.getExchangeData);

export { apiRouter };

Service

Service (or manager) is responsible for handling the business logic. That includes manipulating data, gluing different parts of data together and creating function that bring value to the company. Service is called from Router.

 
import moment from "moment";
import {
  CurrencyRate,
  ExchangeRateData,
  ExchangeRateMap,
  NewExchangeRateData,
} from "../models/exchangeRatesModel";
import { ExchangeRateRepo } from "../repositories/exchangeRateRepo";

export class ExchangeRateService {
  dataset: ExchangeRateMap;
  transformedData: NewExchangeRateData[];

  constructor(repo: ExchangeRateRepo) {
    this.dataset = repo.dataset;
    this.getExchangeData = this.getExchangeData.bind(this);
    this.transformedData = this.transformExRateData(this.dataset);
  }

  async getAllDates(): Promise<string[]> {
    let dates: string[] = [];
    for (let i = 0; i < this.transformedData.length; i++) {
      dates.push(this.transformedData[i].date);
    }
    return dates;
  }

  async getExchangeData(date: string): Promise<NewExchangeRateData> {
    const data = this.transformedData;
    const exchangeRateByDate = data.find((item) => item.date === date);
    if (!exchangeRateByDate) {
      throw new Error("Incorrect date selected");
    }
    return exchangeRateByDate;
  }

  transformExRateData(data: ExchangeRateMap): NewExchangeRateData[] {
    // Make an arr of dates to loop
    const currencies = Object.keys(data);
    if (currencies.length === 0) {
      // Handle the case where there is no data in the ExchangeRateMap
      return [];
    }

    let transformed: NewExchangeRateData[] = [];
    for (let i = 0; i < currencies.length; i++) {
      const currentCurrency = currencies[i];
      const currentData = data[currentCurrency];
      let previousData: ExchangeRateData;
      // We cannot get % ch for the first day, 0
      if (i === 0) {
        previousData = currentData;
      } else {
        previousData = data[currencies[i - 1]];
      }

      // calculatePercentChange and then push to DS
      const percentChangeRates = this.calculatePercentageChange(
        previousData.rates,
        currentData.rates,
      );
      //format dates
      const formatDates = moment(currencies[i], "YYYY-MM-DD").format(
        "DD-MM-YYYY",
      );
      transformed.push({
        date: formatDates,
        timestamp: currentData.timestamp,
        base: currentData.base,
        rates: percentChangeRates,
      });
    }
    return transformed;
  }

  calculatePercentageChange(
    previousRates: Record<string, number>,
    currentRates: Record<string, number>,
  ): CurrencyRate[] {
    const result: CurrencyRate[] = [];

    for (const currency in currentRates) {
      if (previousRates[currency] !== undefined) {
        const currentPrice = currentRates[currency];
        const previousPrice = previousRates[currency];
        const percentChange = Number(
          (((currentPrice - previousPrice) / previousPrice) * 100).toFixed(1),
        );

        result.push({
          name: currency,
          price: currentPrice,
          percentChange,
        });
      }
    }

    return result;
  }
}

Repository

Repository handles all the communication with the database, and allows for clear separation between development, staging, production databases. The idea is to create a Class such as that handles communications with one entity. For example, UsersRepository, PostsRepository, and so on. Repository classes have an exclusive access to the database, which is strictly structured by migrations. It is a single source of truth that interacts with the database. This point is crucial in effectively utilising the Service Repository pattern.

 
import { readFileSync } from "fs";
import { ExchangeRateMap } from "../models/exchangeRatesModel";

export class ExchangeRateRepo {
  jsonFilename: string;
  dataset: ExchangeRateMap;

  constructor(filename: string) {
    this.jsonFilename = filename;
    this.dataset = JSON.parse(readFileSync(this.jsonFilename, "utf-8"));
  }
}

Service repository pattern presents a very intuitive workflow for testing. As a result of using the pattern, logic is effectively decoupled and mock testing becomes a breeze:

• Router class utilises Service classes.
• Services class utilises only the repository.
• Repository class only needs a link to the specified database.

Summary

As a result of using service repository pattern, the application benefits from a clean separation of concerns, making it easier to test, maintain, and extend. Give service repository pattern a go, as it tends nicely to small and large projects alike. If you became interested in Node.js patterns, different approaches to testing and scaling your app: highly recommended reading on Node.js patterns by Mario Casciaro and Luciano Mammino. The book does not cover service repository pattern, but offers a great deal of depth in how structure Node.js applications.