Today’s new knowledge #10 (Building a Flexible Query Builder for MongoDB with Mongoose)

Today’s Overview:

Kon'nichiwa, everyone! I hope you're all doing great. Recently I learned an effective way to querying in a MongoDB database and I wanted to share it with you. The process of creating a custom QueryBuilder class can be extremely useful. This class helps you build flexible, reusable queries that can handle filtering, sorting, pagination, and more. Hope you find this useful in your own projects. so let's get started!

Table of contents

• The QueryBuilder Class
  • Implementation of QueryBuilder
• UseCase of QueryBuilder
  • Search Functionality
  • Filter Functionality
  • Sort Functionality
  • Pagination Functionality
  • Limit Fields Functionality
• Benefits of QueryBuilder
• Conclusion

The QueryBuilder Class

import { FilterQuery, isValidObjectId, Model, Query } from 'mongoose';

class QueryBuilder<T> {
  public modelQuery: Query<T[], T>;
  public query: Record<string, unknown>;
  private allowedFields = [
    'searchTerm',
    'sort',
    'page',
    'limit',
    'fields',
    'id',
  ];

  constructor(
    modelQuery: Query<T[], T>,
    query: Record<string, unknown>,
    model: Model<T>,
  ) {
    this.modelQuery = modelQuery;
    this.query = query;
    this.allowedFields = [
      ...Object.keys(model.schema.paths),
      ...this.allowedFields,
    ];
  }

  private isPositiveInteger(value: unknown): boolean {
    return Number.isInteger(Number(value)) && Number(value) > 0;
  }
  private escapeString(value: string): string {
    return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
  }

  private validateQuery(
    query: Record<string, unknown>,
    strictValidation = true,
  ): Record<string, unknown> {
    const sanitizedQuery: Record<string, unknown> = {};

    for (const key in query) {
      if (!this.allowedFields.includes(key)) {
        if (strictValidation) {
          throw new Error(Invalid query parameter: ${key});
        }
        continue;
      }

      const value = query[key];

      if (key === 'page' || key === 'limit') {
        sanitizedQuery[key] = this.isPositiveInteger(value) ? Number(value) : 1;
      } else if (key === 'id' && typeof value === 'string') {
        sanitizedQuery[key] = isValidObjectId(value) ? value : null;
      } else if (typeof value === 'string') {
        sanitizedQuery[key] = this.escapeString(value);
      } else {
        sanitizedQuery[key] = value;
      }
    }
    return sanitizedQuery;
  }

  search(searchableFields: string[] ) {
    const searchTerm = this.query.searchTerm as string;
    if (searchTerm && searchableFields.length > 0) {
      this.modelQuery = this.modelQuery.find({
        $or: searchableFields.map((field) => ({
          [field]: { $regex: searchTerm, $options: 'i' },
        })),
      });
    }else {
      // If no searchable fields are specified, search across all fields
      this.modelQuery = this.modelQuery.find({
        $text: { $search: searchTerm } // Use text index if available
      });
    }
    return this;
  }

  filter() {
    const sanitizedQuery = this.validateQuery(this.query);
    const queryObj = { ...sanitizedQuery }; // copy

    // Filtering
    const excludeFields = ['searchTerm', 'sort', 'limit', 'page', 'fields'];

    excludeFields.forEach((el) => delete queryObj[el]);

    this.modelQuery = this.modelQuery.find(queryObj as FilterQuery<T>);

    return this;
  }

  sort(defaultSort = '-createdAt') {
    const sort =
      (this?.query?.sort as string)?.split(',')?.join(' ') || defaultSort;
    this.modelQuery = this.modelQuery.sort(sort as string);

    return this;
  }

  paginate(defaultPage = 1, defaultLimit = 10) {
    const page = Number(this?.query?.page) || defaultPage;
    const limit = Number(this?.query?.limit) || defaultLimit;
    const skip = (page - 1) * limit;

    this.modelQuery = this.modelQuery.skip(skip).limit(limit);

    return this;
  }

  limitFields(defaultFields = '-__v') {
    const fields =
      (this?.query?.fields as string)?.split(',')?.join(' ') || defaultFields;

    this.modelQuery = this.modelQuery.select(fields);
    return this;
  }
}

export default QueryBuilder;

The QueryBuilder class is generic and can handle any Mongoose model.

The constructor accepts:

1. modelQuery: The Mongoose query object that will be modified by the builder.
2. query: The query parameters passed by the user, typically from the request URL.
3. model: The Mongoose model, which helps determine the allowed fields based on the schema.

Implementation of QueryBuilder

It's simple. Imagine you have a Mongoose model called UserModel.

import QueryBuilder from "./QueryBuilder";
import UserModel from "./models/User";

const queryBuilder = new QueryBuilder(UserModel.find(), req.query, UserModel);

const results = await queryBuilder
  .search(["name.first", "name.last", "email"]) // Search in nested fields
  .filter()
  .sort()
  .paginate()
  .limitFields()
  .modelQuery.exec();

console.log(results);

UseCase of QueryBuilder

Search Functionality

The search() method allows you to perform full-text searches across multiple fields. You can specify the fields you want to search using the searchableFields parameter. I recommend using the searchableFields parameter to limit the fields you wish to search for performance and security reasons. searchTerm is used for partial matches.

The example endpoint is:GET

http://localhost:3000/user?searchTerm=John

Filter Functionality

The filter() method allows you to apply filters to your query. It uses the validateQuery() method to sanitize the query parameters and then applies the filters to the query. It is used for exact matches.

The example endpoint is:GET
http://localhost:3000/user?name.first=John&name.last=Doe

So in this example, you will get all users with the first name John and last name Doe.

Sort Functionality

The sort() method allows you to sort the results of your query. It uses the sort() method to apply the sort to the query. In the query parameter, you can specify the sort order using the sort parameter.

The example endpoint is:GET
http://localhost:3000/user?sort=name
http://localhost:3000/user?sort=-name

For ascending order you should use sort=fieldName and for descending order, you should use sort=-fieldName. Note that you have to use - before the field name.

Pagination Functionality

Pagination is essential when dealing with large datasets. The paginate() method accepts parameters for the current page and the number of items per page (limit). It calculates the skip value and limits the results accordingly.

The example endpoint is:GET
http://localhost:3000/user?page=2&limit=10

Limit Fields Functionality

The limitFields() method allows you to specify the fields you want to include in the results.

The example endpoint is:GET
http://localhost:3000/user?fields=name.first,email

Note that you have to use , after each field name.

Benefits of QueryBuilder

• It provides a flexible way to build complex queries.
• It handles validation and sanitization of query parameters.
• It ensures that only allowed fields are included in the query.
• It provides a consistent way to apply filters, sorting, pagination, and limiting fields to your queries.
• It helps to keep your code DRY and maintainable.
• It makes it easy to add new features to your queries without modifying the existing code.
• It helps to ensure that your queries are secure and efficient.
• It makes the code more readable and maintainable.

Conclusion

The QueryBuilder class provides a powerful way to manage MongoDB queries with Mongoose. A query builder like this is a great tool to simplify database queries while ensuring security and efficiency. So that's it. I hope you find this useful in your projects. Don't forget to give like and share it if you find this useful. Thanks for reading. Talk to you soon.