🔧 Dynamic NestJS Listeners: Discover the Power of Lazy Loading
Nachrichtenbereich: 🔧 Programmierung
🔗 Quelle: dev.to
In this post, I will show you how to register HTTP routes and message consumers in NestJS applications dynamically.
This demonstration will start by uncovering some NestJS internals, particularly an undocumented feature: the DiscoveryModule
🧐.
By dynamic routes, I mean routes unknown at the time of development but defined at runtime, late in the boot process.
If reading bores you, skip the talk and jump directly to the repository.
Problem
The declarative approach is great but can also bring some limitations.
In the routing case, the decorators used to declare HTTP routes and microservices listeners (@Get
, @Post
, @EventPattern
, @MessagePattern
) require static values known during development time and cannot be late-bound.
Examples:
-
@Get('users')
defines a static route that can only change by modifying the code. -
@MessagePattern('user.created')
is not adaptable to dynamic topic structures.
Use cases
Scenario | Business Impact | Description |
---|---|---|
Modular Plugin Architecture | Allow developers to create modular plugins that can be easily integrated into a core application without modifying the underlying code. | A web development platform allows users to install plugins that add new features, each with its own set of HTTP routes. |
White Labeling for E-commerce Platforms | Enable e-commerce platforms to offer white-labeled application versions to different clients without modifying the underlying codebase. | A popular e-commerce platform allows its clients (e.g., online retailers) to create custom platform versions with unique routes and listeners. |
Customizable Integration Layers | Allow customers to customize the integration layer of an application based on their specific business needs without requiring code changes. | A CRM system allows users to integrate it with various external services (e.g., email marketing platforms and sales tools). |
Feature Flags for Enterprise Applications | Enable enterprise applications to easily toggle certain features on or off based on user roles or organizational policies. | A large enterprise application allows administrators to toggle specific features (e.g., access to advanced analytics tools) on or off for different departments. |
Multi-Tenancy in SaaS Applications | Allow Software-as-a-Service (SaaS) applications to support multiple tenants with their custom routes and listeners. | A SaaS-based customer engagement platform allows clients to create custom application versions with unique routes and listeners. |
NestJS scanning and registration lifecycle
Word to the wise: to declare dynamic routes and listeners, we must first understand how NestJS inspects modules and scans the metadata to register them.
For both HTTP routes and microservices listeners, NestJS will scan the metadata and register the routes and listeners during the initialization phase when NestApplication.init
and NestMicroservice.init
are called.
However, each of them has its own registration process and components:
HTTP routes
Microservices listeners
When calling
NestApplication.listen
andNestMicroservice.listen
, theinit
methods will be called if they were not before.
We have the following options to register dynamic routes and listeners:
- create custom explorers as providers that scan the metadata and replace the placeholders with actual values
- create custom adapters for HTTP API by implementing the
HttpServer
interface - create custom transport strategies for microservices by extending
Server
Option 1 is the most straightforward and will be the focus of this article.
Create a Proof of Concept
The demonstration is a simple NestJS application with a single HTTP route, a single message consumer, and the following requirements:
- Enable lazy evaluation of routes or message patterns with custom decorators and metadata explorers
- Support for both HTTP routes and message consumers
- Use environment variables to define the route and message pattern prefixes
- Replace the placeholders pattern before registering the routes and listeners (before
NestJSApplication.init
andapp.connectMicroservice
)
Seems like a good plan, but how can we set up the placeholders and replace them with actual values?
We could use the reflect-metadata library to store the metadata and then scan it manually to replace the placeholders with actual values, but there is a better way.
During my NestJS codebase review, I started following the MetadataScanner
thread and noticed it was also used by the DiscoveryModule
. This module provides a powerful tool for exploring and inspecting the providers, controllers, and modules in your NestJS application, and it turns out to be the perfect tool for our use case.
Funny enough, it has been in the NestJS core for several years but is not used internally. One could think it is part of the public API, but surprisingly, it is still undocumented. Do you also get this treasure hunt feeling? 🏴☠️
Since the approach is very similar for HTTP and microservices listeners, I only explain the registration process of HTTP routes. You can use the same approach to register the microservice listeners, as can be seen in this internal library.
The approch is based on the following steps:
- Create a custom decorator
- Create the metadata scanner
- Declare the dynamic route
- Import the custom explorers
- Update ConfigService type and validator (optional)
- Testing (optional, or not 😏)
1. Create a custom decorator
For the HTTP routes, the starting point is the custom decorator: CustomHttpMethod
, which takes a method and a path as arguments:
- The path is a template string (e.g. ,
$HTTP_ROUTE_PREFIX/:id
) containing placeholders that actual values will replace. - The method is one of the HTTP methods (e.g., GET, POST, etc).
This decorator is used above the controller methods to define the dynamic routes (e.g., @CustomHttpMethod({ method: 'GET', path: '$HTTP_ROUTE_PREFIX/:id' })
).
import { DiscoveryService } from '@nestjs/core';
import type { Method } from 'axios';
export const CustomHttpMethod = DiscoveryService.createDecorator<{
method: Method;
path: string;
}>();
You can also find an example using only the Reflect API from
reflect-metadata
library, in the example repository.
2. Create the metadata scanner
Register the custom explorer
The scanner is exposed as a dynamic NestJS module—CustomHttpMethodModule
—that can be imported into any module to enable dynamic HTTP routes.
Dynamic module enhances reusability and allows for configuration from the outside.
The module is configurable with:
- a
store
: a map of keys (placeholders) and values (replacements). - a list of
modules
containing the controllers to scan for the custom decorators.
export interface ICustomHttpMethodModuleOptions {
store: Map<string, string>;
modules: Type<unknown>[];
}
- The module can be created using the
forRoot
andforRootAsync
methods, which return aDynamicModule
object containing the module configuration. - The module will import the
DiscoveryModule
to enable the scanning of the controllers and methods. - It provides and exports the
CustomHttpMethodExplorer
service, which injects theDiscoveryService
andMetadataScanner
.
@Module({})
export class CustomHttpMethodModule {
static forRoot(
options: ICustomHttpMethodModuleOptions,
isGlobal?: boolean
): DynamicModule {
return {
module: CustomHttpMethodModule,
imports: [DiscoveryModule],
providers: [
{ provide: CustomHttpMethodModuleOptions, useValue: options },
CustomHttpMethodExplorer,
],
exports: [CustomHttpMethodExplorer],
global: isGlobal,
};
}
static forRootAsync(
options: CustomHttpMethodModuleAsyncOptions,
isGlobal?: boolean
): DynamicModule {
return {
module: CustomHttpMethodModule,
imports: options.imports
? [...options.imports, DiscoveryModule]
: [DiscoveryModule],
providers: [
...this.createAsyncProviders(options),
CustomHttpMethodExplorer,
],
exports: [CustomHttpMethodExplorer],
global: isGlobal,
};
}
private static createAsyncProviders(
options: CustomHttpMethodModuleAsyncOptions
): Provider[] {
if (options.useFactory) {
return [
{
provide: CustomHttpMethodModuleOptions,
useFactory: options.useFactory,
inject: options.inject ?? [],
},
];
}
throw new Error('Invalid CustomHttpMethodModuleAsyncOptions');
}
}
Iterate over the controllers and methods
The CustomHttpMethodExplorer
is a provider that will:
- retrieve the controllers attached to the
modules
provided in the options - scan and iterate over each controller's methods
- search for methods that use the
CustomHttpMethod
decorator - substitute the placeholders with actual values from the
store
- decorate the method with the NestJS HTTP method decorator (e.g.,
@Get
,@Post
, etc.)
One of the benefits of using the DiscoveryService
is that it provides methods to explore the metadata of the controllers and methods [getProviders
, getControllers
, getAllMethodNames
and getMetadataByDecorator
], without having to use low-level APIs like Reflect
.
Bonus: the discoveryService.getMetadataByDecorator
infers the type of the metadata.
The
CustomHttpMethodExplorer
service is instantiated when the module is imported and will scan the controllers and methods to replace the placeholders with actual values. > Alternatively, you can manually trigger the exploration in themain.ts
file.
@Injectable()
export class CustomHttpMethodExplorer {
readonly logger = new Logger(CustomHttpMethodExplorer.name);
private readonly methodsMap = new Map(
Object.entries({
GET: Get,
POST: Post,
PUT: Put,
DELETE: Delete,
PATCH: Patch,
OPTIONS: Options,
HEAD: Head,
} as const)
);
constructor(
@Inject(CustomHttpMethodModuleOptions)
private readonly options: CustomHttpMethodModuleOptions,
private readonly discoveryService: DiscoveryService,
private readonly metadataScanner: MetadataScanner
) {
this.process();
}
get store(): Map<string, string> {
return this.options.store;
}
get modules(): Type<unknown>[] {
return this.options.modules;
}
private substituteValues(input: string): string {
return input.replace(
/\$(\w+)/g,
(match, p1) => this.store.get(p1) || match
);
}
private getMethodDecorator(
method: string
): (path: string | string[]) => MethodDecorator {
return this.methodMap.get(method.ToUpperCase()) || Get;
}
process() {
const instances = this.discoveryService.getControllers({
include: this.modules,
});
for (const wrapper of instances) {
const handlers = this.metadataScanner.getAllMethodNames(
wrapper.metatype.prototype
);
this.logger.log(`${wrapper.name}:`);
for (const handler of handlers) {
const metadata = this.discoveryService.getMetadataByDecorator(
CustomHttpMethod,
wrapper,
handler
);
if (!metadata) {
continue;
}
const { method, path } = metadata;
const fulfilledPath = this.substituteValues(path);
const decorator = this.getMethodDecorator(method);
decorator(fulfilledPath)(
wrapper.metatype,
handler,
Object.getOwnPropertyDescriptor(
wrapper.metatype.prototype,
handler
) as PropertyDescriptor
);
this.logger.log(`Mapped {${method} ${fulfilledPath}} route`);
}
}
}
}
If you like it more low level, you can also find an example using only Reflect API, in the example repository
3. Declare the dynamic route
@Controller()
export class AppController {
constructor(private readonly appService: AppService) {}
@CustomHttpMethod({ method: 'GET', path: '$HTTP_METHOD_PREFIX/:id' })
getData(@Param('id') id: string) {
return this.appService.getData(id);
}
//...
}
4. Import the custom explorers
In app.module.ts
, add the custom explorers to the imports array.
@Module({
imports: [
// ...
CustomHttpMethodModule.forRootAsync({
inject: [ConfigService],
useFactory: (
configService: ConfigService<EnvironmentVariables, true>
) => {
const store = new Map<string, string>();
store.set(
'HTTP_METHOD_PREFIX',
configService.get('HTTP_METHOD_PREFIX')
);
return { store, modules: [AppModule] };
},
}),
// ...
],
controllers: [AppController],
providers: [AppService],
})
export class AppModule {}
5. Update ConfigService type and validator
To provide a better developer experience, we can use the class-transformer
and class-validator
libraries to validate the configuration object and provide default values (any other validation library will do).
import { Expose } from 'class-transformer';
import { IsInt, IsPositive, IsString, IsUrl, Min } from 'class-validator';
export class EnvironmentVariables {
//...
@Expose()
@IsString()
HTTP_METHOD_PREFIX = 'http-demo-1';
//...
}
- The
ConfigModule
is updated to use theplainToInstance
function fromclass-transformer
to transform the configuration object into an instance of theEnvironmentVariables
class. - The
validateSync
function fromclass-validator
validates the instance of theEnvironmentVariables
class. - After this validation we can increase the
ConfigService
type safety by including theEnvironmentVariables
class (e.g.,ConfigService<EnvironmentVariables, true>
).
@Module({
imports: [
ConfigModule.forRoot({
cache: true,
isGlobal: true,
validate: (config) => {
const validatedConfig = plainToInstance(EnvironmentVariables, config, {
excludeExtraneousValues: true,
exposeDefaultValues: true,
});
const errors = validateSync(validatedConfig, {
skipMissingProperties: false,
});
if (errors.length > 0) {
throw new Error(errors.toString());
}
return validatedConfig;
},
}),
//...
],
controllers: [AppController],
providers: [AppService],
})
export class AppModule {}
6. Testing
The workspace includes E2E tests that will send:
- HTTP request to the dynamic route
- MQTT message to the dynamic listener
Feel free to run the tests to see the demonstration in action.
docker compose up -d
npx nx run demo-1:serve
# in another terminal
nx run demo-1-e2e:e2e
Going further
With this knowledge, you can implement the following scenarios:
- Replace the template string with a
printf
-like syntax - Fetch routes and listeners from a remote service or a database
- Build your explorer modules to create a plugin system
- Create a reusable generic controller to avoid repeating decorators
- Create groups of controllers/providers
- Attach decorators to all controllers
Conclusion
I hope you enjoyed this demonstration and learned something new about NestJS internals.
If you want to show your appreciation, you can find the code in this repository and give it a star ⭐️.
...
🔧 Lazy loading vs Eager loading
📈 37.15 Punkte
🔧 Programmierung
🔧 Có thể bạn chưa biết (Phần 1)
📈 34.69 Punkte
🔧 Programmierung
🔧 Lazy-loading the animations package in Angular
📈 25.81 Punkte
🔧 Programmierung
🔧 Image Lazy Loading
📈 25.81 Punkte
🔧 Programmierung
🔧 Angular: Lazy Loading
📈 25.81 Punkte
🔧 Programmierung
🔧 How Lazy Loading Works in Web Development
📈 25.81 Punkte
🔧 Programmierung
🔧 React.js Lazy Loading: EXPLAINED
📈 25.81 Punkte
🔧 Programmierung
🔧 Attempt #20 - Lazy Loading
📈 25.81 Punkte
🔧 Programmierung