Nested Dynamic Modules in NestJS

I spent a few hours debugging an incident caused by a misunderstanding of dynamic modules this week, and want to share my learnings to help others avoid my mistake.

Dynamic modules allow modules to be imported with customized configuration. In our case we use TemporalModule.registerWorkerAsync from the nestjs-temporal library to register a Temporal worker. I won’t go into the details of Temporal since they aren’t directly relevant, but definitely check it out if you’re looking for asynchronous workflow orchestration. What’s important is that we need to register multiple workers, so we import multiple instantiations of the dynamic module. This typically looks like:

import { Module } from '@nestjs/common';
import { TemporalModule } from 'nestjs-temporal';

@Module({
  imports: [
    TemporalModule.registerWorkerAsync({
      useFactory: async () => {
        const sharedWorkerOptions = await getWorkerOptions();

        return {
          workerOptions: <WorkerOptions>{
            ...sharedWorkerOptions,
            taskQueue: 'loyalty',
            workflowsPath: require.resolve('./temporal/workflow'),
          },
          activityClasses: [LoyaltyActivity],
        };
      },
    })
  ],
})

export class LoyaltyWorkerModule {}

When our application loads, Nest calls useFactory to render the various options passed to nestjs-temporal, which in turn runs some Temporal code. Again, we have about a dozen of these with responsibilities ranging from POS data synchronization to updating electronic shelf labels.

There is a lot of duplicated code in that call that we don’t expect to ever change. This code also allows for the potential to use an incorrect task queue name that can result in workflows not being executed.

Make it easy to do the right thing, and hard to do the wrong thing.

I first heard the saying, “make it easy to do the right thing, and hard to do the wrong thing,” from a security team. It’s something I think about as I write code I know will become a pattern reused elsewhere. Given this, I opted to write a function that could be called to create the dynamic module:

export function registerTemporalWorker({  
  taskQueue,  
  workflowsPath,  
  activityClasses,  
}: {  
  taskQueue: string;
  workflowsPath: string;  
  activityClasses: object[];  
}): DynamicModule {
  // NOTE: The code to avoid duplicate registrations will be shown later.
  return TemporalModule.registerWorkerAsync({
    useFactory: async () => {  
      const workerOptions = await getNestWorkerOptions({
        taskQueue,
        workflowsPath,  
      });  
      return { workerOptions, activityClasses };  
    },  
  });  
}

Here is what a registration looks like:

import { Module } from '@nestjs/common';

@Module({
  imports: [
	registerTemporalWorker({  
	  taskQueue: 'loyalty',
	  workflowsPath: require.resolve('./temporal/workflow'),
	  activityClasses: [LoyaltyActivity],
    }),
  ],
})
export class LoyaltyWorkerModule {}

This does not work!

The application will start but only one worker will be created. If you have a single application servicing multiple queues, thus requiring multiple workers, only the worker imported/created first will be created. The workflows of the other queues will remain untouched.

Why? I still don’t fully understand. After many hours of searching Google and Discord, writing prompts for ChatGPT and Gemini, and trial-and-error, I learned the output of the first call is cached by Nest’s dependency injection (DI) mechanism, but it’s unclear what it’s keying this cache on.

However, while looking through some of the source code, I came across a property on the ConfigurableModuleAsyncOptions interface:

interface ConfigurableModuleAsyncOptions {
  /**
   * List of parent module's providers that will be filtered to only provide necessary
   * providers for the 'inject' array
   * useful to pass options to nested async modules
   */
  provideInjectionTokensFrom?: Provider[];
}

I must admit I didn’t see the line, “useful to pass options to nested async modules,” until writing this post. Even now, I’m still not sure it would have guided me to my solution sooner. Regardless, I decided bo wrap the dynamic module in another dynamic module:

import { DynamicModule, Module } from '@nestjs/common';
import { WorkerOptions } from '@temporalio/worker';
// eslint-disable-next-line no-restricted-imports  
import { TemporalModule as NestTemporalModule } from 'nestjs-temporal';

export interface TemporalModuleOptions {
  taskQueue: string;
  workflowsPath: string;
  activityClasses: object[];
}

@Module({})
export class TemporalModule {
  private static OPTIONS_PROVIDER_KEY = 'TemporalModuleOptions';
  protected static registeredWorkers: Set<string> = new Set<string>([]);

  /**
   * Registers a Temporal worker.   
   *   
   * @note This should only be called once per task queue.
   */  
  public static registerWorker(options: TemporalModuleOptions): DynamicModule {
    const taskQueue = options.taskQueue;
    if (this.registeredWorkers.has(taskQueue)) {
      // NOTE: There should only be one worker registered per queue; 
      //  otherwise, some workflows will be ignored.  
      throw new Error(
        `A Temporal worker is already registered for task queue ${taskQueue}! Either combine the code/modules or use a different task queue.`
      );
    }

    this.registeredWorkers.add(taskQueue);

    return {
      module: TemporalModule,
      imports: [
        NestTemporalModule.registerWorkerAsync({
          inject: [this.OPTIONS_PROVIDER_KEY],
          // NOTE (clintonb): This is *key* to ensuring the factory is not cached, 
          //  and multiple workers are created/run when this method is called by 
          //  multiple modules. Far too many hours were spent discovering this fact. 
          //  Be careful not to remove this!          
          provideInjectionTokensFrom: [
            {
              // NOTE: This value should not be exposed outside of this dynamic module.  
              provide: this.OPTIONS_PROVIDER_KEY,
              useValue: options,
            },
          ],
          useFactory: async (
            options: TemporalModuleOptions
          ) => {
            const { activityClasses } = options;
            const workerOptions = await this.getNestWorkerOptions(options);
            return { workerOptions, activityClasses };
          },
        }),
      ],
    };
  }

  protected static async getNestWorkerOptions(
    {
      taskQueue,
      workflowsPath,
    }: {
      taskQueue: string;
      workflowsPath: string;
    }): Promise<WorkerOptions> {
    const sharedWorkerOptions = await getWorkerOptions();
    return <WorkerOptions>{
      ...sharedWorkerOptions,
      taskQueue,
      workflowsPath,
    };
  }
}

The options passed to useFactory are now injected by the provider defined in provideInjectionTokensFrom. As I understand it—I haven’t dug into the internals of Nest’s DI implementation—this forces a new module to be created via useFactory because the dependencies differ compared to the calls made for other queues/workers. I’ve roughly confirmed this by calling registerWorker twice with the exact same parameters (after removing my code to avoid duplicate registration) and confirming useFactory is only called once and only one worker is created.

Creating a worker now looks like this:

import { Module } from '@nestjs/common';
import { TemporalModule } from '@vori/nest/libs/temporal/temporal.module';

@Module({
  imports: [
    TemporalModule.registerWorker({  
	  taskQueue: 'loyalty',  
	  workflowsPath: require.resolve('./temporal/workflow'),  
	  activityClasses: [PosActivity],  
	}),
  ],
})
export class LoyaltyWorkerModule {}

This is much safer in terms of preventing invalid queue configurations and minimizes duplication. I also added an ESLint rule to prevent importing TemporalModule from nestjs-temporal, so there is minimal risk of a developer using the incorrect module or worker options:

module.exports = {
  rules: {  
    'no-restricted-imports': [  
      'error',  
      {  
        paths: [  
          {  
            name: 'nestjs-temporal',  
            importNames: ['TemporalModule'],  
            message:  
              'Please use import from `@vori/nest/libs/temporal/temporal.module` instead.',  
          },  
        ],  
      },  
    ],  
  },  
};

As of this writing, a search on Google for provideInjectionTokensFrom yields about six hits. One goes to a closed GitHub ticket that was not helpful. The others are effectively spam. Hopefully this will be number seven, and saves someone else from the hours of anguish I experienced.

If you have a better understanding of dependency caching, please drop a line explaining why this works. Better yet, write a blog post—or something somewhere indexable—where others can learn.

← Previous Post