Open Telemetry makes Next.js initialization extremely slow

Question

When initializing Nextjs via node -r and node --require the application takes 4-5 minutes to load. The telemetry script loads within the first 5 seconds so this issue is likely related to Nextjs or node. This contrasts calling without node require module's 30 second load time.

Without node require module:

"dev": "env-cmd -f environments/.env.development next dev",

With node require module:

"dev": "env-cmd -f environments/.env.development node --require ./tracing.js ./node_modules/next/dist/bin/next dev",

This implementation is based on ross-hagan's blog about instrument-nextjs-opentelemetry

Alternative to a custom server

I originally started off with a completely separate tracing.js script with the contents of our start.js script without the startServer call.

This separates the telemetry SDK startup from the server. You can then keep the Next.js built-in startup behaviours by using the node --require (-r) to load in a module before starting the Next app.

In your npm run dev script in your package.json this looks like:

node -r tracing.js ./node_modules/.bin/next dev

I switched away from this after frustration getting the node command to run in a Dockerfile as this was destined for a Google Kubernetes Engine runtime. Also, some concern about use of the --require flag.

See if it works for you to do it this way, as a Next.js custom server comes with some consequences documented in their docs!

---

I've tried two separate tracing.js without success in reducing load times.

tracing.js provided by open telemetry:

/* tracing.js */

// Require dependencies
const opentelemetry = require("@opentelemetry/sdk-node");
const { getNodeAutoInstrumentations } = require("@opentelemetry/auto-instrumentations-node");
const { diag, DiagConsoleLogger, DiagLogLevel } = require('@opentelemetry/api');

// For troubleshooting, set the log level to DiagLogLevel.DEBUG
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO);

const sdk = new opentelemetry.NodeSDK({
  traceExporter: new opentelemetry.tracing.ConsoleSpanExporter(),
  instrumentations: [getNodeAutoInstrumentations()]
});

sdk.start()

As well as the customized tracing.js for jaeger:

const process = require('process');
const opentelemetry = require('@opentelemetry/sdk-node');
const {
  getNodeAutoInstrumentations,
} = require('@opentelemetry/auto-instrumentations-node');
const { Resource } = require('@opentelemetry/resources');
const {
  SemanticResourceAttributes,
} = require('@opentelemetry/semantic-conventions');
const { JaegerExporter } = require('@opentelemetry/exporter-jaeger');

const hostName = process.env.OTEL_TRACE_HOST || 'localhost';

const options = {
  tags: [],
  endpoint: `http://${hostName}:1234/api/traces`,
};
const traceExporter = new JaegerExporter(options);

// configure the SDK to export telemetry data to the console
// enable all auto-instrumentations from the meta package
const sdk = new opentelemetry.NodeSDK({
  resource: new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: 'my_app',
  }),
  traceExporter,
  instrumentations: [
    getNodeAutoInstrumentations({
      // Each of the auto-instrumentations
      // can have config set here or you can
      // npm install each individually and not use the auto-instruments
      '@opentelemetry/instrumentation-http': {
        ignoreIncomingPaths: [
          // Pattern match to filter endpoints
          // that you really want to stop altogether
          '/ping',

          // You can filter conditionally
          // Next.js gets a little too chatty
          // if you trace all the incoming requests
          ...(process.env.NODE_ENV !== 'production'
            ? [/^\/_next\/static.*/]
            : []),
        ],

        // This gives your request spans a more meaningful name
        // than `HTTP GET`
        requestHook: (span, request) => {
          span.setAttributes({
            name: `${request.method} ${request.url || request.path}`,
          });
        },

        // Re-assign the root span's attributes
        startIncomingSpanHook: (request) => {
          return {
            name: `${request.method} ${request.url || request.path}`,
            'request.path': request.url || request.path,
          };
        },
      },
    }),
  ],
});

// initialize the SDK and register with the OpenTelemetry API
// this enables the API to record telemetry
sdk
  .start()
  .then(() => console.log('Tracing initialized'))
  .catch((error) =>
    console.log('Error initializing tracing and starting server', error)
  );

// gracefully shut down the SDK on process exit
process.on('SIGTERM', () => {
  sdk
    .shutdown()
    .then(() => console.log('Tracing terminated'))
    .catch((error) => console.log('Error terminating tracing', error))
    .finally(() => process.exit(0));
});

---

Separately, building and then serving does not speed up the load times either.

Answer 1

Check to see if you were experiencing the issue that was reported here [@opentelemetry/instrumentation] require performance grows linearly with each instrumentation plugin. Within this issue, there was a bug that caused instrumentation to be layered on top of itself repeatedly. They have since fixed the issue.

See also this answer.