CODEWITHSUNDEEP
open-telemetryopen-telemetry-collector
Eitan
Eitan

Reputation: 173

set Timestamp field on log data model in opentelemetry collector using transform processor

I've configured my OpenTelemetry Collector (otel/opentelemetry-collector-contrib:0.82.0) to receive events via the FluentForward receiver. The incoming events have a structure like this:

{
  "message": "hello world",
  "timestamp": "1692092369949725000"
}

My goal is to modify the final OTLP payload before it's sent to the exporter, so that it contains my message timestamp (which is in epoch format in nanoseconds) rather than the ingested time.

Here's the configuration I've tried:

receivers:
  fluentforward:
    endpoint: 0.0.0.0:8006

processors:
  transform/example:
    log_statements:
      - context: log
        statements:
          - set(timestamp, attributes["timestamp"])

exporters:
  logging:
    loglevel: debug

service:
  pipelines:
    logs:
      receivers: [fluentforward]
      processors: [transform/example]
      exporters: [logging]

However, I'm encountering an error:

Error: invalid configuration: processors::transform/example: unable to parse OTTL statement "set(timestamp, attributes["timestamp"])". Error details: error while parsing arguments for call to "set": invalid argument at position 0: invalid path expression [{timestamp []}]

Upvotes: 4

Views: 3671

Answers (2)

Tyler B
Tyler B

Reputation: 358

The path timestamp is not a valid path for the logs context.
All valid paths can be found here.
Also, your timestamp attribute appears to be a string, so you'll need to convert it to an int.

The following statement should work:

set(time_unix_nano, Int(attributes["timestamp"]))

Upvotes: 6

Craig Ringer
Craig Ringer

Reputation: 324475

For original timestamp, the field name you want is observed_time.

Why it's not called observed_timestamp when the docs call it Observed Timestamp, I don't know.

set(observed_time, attributes["ts"])

Upvotes: 2

Related Questions