drago
drago

Reputation: 1

Convert Arrays to Rows in Opensearch table/viz

I have an index with many records similar to (truncated):

{
"_source": {
    "@timestamp": "2025-01-18T17:21:45Z",
    "calculated_fields": {
      "affectedMessages": [
        {
          "Id": "AAkALgAAAAAAHYQDEapAkALgAAAAAAHYQDEX6dxDUafP2CIgAkALgAAAAAAHYQDEgAA",
          "Path": "messages",
          "MessageItemId": "AAkALgADEapAkALgAAAAAAHYQDEX6dQDEapAkALgAAAAAAHYQDEX6dxDUafP2CIgAkALgAAAAAAHYQDEgAC4GCAaR-tbRLVHWIZgnBGe",
          "SizeInBytes": 22189
        },
        {
          "Id": "AAkALgAAAAAAHYQDEapDEapAkALgAAAAAAHYQDEX6dxDUafP2CIgAkAjgAA",
          "Path": "messages",
          "MessageItemId": "AAkALgAAAHYQDEapDEapAkALgAAAAAAHYQDCpvyX6dxDUafP2CIgdPO8AALD8vtjgAAARIAEAB-igocZ5K0S43VR5qG8mHr",
          "SizeInBytes": 2521
        },
        {
          "Id": "AAkALgDEapAkALgAAAAAAHYQDEX6dDEapAkALgAAAAAAHYQDEX6dAHYQDEapDEapAkALgAAAAAAHYQDafP2CIgdPO8AALD8vtjgAA",
          "Path": "messages",
          "MessageItemId": "AAkALDEapAkAHYQDEapDEapAkALgAAAAAAHYQDAkAEAA7FZgRR2i8RqNwTbKbT27f",
          "SizeInBytes": 2593
        }
      ]
    }
  }
} 

when rendered in a saved search table, each item in affectedMessages is rendered as arrays across the four columns corresponding to the array objects e.g.:

data table example

Two main things we want to accomplish:

  1. generate a time-ordered table of all results, but split the affectedMessages array across multiple table rows. e.g. one row per item in affectedMessages
  2. export the resulting data table to CSV

Is there any way to accomplish this in OpenSearch 2.17? I think the solution is the long-sought pivot table functionality, which I'm not holding my breath for.

I've tried doing this with a saved search, but there's no way to split the results on affectedMessages that I've been able to find. I've tried building a visualization, but the closest viz is DataTable, and that requires results to be bucketized, including @timestamp, but we ultimately want a full cronological export of all these events and don't want some uncategorized in an "Others" bucket

Other possible options don't allow for data export (e.g. a custom vega chart)

Upvotes: 0

Views: 50

Answers (0)

Related Questions