CODEWITHSUNDEEP
jsonstreamuniquejq
yregex2017
yregex2017

Reputation: 87

Get unique strings from group of strings using JQ

I have strings extracted from jq , from which I just want to get unique values.

"a-b-c-v001"
"a-b-c-v002"
"a-b-c-v001"
"a-b-c-v003"
"a-b-c-v002"

I just need 3 results, unique

"a-b-c-v001"
"a-b-c-v002"
"a-b-c-v003"

I have tried unique & sort did not work - https://jqplay.org/s/xjND6Iv60T

Upvotes: 2

Views: 6996

Answers (2)

peak
peak

Reputation: 116740

Make sure your jq expression produces an array. For example, if your jq expression (the one producing a stream of strings) is E, then you could modify it to:

[E] | unique | .[]

or just:

[E] | unique[]

Avoiding a sort

unique involves a sort, which can easily be avoided if E as above produces a stream of strings:

INDEX(E;.)[]

So, in answer to a Q in a comment, a sort-free way to get the count of distinct strings would be:

def count(s): reduce s as $x (0;.+1);
count( INDEX(E;.)[] )

Upvotes: 9

RomanPerekhrest
RomanPerekhrest

Reputation: 92854

jq processor itself has unique function:

Let's assume the following input.json:

[
  "a-b-c-v001",
  "a-b-c-v002",
  "a-b-c-v001",
  "a-b-c-v003",
  "a-b-c-v002"
]

jq 'unique' input.json

The output:

[
  "a-b-c-v001",
  "a-b-c-v002",
  "a-b-c-v003"
]

Upvotes: 1

Related Questions