CODEWITHSUNDEEP
google-cloud-dataflowapache-beam
Anna Kasikova
Anna Kasikova

Reputation: 37

BeamSql support for arrays and maps

We pulled down the BeamSql code from the master branch on GitHub, since it seems to support Array, Map, and Row type fields. Does it have a way of querying those fields? or do we need to write UDFs for that?

Upvotes: 0

Views: 248

Answers (1)

Anton
Anton

Reputation: 2539

You should be able to query them, see the examples in the tests:

  • arrays: "SELECT f_stringArr[0] FROM PCOLLECTION";
  • nested rows and arrays: "SELECT PCOLLECTION.f_nestedRow.f_nestedArray[1] FROM PCOLLECTION";
  • maps: "SELECT f_intStringMap['key11'] FROM PCOLLECTION";

Caveats:

  • you may need to fully qualify the fields;
  • complex types support is under active development and there are known issues around it, e.g. BEAM-4161 or BEAM-4167;

Upvotes: 1

Related Questions