CODEWITHSUNDEEP
dictionarytypeshivecastinghiveql
Zachary Oldham
Zachary Oldham

Reputation: 868

How do I convert a `string` field to a `map<int, int>` field using hive?

I have a field that is a string representation of a map, as an example one of the rows is "1:2,3:4". Using the function str_to_map I have managed to convert that field to map<string, string>, where that example would be represented as {"1":"2", "3":"4"}. Unfortunately, I need this field to be map<int, int> (or really any integer types), where once again the example would be represented as {1:2, 3:4}. Is there any way to accomplish this in Hive without using a UDF? If not, is there a simple UDF that could accomplish this?

Upvotes: 1

Views: 963

Answers (1)

leftjoin
leftjoin

Reputation: 38325

Unfortunately, it is not possible to convert string or map<string, string> to map<int, int> using Hive native functions only. You can use brickhouse collect function. For example if you have table mytable with column str:

add jar '~/brickhouse/target/brickhouse-0.6.0.jar'; --check brickhouse site https://github.com/klout/brickhouse for instructions

create temporary function collect as 'brickhouse.udf.collect.CollectUDAF';
 
select d.str, collect(int(key), int(value)) as result
  from mytable d
       lateral view outer explode(str_to_map(str)) e as key, value
group by d.str;

Also they have another very useful function json_map, which can be used to cast JSON string to map of required type even simpler without exploding initial map. Your string can be converted to JSON by concatenating with { and }, then apply json_map function, see instructions on brickhouse site:

json_map(concat('{', str, '}'), 'int,int')

Upvotes: 1

Related Questions