CODEWITHSUNDEEP
mapreduceriak
Rambo
Rambo

Reputation: 159

Riak.mapValuesJson failing when using a secondary index in mapreduce

I have a Riak DB running eLevelDB backend and when I query is with this map reduce, I get all the data returned as a JSON string (As expected):

{
"inputs":"TWEETS_BY_ID",
"query":[
    {"map":
        {"language":"javascript",
         "name":"Riak.mapValuesJson"
        }
    }]
}

This query is slow and I really only want to query a small subset of the entire bucket. I have a secondary index with a timestamp, so I try and run this mapreduce:

{
"inputs":
    {
        "bucket":"TWEETS_BY_ID",
        "index":"timestamp_int",
        "start":"1375736484000",
        "end":"1375736485000"
    },
"query":[
    {"map":
        {"language":"javascript",
         "name":"Riak.mapValuesJson",
         "keep":true
        }
    }
]
}

I get this error:

{error,{exit,{json_encode,{bad_term,{1375736484000,<<"364491348659142656">>}}},
             [{mochijson2,json_encode,2,
                          [{file,"src/mochijson2.erl"},{line,149}]},
              {mochijson2,'-json_encode_proplist/2-fun-0-',3,
                          [{file,"src/mochijson2.erl"},{line,167}]},
              {lists,foldl,3,[{file,"lists.erl"},{line,1197}]},
              {mochijson2,json_encode_proplist,2,
                          [{file,"src/mochijson2.erl"},{line,170}]},
              {mochijson2,'-json_encode_proplist/2-fun-0-',3,
                          [{file,"src/mochijson2.erl"},{line,167}]},
              {lists,foldl,3,[{file,"lists.erl"},{line,1197}]},
              {mochijson2,json_encode_proplist,2,
                          [{file,"src/mochijson2.erl"},{line,170}]},
              {mochijson2,'-json_encode_array/2-fun-0-',3,
                          [{file,"src/mochijson2.erl"},{line,157}]}]}}

I did upgrade to 1.4.1 that was released today and had a bug report in it about secondary indexes and Javascript mapreduces, but that does not seem to have changed my error. I'm not even really sure how to start debugging this. Any suggestions?

Upvotes: 3

Views: 140

Answers (1)

Joe
Joe

Reputation: 28336

You have found a bug. There is new behavior that returns the matched term when you do a range query on an index that currently makes it incompatible with mapreduce. To workaround, first get the source tar if you don't have it already. Untar the file and edit the file riak_kv_index.hrl in riak-1.4.1/deps/riak_kv/include, change the line

          return_terms=true :: boolean(), %% Note, should be false for an equals query

to

          return_terms=false :: boolean(), %% Note, should be false for an equals query

To compile just the index module, run

 /usr/lib/riak/erts-5.9.1/bin/erlc -I riak-1.4.1/deps/riak_kv/include riak-1.4.1/deps/riak_kv/src/riak_index.erl

That should create a new riak_index.beam file for you.

Locate your old riak_index.beam, make a backup of it, replace it with the new one, stop then start riak. Repeat this on all of your nodes, and you should be back in business.

Upvotes: 2

Related Questions