CODEWITHSUNDEEP
pythonpython-3.xgrequests
bad_keypoints
bad_keypoints

Reputation: 1400

POSTing json data with grequests?

I need to make async POST requests with grequests.

My post body (in json), is this:

[{'params': {'source': 'widget',
   'id': 'http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif',
   'groupId': '@self',
   'nolog': 'true',
   'userId': '@viewer'},
  'method': 'pos.plusones.get',
  'id': 'p',
  'jsonrpc': '2.0',
  'apiVersion': 'v1',
  'key': 'p'}]

I need to change [0]['params']['id'] key's value to different URLs for the many POSTs I'll make.

So I'm doing:

myrequests = (grequests.post(POST_URL, data=fgp(a_url) for a_url in all_urls)

The fgp() method in my generator comprehension is a method that changes the [0]['params']['id'] to the a_url passed to it, in the POST body I'm sending.

And when I map the requests to responses:

myresponses = grequests.map(myrequests)

This is what I get, as many times as there are requests (obviously).

Traceback (most recent call last):
  File "/home/ashk/.virtualenvs/cosignp3/src/gevent/gevent/greenlet.py", line 340, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/ashk/.virtualenvs/cosignp3/lib/python3.4/site-packages/grequests.py", line 71, in send
    self.url, **merged_kwargs)
  File "/home/ashk/.virtualenvs/cosignp3/lib/python3.4/site-packages/requests/sessions.py", line 451, in request
    prep = self.prepare_request(req)
  File "/home/ashk/.virtualenvs/cosignp3/lib/python3.4/site-packages/requests/sessions.py", line 382, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/home/ashk/.virtualenvs/cosignp3/lib/python3.4/site-packages/requests/models.py", line 307, in prepare
    self.prepare_body(data, files, json)
  File "/home/ashk/.virtualenvs/cosignp3/lib/python3.4/site-packages/requests/models.py", line 456, in prepare_body
    body = self._encode_params(data)
  File "/home/ashk/.virtualenvs/cosignp3/lib/python3.4/site-packages/requests/models.py", line 89, in _encode_params
    for k, vs in to_key_val_list(data):
ValueError: too many values to unpack (expected 2)
<Greenlet at 0x7f0f7cbf33d8: <bound method AsyncRequest.send of <grequests.AsyncRequest object at 0x7f0f7c90d080>>(stream=False)> failed with ValueError

EDIT: Problem solved :-

I got to playing around, and put in the headers just like you would in the requests module.

I'm setting the headers kwarg argument to make encoding none, and content-type:

(grequests.post(POST_URL, data=fgp(a_url, j=True), headers={'Accept-Encoding':'none', 'Content-Type':'application/json'}) for a_url in urls)

And now I'm getting the output correctly:

In [64]: resps[0].json()
Out[64]: 
{'result': {'isSetByViewer': False,
  'kind': 'pos#plusones',
  'metadata': {'type': 'URL', 'globalCounts': {'count': 0.0 }},
  'id': 'http://example.com/dfg',
  'abtk': 'xxxxxxxxxxxxxxx'},
 'id': 'p'}

Note: Output edited a little bit to hide some data.

Upvotes: 1

Views: 6305

Answers (2)

Martijn Pieters
Martijn Pieters

Reputation: 1124748

You are trying to send JSON data; have requests encode this for you (and set the correct content type), by using the json argument instead of data:

myrequests = (grequests.post(POST_URL, json=fgp(a_url)) for a_url in all_urls)

Upvotes: 3

Daniel Roseman
Daniel Roseman

Reputation: 600026

You're missing a close paren in that line of grequests.post that you've shown. However, assuming it's actually there at the end of the line:

myrequests = (grequests.post(POST_URL, data=fgp(a_url) for a_url in all_urls))

that means that you're generating only a single post call but with multiple values for data. I think what you mean instead is:

myrequests = (grequests.post(POST_URL, data=fgp(a_url)) for a_url in all_urls)

Upvotes: 5

Related Questions