CODEWITHSUNDEEP
pythonpandasdataframeinfluxdbinfluxdb-python
lukewitmer
lukewitmer

Reputation: 1183

InfluxDB DataFrameClient KeyError: 83

I am building a somewhat automated query builder for a process I have that needs to query data from influx for a time range. Towards that end, I need to filter by time, which means I need epoch millisecond strings in my query. This code throws the following error.

import pandas as pd

ts = pd.Timestamp(pd.Timestamp('now', tz='US/Pacific').date(), tz='US/Pacific')
ts_str = str(int(ts.value / (10 ** 6)))
query_str = 'SELECT * FROM "table"."measurement" WHERE time <= ' + \
            ts_str + 'ms LIMIT 10'
json_payload = client.query(query_str)
print(json_payload)

and the stack trace:

Traceback (most recent call last):
  File "my_code.py", line 31, in query
    return self.client.query(query)
  File "/usr/local/lib/python2.7/site-packages/influxdb/_dataframe_client.py", line 138, in query
    results = super(DataFrameClient, self).query(query, database=database)
  File "/usr/local/lib/python2.7/site-packages/influxdb/client.py", line 339, in query
    expected_response_code=expected_response_code
  File "/usr/local/lib/python2.7/site-packages/influxdb/client.py", line 239, in request
    timeout=self._timeout
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 474, in request
    prep = self.prepare_request(req)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 407, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 302, in prepare
    self.prepare_url(url, params)
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 426, in prepare_url
    enc_params = self._encode_params(params)
  File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 104, in _encode_params
    return urlencode(result, doseq=True)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1353, in urlencode
    v = quote_plus(v)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1308, in quote_plus
    s = quote(s, safe + ' ')
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1303, in quote
    return ''.join(map(quoter, s))
KeyError: 83

Upvotes: 0

Views: 1571

Answers (2)

C.J
C.J

Reputation: 55

I can confirm your findings in a similar context: concatenating strings using str() from the future package and standard Python 2.7 strings. The following function can be given epoch floats as arguments to define a time window. The use of str() would lead to a KeyError: 83 during the query.

from builtins import *
# no 'from __future__ import unicode_literals' for some reason
from influxdb import InfluxDBClient

client = InfluxDBClient(host=db_IP, port=db_port, database=db_name)

def get_measurement(start='now() - 1d', stop='now()'):
    # Checks the arguments
    if start != 'now() - 1d':
        start = str(int(start)) + 's'     #Replace str() with unicode()
    if stop != 'now()':
        stop = str(int(stop)) + 's'       #Replace str() with unicode()
    # Format the database query and query
    query = "SELECT * FROM table WHERE (time > " + start + ") AND (time < " + stop + ")"
    response = client.query(query)

Using unicode() instead of str() solved it.
It is instructing to look at the types of start and stop once parsed. str() returns a <class 'future.types.newstr.newstr'> type while unicode() returns a <type 'unicode'>. It seems that the query string was ultimately messed up but I am not sure of the details.

EDIT: Warning unicode() does not exist in Python 3.x. My solution is not future-proof and should only be considered as relevant for Python 2.x. Any suggestion to improve that is appreciated.

EDIT #2: Update for Python 2/3 compatibility. I added exception management in the two if statements to deal with compatibility. Any solution that would be more "pythonic" or elegant is appreciated.

if start != 'now() - 1d':
    try:
        start = unicode(int(start)) + 's'   # no unicode() in Python 3 !
    except NameError:
        start = str(int(start)) + 's'
if stop != 'now()':
    try:
        stop = unicode(int(stop)) + 's'
    except NameError:
        stop = str(int(stop)) + 's'

Upvotes: 1

lukewitmer
lukewitmer

Reputation: 1183

Influx is very picky about unicode, single quotes, double quotes, etc. Swapping out str for unicode for ts_str works. Printing out the query_str before and after looks the same, but under the hood the type matters. 

import pandas as pd

ts = pd.Timestamp(pd.Timestamp('now', tz='US/Pacific').date(), tz='US/Pacific')
ts_str = unicode(int(ts.value / (10 ** 6)))
query_str = 'SELECT * FROM "table"."measurement" WHERE time <= ' + \
            ts_str + 'ms LIMIT 10'
json_payload = client.query(query_str)
print(json_payload)

Upvotes: 0

Related Questions