CODEWITHSUNDEEP
pythonpython-3.x
Dean Rowe
Dean Rowe

Reputation: 17

TypeError when concatenating bytes

I'm having problems with a bytes encoding problem and hoping for help please as stuck (python beginner, trying to adapt some code from github to python3). I 'think' the issue is that I need to encode the 'data' variable to bytes, however I'm struggling to do so as .encode() isn't working. The error returned is:

Traceback (most recent call last):
  File "C:/Users/DR/Desktop/iqfeed3.py", line 51, in <module>
    data = read_historical_data_socket(sock)
  File "C:/Users/DR/Desktop/iqfeed3.py", line 21, in read_historical_data_socket
    buffer += data
TypeError: can only concatenate str (not "bytes") to str

The underlying code (which attempts to buffer stock price data and write to csv) is below (highlighted lines 21 & 51 with comments):

# iqfeed3.py

import sys
import socket

# iqfeed3.py

def read_historical_data_socket(sock, recv_buffer=4096):
    """
    Read the information from the socket, in a buffered
    fashion, receiving only 4096 bytes at a time.

    Parameters:
    sock - The socket object
    recv_buffer - Amount in bytes to receive per read
    """
    buffer = ""
    data = ""
    while True:
        data = sock.recv(recv_buffer)
        buffer += data #TRACEBACK ERROR LINE 21

        # Check if the end message string arrives
        if "!ENDMSG!" in buffer:
            break

    # Remove the end message string
    buffer = buffer[:-12]
    return buffer

if __name__ == "__main__":
    # Define server host, port and symbols to download
    host = "127.0.0.1"  # Localhost
    port = 9100  # Historical data socket port
    syms = ["AAPL", "GOOG", "AMZN"]

    # Download each symbol to disk
    for sym in syms:
        print ("Downloading symbol: %s..." % sym)

        # Construct the message needed by IQFeed to retrieve data
        message = "HIT,%s,60,20180601 075000,,,093000,160000,1\n" % sym

        # Open a streaming socket to the IQFeed server locally
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.connect((host, port))

        # Send the historical data request
        # message and buffer the data
        sock.sendall(message.encode()) #Formerly gave error "TypeError: a bytes-like object is required, not 'str'" before adding .encode()
        data = read_historical_data_socket(sock) #TRACEBACK ERROR LINE 51
        sock.close

        # Remove all the endlines and line-ending
        # comma delimiter from each record
        data = "".join(data.split("\r"))
        data = data.replace(",\n","\n")[:-1]

        # Write the data stream to disk
        f = open("%s.csv" % sym, "w")
        f.write(data)
        f.close()

Thanks in advance for any help or pointers.

Dean

Upvotes: 1

Views: 535

Answers (1)

grapes
grapes

Reputation: 8646

You are confusing strings and raw bytes. Your buffer is a string, while socket input is normally bytes.

Best approach is simply convert received bytes into string.

Try this code:

data = sock.recv(recv_buffer)
buffer += data.decode()

This will use default utf-8 encoding. If you want another, specify it explicitly.

Upvotes: 3

Related Questions