CODEWITHSUNDEEP
pythontcp
cog_n1t1v3
cog_n1t1v3

Reputation: 229

python tcp put connection reset

def put_data(line = '', dest_host = 'server', dest_port = '876'):
    ''' This method once invoked pushes data to dest_host and dest_port  '''
    if line:
        cmd = '/bin/echo put ' + line + '| ' + '/bin/nc -w 15 server port'
        print('TCP put using command: ' + cmd)
        os.system(cmd)

I am using the above method to TCP put lines of data to an opentsdb driver on top of a HBASE.

This method as can be seen, calls the the netcat utility for each line of data that is being pushed out.

Questions: 1.) Is there a better way to accomplish this task by not having to open a TCP connection each time for every line of data I want to push? I run this every minute and this is causing a large number of connections in TIME_WAIT state.

2.) Can I push say 1000 lines at a time using a persistent TCP socket? I have looked at the netcat implementations in python but they seem to cause a connection reset from the peer when I push all the lines

For example: When I try to use the below method, I keep getting the error: error(104, 'Connection reset by peer')

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(server, port)
for line in data_list:
    s.sendall(line)
s.close()

error(104, 'Connection reset by peer')

Sample lines of data being pushed:

line1 = 'fin.wait2 1385428269 0.0 host=srv1 testname=tcp source=nag03 product=rebuild'
line2 = 'established 1385428269 11.0 host=srv2 testname=tcp source=nag03 product=rebuild'
line3 = 'fin.wait1 1385428269 0.0 host=srv3 testname=tcp source=nag03 product=rebuild'
line4 = 'last.ack 1385428269 0.0 host=srv4 testname=tcp source=nag03 product=rebuild'
line5 = 'unknown 1385428269 0.0 host=srv5 testname=tcp source=nag03 product=rebuild'
line6 = 'syn.recv 1385428269 0.0 host=srv6 testname=tcp source=nag03 product=rebuild'

Upvotes: 0

Views: 711

Answers (1)

abarnert
abarnert

Reputation: 366073

The problem is that your lines don't have newlines on the end.

Your netcat-based code is passing the lines to netcat using echo, which, without any arguments, will add a newline to the end, so it works.

But your Python socket-based code is not adding any newlines.

So, you wanted to send it five commands like this:

put fin.wait2 1385428269 0.0 host=srv1 testname=tcp source=nag03 product=rebuild
put established 1385428269 11.0 host=srv2 testname=tcp source=nag03 product=rebuild
put fin.wait1 1385428269 0.0 host=srv3 testname=tcp source=nag03 product=rebuild
put last.ack 1385428269 0.0 host=srv4 testname=tcp source=nag03 product=rebuild
put unknown 1385428269 0.0 host=srv5 testname=tcp source=nag03 product=rebuild
put syn.recv 1385428269 0.0 host=srv6 testname=tcp source=nag03 product=rebuild

… but instead, you're sending it one big line like this:

put fin.wait2 1385428269 0.0 host=srv1 testname=tcp source=nag03 product=rebuildput established 1385428269 11.0 host=srv2 testname=tcp source=nag03 product=rebuildput fin.wait1 1385428269 0.0 host=srv3 testname=tcp source=nag03 product=rebuildput last.ack 1385428269 0.0 host=srv4 testname=tcp source=nag03 product=rebuildput unknown 1385428269 0.0 host=srv5 testname=tcp source=nag03 product=rebuildput syn.recv 1385428269 0.0 host=srv6 testname=tcp source=nag03 product=rebuild

Either it's getting confused by your data and hanging up, rejecting your overly long line as a DoS attempt, or just waiting around for the newline that's supposed to end the line and never getting it and eventually timing out.

Whatever the underlying problem, the root cause is that lack of newlines, and the fix is to put them in there.

While we're at it, your netcat code appends a put at the start of each command, and your socket code doesn't, and you'll probably need to fix that as well. So:

for line in data_list:
    s.sendall('put {}\n'.format(line))

Upvotes: 1

Related Questions