CODEWITHSUNDEEP
pythonipurllib
NedStarkOfWinterfell
NedStarkOfWinterfell

Reputation: 5153

Spoof IP address in Python

There is a certain data API which I need to call to fetch some data. Imagine there are few million such records. For each such record, I construct the data URL (that returns a JSON response) from the record id and API key, fetch the response and put it to my local database. Now if I do it on my Mac, even with a 16 GB RAM, this will take months. Whereas if I try it on a 64 GB multicore EC2 desktop provided by AWS, it works lightning fast..except there is a problem.

There is a rate limit on the number of queries the API server can answer. If it goes beyond 40 any minute, it starts returning error messages in place of the JSON response. And on the EC2 machine, it ends up going upto 600 requests per minute. If it works correctly, I will have all the data in 2-3 hours. The funny thing is that the rate limit is imposed by IP, not by API key. So if I can somehow spoof the IP address (say from a list of 15 IPs in a round-robin manner) for the requests, it will stay within the limits. How do I do it? I am using urllib. Here's my sample code:

url = urltemplate % (list_of_params_including_API_key)
data = json.load(urllib.urlopen(url))
//parse the data and load it into database

Upvotes: 0

Views: 4689

Answers (1)

Barmar
Barmar

Reputation: 780869

Use the socket.bind method to specify the source address of the network connection.

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# bind socket to a particular IP. port 0 allows it to select an unused local port
s.bind((spoofed_ip, 0))
s.connect((server_ip, 80))

Upvotes: 2

Related Questions