Reputation: 1913
Here is the function I use to make the API operation 10 times faster:
def load_url(req, id, data, timeout):
headers = {'Authorization': 'AT-API 111111222222333333344444445555555'}
r = req.post("https://service.com/api/v1/compare", headers=headers, data=data, timeout=timeout)
data = r.json()
print id
if data['error']:
print data['error']
else:
c.execute("UPDATE offers SET valid = ? WHERE id = ?", ('valid' if data['data']['success'] else 'invalid', id))
print data['data']['success']
print data['data']['count']
return r.json()
if __name__ == '__main__':
...
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
future_to_url = {executor.submit(load_url, s, data['id'], data, 120): data for data in datas}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
try:
data = future.result()
We're running this script periodically every 4 hours. In load_url, we print and check the statuses of the request, data['data']['count'] - is an amount of calls remain for a month. I want to send an email notification after we reached 5000 calls or less, but ONLY ONCE. How to implement the sending only one message, but not 5 and not 50? We're using Sqlite3 to store the data.
We're using Mailgun for sending emails:
def send_simple_message(email_list):
for email in email_list:
response = requests.post(
"https://api.mailgun.net/v3/newsletter.company.com",
auth=("api", "key-1234567"),
data={"from": "Mailgun Sandbox <[email protected]>",
"to": email,
"subject": "Agent - we reached the limit by API",
"html": "We reached the limit for agent" })
Upvotes: 2
Views: 462
Reputation: 3094
You could use a lock mechanism to prevent multiple threads from executing the send_simple_message
function, and set a synchronized value that keeps track of whether the mail has been sent or not.
import threading
lock = threading.Lock()
has_been_sent = False
# then in your load_url function you could do something like
if condition on count:
with lock:
if not has_beend_sent:
# send mail
has_been_sent = True
Upvotes: 2