giving me a error "The request failed: Google returned a response with code 429".It gives me this response on every attempt. Pytrend Trendreq class

Question

Initial conditions: Windows 11, python 3.11.2

I have been using pytrends successfully for over a year now. Around 2 weeks ago, just before my pc crashed, it started giving me a error "The request failed: Google returned a response with code 429".It gives me this response on every attempt. I started fresh on a new PC, and got the same results.

I have tried creating my own custom header: -went to google trends site and found get request. -copied get request and used this site to build python code https://curlconverter.com/ -copied request header into request_args and input it That didn't work

I have tried going back in time to previous versions for certifi,urllib3,requests,pytrends. None have worked.

I bought a proxy and after integrating I am getting the response back of just "0". I ensured the proxy was from US.

'''

from pytrends.request import TrendReq

import urllib.request
import random
username = 'USER'
password = 'PASS'
state = 'us_texas'
entry = ('http://customer-%s-st-%s:%s@pr.oxylabs.io:7777' %
    (username, state, password))
query = urllib.request.ProxyHandler({
    'http': entry,
    'https': entry,
})
execute = urllib.request.build_opener(query)
proxy_ip = execute.open('https://ipinfo.io').read()
proxy_ip=str(proxy_ip)
proxy_ip=proxy_ip.split("ip",1)[1]
proxy_ip=proxy_ip.split(",",1)[0]
proxy_ip=proxy_ip.replace("\"","")
proxy_ip=proxy_ip.replace(":","")
print(proxy_ip)
proxy='https://'+proxy_ip
proxy={1: proxy}
print(proxy)


pytrends = TrendReq(hl='en-US', tz=360, timeout=(10,25), retries=2, backoff_factor=0.1,proxies=proxy,requests_args={'verify':False})

'''

Here is my entire pip list

C:\Windows\System32>pip list Package Version

---

aiohttp 3.8.4 aiosignal 1.3.1 appdirs 1.4.4 async-timeout 4.0.2 attrs 22.2.0 beautifulsoup4 4.12.0 bs4 0.0.1 certifi 2022.12.7 cffi 1.15.1 charset-normalizer 2.1.1 click 8.1.3 colorama 0.4.6 contourpy 1.0.7 cryptography 39.0.2 cssselect 1.2.0 cycler 0.11.0 DateTime 5.1 fake-useragent 1.1.3 feedparser 6.0.10 finviz 1.4.4 finvizfinance 0.14.5 fonttools 4.39.2 frozendict 2.3.6 frozenlist 1.3.3 get-all-tickers 1.8 html5lib 1.1 idna 3.4 importlib-metadata 6.1.0 joblib 1.2.0 kiwisolver 1.4.4 lxml 4.9.2 matplotlib 3.7.1 multidict 6.0.4 multitasking 0.0.11 nltk 3.8.1 numpy 1.24.2 packaging 23.0 pandas 1.5.3 parse 1.19.0 pause 0.3 Pillow 9.4.0 pip 23.0.1 pycparser 2.21 pyee 8.2.2 pyparsing 3.0.9 pyppeteer 1.0.2 pyquery 2.0.0 python-dateutil 2.8.2 pytickersymbols 1.13.0 pytrends 4.7.3 pytz 2022.7.1 PyYAML 6.0 regex 2022.10.31 requests 2.28.1 requests-futures 1.0.0 requests-html 0.10.0 scikit-learn 1.2.2 scipy 1.10.1 setuptools 65.5.0 sgmllib3k 1.0.0 six 1.16.0 sklearn 0.0.post1 soupsieve 2.4 tenacity 8.2.2 textblob 0.17.1 threadpoolctl 3.1.0 tqdm 4.65.0 urllib3 1.26.11 user-agent 0.1.10 w3lib 2.1.1 webencodings 0.5.1 websockets 10.4 wheel 0.38.4 yahoo-earnings-calendar 0.6.0 yahooquery 2.2.15 yarl 1.8.2 yfinance 0.2.13 zipp 3.15.0

Answer 1

I found the answer. It isn't ideal because it takes 20s to get data, faster with faster internet, but it is better than nothing. It uses selenium to open google trends in browser, logged in as you. If you want to use, change directory path to your own user.

'''

import time, json
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from parsel import Selector
from bs4 import BeautifulSoup
import pandas as pd

GEO = "US"
search_term= "AMC"
search_term2= "stock"
HL="en"
URL=f"https://trends.google.com/trends/explore?q={search_term}%20{search_term2}&date=now%201-d&geo={GEO}&hl={HL}"
# graph is in units of 8 minute increments
timeframe=60 # set to 8 hrs

def Google_Trend_Data(url,timeframe):

    service = Service(ChromeDriverManager().install())
    options = webdriver.ChromeOptions()
    options.add_argument("--headless")
    options.add_argument("--lang=en")
    options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36")
    userdatadir = 'C:/Users/jonat/AppData/Local/Google/Chrome/User Data'
    options.add_argument(f"--user-data-dir={userdatadir}")
    driver = webdriver.Chrome(service=service,options=options)
    driver.get(url)
    WebDriverWait(driver, 10000).until(EC.visibility_of_element_located((By.TAG_NAME, 'body')))
    time.sleep(5)
    selector = driver.page_source
    soup=BeautifulSoup(selector,'html.parser')
    columns=[]
    df=pd.DataFrame(columns=columns)
    table=soup.select('td')
        
    odd_i = []
    even_i = []
    for i in range(0, len(table)):
        if i % 2:
            even_i.append(table[i])
        else :
            odd_i.append(table[i])
 
   
    date_table=odd_i
    value_table=even_i
    date_table=date_table[::-1]
    int_value_table=[]
    for item in value_table:
        item=str(item)
        item=item.replace("<td>","")
        item=item.replace("</td>","")
        item=int(item)
        int_value_table.append(item)
    int_value_table.reverse()
    df['Date']=date_table[0:timeframe]
    df['Interest over Time']=int_value_table[0:timeframe]
    mean=df['Interest over Time'].mean()

    print(df.to_string())
    print(mean)
    driver.quit()

    return df,mean


if __name__ == "__main__":
    frame, result = Google_Trend_Data(URL,timeframe)

'''