CODEWITHSUNDEEP
pythonautomationwebbrowser-controlpyautogui
FastRx3
FastRx3

Reputation: 1

Controlling the mouse and browser with pyautogui for process automation

I'm new at Python and I need expert guidance for the project I'm trying to finish at work, as none of my coworkers are programmers.

I'm making a script that logs into a website and pulls a CSV dataset. Here are the steps that I'd like to automate:

  1. Open chrome, go to a website
  2. Login with username/password
  3. Navigate to another internal site via menu dropdown
  4. Input text into a search tag box or delete search tags, e.g. "Hours", press "Enter" or "Tab" to select (repeat this for 3-4 search tags)
  5. Click "Run data"
  6. Wait until data loads, then click "Download" to get a CSV file with 40-50k rows of data
  7. Repeat this process 3-4 times for different data pulls, different links and different search tags

This process usually takes 30-40 minutes for a total of 4 or 5 data pulls each week so it's like watching paint dry.

I've tried to automate this using the pyautogui module, but it isn't working out for me. It works too fast, or doesn't work at all. I think I'm using it wrong.

This is my code:

import webbrowser
import pyautogui
#pyautogui.position()
#print(pyautogui.position())

#1-2
pyautogui.FAILSAFE = True
chrome_path = 'open -a /Applications/Google\ Chrome.app %s'

#2-12
url = 'http://Google.com/'
webbrowser.get(chrome_path).open(url)
pyautogui.moveTo(185, 87, duration=0.25)
pyautogui.click()
pyautogui.typewrite('www.linkedin.com')
pyautogui.press('enter')
#loginhere? Research

In case pyautogui is not suited for this task, can you recommend an alternative way?

Upvotes: 0

Views: 4835

Answers (1)

Latronis
Latronis

Reputation: 71

The way you are going about grabbing your data is very error prone and not how people generally go about grabbing data from websites. What you want is a web scraper, which allows you to grab information from websites or some companies provide API's that allow you easier access to the data.

To grab information from LinkedIn it has a built in API. You did mention that you were navigating to another site though in which case I would see if that site has an API or look into using Scrapy, a web scraper that should allow you to pull the information you need.

Sidenote: You can also look into synchronous and asynchronous programming with python to make multiple requests faster/easier

Upvotes: 1

Related Questions