wait for an element to be displayed and not displayed, using selenium, python and whatsapp-web

Question

I'm trying to use "whatsapp-web", "selenium" and "python 3" to know whenever a whatsapp user comes online or go offline.

to explain more , this is how i want the script to work :

the script will be listening for a span (with title=online) to be displayed , when the span is displayed (it means the user comes online) i want the time at this moment to be printed, then the script will keep listening again for the span to disappear, when it disappears the script print the time of disappearance, and so on.

this is my code :

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import datetime

driver = webdriver.Chrome('C:/webdrivers/chromedriver.exe')
driver.get('https://web.whatsapp.com/')

# do nothing until QR code scanned and whatsapp-web is accessed
input('Enter anything after scanning QR code')

# Input the name of the user to track
name = input('Enter the name of the user : ')

# find the whatsapp user to be tracked then a click to enter the conversation
user = driver.find_element_by_xpath("//span[@title = '{}']".format(name))
user.click()

while True:
   # in the conversation page, a span with title online is diplayed when user is online.
   #the web driver will wait 8hrs=28800s, if user not online all this time script will be killed by webdriverWait
   element = WebDriverWait(driver, 28800).until(
      EC.visibility_of_element_located(
         (By.XPATH, "//span[@title = 'online']")))

   #Moment the user came online
   now = datetime.datetime.now()
   print("online at : ")
   print(now.strftime("%H:%M:%S"))

   element = WebDriverWait(driver, 28800).until(
      EC.invisibility_of_element_located(
         (By.XPATH, "//span[@title = 'online']")))

   #Moment the user went offline
   now = datetime.datetime.now()
   print("offline at : ")
   print(now.strftime("%H:%M:%S"))
   print("************")

My script works but, I want it to be running for hours , like 8 hour or maybe more, but i read that it is a bad practice to use WebDriverWait with a high number of seconds (28800s in my case).

So is there any other better way to achieve this ?

how can i write my output to a txt or word file?

Any suggestion to make my code better ?

How to prevent CPU slamming ? or any possible problem that may happen

Answer 1

One thing that i should suggest is that in your program you need to scan whatsapp QR everytime you execute this program, just replace this line

driver = webdriver.Chrome('C:/webdrivers/chromedriver.exe')

with this

---

driver = webdriver.Chrome('C:/webdrivers/chromedriver.exe', options="user-data-dir=C:\\Users\\<username>\\AppData\\Local\\Google\\Chrome\\User Data\\whtsap")

---

this way you'll need to scan the QR but only once.

Answer 2

WebDriverWait is nothing more than a (quite) fancy while/catch/sleep loop; in your particularly case you might want to replicate it yourself, for one simple reason - it polls every 500ms, which is probably too detailed resolution for this task. It also shields you from a bit more granular control.

Here's how to do the logic yourself - have a boolean variable is the user online or not; based on it's value, check is the element visible (.is_displayed()), sleep X time and repeat. An exception NoSuchElementException, StaleElementReferenceException will count as the the user offline/the boolean value false.

At the end, you're code will be pretty close to the logic in WebDriverWait - still, yours and more flexible if needed to be.

---

Alternatively, just pass a bigger polling internal in WebDriverWait in the current code - it's the poll_frequency argument of the call :)

WebDriverWait(driver, 28800, 5)  # the value is in seconds

---

I can't know where and what you've read it's bad practice to use WebDriverWait with a high number of seconds; as you see in its code it's just how much time the method is given to be running.
I presume the advice was in the tone "it is a bad practice to use WebDriverWait with a high number of seconds, because if the condition is not fulfilled in X seconds, it won't be ever fulfilled and your code will just spin and spin.". Which is actually the desired behavior for you :)

I also wouldn't worry taxing the cpu - these checks are very lightweight, nothing harmful. For this big runtime, what would worry me is memory leaks in the browser itself ;)

---

As for optimizing the code - what I would do is to cut on the statements repetitions; with the downside of decreasing its readability a bit. My take for the loop:

user_online = False

while True:
    # we'll be checking for the reverse of the last status of the user
    check_method = EC.visibility_of_element_located if not user_online else EC.invisibility_of_element_located

    # in the conversation page, a span with title online is diplayed when user is online.
    # the web driver will wait 8hrs=28800s for the user status to change all
    # the script will be killed by webdriverWait if that doesn't happen
    element = WebDriverWait(driver, 28800, 5).until(
            check_method((By.XPATH, "//span[@title = 'online']")))

    # The moment the user changed status
    now = datetime.datetime.now().strftime("%H:%M:%S")
    print("{} at : {}".format('online' if not user_online else 'offline', now))   # if you're using python v3.6 or more, the fstrings are much more convenient for this
    print("************")

    user_online = not user_online   # switch, to wait for the other status in the next cycle

Finally, code-wise - the script can't be left running "endlessly". Why? Because if the user doesn't change status in 8 hours, WebDriverWait will stop. To salvage that, wrap the loop body in try/except:

from selenium.common.exceptions import TimeoutException  # put this in the beginning of the file

while True:
    try:
        # the code from above
    except TimeoutException:
        # the status did not change, repeat the cycle
        pass

Writing to a file

You might want to read a bit how to do that - it's quite a simple operation.

Here's a sample - open a file for appending (so there previous logs are preserved), wrapping the while loop:

with open("usermonitor.log", "a") as myfile:
    while True:
        # the other code is not repaeted for brevity
        # ...
        output = "{} at : {}".format('online' if not user_online else 'offline', now)
        print(output)
        myfile.write(output + "\n")  # this will write (append as the last line) the same text in the file
        # write() does not append newlines by itself - you have to do it yourself