PascalVKooten
PascalVKooten

Reputation: 21451

Detect user key/mouse in Python Selenium

I'm using Selenium Browser for day to day browsing, and I'd like to fire some code when I press some keys on any page. At first I thought I can just load javascript on every page that registers keys/mouse input, but I'd actually really prefer to have some python list available with past keys/mouse clicks, e.g. my key example in javascript:

var myhistory = []

document.addEventListener("keydown", keyDownTextField, false);

function keyDownTextField(e) {
var keyCode = e.keyCode;
  myhistory.push(keyCode)
}

Is there any way to do this in pure Python/Selenium?

Upvotes: 5

Views: 3468

Answers (3)

AturSams
AturSams

Reputation: 7952

boppreh/keyboard would let you do that.
You install it. pip install keyboard
You import it. import keyboard
You use it. keyboard.add_hotkey('left', print, args=['You pressed the left arrow key'])
Then you disable it. keyboard.remove_all_hotkeys()

Upvotes: 1

Abr001am
Abr001am

Reputation: 591

Well, in that case you had to choose the right tool for the job, i advice puppeteer a web-automation family instrument a pure-made JS, which can easily interact with the browser ( from js to js ) and catch events directly from the other side without any mediation.

Yet with selenium you can still achieve this transitively without messing too much with the pages's code or overcharging it with unnecessary tasks, also reloading the page content resets all its variables, which means it's lossy approach. The best closest way is to set an eventhandler internally and directly catch it from outside using Runtime.evaluate instead because it doesn't affect the page content and specifically it sticks to the function until it yields something using promise calls, it's better away than probing around some global variable over and over which is seen a bad practise see here.

myhistory = []
evt_handler = """
new Promise((rs,rj) => window.onkeydown= e => rs(e.keyCode) )
"""
def waitforclick():
    try:
        myhistory.append(browser.execute_cdp_cmd('Runtime.evaluate', {'expression': evt_handler, 'awaitPromise': True,'returnByValue': True})['result']['value'])
    except:
        waitforclick()

To avoid locking out the cpu you need to fork a thread in parallel.

from threading import Timer 
t = Timer(0.0, waitforclick)

then t.start() instead of waitforclick().

Also you can use timeout if you want to reject the promise with a zero value after some time.

Upvotes: 0

User
User

Reputation: 14873

What I would try:

  1. Execute a javascript that registers at the document body

    <body onkeyup="my_javasctipt_keyup()" and onkeydown="my_javasctipt_keydown()"> 
    

    using browser.execute_script. (partially solved, see question)

  2. Save the key up and keydown events in a variable in javascript. (solved, see question)

  3. use browser.execute_script to return the variables.

What I am uncertain about:

  • The return value of browser.execute_script may return json serializable objects or strings only
  • keyup and keydown in body may not work if they are used in child elements that define their own event listeners

Hopefully this is of help. If any code results form this I would be interested in knowing.

This code is what I feel should work:

from selenium import webdriver
browser = webdriver.Firefox()
browser.execute_script("""var myhistory = []

document.addEventListener("keydown", keyDownTextField, false);

function keyDownTextField(e) {
var keyCode = e.keyCode;
  myhistory.push(keyCode)
}""")
def get_history():
    return browser.execute_script("myhistory")

# now wait for a while and type on the browser
import time; time.sleep(5000)

print("keys:", get_history())

The point is that the code of selenium can never run at the same time as the browser handles keyboard input. As such, events need to be handled in javascript, the result saved, e.g. in an array and then, when selenium is asked, the array is returned to Python.

Upvotes: 3

Related Questions