CODEWITHSUNDEEP
pythonseleniumselenium-webdriver
Mike C.
Mike C.

Reputation: 1921

Python Selenium Get Data From Two Combo Boxes

I have two combo boxes from which I need to get data. Let's call them 'Manufacturers' and 'Models'

When you select a manufacturer from one box the other box populates with the models that the manufacturer produces. This works fine manually but when I select a manufacturer programmatically with Selenium the 'Model' box does not re-populate. Here's the code.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
driver = webdriver.Firefox()
#Base URL
driver.get("https://example.com/")
def scrape():    
    #get manufacturer list
    select1 = driver.find_element_by_name("mnfr")
    makes = [x.text for x in select1.find_elements_by_tag_name("option")]
    print(makes)
    #get models list
    for m in makes:
        select1a = Select(driver.find_element_by_name("makeCodeListPlaceHolder"))           
        select1a.select_by_visible_text(m)        
        select2 = driver.find_element_by_name("models")
        models = [x.text for x in select2.find_elements_by_tag_name("option")]
        print(models)
scrape()

Upvotes: 0

Views: 64

Answers (2)

supputuri
supputuri

Reputation: 14135

Have you tried the below approach?

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
driver = webdriver.Firefox()
#Base URL
driver.get("https://example.com/")
def scrape():    
    #get manufacturer list
    select1 = driver.find_element_by_name("mnfr")
    makes = select1.find_elements_by_tag_name("option") # get the option elements
    print(makes)
    #get models list
    for m in makes:
        #click on list option
        m.click()
        select2 = driver.find_element_by_name("models")
        models = [x.text for x in select2.find_elements_by_tag_name("option")]
        print(models)
scrape()

Upvotes: 0

Asyranok
Asyranok

Reputation: 985

So there are some oddities with Selenium clicks not triggering all DOM events attached to an object. It normally occurs due to, let's say, un-optimally written front-end code.

But not making any judgements here, as Selenium can just encounter problems that it really shouldn't have.

So here is one suggestion. Go into the browser dev tools > console, and try the following example. I will write this in JQuery, but if you do not have JQuery, feel free to inject it on the page, or change the code to regular document.getElement code.

$("#FirstComboBoxOption").click();

Does that trigger all the expected events? Does the second Combobox populate correctly?

There is no dishonor using that to click an element. I use Selenium's click whenever possible, but Selenium's click is not literally identical to a real human click anyway, as the mouse/user32.dll is not used to perform a click. So using javascript, while not ideal, is not wrong if it is the only way to trigger all the events on the object.

The best situation is that the event logic is hooked up in a way that this is not necessary, but I am going to assume that is not an option here. Instead, I would do this (note that I use C# for Selenium, so this is a general guess on the syntax. Consider it kinda pseudo code)

browser.execute_script("$('#FirstComboBoxOption').click()");

Next, I am going to assume this doesn't work for you, so here is the next option.

First, find all events attached to the first combo box, that ostensibly are triggering the population of the second combo box. You can do this, using JQuery, like this:

var elem = $('#FirstComboBoxOption')[0];
$._data(elem, "events");

Do a little playing around to make sure you know what events are going to be triggered by a real click. Then, do the following:

$('#FirstComboBoxOption').trigger('SomeExpectedEvent');

Do one line for each of the events, if there is more than one event. I've only ever had to do the event example above once. It is a last resort. There really isn't anything wrong with it in the grand scheme of things, but whenever possible, it is obviously a superior option to trigger things using means as close to a real user's interactions as possible.

Upvotes: 1

Related Questions