Browser Automation with Python and Selenium — 4: Locating Elements

How to locate elements on the page?

In the previous post, we looked at the high-level architecture of Selenium applications. This post will be about locating elements on a web page to interact with them.

We should first locate elements on a page to perform some operations on them. We can locate elements by id attribute, name attribute, css selector, class name, tag name, xpath, and full or partial link text.

Locating Strategies

Python api provides the following methods to find elements on a page.

• find_element_by_id
• find_element_by_name
• find_element_by_xpath
• find_element_by_link_text
• find_element_by_partial_link_text
• find_element_by_tag_name
• find_element_by_class_name
• find_element_by_css_selector

These methods return an object in WebElement (represents a particular DOM node) type or raise NoSuchElementException if it is not found.

You can also find multiple elements with the following versions of find methods.

• find_elements_by_id
• find_elements_by_name
• find_elements_by_xpath
• find_elements_by_link_text
• find_elements_by_partial_link_text
• find_elements_by_tag_name
• find_elements_by_class_name
• find_elements_by_css_selector

These methods return a collection of elements. If only one element is found, it will still return a list (of one element). If no element can be found, an empty list is returned.

There are two more methods you can use with the help of By class.

• find_element
• find_elements

Attributes available for By class are

• ID
• XPATH
• LINK_TEXT
• PARTIAL_LINK_TEXT
• NAME
• TAG_NAME
• CLASS_NAME
• CSS_SELECTOR

They are used like in the below code snippet.

from selenium.webdriver.common.by import By

element = driver.find_element(By.ID, "main_content")
list_of_elements = driver.find_elements(By.CLASS_NAME, "links")

All find_* methods can be called on both WebDriver and WebElement instances that mean if you locate a container element by calling find_* method on the WebDriver instance, you can call find_* on this element to further search elements inside it.

Example

The following example includes all location strategies and handling of NoSuchElementException case.

You can also find the codes here.

# Output

Mail input field placeholder: [email protected]
Submit button text: SUBMIT QUESTION
Second link text: Hitchhiker’s Guide to Python
Python Official link text: Python Official
Link text - 1: Python Official
Link text - 2: Hitchhiker’s Guide to Python
Link text - 3: Dive Into Python
Link text - 4: Real Python
Link text - 5: Python Subreddit
Link text - 6: Talk Python Podcast
Link text - 7: Awesome Python
Question text area placeholder: Your question...
Navigation item - 1: About
Navigation item - 2: BOPI
Navigation item - 3: Feedback
Support button text: Support Me
There is no h1 tag in the document!

Tips on performance

• If an id is available for the element to be located, prefer using it. Since id must be unique on a page, the performance will be better.
• If id is not available, prefer a well-written, targeted css selector.
• Traversing the DOM is an expensive operation, so you should narrow the scope of your search as much as possible to get better performance.

Things to Remember

• You can locate elements by id attribute, name attribute, css selector, class name, tag name, xpath, and full or partial link text.
• Selenium Python api provides find_element(s) and find_element(s)_by_* methods to locate elements on a page.
• All find_* methods can be called on both WebDriver and WebElement instances.
• In case of failing to find, these methods return an empty list or raise NoSuchElementException

In the next post, I will write about waits in Selenium.

References

1. https://www.selenium.dev/documentation/en/getting_started_with_webdriver/locating_elements/
2. https://selenium-python.readthedocs.io/locating-elements.html
3. https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors

Thank you for your time.

If you want to read the previous posts, check out the following links.