Selenium Automation in Practice: Reliable UI Tests with Page Objects, Waits, and Useful Debug Artifacts

Pictalingo

·

Selenium Automation in Practice: Reliable UI Tests with Page Objects, Waits, and Useful Debug Artifacts

Selenium is great for automating real browsers, but beginner test suites often become flaky: “element not found”, timing issues, random failures in CI, and no clue why. This hands-on guide shows a practical pattern for building reliable Selenium automation in Python using:

  • Explicit waits (instead of time.sleep())
  • Page Object Model for maintainable selectors
  • Stable locators with data-testid
  • Debug artifacts (screenshots + HTML dumps) on failure
  • Headless mode that works in CI

We’ll automate a common flow: log in, create an item, and verify it appears in a list.

Project Setup

Install dependencies:

python -m venv .venv source .venv/bin/activate # (Windows: .venv\Scripts\activate) pip install selenium pytest

Use Selenium Manager (built into Selenium 4.6+) so you don’t manually install chromedriver:

python -c "from selenium import webdriver; driver = webdriver.Chrome(); driver.quit()"

If that opens and closes Chrome cleanly, you’re good.

Prefer Stable Selectors (Add data-testid)

Flaky tests often come from brittle selectors like .btn:nth-child(2) or text-only XPath that changes when wording changes. If you control the app, add attributes like:

<input data-testid="email" /> <input data-testid="password" /> <button data-testid="login-submit">Sign in</button>

Then your tests can target those reliably.

Create a Robust WebDriver Fixture

Put this in tests/conftest.py. It configures headless Chrome, sensible defaults, and ensures cleanup.

import os import pytest from selenium import webdriver from selenium.webdriver.chrome.options import Options @pytest.fixture def driver(): options = Options() # Headless is ideal for CI. For local debugging, set HEADLESS=0 headless = os.getenv("HEADLESS", "1") == "1" if headless: options.add_argument("--headless=new") # Common stability flags for CI/Linux containers options.add_argument("--disable-gpu") options.add_argument("--no-sandbox") options.add_argument("--disable-dev-shm-usage") # Make viewport consistent (helps with responsive layouts) options.add_argument("--window-size=1280,900") driver = webdriver.Chrome(options=options) driver.set_page_load_timeout(20) driver.implicitly_wait(0) # IMPORTANT: prefer explicit waits only yield driver driver.quit()

Why disable implicit waits? Mixing implicit and explicit waits can create confusing timing behavior. Stick to explicit waits so you control exactly what you’re waiting for.

Build a Small “Wait Helpers” Module

Create tests/ui/waits.py with a few reusable waits. This is the foundation of non-flaky Selenium.

from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC DEFAULT_TIMEOUT = 10 def wait_visible(driver, locator, timeout=DEFAULT_TIMEOUT): return WebDriverWait(driver, timeout).until( EC.visibility_of_element_located(locator) ) def wait_clickable(driver, locator, timeout=DEFAULT_TIMEOUT): return WebDriverWait(driver, timeout).until( EC.element_to_be_clickable(locator) ) def wait_url_contains(driver, fragment, timeout=DEFAULT_TIMEOUT): return WebDriverWait(driver, timeout).until( EC.url_contains(fragment) ) def wait_gone(driver, locator, timeout=DEFAULT_TIMEOUT): return WebDriverWait(driver, timeout).until( EC.invisibility_of_element_located(locator) )

These helpers keep tests readable and consistent.

Page Object Model: Keep Selectors Out of Tests

When selectors live in tests, every UI change forces you to edit many files. Page Objects centralize them and provide higher-level actions like “login” or “create item”.

Create tests/ui/pages/base.py:

from pathlib import Path from datetime import datetime class BasePage: def __init__(self, driver, base_url): self.driver = driver self.base_url = base_url.rstrip("/") def open(self, path): self.driver.get(f"{self.base_url}{path}") return self def save_debug_artifacts(self, name_prefix="failure"): out_dir = Path("test-artifacts") out_dir.mkdir(exist_ok=True) ts = datetime.utcnow().strftime("%Y%m%d-%H%M%S") png = out_dir / f"{name_prefix}-{ts}.png" html = out_dir / f"{name_prefix}-{ts}.html" self.driver.save_screenshot(str(png)) html.write_text(self.driver.page_source, encoding="utf-8") # Returning paths helps you print/log them if you want return str(png), str(html)

Now a LoginPage in tests/ui/pages/login.py:

from selenium.webdriver.common.by import By from tests.ui.waits import wait_visible, wait_clickable, wait_url_contains from tests.ui.pages.base import BasePage class LoginPage(BasePage): EMAIL = (By.CSS_SELECTOR, '[data-testid="email"]') PASSWORD = (By.CSS_SELECTOR, '[data-testid="password"]') SUBMIT = (By.CSS_SELECTOR, '[data-testid="login-submit"]') ERROR = (By.CSS_SELECTOR, '[data-testid="login-error"]') def go(self): return self.open("/login") def login_as(self, email, password): wait_visible(self.driver, self.EMAIL).clear() self.driver.find_element(*self.EMAIL).send_keys(email) wait_visible(self.driver, self.PASSWORD).clear() self.driver.find_element(*self.PASSWORD).send_keys(password) wait_clickable(self.driver, self.SUBMIT).click() return self def assert_logged_in(self): wait_url_contains(self.driver, "/app")

And an ItemsPage in tests/ui/pages/items.py:

from selenium.webdriver.common.by import By from tests.ui.waits import wait_visible, wait_clickable, wait_gone from tests.ui.pages.base import BasePage class ItemsPage(BasePage): NEW_BUTTON = (By.CSS_SELECTOR, '[data-testid="item-new"]') NAME_INPUT = (By.CSS_SELECTOR, '[data-testid="item-name"]') SAVE_BUTTON = (By.CSS_SELECTOR, '[data-testid="item-save"]') TOAST = (By.CSS_SELECTOR, '[data-testid="toast"]') def go(self): return self.open("/app/items") def create_item(self, name): wait_clickable(self.driver, self.NEW_BUTTON).click() wait_visible(self.driver, self.NAME_INPUT).send_keys(name) wait_clickable(self.driver, self.SAVE_BUTTON).click() # Example: wait for a toast to appear and then disappear wait_visible(self.driver, self.TOAST) wait_gone(self.driver, self.TOAST) return self def assert_item_listed(self, name): # For dynamic lists, locate by a stable test id when possible. # If you must search by text, keep it narrow. row = (By.XPATH, f'//*[@data-testid="item-row" and .//*[normalize-space()="{name}"]]') wait_visible(self.driver, row)

Write the Test (Readable and Maintainable)

Create tests/test_items_flow.py:

import os import uuid import pytest from tests.ui.pages.login import LoginPage from tests.ui.pages.items import ItemsPage BASE_URL = os.getenv("BASE_URL", "http://localhost:3000") EMAIL = os.getenv("E2E_EMAIL", "[email protected]") PASSWORD = os.getenv("E2E_PASSWORD", "secret123") @pytest.mark.e2e def test_user_can_create_item(driver): item_name = f"Item-{uuid.uuid4().hex[:8]}" login = LoginPage(driver, BASE_URL).go() login.login_as(EMAIL, PASSWORD).assert_logged_in() items = ItemsPage(driver, BASE_URL).go() items.create_item(item_name).assert_item_listed(item_name)

This test reads like a script a human would follow, while the complicated selector/wait logic stays in the Page Objects.

Capture Screenshots + HTML on Failure

When a test fails in CI, a screenshot and DOM dump are gold. Add a pytest hook in tests/conftest.py to auto-save artifacts when a test fails.

import pytest from tests.ui.pages.base import BasePage @pytest.hookimpl(hookwrapper=True) def pytest_runtest_makereport(item, call): outcome = yield report = outcome.get_result() if report.when == "call" and report.failed: driver = item.funcargs.get("driver") if driver: page = BasePage(driver, base_url="") # base_url not needed for artifacts png, html = page.save_debug_artifacts(name_prefix=item.name) # Attach paths to report (shows in terminal output if printed) report.extra = getattr(report, "extra", []) report.extra.append(("screenshot", png)) report.extra.append(("html", html))

Now failed runs create files under test-artifacts/.

Common Flakiness Fixes You’ll Actually Use

  • Wait for “ready” UI states: after navigation, wait for a page-specific element, not just URL changes.
  • Wait for spinners/overlays to disappear: if your app uses a loading overlay, add a wait_gone() for it before clicking.
  • Don’t click too early: prefer element_to_be_clickable over presence_of_element_located.
  • Prefer data-testid locators: class names and DOM structure change a lot; test IDs should not.
  • Keep tests independent: generate unique names (like the UUID example) and avoid relying on previous test state.

Run Locally and in Headless Mode

Run normally (headless by default from our fixture):

pytest -m e2e -q

Run with a visible browser for debugging:

HEADLESS=0 pytest -m e2e -q

Point at another environment (staging, preview, etc.):

BASE_URL="https://staging.example.com" E2E_EMAIL="..." E2E_PASSWORD="..." pytest -m e2e

Wrap-Up: A Pattern That Scales

If you take only three things from this article, take these:

  • Use explicit waits everywhere and delete time.sleep() from your vocabulary.
  • Adopt Page Objects so tests stay readable and selectors stay centralized.
  • Collect failure artifacts so CI failures are actionable, not mysterious.

With this foundation, you can add more flows (checkout, profile updates, role permissions), run tests in parallel later, and keep the suite stable as the UI evolves.

Leave a Reply

Your email address will not be published. Required fields are marked *