Lyra Testing Best Practices¶

Lyra Testing Best Practices

This guide covers best practices for writing, organizing, and maintaining automated tests with Lyra. It is aimed at developers, QA engineers, and anyone starting with Lyra for the first time. The principles here are grounded in real experience with FWD canvas-based applications and informed by established practices from the broader test automation community, adapted to the unique nature of FWD's 2D canvas-rendered UI.

See also: the official Lyra Automated Testing Evolved documentation.

Understanding What Makes FWD Testing Different¶

Before diving into practices, it is important to understand why testing FWD applications is fundamentally different from testing traditional web applications.

In a traditional web application, the UI is made of DOM elements — buttons, inputs, divs — that can be queried with CSS selectors or semantic locators like getByRole(). In FWD, the entire application UI is rendered onto an HTML5 Canvas. There are no DOM elements to inspect. A button on screen is just pixels painted on a canvas surface.

This means:

No DOM selectors. You cannot use CSS selectors, XPath, or accessibility-based locators to find UI elements.
Stable IDs are your locators. Lyra resolves widgets through FWD's widget tree using Stable IDs — hierarchical paths like win:Hotel Demo/frame:fmain/btn:Search that identify widgets by their type, label, and position in the hierarchy.
Assertions are semantic, not pixel-based. Lyra's Selection Assert captures the widget tree structure (types, text, visibility, enabled state) within a rectangular area — not a screenshot comparison. This makes assertions resilient to font rendering differences while still catching meaningful UI changes.
No auto-waiting on DOM state. Traditional frameworks like Playwright auto-retry assertions until a DOM element appears. Lyra uses waitForInputReady() and waitForP2JReady() to synchronize with the FWD application's internal state instead.

Keep these differences in mind as you read every section below. Practices that work for DOM-based testing often need adaptation — or outright replacement — for canvas-based FWD applications.

Recording Tests: Start Here, Don't Stop Here¶

The Lyra Recorder is the fastest way to bootstrap a test. Launch it, interact with the application, and it generates a complete Playwright test script. But a raw recorded script is a starting point, not a finished test.

What the Recorder Does Well¶

Captures every user interaction (clicks, keystrokes, navigation) with precise timing.
Resolves Stable IDs automatically — you get clickAtWidget('win:Hotel Demo/frame:fmain/btn:Search') instead of fragile coordinate-based clicks.
Captures mouse-down screenshots in the captures/ directory for visual reference.
Generates a standalone script that can be replayed immediately.

What the Recorder Cannot Do for You¶

Add assertions. A recorded script without assertions only proves the application does not crash — it does not prove it works correctly. You must interrupt recording at key points and use Selection Assert to verify UI state.
Remove noise. The recorder captures everything, including unnecessary waits, accidental clicks, and hesitation pauses. These must be cleaned up.
Structure for reuse. A raw recording is a monolithic script. You must manually extract it into test cases and compose them into suites.

The Recording Workflow¶

Plan before recording. Know exactly what workflow you want to capture. Write down the steps. A focused recording session produces cleaner scripts than an exploratory one.
Record one test case at a time. Do not record an entire 30-minute workflow in one session. Record small, focused test cases — login, search for a customer, create an invoice — each as a separate script.
Pause and assert at key states. After every significant action (a form loads, search results appear, a record is saved), pause the recording and use Selection Assert (the target icon) to verify the UI state.
Insert performance timers around critical operations. Use startTimer('Search') before clicking Search and endTimer('Search', 3000) after results load.
Stop cleanly. End the recording with the application in a known state (see "Returning to a Known State" below).

Refactoring Recorded Scripts¶

After recording, edit the generated script before considering it done. This is expected — Lyra scripts are meant to be hand-edited.

Remove unnecessary waits. The recorder inserts waitForTimeout() calls matching your actual think time. Most of these can be shortened, but keep realistic user wait times. If you need a mandatory wait that cannot be skipped during fast replay, pass true as the second argument:

// Removable think time (skipped with --skip-waits)
await waitForTimeout(2000);

// Mandatory wait for server processing (never skipped)
await waitForTimeout(2000, true);

Use the --skip-waits flag during replay to skip non-mandatory waits and speed up test execution. This is especially useful in CI or load testing where you want to minimize idle time.

Remove accidental interactions. If you clicked the wrong button and corrected yourself during recording, delete those events from the script.

Add comments explaining intent. The recorder adds comments describing what happens (// click (stableId: win:Hotel Demo/btn:OK)). Add comments explaining why:

// Dismiss the confirmation dialog after saving the customer record
await clickAtWidget('win:Hotel Demo/alert:Saved/btn:OK');

Externalize credentials and test data. Never leave usernames and passwords hardcoded in test scripts. Use datasets instead (see "Managing Test Data" below).

Project Structure and File Organization¶

A consistent directory structure makes tests discoverable, maintainable, and easy to run selectively.

Recommended Directory Layout¶

tests/
  cases/                          -- Individual test case scripts
    customer/
      customer-search.js
      customer-create.js
      customer-edit.js
      customer-delete.js
    invoice/
      invoice-create.js
      invoice-void.js
    login.js
    navigate-to-main-menu.js
  suites/                         -- Suite files that compose test cases
    smoke.js
    customer-crud.js
    full-regression.js
  data/                           -- Datasets (JSON files)
    users.json
    customers.json
    products.json
  captures/                       -- Assertion screenshots (auto-generated)
  profiles/                       -- Load test profiles (JSON)
    steady-state.json
    ramp-up.json
  lyra-test-helpers-X.Y.Z.js     -- Shared helper library (auto-generated)

Key Organizational Principles¶

Group test cases by feature or business domain, not by page or screen. All customer-related tests go in cases/customer/, regardless of which screens they touch.
One test case per file. Each .js file in cases/ should contain exactly one complete, minimal test scenario.
Suites are composition files. A suite .js file requires and executes multiple test cases in sequence. Suites handle login and teardown; individual test cases do not.
Keep datasets separate from scripts. Test data lives in data/, not hardcoded in test files.

File Naming Conventions¶

Use descriptive, lowercase, dash-separated names that communicate the test's purpose at a glance.

Type	Convention	Examples
Test case	`<entity>-<action>.js`	`customer-search.js`, `invoice-create.js`
Suite	`<scope>.js` or `<feature>-<scope>.js`	`smoke.js`, `customer-crud.js`, `full-regression.js`
Dataset	`<entity-plural>.json`	`users.json`, `customers.json`
Load profile	`<pattern-description>.json`	`steady-state.json`, `ramp-up-100.json`
Timer labels	`<Entity> <Action>` (title case)	`'Customer Search'`, `'Invoice Save'`

Structuring Test Cases and Suites¶

The Test Case Contract¶

Every individual test case must follow this contract:

Assume you start at a known screen. Typically the main menu. The test case does not log in — that is the suite's responsibility.
Perform exactly one workflow. Search for a customer, create an invoice, edit a record — one logical operation per test case.
Assert the outcome. Use assertUiState() to verify the result.
Return to the known starting screen. Navigate back to the main menu (or whatever the agreed starting point is) before the script ends.

This contract enables test cases to be composed in any order within a suite.

Example: A Minimal Test Case¶

/**
 * Test Case: Customer Search
 * Precondition: Main menu is displayed
 * Postcondition: Main menu is displayed
 */

const {
  clickAtWidget, keyPressString, keyPress,
  assertUiState, waitForInputReady, waitForTimeout, log
} = require('../lyra-test-helpers-1.1.0.js');

async function runTest(page, context) {
  log('Starting: Customer Search');

  // Navigate to customer search from main menu
  await clickAtWidget('win:Main/frame:fmain/btn:Customers');
  await waitForInputReady();

  // Enter search criteria
  await clickAtWidget('win:Customer Search/frame:fsearch/fill:Name');
  await keyPressString('Smith');
  await keyPress('Enter');
  await waitForInputReady();

  // Assert: search results are displayed with the expected columns
  await assertUiState(
    {"sid":"win:Customer Search","widgets":[
      {"sid":"win:Customer Search/frame:fresults","widgets":[
        {"sid":"win:Customer Search/frame:fresults/browse:results","widgets":[
          {"sid":"win:Customer Search/frame:fresults/browse:results/brcol:Name"},
          {"sid":"win:Customer Search/frame:fresults/browse:results/brcol:Account"}
        ]}
      ]}
    ]}
  );

  // Return to main menu
  await keyPress('F4');
  await waitForInputReady();

  log('Completed: Customer Search');
}

module.exports = { testFlow: async (page, ctx) => { await runTest(page, ctx); } };

if (require.main === module) {
  const { firefox } = require('playwright');
  const { wrapTestExecution } = require('../lyra-test-helpers-1.1.0.js');
  wrapTestExecution(firefox, async (browser, context, page) => {
    await runTest(page, context);
  });
}

Example: A Suite File¶

A suite composes test cases into a logical group. The suite handles login and logout.

/**
 * Suite: Customer CRUD
 * Runs all customer-related test cases.
 */

const { firefox } = require('playwright');
const {
  wrapTestExecution, keyPressString, keyPress, clickAt,
  waitForP2JReady, waitForInputReady, loadDataset, getData, log
} = require('../lyra-test-helpers-1.1.0.js');

// Import test cases
const customerSearch = require('../cases/customer/customer-search.js');
const customerCreate = require('../cases/customer/customer-create.js');
const customerEdit   = require('../cases/customer/customer-edit.js');

// Load test data
loadDataset('users', '../data/users.json');

wrapTestExecution(firefox, async (browser, context, page) => {
  const user = getData('users');

  // --- Login ---
  log('Suite: Logging in');
  await page.setViewportSize({ width: 1536, height: 778 });
  await page.goto('https://localhost:7443/gui');

  await clickAt('#user-input', { x: 100, y: 10 }, '#main-iframe');
  await page.frameLocator('#main-iframe').locator('#user-input').fill(user.username);
  await keyPress('Tab');
  await page.frameLocator('#main-iframe').locator('#password-input').fill(user.password);
  await keyPress('Enter');
  await waitForP2JReady();

  log('Suite: Login complete, running test cases');

  // --- Run test cases ---
  await customerSearch.testFlow(page, context);
  await customerCreate.testFlow(page, context);
  await customerEdit.testFlow(page, context);

  // --- Logout ---
  log('Suite: Logging out');
  await keyPress('F4');
  await waitForInputReady();

  log('Suite: Complete');
});

Returning to a Known State¶

Every test case must leave the application in the same state it found it. This is the most critical rule for composable test suites. Strategies for returning to a known state:

Use keyboard shortcuts. Most FWD applications support F4 or Escape to close the current window and return to the previous screen. Navigate back step by step.
Use LQL to verify you arrived. After navigating back, use isWidgetVisible() to confirm you are on the expected screen:

// Navigate back to main menu
await keyPress('F4');
await waitForInputReady();

// Verify we are back at the main menu
const atMainMenu = await isWidgetVisible('win:Main/frame:fmain');
if (!atMainMenu) {
  throw new Error('Failed to return to main menu');
}

Handle unexpected dialogs. If the application shows confirmation dialogs ("Save changes?") when navigating away, your test case must dismiss them. Use isWidgetVisible('//alert:*') to check for dialogs and handle them.

Writing Effective Assertions¶

Assertions are the difference between a test that proves something works and a script that just clicks around.

Assert Only Key Screen Parts¶

The most important assertion principle: verify the parts that matter, ignore the parts that change.

Good assertion targets:

The presence of expected widgets (buttons, fields, browse columns).
Text content that confirms correct data (customer name, account number, status).
Window and frame titles that confirm correct navigation.

Bad assertion targets:

Timestamps, dates, or auto-generated IDs that change every run.
Dynamic counters ("Showing 1-20 of 347 results").
Widgets in areas of the screen unrelated to what you are testing.

Selection Assert: How to Use It Well¶

When using the Selection Assert tool during recording:

Draw tight rectangles. Only include the widgets you want to verify. A smaller selection area means fewer widgets in the assertion and fewer things that can cause false failures.
Focus on structural correctness. The assertion captures widget types, labels, and hierarchy. It verifies that a "Save" button exists and says "Save" — not that it is rendered with a specific font size.
Add multiple assertions per test case. Assert after each significant step: after navigation, after search, after saving a record.

Using LQL for Dynamic Assertions¶

For scenarios where you need more flexible verification than a static widget tree comparison, use the LQL query functions:

// Verify a specific window is displayed
const isVisible = await isWidgetVisible('win:Customer Details');
if (!isVisible) {
  throw new Error('Customer Details window did not appear');
}

// Verify the browse grid has the expected columns
const columns = await queryVisibleWidgets('//frame:fresults/browse:*/brcol:*');
if (columns.length < 3) {
  throw new Error('Expected at least 3 browse columns, found ' + columns.length);
}

// Verify no error dialogs are showing
const hasAlert = await isWidgetVisible('//alert:*');
if (hasAlert) {
  throw new Error('Unexpected alert dialog appeared');
}

LQL is particularly useful for:

Conditional branching. Check if a dialog appeared before trying to dismiss it.
Counting widgets. Verify that a browse grid has the expected number of columns.
Negative assertions. Verify that something is not visible (no error dialogs, no unexpected windows).

When to Use Static Assertions vs. LQL¶

Situation	Approach
Verify a specific screen loaded correctly with all expected elements	Static `assertUiState()` via Selection Assert
Check if a particular widget exists before interacting with it	`isWidgetVisible()`
Count widgets or enumerate what is on screen	`queryVisibleWidgets()` or `countVisibleWidgets()`
Guard against unexpected error dialogs	`isWidgetVisible('//alert:*')`
Debug what is currently visible	`getAllVisibleWidgets()`

Managing Test Data¶

Hardcoded test data leads to brittle, single-use scripts. Lyra's dataset system provides a clean alternative.

Use JSON Dataset Files¶

Keep test data in separate JSON files under data/. This decouples data from test logic and makes it easy to update data without touching scripts.

// data/users.json
[
  { "username": "testuser1", "password": "secret123" },
  { "username": "testuser2", "password": "secret456" }
]

// data/customers.json
[
  { "name": "Smith Industries", "account": "SI-001", "city": "London" },
  { "name": "Jones Corp",       "account": "JC-002", "city": "Manchester" }
]

Load Datasets in Suite Files, Not Test Cases¶

Datasets should be loaded once in the suite file, not in each test case. The suite owns the test data lifecycle.

// In the suite file
loadDataset('users',     'data/users.json');
loadDataset('customers', 'data/customers.json');

// In a test case
const customer = getData('customers');
log('Creating customer: ' + customer.name);

Load Testing and Data Partitioning¶

When running load tests with multiple Virtual Users, the sequence strategy (default) automatically partitions data by VU ID using round-robin:

VU 0 gets index 0
VU 1 gets index 1
VU N gets index N % dataLength

This means each VU logs in as a different user and operates on different data, avoiding collisions. Ensure your dataset has at least as many entries as VUs in your load profile.

For scenarios where you want randomized data per request, use the random strategy:

loadDataset('search-terms', 'data/search-terms.json', { strategy: 'random' });

See Virtual Users for more details.

Never Commit Secrets¶

Never store real passwords, API keys, or other secrets in dataset files that are committed to version control. Use placeholder files and document how to populate them, or load credentials from environment variables in the test script.

Performance Testing with Timers¶

Lyra's timer system lets you measure how long specific operations take and optionally fail tests when operations are too slow.

Timer Best Practices¶

Measure end-to-end user-visible operations, not internal steps. Good timer labels: 'Customer Search', 'Invoice Save', 'Report Generation'. Bad: 'Click Button', 'Wait for Input'.
Place startTimer() immediately before the triggering action and endTimer() after the result is visible (after waitForInputReady() returns):

startTimer('Customer Search');
await clickAtWidget('win:Customer Search/frame:fsearch/btn:Search');
await waitForInputReady();
endTimer('Customer Search', 3000);

Set realistic thresholds. Base thresholds on observed performance, not aspirational targets. A threshold that fails every run is useless. Start with generous thresholds (2x observed time) and tighten them as performance stabilizes.
Use the --fail-timer-threshold flag in CI to enforce performance budgets. Without this flag, thresholds are recorded but do not cause failures.

Timer Naming Conventions¶

Use consistent, descriptive timer labels that read well in reports:

Title case: 'Customer Search', not 'customer_search' or 'CUSTOMER_SEARCH'.
Action-oriented: 'Save Invoice', not 'Invoice Form'.
Include context when ambiguous: 'Search by Name' vs. 'Search by Account'.

Debugging Failed Tests¶

When a test fails, Lyra provides several artifacts to help diagnose the problem.

Failure Artifacts¶

Every failed test run produces:

failure-screenshot.png — A full-page screenshot of the application state at the moment of failure. This is the first thing to check.
failure-diagnostics.json — URL, viewport size, page title, and error details.
assert-failure-*.json — For assertion failures: both the expected and actual widget trees, plus a computed diff.
user_<ID>.log — The complete execution log with timestamps for every action.
user_<ID>_browser.log — Browser console output captured during the run.

Common Failure Patterns¶

Error	Likely Cause	How to Fix
Widget Not Found	Application is on a different screen than expected, or the widget was renamed/removed	Check `failure-screenshot.png`. Verify the preceding navigation steps. Update the Stable ID if the widget was renamed.
Assertion Failure	UI state changed (field text, widget presence, enabled state)	Check `assert-failure-*.json` for the diff between expected and actual. Update the assertion if the change is intentional.
Timeout waiting for p2j.screen	Application failed to load or crashed	Check `user_<ID>_browser.log` for JavaScript errors. Verify the server is running and accessible.
Timer threshold exceeded	Operation took longer than allowed	Check if the threshold is realistic. Look for server-side performance issues.

Debugging Tools¶

--keep-browser-open — Keeps the browser open after a failure so you can inspect the application state interactively.
--debug — Launches the Playwright Inspector for step-by-step execution.
--debug (or -d) — Enables Playwright Inspector for step-by-step debugging and verbose debug logging in helpers.
--show-mouse — Renders a visible cursor overlay during replay, making it easy to see where clicks land.
--highlight-bounds — Highlights widget bounding boxes resolved by FWD before each click.
--debug-widget-bounds — Shows per-area debug overlays with labels for widget bounds, useful for diagnosing area metadata issues.
--step-delay <ms> — Pauses the specified number of milliseconds after each action, useful for slow-motion visual debugging.
getAllVisibleWidgets() — Call this in your script to dump all currently visible Stable IDs. Invaluable when a widget is not where you expect it.

Handling Flaky Tests¶

A flaky test is one that sometimes passes and sometimes fails without any change to the application. Flaky tests erode trust in the test suite and should be treated as bugs.

Common Causes of Flakiness in Lyra Tests¶

Missing waitForInputReady(). If you click or type before the application is ready to receive input, the action may be lost. Lyra's helper functions (clickAtWidget, keyPress, keyPressString) call waitForInputReady() automatically. If you use raw Playwright API calls (e.g., page.keyboard.down()), you must call waitForInputReady() yourself.
Race conditions with server processing. After triggering a server round-trip (saving a record, running a query), the application may need time to process. Use waitForInputReady() or a brief waitForTimeout() before asserting.
Asserting on dynamic content. Assertions that include timestamps, auto-increment IDs, or record counts will fail when the data changes. Tighten your selection area to exclude dynamic content.
Shared state between test runs. If a test creates data but does not clean it up, subsequent runs may encounter different state. Ensure each test run starts from a clean, predictable application state.

How to Handle Flaky Tests¶

Reproduce. Run the test multiple times in quick succession: lyra play my-test.js repeated 5-10 times. If it fails intermittently, it is flaky.
Diagnose. Compare the logs and screenshots from passing and failing runs. Look for timing differences.
Fix at the source. Add missing waits, tighten assertions, or handle unexpected dialogs. Do not add arbitrary waitForTimeout() calls as a band-aid — find the root cause.
If the fix is non-trivial, mark and track it. Move the test to a quarantine/ directory and create a ticket. Do not leave failing tests in the active suite — they obscure real failures.

Test Isolation¶

Test isolation means each test run is independent and does not depend on state left behind by previous runs.

Why Isolation Matters¶

Parallel execution. During load testing, multiple VUs run simultaneously. If they share state (e.g., all edit the same customer record), they will conflict.
Order independence. You should be able to run any test case in any order. If test B only passes when test A runs first, you have a coupling bug.
Reproducibility. A test that depends on external state is inherently non-deterministic.

Isolation Strategies¶

Login fresh every run. Lyra does not reuse session state across runs. Each suite execution logs in, runs its test cases, and logs out. This is by design.
Use per-VU data. In load tests, use the dataset system with sequence strategy to give each VU its own user account and data.
Leave the application in the state you found it. Every test case must navigate back to the known starting screen.
Avoid creating shared data. If a test case creates a record, either delete it at the end of the test or use unique identifiers so it does not collide with other runs.

CI/CD Integration¶

While Lyra is not yet integrated into a CI pipeline, it is designed for headless execution and produces structured artifacts that map naturally to CI workflows.

Running Lyra in CI¶

Lyra's --headless flag runs tests without a visible browser window, which is required for CI environments that do not have a display:

lyra play tests/suites/smoke.js --headless

A minimal CI job looks like this:

# Install dependencies
npm install -g ./fwd-lyra-1.0.0.tgz
npx playwright install-deps && npx playwright install firefox

# Run the smoke suite
lyra play tests/suites/smoke.js --headless --fail-timer-threshold

# Archive artifacts
cp -r test_run_* $CI_ARTIFACT_DIR/

CI Strategy: Layered Test Execution¶

Organize your suites into tiers that match your CI triggers:

Tier	Suite	When to Run	Expected Duration
Smoke	`suites/smoke.js`	Every commit / PR	Under 5 minutes
Regression	`suites/full-regression.js`	Merge to main / nightly	15-30 minutes
Load	`suites/smoke.js` + load profile	Nightly / weekly	30-60 minutes

Smoke tests should cover the most critical paths — login, basic navigation, one or two key workflows. They gate every change.

Regression tests cover all known test cases. They run after changes are merged and catch regressions that smoke tests miss.

Load tests simulate concurrent users and measure performance under stress. They run on a schedule, not per-commit.

Handling Test Failures in CI¶

Fail the build on any test failure. Do not allow failing tests to be ignored.
Archive artifacts. Every test run produces a timestamped directory with logs, screenshots, and reports. Upload these as CI artifacts so developers can diagnose failures without re-running the test.
Use --fail-timer-threshold to enforce performance budgets. A test that passes functionally but takes 30 seconds for a search that should take 2 seconds is still a failure.

Load Testing Practices¶

Lyra's load testing engine spawns multiple Virtual Users, each in its own process and browser instance.

Start Small, Scale Up¶

Do not begin with a 100-user spike test. Start with a constant load profile of 2-3 users and verify that the test runs cleanly in parallel. Then gradually increase:

constant with 2-3 users for 2 minutes — verify basic functionality under concurrency.
ramp-up from 1 to 10 users over 2 minutes — find the first signs of contention.
step profile (10, 25, 50, 100 users) — identify the breaking point.
combined profile (ramp-up, steady state, ramp-down) — simulate a realistic traffic pattern.

Design Tests for Concurrency¶

Each VU must use its own credentials. Use the dataset system with enough user entries for your maximum VU count.
Avoid write conflicts. If two VUs edit the same record simultaneously, the application will likely show lock conflicts or errors. Design test data so each VU operates on its own records.
Use the loop execution mode for sustained load testing. In loop mode, each VU runs the test script repeatedly until the profile duration ends.
Set a thinkTime in loop mode to simulate realistic user pacing. Without think time, VUs will slam the server with requests as fast as possible, which may not match real usage patterns.

Set Failure Thresholds¶

Use the failureThreshold configuration to abort load tests early when error rates are unacceptable:

{
  "failureThreshold": {
    "maxFailureRate": 0.2,
    "windowSize": "30s" 
  }
}

This aborts the test if more than 20% of VU runs fail within any 30-second window. Without this, a broken server can burn through your entire load test duration producing only error logs.

Maintenance and Long-Term Health¶

Keep Assertions Up to Date¶

When the application changes (a button is renamed, a field is added to a form), assertions that cover those widgets will fail. This is expected and desirable — it is the test doing its job. Update the assertion to match the new expected state.

Do not disable or remove assertions just because they failed after an application change. Instead:

Verify the application change is intentional.
Re-record the assertion using Selection Assert, or update the assertUiState() call in the script.
Commit the updated test alongside the application change.

Review Tests Like Code¶

Test scripts are code. They should be reviewed with the same rigor as production code:

Are test case names descriptive?
Do assertions verify the right things?
Is the script free of hardcoded credentials and magic numbers?
Does the test case return to the known starting state?
Are waits justified, or are they band-aids for race conditions?

Keep the Helper Library in Sync¶

The lyra-test-helpers-X.Y.Z.js file is versioned to match your Lyra installation. When you upgrade Lyra, regenerate the helpers:

lyra record --helpers-only --output tests/

Document Non-Obvious Test Dependencies¶

If a test case requires specific application state that is not obvious (e.g., "this test assumes the 'Advanced Search' feature is enabled in settings"), document it in a comment at the top of the test file. Future maintainers will thank you.

Quick Reference Checklist¶

Use this checklist when writing or reviewing a Lyra test:

[ ] Test case file has a descriptive name (customer-search.js, not test1.js).
[ ] Test case starts with a comment documenting preconditions and postconditions.
[ ] Assertions verify key UI state after each significant action.
[ ] Assertions do not include dynamic content (timestamps, auto-generated IDs).
[ ] Test case returns to the known starting screen at the end.
[ ] Credentials come from a dataset file, not hardcoded values.
[ ] Unnecessary waitForTimeout() calls have been removed.
[ ] Performance timers bracket critical operations with realistic thresholds.
[ ] The test case can run independently (does not depend on other test cases).
[ ] The test case works in headless mode (--headless).

Project

General

Profile

FWD

Wiki