web-qa-bot – OpenClaw Skill

---

name: web-qa-bot description: AI-powered web application QA automation using accessibility-tree based testing. Smoke tests, test suites, and PDF reports. version: 0.1.0

web-qa-bot

AI-powered web application QA automation using accessibility-tree based testing.

Overview

This skill provides tools for automated QA testing of web applications. It uses browser accessibility trees for reliable element detection instead of fragile CSS selectors.

Installation

npm install -g web-qa-bot agent-browser
agent-browser install

Commands

Quick Smoke Test

web-qa-bot smoke https://example.com

Runs basic health checks:

• Page loads successfully
• No console errors
• Navigation elements present
• Images have alt text

Run Test Suite

web-qa-bot run ./tests/suite.yaml --output report.md

Generate PDF Report

web-qa-bot report ./results.json -o report.pdf -f pdf

Use Cases

1. Quick Site Health Check

# Smoke test a production URL
web-qa-bot smoke https://app.example.com --checks pageLoad,consoleErrors,navigation

2. Pre-deployment QA

Create a test suite and run before each deployment:

# tests/critical-paths.yaml
name: Critical Paths
baseUrl: https://staging.example.com

tests:
  - name: Login flow
    steps:
      - goto: /login
      - type: { ref: Email, text: test@example.com }
      - type: { ref: Password, text: testpass }
      - click: Sign In
      - expectVisible: Dashboard
      - expectNoErrors: true

web-qa-bot run ./tests/critical-paths.yaml --output qa-report.pdf -f pdf

3. Monitor for Regressions

# Run tests and fail CI if issues found
web-qa-bot run ./tests/smoke.yaml || exit 1

4. Programmatic Testing

import { QABot } from 'web-qa-bot'

const qa = new QABot({
  baseUrl: 'https://example.com',
  headless: true
})

await qa.goto('/')
await qa.click('Get Started')
await qa.snapshot()
qa.expectVisible('Sign Up')
await qa.close()

Integration with agent-browser

This tool wraps agent-browser CLI for browser automation:

# Connect to existing browser session
web-qa-bot smoke https://example.com --cdp 18800

# Run headed for debugging
web-qa-bot run ./tests/suite.yaml --no-headless

Test Results Format

Results are returned as structured JSON:

{
  "name": "Smoke Test",
  "url": "https://example.com",
  "summary": {
    "total": 4,
    "passed": 3,
    "failed": 0,
    "warnings": 1
  },
  "tests": [
    {
      "name": "Page Load",
      "status": "pass",
      "duration": 1234
    }
  ]
}

Tips

1. Use role-based selectors - More reliable than CSS classes
2. Check console errors - Often reveals hidden issues
3. Test both navigation methods - Direct URL and in-app routing
4. Screenshot on failure - Automatic in test suites
5. Monitor for modals - Can block interactions

Report Formats

• Markdown - Default, human-readable
• PDF - Professional reports via ai-pdf-builder
• JSON - Machine-readable for CI/CD

Troubleshooting

"agent-browser not found"

npm install -g agent-browser
agent-browser install

"Element not found"

Take a snapshot first to see available refs:

agent-browser snapshot

"Timeout waiting for element"

Increase timeout or check if element is behind a loading state:

steps:
  - waitMs: 2000
  - waitFor: "Loading" # Wait for loading to appear
  - waitFor: "Content" # Then wait for content

Links

• GitHub
• npm

web-qa-bot

AI-powered web application QA automation using accessibility-tree based testing.

Features

• Accessibility-first testing - Uses browser accessibility tree instead of CSS selectors
• Smart element detection - Role-based locators with auto-wait and retry logic
• Console monitoring - Captures errors, warnings, and custom events
• Modal detection - Automatically detects dialogs and popups
• Stale ref handling - Re-snapshots when elements become stale
• Professional reports - Markdown and PDF output via ai-pdf-builder
• Smoke tests - Quick health checks for any URL
• Built on agent-browser - Fast, reliable browser automation

Installation

npm install -g web-qa-bot

# Peer dependency
npm install -g agent-browser
agent-browser install

Quick Start

Smoke Test

Run quick health checks on any URL:

web-qa-bot smoke https://example.com

Output:

=== Smoke Test Results ===

URL: https://example.com
Duration: 2.34s

✓ Page Load: PASS
✓ Console Errors: PASS
✓ Navigation: PASS
✓ Images: PASS

Total: 4 | Pass: 4 | Fail: 0 | Warn: 0

Test Suite

Create a test suite file (tests/homepage.yaml):

name: Homepage Tests
baseUrl: https://example.com

tests:
  - name: Page loads with title
    steps:
      - goto: /
      - expectVisible: heading
      - expectTitle: "Example Domain"

  - name: Navigation works
    steps:
      - goto: /
      - click: "More information"
      - expectUrl:
          contains: /about

  - name: No console errors
    steps:
      - goto: /
      - expectNoErrors: true

Run the suite:

web-qa-bot run ./tests/homepage.yaml --output report.md

Generate PDF Report

web-qa-bot report ./results.json -o report.pdf -f pdf --company "Acme Corp"

CLI Commands

smoke <url>

Run smoke tests on a URL.

web-qa-bot smoke https://example.com [options]

Options:
  --checks <list>    Comma-separated checks: pageLoad,consoleErrors,navigation,images,forms,accessibility,performance
  --timeout <ms>     Timeout in milliseconds (default: 30000)
  -o, --output       Output report path

run <suite>

Run a test suite from a YAML or JSON file.

web-qa-bot run ./tests/suite.yaml [options]

Options:
  --cdp <port>       Connect to existing browser on CDP port
  --no-headless      Run in headed mode (visible browser)
  --timeout <ms>     Default timeout
  -o, --output       Output report path
  -f, --format       Report format: markdown, pdf, json
  --company          Company name for PDF header
  --verbose          Enable verbose logging

report <results>

Generate a report from test results.

web-qa-bot report ./results.json -o report.pdf

Options:
  -o, --output       Output file path
  -f, --format       Report format: markdown, pdf, json
  --company          Company name for PDF header

Programmatic API

import { QABot, smokeTest } from 'web-qa-bot'

// Quick smoke test
const result = await smokeTest({
  url: 'https://example.com',
  checks: ['pageLoad', 'consoleErrors', 'navigation']
})

console.log(result.summary) // { total: 3, passed: 3, failed: 0, ... }

// Full test suite
const qa = new QABot({
  baseUrl: 'https://example.com',
  headless: true,
  screenshotDir: './screenshots'
})

try {
  // Navigate
  await qa.goto('/')
  
  // Interact
  await qa.click('Login')
  await qa.type('Email', 'user@example.com')
  await qa.type('Password', 'secret')
  await qa.click('Submit')
  
  // Assert
  await qa.snapshot()
  qa.expectVisible('Dashboard')
  qa.expectUrl({ contains: '/dashboard' })
  await qa.expectNoErrors()
  
  // Generate report
  await qa.generateReport('./report.pdf', { format: 'pdf' })
} finally {
  await qa.close()
}

Test Suite Format

Test suites can be written in YAML or JSON:

name: My Test Suite
baseUrl: https://example.com

# Run before all tests
beforeAll:
  - goto: /login
  - type: { ref: Email, text: admin@example.com }
  - type: { ref: Password, text: secret }
  - click: Submit
  - waitFor: Dashboard

# Run before each test
beforeEach:
  - goto: /

# Test cases
tests:
  - name: Homepage loads
    steps:
      - goto: /
      - expectVisible: Welcome
    
  - name: Search works
    steps:
      - type: { ref: Search, text: hello }
      - press: Enter
      - waitFor: Results
      - expectCount: { selector: article, min: 1 }
    
  - name: Known issue (still runs, marked as warning)
    knownIssue: JIRA-123
    steps:
      - click: Broken Button
      - expectVisible: Success

  - name: Skip this test
    skip: true
    steps:
      - goto: /disabled-feature

# Run after each test
afterEach:
  - screenshot: after-test

# Run after all tests
afterAll:
  - click: Logout

Available Assertions

Assertion	Description
expectVisible(selector)	Element exists in accessibility tree
expectNotVisible(selector)	Element does not exist
expectText(selector, text)	Element has matching text
expectUrl({ contains })	URL matches pattern
expectCount(role, count)	Count of elements with role
expectNoErrors()	No console errors
expectConsoleEvent(pattern)	Console event matches pattern
expectModal(present)	Modal is present/absent
expectTitle(text)	Page title matches
expectClickable(selector)	Element is interactive and enabled

Available Actions

Action	Description
goto(url)	Navigate to URL
click(selector)	Click element
type(selector, text)	Type text into element
press(key)	Press keyboard key
hover(selector)	Hover over element
select(selector, value)	Select dropdown option
waitFor(selector)	Wait for element to appear
waitForLoad()	Wait for page load
waitForUrl(pattern)	Wait for URL to match
screenshot(name)	Take screenshot
snapshot()	Take accessibility tree snapshot

Element Selectors

web-qa-bot uses accessibility-tree based selectors:

// By ref ID (from snapshot output)
await qa.click('@e42')

// By text content
await qa.click('Submit')

// By role:name
await qa.click('button:Submit')

// By role with partial name
await qa.waitFor('heading')  // Any heading

Best Practices

Based on Playwright testing best practices:

1. Test user-visible behavior - Not implementation details
2. Use role-based locators - Accessibility refs over CSS classes
3. Web-first assertions - Built-in auto-wait, no manual timeouts
4. Isolated tests - Each test starts fresh
5. No third-party dependencies - Mock external services

Learnings Integrated

This tool incorporates learnings from real-world QA sessions:

• Async data handling - Configurable wait strategies for dynamic content
• Stale ref detection - Automatic re-snapshot when elements change
• Modal detection - Via DOM and console event monitoring
• Route comparison - Test direct URL vs in-app navigation
• Audio/media state - UI state verification for media elements

Requirements

• Node.js 18+
• agent-browser CLI (peer dependency)
• For PDF reports: ai-pdf-builder and LaTeX

License

MIT

Contributing

PRs welcome! Please read the contributing guidelines first.

---

Built by NextFrontierBuilds