FirstQA Documentation

Everything you need to know about Ovi AI and how FirstQA works.

1. What is FirstQA?

FirstQA is your AI-powered QA engineer. Ovi AI analyzes every pull request for bugs, risks, and edge cases — generates prioritized test scenarios — runs them in a real browser — and delivers Playwright test code. When you need deeper coverage, on-demand senior QA engineers are one click away.

2. How It Works

1. Install the GitHub App: One click — select your repos. No code changes, no CI config.
2. Ovi Analyzes Every Change: Automatically on PR open/update, or manually with /qa. Ovi finds bugs, risks, and edge cases.
3. Test Recipe: Prioritized test scenarios with exact steps and expected results, generated from the code changes.
4. Browser Test Execution: Comment /qa testrun — Ovi's AI agent runs every scenario in a real Chromium browser with screenshots and video.
5. Playwright Test Code: Download ready-to-use spec files with accurate selectors and flows.
6. Human QA (on-demand): Senior QA engineers for exploratory testing, pre-launch audits, and complex flows.

3. Getting Started

Option A: GitHub App (PR Analysis + Test Execution)

1. Install the GitHub App: Install FirstQA on your repositories.
2. Open a Pull Request: Create or update a PR in an enabled repository.
3. Get Analysis: Ovi automatically analyzes the PR — bugs, risks, test scenarios, ship score. Or comment /qa to trigger manually.
4. Run Tests: Comment /qa testrun to execute browser tests. Add -env=URL to target a specific environment.
5. Download Playwright Tests: Grab the generated spec files from the analysis comment.

Option B: Chrome Extension (Ticket Analysis)

1. Install the Extension: Add the FirstQA Chrome Extension from your account dashboard.
2. Navigate to a Ticket: Open any Linear or Jira ticket in your browser.
3. Click Analyze: Ovi reviews the ticket — surfacing gaps, edge cases, risks, and test scenarios before coding starts.

4. Commands

5. Ovi AI — Your AI QA Engineer

Ovi works on every PR, automatically or on-demand:

PR Analysis

• Bug Detection: Functional bugs, logic errors, data flow issues, race conditions
• Risk Assessment: Edge cases, state transitions, missing error handling
• UI/UX Review: User-visible issues, confusing flows, inconsistent behavior
• Ship Score: Go/No-Go confidence rating based on the analysis

Test Recipe

• Prioritized Scenarios: High/medium/low priority based on risk and impact
• Exact Steps: Step-by-step instructions with expected results
• User Journey Order: Scenarios ordered by natural user flow (access → create → edit → delete)
• Executability Score: Each scenario scored for browser automation feasibility

Browser Test Execution

• AI Agent: Stagehand agent autonomously navigates and tests in real Chromium
• Cloud Browsers: Runs on Browserbase — no local setup needed
• Evidence: Screenshots and session replay video for every scenario
• Auth Support: Cookie injection via -context or configured credentials

Playwright Test Code

• Ready-to-Use: Downloadable .spec.js files with accurate selectors
• Based on Real Code: Selectors and flows extracted from your actual codebase
• Copy & Integrate: Drop into your test suite and run locally or in CI

Ship Score

6. Human QA Engineers (On-Demand)

When you need deeper coverage — before a major release, for complex user flows, or when AI isn't enough:

• Exploratory Testing: Senior QA engineers test your app before critical releases
• Human Judgment: Complex flows, edge cases, and UX that require human evaluation
• Pre-Launch Audits: Go/no-go confidence on critical releases
• QA Strategy: Process setup, test planning, and embedded QA support
• Pay Per Engagement: No contracts — scale up when it matters

Request Human QA or Book a Call

7. Test Execution Authentication

If your app requires login, you have two options to authenticate the test agent:

Option A: Cookie Injection (via command)

Pass auth cookies directly in the /qa testrun command:

/qa testrun -env=https://staging.myapp.com -context cookie:session_id=abc123;auth_token=xyz789

Extract cookies from your browser's DevTools (Application → Cookies) while logged in.

Option B: Configured Credentials (via dashboard)

Set TEST_USER_EMAIL and TEST_USER_PASSWORD in your FirstQA dashboard settings. The agent will attempt deterministic login with these credentials.

8. Integrations

9. FAQ

10. Need Help?

• Email us at support@firstqa.dev
• Schedule a demo call
• Visit the Support page