-
Notifications
You must be signed in to change notification settings - Fork 65
docs(ai-docs): playwright ai docs added #623
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
3b19a9b
feat: added agents.md and spec generator for playright e2e
rsarika b947723
Merge branch 'next' of https://github.com/webex/widgets into playrigh…
rsarika c998f85
docs: update playwright AGENTS and multiparty conference spec
rsarika 4f9eb65
fix: updated spec
rsarika 701302d
docs: simplify playwright templates and consolidate ai-docs
rsarika 813d619
revert(docs): remove web-component-patterns file
rsarika 46f5222
docs(playwright): update AGENTS with setup and env corrections
rsarika 9b3b4ed
docs(playwright): align architecture with page types and constants
rsarika 8b7aec5
Merge upstream/next into playright-spec (take upstream AGENTS.md)
rsarika 2a910fb
feat: multiparty conference tests updated
rsarika adb3d19
docs(ai-docs): keep generic playwright docs and drop code deltas
rsarika 058e6ba
docs(playwright): address ciscoRankush review feedback
rsarika File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| # Playwright Test Work - Master Template | ||
|
|
||
| ## Purpose | ||
|
|
||
| Orchestrator for Playwright E2E work in this repository. | ||
|
|
||
| --- | ||
|
|
||
| ## Workflow | ||
|
|
||
| 1. Gather complete requirements -> [01-pre-questions.md](./01-pre-questions.md) | ||
| 2. Implement tests/suites/sets -> [02-test-implementation.md](./02-test-implementation.md) | ||
| 3. Apply framework updates and documentation alignment -> [03-framework-and-doc-updates.md](./03-framework-and-doc-updates.md) | ||
| 4. Validate before completion -> [04-validation.md](./04-validation.md) | ||
|
|
||
| Doc alignment is mandatory when any suite/test/set/framework behavior changes. | ||
|
|
||
| --- | ||
|
|
||
| ## Task Types | ||
|
|
||
| Use one or more values from: | ||
|
|
||
| - `new_test_suite` | ||
| - `new_test_file` | ||
| - `add_scenarios_to_existing_test` | ||
| - `update_existing_test_logic` | ||
| - `stabilize_flaky_tests` | ||
| - `add_or_update_user_set` | ||
| - `add_or_update_utils` | ||
| - `add_or_update_test_manager` | ||
| - `add_or_update_constants` | ||
| - `add_or_update_global_setup` | ||
| - `update_playwright_docs` | ||
|
|
||
| --- | ||
|
|
||
| ## Current Set-to-Suite Mapping | ||
|
|
||
| Use `playwright/test-data.ts` as the source of truth for current set/suite mappings. | ||
|
|
||
| Guidance: | ||
| - Read existing `USER_SETS` entries before proposing changes | ||
| - Reuse an existing set when possible | ||
| - Create a new set only when required by scenario isolation, credentials, or participant count | ||
| - Keep set design generic so any scenario can be added via requirements intake | ||
|
|
||
| --- | ||
|
|
||
| ## Execution Guidance | ||
|
|
||
| - Always start with [01-pre-questions.md](./01-pre-questions.md). | ||
| - Keep implementations aligned with current framework files (`playwright/suites`, `playwright/tests`, `playwright/Utils`, `playwright/test-manager.ts`, `playwright/test-data.ts`, `playwright/constants.ts`, `playwright/global.setup.ts`, `playwright.config.ts`). | ||
| - For all test/suite/set/framework changes, update `playwright/ai-docs/AGENTS.md` and/or `playwright/ai-docs/ARCHITECTURE.md` as part of the same task. | ||
| - End with [04-validation.md](./04-validation.md). | ||
|
|
||
| --- | ||
|
|
||
| _Last Updated: 2026-03-05_ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| # Playwright Pre-Questions | ||
|
|
||
| ## Purpose | ||
|
|
||
| Mandatory intake checklist to ensure the user provides complete implementation details before Playwright changes. | ||
|
|
||
| --- | ||
|
|
||
| ## Core Principles | ||
|
|
||
| ### 1. Root Cause Analysis Over Symptomatic Fixes | ||
| - **NEVER** increase timeouts as a first response to flaky tests | ||
| - **ALWAYS** investigate WHY a test is failing before applying fixes | ||
| - Timeouts mask underlying issues (race conditions, incorrect selectors, state leakage) | ||
| - Document the specific reason if timeout increases are necessary | ||
|
|
||
| ### 2. No Lazy Reasoning | ||
| - **NEVER** assume "it probably works this way" - verify by reading code or testing | ||
| - **NEVER** copy-paste patterns without understanding them | ||
| - **NEVER** skip edge cases or error handling | ||
| - **ALWAYS** ask clarifying questions when requirements are unclear | ||
|
|
||
| ### 3. Timeout Justification Required | ||
| - Default timeouts should be sufficient for most operations | ||
| - When increasing timeouts, document: | ||
| - What specific operation needs more time | ||
| - Why this operation requires more time (measured/observed duration) | ||
| - Why the operation cannot be optimized instead | ||
| - Example GOOD reason: "Widget initialization requires loading external config (observed 15-20s in production)" | ||
| - Example BAD reason: "Test was flaky, increased timeout to make it pass" | ||
|
|
||
| ### 4. Requirements Gathering First | ||
| - Use questionnaire-driven approach | ||
| - Understand full scope before implementation | ||
| - Identify edge cases and error scenarios upfront | ||
| - Don't start coding until requirements are clear | ||
|
|
||
| --- | ||
|
|
||
| ## Required Questions | ||
|
|
||
| ### 1. Task Scope | ||
|
|
||
| - What exactly should be changed? (one sentence) | ||
| - Is this a new suite, new test, update, or flaky-test stabilization? | ||
| - Which current suite is impacted? (`playwright/suites/*.spec.ts`) | ||
|
|
||
| ### 2. Coverage Details | ||
|
|
||
| - Exact scenarios to add/update (list each scenario title) | ||
| - Expected assertions for each scenario | ||
| - Out-of-scope scenarios | ||
|
|
||
| ### 3. Execution Target | ||
|
|
||
| - Which set(s) should run this? (`SET_1`..`SET_N` or new set) | ||
| - If new set is needed: | ||
| - What suite filename should `TEST_SUITE` map to? | ||
| - How many agents are required? (1-4) | ||
| - What entry points/queues/URLs are needed? | ||
|
|
||
| ### 4. Data and Environment | ||
|
|
||
| - Required env keys (entry points, queue, chat URL, dial number, etc.) | ||
| - How many agents are needed? (1-4) | ||
| - Which roles are needed? (Agent1/Agent2/Agent3/Agent4/caller/extension/multi-session) | ||
| - Any OAuth or token-related constraints? | ||
| - Are there specific user credentials or permissions required? | ||
|
|
||
| ### 5. Setup and Utilities | ||
|
|
||
| - Preferred `TestManager` setup method (if known) | ||
| - Can existing `Utils/*` be reused, or is a new helper needed? | ||
|
|
||
| ### 6. Stability Expectations | ||
|
|
||
| - Known flaky points today and their root causes (if known) | ||
| - Have existing flaky tests been investigated for root cause? | ||
| - Are there specific operations that legitimately require longer timeouts? (with justification) | ||
| - What cleanup is required? (wrapup/end-state/agent cleanup) | ||
| - Are there race conditions or state dependencies to address? | ||
|
|
||
| **Important:** If user mentions timeout issues: | ||
| 1. First investigate: console logs, network traces, application state | ||
| 2. Identify root cause: race condition? slow operation? incorrect selector? | ||
| 3. Fix root cause: add proper state checks, optimize operation, fix selector | ||
| 4. Only increase timeout if justified and documented | ||
|
|
||
| ### 7. Documentation Expectations | ||
|
|
||
| - Should `playwright/ai-docs/AGENTS.md` be updated? | ||
| - Should `playwright/ai-docs/ARCHITECTURE.md` be updated? | ||
|
|
||
| --- | ||
|
|
||
| ## Completion Gate | ||
|
|
||
| Do not start implementation until required questions are answered or explicitly assumed and documented. | ||
|
|
||
| --- | ||
|
|
||
| _Last Updated: 2026-03-04_ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # Playwright Test Implementation | ||
|
|
||
| ## Purpose | ||
|
|
||
| Implement new or updated Playwright suites/tests/sets using current framework conventions. | ||
|
|
||
| --- | ||
|
|
||
| ## Implementation Paths | ||
|
|
||
| ### A. New Suite | ||
|
|
||
| - Add `playwright/suites/<name>-tests.spec.ts` | ||
| - Import one or more test factories from `playwright/tests/` | ||
| - Wire `TEST_SUITE` in `playwright/test-data.ts` | ||
|
|
||
| ### B. New Test File | ||
|
|
||
| - Add `playwright/tests/<feature>-test.spec.ts` | ||
| - Export default factory: `create...Tests` | ||
| - Register factory in target suite with `test.describe(...)` | ||
|
|
||
| ### C. Update Existing Tests | ||
|
|
||
| - Update scenario logic in `playwright/tests/*.spec.ts` | ||
| - Preserve existing factory export shape | ||
| - Keep deterministic setup and cleanup | ||
|
|
||
| ### D. Add/Update User Set | ||
|
|
||
| - Update `USER_SETS` in `playwright/test-data.ts` | ||
| - Provide required set fields: | ||
| - `AGENTS` | ||
| - `QUEUE_NAME` | ||
| - `CHAT_URL` | ||
| - `EMAIL_ENTRY_POINT` | ||
| - `ENTRY_POINT` | ||
| - `TEST_SUITE` | ||
|
|
||
| --- | ||
|
|
||
| ## Required Patterns | ||
|
|
||
| - Use `TestManager` convenience setup methods when possible | ||
| - Reuse `playwright/Utils/*.ts` before adding new helper logic | ||
| - Use shared constants from `playwright/constants.ts` | ||
| - Keep set-scoped env access via `${testManager.projectName}_...` | ||
| - For flaky test fixes, record root cause and update docs if behavior/expectations changed | ||
|
|
||
| --- | ||
|
|
||
| ## Done Criteria | ||
|
|
||
| - [ ] Suite/test wiring complete | ||
| - [ ] Set mapping correct (if changed) | ||
| - [ ] Cleanup path present for new/changed scenarios | ||
| - [ ] `playwright/ai-docs/AGENTS.md` baseline updated (if suites/sets/workflow changed) | ||
| - [ ] `playwright/ai-docs/ARCHITECTURE.md` topology/mapping/runtime docs updated (if files/sets/framework behavior changed) | ||
| - [ ] Fix/stabilization changes include a doc update loop when behavior or constraints changed | ||
|
|
||
| --- | ||
|
|
||
| _Last Updated: 2026-03-05_ | ||
42 changes: 42 additions & 0 deletions
42
ai-docs/templates/playwright/03-framework-and-doc-updates.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # Playwright Framework and Docs Updates | ||
|
|
||
| ## Purpose | ||
|
|
||
| Apply shared framework updates and keep Playwright docs aligned in the same task. | ||
|
|
||
| --- | ||
|
|
||
| ## Framework Files | ||
|
|
||
| Update these as needed for reusable behavior changes: | ||
|
|
||
| - `playwright/Utils/*.ts` | ||
| - `playwright/test-manager.ts` | ||
| - `playwright/constants.ts` | ||
| - `playwright/global.setup.ts` | ||
| - `playwright.config.ts` | ||
|
|
||
| --- | ||
|
|
||
| ## Doc Files (Required Alignment) | ||
|
|
||
| - `playwright/ai-docs/AGENTS.md` | ||
| - `playwright/ai-docs/ARCHITECTURE.md` | ||
|
|
||
| `playwright/ai-docs/ARCHITECTURE.md` is the single source of truth for framework internals and scenario-level technical reference. | ||
|
|
||
| If a task adds/changes suites, tests, sets, setup flows, runtime flags, or stability constraints, doc updates are mandatory. | ||
|
|
||
| --- | ||
|
|
||
| ## Checklist | ||
|
|
||
| - [ ] Shared changes are reusable and not test-case specific | ||
| - [ ] Existing tests remain backward compatible or intentionally migrated | ||
| - [ ] AGENTS baseline reflects current suites/sets/workflow | ||
| - [ ] ARCHITECTURE topology/mapping/runtime sections match current code | ||
| - [ ] No references to non-existent files/suites/sets | ||
|
|
||
| --- | ||
|
|
||
| _Last Updated: 2026-03-05_ |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
M8: Governance Gap — Doc Updates Not Enforced
This Done Criteria is missing mandatory doc update steps. Currently has 3 items, none mention AGENTS.md or ARCHITECTURE.md.
Should include:
- [ ] playwright/ai-docs/AGENTS.md baseline updated (if new suite/set added)- [ ] playwright/ai-docs/ARCHITECTURE.md file topology and set mapping updated (if new files/sets added)Also:
03-framework-and-doc-updates.mdsays "update only if required" — should be mandatory in the 00-master workflow when adding new tests/suites/sets04-validation.mdshould explicitly check:- [ ] AGENTS.md baseline matches actual suitesand- [ ] ARCHITECTURE.md file topology matches actual filesplaywright/ai-docs/specs/multiparty-conference.spec.mdwas added — but this file is not in the diff. Either implement the specs/ concept or remove the claim.