Conversation
|
/azp run |
|
Azure Pipelines failed to run 2 pipeline(s). |
There was a problem hiding this comment.
Pull Request Overview
This PR adds accessibility testing capabilities to the React Native Windows CI/CD pipeline by integrating automated accessibility checks during pull request validation.
- Adds a new accessibility testing stage that runs on PR builds
- Implements comprehensive accessibility validation using axe-core and custom scanning
- Integrates with React Native Gallery for real-world component testing
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| .ado/stages.yml | Adds the new accessibility testing job to the build pipeline |
| .ado/jobs/accessibility-tests.yml | Implements the complete accessibility testing workflow with Gallery integration |
| - script: | | ||
| echo "Checking PR changes for accessibility compliance..." | ||
| git diff origin/main --name-only | grep -E "\.(tsx?|jsx?)$" | head -20 | while read file; do | ||
| if [ -f "$file" ]; then | ||
| echo "Scanning $file for accessibility patterns..." | ||
| # Check for accessibility props | ||
| grep -n "accessibility\|testID\|aria-" "$file" || echo "No accessibility props found in $file" | ||
| fi | ||
| done |
There was a problem hiding this comment.
This bash-style script with pipes and grep commands will fail on Windows agents. The pipeline should use PowerShell commands or cross-platform alternatives since this is targeting Windows build agents.
| - script: | | |
| echo "Checking PR changes for accessibility compliance..." | |
| git diff origin/main --name-only | grep -E "\.(tsx?|jsx?)$" | head -20 | while read file; do | |
| if [ -f "$file" ]; then | |
| echo "Scanning $file for accessibility patterns..." | |
| # Check for accessibility props | |
| grep -n "accessibility\|testID\|aria-" "$file" || echo "No accessibility props found in $file" | |
| fi | |
| done | |
| - powershell: | | |
| Write-Host "Checking PR changes for accessibility compliance..." | |
| $files = git diff origin/main --name-only | Select-String -Pattern '\.(ts|tsx|js|jsx)$' | Select-Object -First 20 | |
| foreach ($fileObj in $files) { | |
| $file = $fileObj.ToString().Trim() | |
| if (Test-Path $file) { | |
| Write-Host "Scanning $file for accessibility patterns..." | |
| $matches = Select-String -Pattern 'accessibility|testID|aria-' -Path $file -SimpleMatch | |
| if ($matches) { | |
| $matches | ForEach-Object { Write-Host $_ } | |
| } else { | |
| Write-Host "No accessibility props found in $file" | |
| } | |
| } | |
| } |
| testResultsFormat: 'JUnit' | ||
| testResultsFiles: '**/accessibility-*.xml' |
There was a problem hiding this comment.
The pipeline expects JUnit XML files but the accessibility tools are configured to generate JSON files (accessibility-results.json). Either change the expected format to match the generated files or configure the tools to output XML format.
| - script: | | ||
| echo "Installing accessibility testing tools..." | ||
| yarn add --dev @axe-core/cli axe-playwright @playwright/test | ||
| yarn add --dev accessibility-checker |
There was a problem hiding this comment.
Installing dependencies during the build process is inefficient and unreliable. These dependencies should be added to package.json and installed during the standard dependency installation step.
| yarn add --dev accessibility-checker |
| cd $(GalleryPath) | ||
| npx react-native run-windows --no-launch --no-packager --arch x64 --release | ||
| displayName: 'Build Gallery Windows App' | ||
|
|
There was a problem hiding this comment.
The Gallery app is built with --no-packager flag but later the accessibility check tries to connect to localhost:8081, which is typically the Metro bundler port. This creates an inconsistency since the packager won't be running.
| # Start Metro bundler (packager) in background | |
| - script: | | |
| echo "Starting Metro bundler..." | |
| cd $(GalleryPath) | |
| start "" cmd /c "npx react-native start" | |
| timeout /t 10 /nobreak | |
| displayName: 'Start Metro Bundler' |
There was a problem hiding this comment.
Why does it say Azure Pipelines failed to run 2 pipeline(s)?
There was a problem hiding this comment.
@copilot why did Azure Pipelines failed to run
|
/azp run |
|
Azure Pipelines failed to run 2 pipeline(s). |
| npx axe --chrome-options="--no-sandbox --disable-dev-shm-usage" --save accessibility-results.json --reporter json http://localhost:8081 || true | ||
|
|
||
| # Also run accessibility checks on any locally running RNW components |
There was a problem hiding this comment.
The axe command is trying to test http://localhost:8081 but there's no evidence that a web server is started on this port. The Gallery app is launched as a native Windows executable, not a web server.
| npx axe --chrome-options="--no-sandbox --disable-dev-shm-usage" --save accessibility-results.json --reporter json http://localhost:8081 || true | |
| # Also run accessibility checks on any locally running RNW components | |
| # The axe CLI cannot be used directly on the native Windows app, as it is not served on http://localhost:8081. | |
| # To run accessibility checks, use Playwright or another tool to automate the native app UI. | |
|
#15178 has the same title. Is one of them a duplicate? |
|
Please fix PR title: Add Accessibility Checks and ensure |
2 participants
Description
adding accessibility checks in pipeline
Type of Change
Why
adding accessibility checks in pipeline
Resolves [Add Relevant Issue Here]
What
adding accessibility checks in pipeline
Screenshots
Add any relevant screen captures here from before or after your changes.
Testing
If you added tests that prove your changes are effective or that your feature works, add a few sentences here detailing the added test scenarios.
Optional: Describe the tests that you ran locally to verify your changes.
Changelog
Should this change be included in the release notes: indicate yes or no
Add a brief summary of the change to use in the release notes for the next release.
Microsoft Reviewers: Open in CodeFlow