vscode(feat): Add VS Code extension E2E tests and CI pipeline by lambrianmsft · Pull Request #8884 · Azure/LogicAppsUX

lambrianmsft · 2026-03-05T02:30:14Z

ExTester UI tests: workspace creation, designer actions, inline JavaScript, stateless variables, designer view extensions, keyboard navigation, data mapper
Workspace conversion tests: dialog dismiss, create workspace from legacy project, multiple designers simultaneously, right-click add workflow, subfolder prompt
Shared helper modules: helpers.ts, designerHelpers.ts, runHelpers.ts
GitHub Actions workflow for CI (vscode-e2e.yml)
CLI integration tests: workspace conversion, project outside workspace
Documentation: SKILL.md, CLAUDE.md updates, copilot-instructions
Product fix: silentAuth support in getAuthorizationToken.ts
createWorkspace validation improvements

ADO #31054994: Full coverage of all 15 steps (workspace requirement)

Commit Type

feature - New functionality
fix - Bug fix
refactor - Code restructuring without behavior change
perf - Performance improvement
docs - Documentation update
test - Test-related changes
chore - Maintenance/tooling

Risk Level

Low - Minor changes, limited scope
Medium - Moderate changes, some user impact
High - Major changes, significant user/system impact

What & Why

Impact of Change

Users:
There are no user facing changes.
Developers:
Developers will have better visibility on how their changes affect the vscode experience.
System:
Test architecture is now set up for vscode webviews so new functionality can have e2e tests written for them.

Test Plan

Unit tests added/updated
E2E tests added/updated
Manual testing completed
Tested in:

Contributors

Screenshots/Videos

github-actions · 2026-03-05T02:34:12Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 504

Copilot

Pull request overview

Adds end-to-end test coverage for the Logic Apps VS Code extension (both UI-driven ExTester/Selenium suites and extension-host CLI-style tests), plus a CI workflow to execute them, while also tightening Create Workspace validation and enabling optional silent auth for automation scenarios.

Changes:

Introduces extensive ExTester UI E2E suites (workspace creation/conversion + designer lifecycle scenarios) with shared helpers and documentation.
Adds a GitHub Actions workflow to run the VS Code extension E2E pipeline and upload screenshots as artifacts.
Improves Create Workspace validation (notably for function name/namespace) and adds silentAuth support in token acquisition.

Reviewed changes

Copilot reviewed 53 out of 56 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
apps/vs-code-react/src/app/createWorkspace/validation/helper.ts	Adds C#-safe function name regex + stronger function field validation messages.
apps/vs-code-react/src/app/createWorkspace/validation/test/helper.test.ts	Adds Vitest coverage for new validation behaviors.
apps/vs-code-react/src/app/createWorkspace/steps/dotNetFrameworkStep.tsx	Revalidates function namespace/name on state changes.
apps/vs-code-react/src/app/createWorkspace/createWorkspace.tsx	Uses namespace regex for navigation gating during wizard progression.
apps/vs-code-designer/tsconfig.e2e.json	Adds TypeScript config for extension-host E2E tests output.
apps/vs-code-designer/src/test/ui/workspaceManifest.ts	Shared manifest format + cleanup helpers for cross-phase UI tests.
apps/vs-code-designer/src/test/ui/workspaceConversionYes.test.ts	UI test validating “Open Workspace” prompt and accepting it.
apps/vs-code-designer/src/test/ui/workspaceConversionSubfolder.test.ts	UI test validating prompt when opening a logic app subfolder and dismissing it.
apps/vs-code-designer/src/test/ui/workspaceConversionNo.test.ts	UI test validating prompt dismissal (“No”/escape/close) and VS Code remains usable.
apps/vs-code-designer/src/test/ui/workspaceConversionCreate.test.ts	UI test creating workspace from a legacy project and verifying source is untouched.
apps/vs-code-designer/src/test/ui/test-settings.json	Disables updates/telemetry/trust prompts to stabilize UI automation.
apps/vs-code-designer/src/test/ui/statelessVariables.test.ts	UI E2E around variable actions + debug/run verification.
apps/vs-code-designer/src/test/ui/runHelpers.ts	Shared debug/overview/run-history helper utilities for UI tests.
apps/vs-code-designer/src/test/ui/run-clean.ps1	Local helper script to clean processes and run UI tests.
apps/vs-code-designer/src/test/ui/pin-ext-dev-path.js	Preload hook preventing ExTester from clobbering `EXTENSION_DEV_PATH`.
apps/vs-code-designer/src/test/ui/keyboardNavigation.test.ts	UI E2E for canvas keyboard navigation.
apps/vs-code-designer/src/test/ui/inlineJavascript.test.ts	UI E2E for inline JavaScript action + run verification.
apps/vs-code-designer/src/test/ui/helpers.ts	Shared UI test helpers (screenshots, dismissal, activity bar/tree helpers).
apps/vs-code-designer/src/test/ui/designerViewExtended.test.ts	UI E2E for parallel branches + run-after settings.
apps/vs-code-designer/src/test/ui/dataMapper.test.ts	UI E2E smoke coverage for Data Mapper integration points.
apps/vs-code-designer/src/test/ui/WORKSPACE_CONVERSION_TEST_PLAN.md	Documents mapping from ADO steps → test files/phases.
apps/vs-code-designer/src/test/ui/README.md	Updates UI E2E documentation (phases, build/run, gotchas).
apps/vs-code-designer/src/test/e2e/runTest.ts	Adds a “runner entry point” placeholder for extension-host tests.
apps/vs-code-designer/src/test/e2e/integration/workflow.test.ts	Adds extension-host integration tests around workflow file access.
apps/vs-code-designer/src/test/e2e/integration/projectOutsideWorkspace.test.ts	Adds extension-host integration tests for workspace/project prompt behavior.
apps/vs-code-designer/src/test/e2e/integration/designerOpens.test.ts	Adds extension-host tests simulating webview panel creation/caching/protocol.
apps/vs-code-designer/src/test/e2e/integration/designer.test.ts	Adds basic extension-host webview panel API tests.
apps/vs-code-designer/src/test/e2e/integration/createWorkspace.test.ts	Adds extension-host tests that validate expected workspace structure.
apps/vs-code-designer/src/test/e2e/extension.test.ts	Adds basic extension-host activation tests.
apps/vs-code-designer/src/test/e2e/commands.test.ts	Adds extension-host command enumeration/smoke tests.
apps/vs-code-designer/src/test/e2e/README.md	Documents CLI-based E2E structure and usage.
apps/vs-code-designer/src/app/utils/codeless/getAuthorizationToken.ts	Adds `silentAuth` option support to avoid interactive auth prompts in automation.
apps/vs-code-designer/e2e/test-workspace/package.json	Adds a fixture workspace for extension-host E2E runs.
apps/vs-code-designer/e2e/test-workspace/Workflows/TestWorkflow/workflow.json	Adds a sample workflow fixture.
apps/vs-code-designer/e2e/test-workspace/.vscode/settings.json	Adds fixture workspace VS Code settings.
apps/vs-code-designer/e2e/test-workspace/.vscode/extensions.json	Adds fixture workspace extension recommendations.
apps/vs-code-designer/CLAUDE.md	Documents E2E test phases/helpers and authoring rules.
apps/vs-code-designer/.vscode/tasks.json	Adds VS Code tasks for compiling E2E tests / watch.
apps/vs-code-designer/.vscode/launch.json	Adds launch configs for extension + tests.
apps/vs-code-designer/.vscode-test.mjs	Adds `@vscode/test-cli` configuration profiles.
apps/vs-code-designer/.github/copilot-skills/vscode-e2e-testing.md	Adds a “skill” doc describing E2E conventions and architecture.
CLAUDE.md	Links repo-level guidance to VS Code extension E2E docs/tools.
.gitignore	Ignores additional E2E artifacts (.vscode-test, test-extensions).
.github/workflows/vscode-e2e.yml	Adds CI job to build extension and run VS Code E2E tests under Xvfb.
.github/copilot-instructions.md	Adds E2E run/build guidance + formatting/linting expectations.

Comments suppressed due to low confidence (1)

apps/vs-code-designer/src/app/utils/codeless/getAuthorizationToken.ts:22

When silentAuth is enabled, getSessionFromVSCode(..., { silent: true }) can return undefined (per the comment). In that case getAuthorizationToken() returns Bearer undefined, which is almost certainly not a valid token and will cause confusing downstream failures. Handle the no-session case explicitly (e.g., throw a localized error, or fall back to createIfNone: true depending on desired behavior).

export async function getAuthData(tenantId?: string): Promise<AuthenticationSession> {
  // When silentAuth is enabled (e.g. in automated test environments),
  // use { silent: true } to avoid showing the "wants to sign in" dialog.
  // This returns undefined if no cached session exists, instead of prompting.
  const silentAuth = vscode.workspace.getConfiguration('azureLogicAppsStandard').get<boolean>('silentAuth', false);
  if (silentAuth) {
    return await getSessionFromVSCode(undefined, tenantId, { silent: true });
  }
  return await getSessionFromVSCode(undefined, tenantId, { createIfNone: true });
}

export async function getAuthorizationToken(tenantId?: string): Promise<string> {
  const authData = await getAuthData(tenantId);
  return `Bearer ${authData?.accessToken}`;
}

Copilot · 2026-03-05T02:38:30Z

apps/vs-code-designer/src/test/ui/workspaceConversionYes.test.ts

+import * as assert from 'assert';
+import { type WebDriver, VSBrowser, ModalDialog, By } from 'vscode-extension-tester';
+import { WORKSPACE_MANIFEST_PATH, loadWorkspaceManifest } from './workspaceManifest';


Key is used later (e.g., Key.ESCAPE), but it is not imported from vscode-extension-tester in this file. This will fail compilation/runtime for the test. Add Key to the import list.

Copilot · 2026-03-05T02:38:30Z

apps/vs-code-designer/src/test/ui/run-clean.ps1

+Write-Host "`n=== Compiling test ==="
+Set-Location d:\dev\LogicAppsUX\apps\vs-code-designer
+npx tsc src/test/ui/createWorkspace.test.ts --outDir out/test --esModuleInterop --module commonjs --target es2020 --moduleResolution node --skipLibCheck
+Write-Host "Compile exit code: $LASTEXITCODE"


This helper script hard-codes local absolute paths (e.g., d:\dev\LogicAppsUX\...). That makes it unusable for other devs and in CI. Prefer resolving paths relative to the repo/app directory (or accept a parameter / use $PSScriptRoot) so the script works on any machine.

Copilot · 2026-03-05T02:38:30Z

apps/vs-code-designer/.vscode/launch.json

+        "--extensionTestsPath=${workspaceFolder}/out/test/e2e",
+        "${workspaceFolder}/e2e/test-workspace"
+      ],
+      "outFiles": ["${workspaceFolder}/out/**/*.js"],
+      "preLaunchTask": "npm: test:e2e-cli:compile"


This launch config depends on the npm: test:e2e-cli:compile task, but that task/script is not defined in apps/vs-code-designer/package.json, so debugging will fail. Also, --extensionTestsPath typically points to a specific compiled test runner module, not a directory; verify the intended runner path and update accordingly.

Suggested change

"--extensionTestsPath=${workspaceFolder}/out/test/e2e",

"${workspaceFolder}/e2e/test-workspace"

],

"outFiles": ["${workspaceFolder}/out/**/*.js"],

"preLaunchTask": "npm: test:e2e-cli:compile"

"--extensionTestsPath=${workspaceFolder}/out/test/e2e/index",

"${workspaceFolder}/e2e/test-workspace"

],

"outFiles": ["${workspaceFolder}/out/**/*.js"]

Copilot · 2026-03-05T02:38:31Z

apps/vs-code-react/src/app/createWorkspace/createWorkspace.tsx

+          const functionNamespaceValid = functionNamespace.trim() !== '' && namespaceValidation.test(functionNamespace.trim());
          const functionNameValid = functionName.trim() !== '' && nameValidation.test(functionName.trim());



functionNameValid still uses nameValidation, which allows hyphens. This makes the step navigation validation inconsistent with validateFunctionName() (which uses functionNameValidation) and can allow invalid C# identifiers to pass the wizard gating. Use the same functionNameValidation regex here (and import it) so navigation blocking matches the field-level validation.

Copilot · 2026-03-05T02:38:31Z

apps/vs-code-react/src/app/createWorkspace/createWorkspace.tsx

            functionFolderName.trim().toLowerCase() !== logicAppName.trim().toLowerCase();
-          const functionNamespaceValid = functionNamespace.trim() !== '' && nameValidation.test(functionNamespace.trim());
+          const functionNamespaceValid = functionNamespace.trim() !== '' && namespaceValidation.test(functionNamespace.trim());
          const functionNameValid = functionName.trim() !== '' && nameValidation.test(functionName.trim());

          const functionFieldsValid = functionNamespaceValid && functionNameValid && functionFolderNameValid;


Same issue as above: functionNameValid uses nameValidation, so hyphens can be treated as valid during step completion checks even though the function-name field validation rejects them. Align this to functionNameValidation to keep step completion logic consistent with the actual function name rules.

Copilot · 2026-03-05T02:38:31Z

apps/vs-code-designer/.vscode/tasks.json

+      "script": "test:e2e-cli:compile",
+      "group": "build",
+      "label": "npm: test:e2e-cli:compile",
+      "detail": "Compile e2e tests"
+    },
+    {
+      "type": "npm",
+      "script": "watch",
+      "isBackground": true,
+      "group": {
+        "kind": "build",
+        "isDefault": true
+      },
+      "label": "npm: watch"


This task references npm scripts (test:e2e-cli:compile and watch) that don’t exist in apps/vs-code-designer/package.json scripts. As-is, running the VS Code task will fail. Add the corresponding scripts or update the task names to match existing scripts (e.g., build:ui, etc.).

Suggested change

"script": "test:e2e-cli:compile",

"group": "build",

"label": "npm: test:e2e-cli:compile",

"detail": "Compile e2e tests"

},

{

"type": "npm",

"script": "watch",

"isBackground": true,

"group": {

"kind": "build",

"isDefault": true

},

"label": "npm: watch"

"script": "test:e2e",

"group": "build",

"label": "npm: test:e2e",

"detail": "Run e2e tests"

},

{

"type": "npm",

"script": "build:extension",

"isBackground": true,

"group": {

"kind": "build",

"isDefault": true

},

"label": "npm: build:extension"

Copilot · 2026-03-05T02:38:32Z

apps/vs-code-designer/.vscode-test.mjs

+import { defineConfig } from '@vscode/test-cli';
+import * as path from 'path';
+import { fileURLToPath } from 'url';
+


.vscode-test.mjs depends on @vscode/test-cli, but this repo/package doesn’t declare it as a dependency (and it isn’t present in pnpm-lock.yaml). This will fail at runtime in CI/dev. Add @vscode/test-cli to the appropriate package.json (and lockfile) or switch this config to the existing @vscode/test-electron tooling already in use.

Suggested change

import { defineConfig } from '@vscode/test-cli';

import * as path from 'path';

import { fileURLToPath } from 'url';

import * as path from 'path';

import { fileURLToPath } from 'url';

const defineConfig = (config) => config;

Copilot · 2026-03-05T02:38:32Z

apps/vs-code-react/src/app/createWorkspace/steps/dotNetFrameworkStep.tsx

+  useEffect(() => {
+    if (functionNamespace) {
+      setFunctionNamespaceError(validateFunctionNamespace(functionNamespace, intlText));
+    }
+  }, [functionNamespace, intlText]);
+
+  useEffect(() => {
+    if (functionName) {
+      setFunctionNameError(validateFunctionName(functionName, intlText));
+    }
+  }, [functionName, intlText]);


These effects only re-validate when the value is truthy. If the field becomes empty via external state updates (or intlText changes due to locale change), the error message won’t be recomputed/cleared/set to the empty-field message. Consider running validation unconditionally on change (or explicitly handling the empty string) so the error state always matches the current input value.

- ExTester UI tests: workspace creation, designer actions, inline JavaScript, stateless variables, designer view extensions, keyboard navigation, data mapper - Workspace conversion tests: dialog dismiss, create workspace from legacy project, multiple designers simultaneously, right-click add workflow, subfolder prompt - Shared helper modules: helpers.ts, designerHelpers.ts, runHelpers.ts - GitHub Actions workflow for CI (vscode-e2e.yml) - CLI integration tests: workspace conversion, project outside workspace - Documentation: SKILL.md, CLAUDE.md updates, copilot-instructions - Product fix: silentAuth support in getAuthorizationToken.ts - createWorkspace validation improvements ADO #31054994: Full coverage of all 15 steps (workspace requirement)

github-actions · 2026-03-05T02:52:41Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 500

ExTester's page-objects use '.editor-instance' CSS selector which doesn't exist in VS Code 1.102.0 (the version CI downloaded with 'max'). Pin to 1.108.0 which is known to work with ExTester 8.21.0. This fixes all 13 Phase 4.1 workspace creation test failures on CI.

…onflict)

github-actions · 2026-03-05T03:16:45Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 504

…tSettings, conversion assertions - Add writeTestSettings() function for per-phase VS Code settings control - Disable autoStartDesignTime for conversion tests (4.8a-4.8e) - Add JS-based QuickPick dismissal for Azure connector wizard prompts - Fix waitForDesignerWebviewTab to skip command palette detection - Fix designerActions local waitForDependencyValidation timeout - Relax conversion file assertion: only check for removed files - Replace this.skip() with assert.fail() across all test files - Add retry logic for legacy project cleanup in 4.8b

github-actions · 2026-03-05T21:46:56Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 504

…fallback - Phase 4.8b: enable validateDependencies so extension activates for dialog - Phase 4.8c: restore full design-time settings (needs designer) - Phase 4.8d/4.8e: restore conversion settings after 4.8c - Manifest lookups: fall back to Stateless when Stateful unavailable - clearAndType: scrollIntoView + JS click fallback for overlay/iframe issues - wsDir/appDir lookups: fall back to any standard entry (not just Stateful)

github-actions · 2026-03-05T23:57:24Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 500

- clearAndType: use VSBrowser.instance.driver (not describe-scoped var) - executeOpenDesignerCommand: 10 retries × 5s delay (was 5 × 3s) - ALL conversion tests: validateDependencies=true (extension must activate) - conversiononly mode: same fix for validateDependencies

github-actions · 2026-03-06T00:44:52Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 504

Root cause: opening a .code-workspace file restarts the extension host. On CI, the extension re-downloads/validates dependencies (NodeJs, FuncCoreTools, DotNetSDK) which takes 30-120s. Without waiting for this second activation cycle, 'Open Designer' is never found because the extension hasn't finished registering its commands. Fix: openWorkspaceFileInSession now calls waitForDependencyValidation after the workspace switch, polling up to 300s for the extension to re-activate. Applied to both designerHelpers.ts (shared) and designerActions.test.ts (local).

Root cause: each phase launches VS Code with the workspace as a startup resource, but then openDesignerForEntry re-opens the same workspace via openWorkspaceFileInSession. This triggers an unnecessary extension host restart + dependency re-validation (60s+ on CI each time). Fix: check VS Code's window title before switching. If the workspace is already open, skip the switch and go directly to opening workflow.json. Also increase CI job timeout from 60→90 min as safety margin.

github-actions · 2026-03-06T02:08:27Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 500

Root cause: waitForDependencyValidation(300s) in openWorkspaceFileInSession created dangling Promises that outlived Mocha test timeouts. These Promises continued polling the dead WebDriver session after the browser shut down, causing 'invalid session id' errors to leak into the next phase. Fix: - Remove waitForDependencyValidation from openWorkspaceFileInSession entirely - Use simple sleep(5s) for workspace settle instead of 300s blocking poll - Increase executeOpenDesignerCommand to 20 retries × 10s = 200s - Extension activation wait now happens at command-palette level, not at workspace-switch level (no dangling Promises)

github-actions · 2026-03-06T03:32:09Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 504

ROOT CAUSE: prepareFreshSession used Windows-only PowerShell to kill lingering VS Code processes. On Linux CI, this silently failed, leaving the Phase 4.1 VS Code window running. When Phase 4.2+ launched a NEW VS Code, ExTester's 'code -r wsFilePath' sent the workspace-open command to the OLD window. The new window stayed bare (no folder), which is why all screenshots showed empty VS Code with no extension behavior. FIX: - Use pkill on Linux/Mac to kill test-resources VS Code processes - Also kill lingering chromedriver processes between phases - Keep PowerShell for Windows compatibility - Add step-by-step screenshots to standard workspace creation test - Add failure screenshots in afterEach hook - Revert excessive retry counts back to 5x3s (root cause was stale processes, not timing)

github-actions · 2026-03-06T03:59:42Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 500

Root causes fixed: 1. switchToWebviewFrame: 10 retries x 3s (was 5x2s) for cold CI webview render 2. openDesignerForEntry: uses Explorer right-click instead of command palette - No timing dependency on command registration after workspace switch - Context menu available as soon as extension loads - Mirrors real user behavior 3. clearAndType: add VSBrowser import for JS-click fallback on rules engine 4. waitForDependencyValidation: added to before hooks of 4 test files (inlineJavascript, statelessVariables, designerViewExtended, keyboardNavigation) 5. getPhase2Resources: aligned fallback with test files (prefer standard+Stateful → standard → manifest[0])

Root cause analysis: code -r (openResources) works intermittently on CI. Phases 4.2, 4.4, 4.6 succeed while 4.3, 4.5, 4.8c fail — alternating pattern suggests race condition with stale IPC socket files from killed processes. Fix: - Clean up *.sock files in settings dir after killing processes - Increase post-kill wait from 3s to 5s for full process exit - Revert appDir startup resource change (would trigger conversion prompt) - Keep wsFilePath for startup resources

github-actions · 2026-03-06T22:01:51Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 500

Adds diagnostic logging to understand why openResources (code -r) doesn't work on Linux CI: - VS Code title before/after openResources - Explorer sidebar row count and contents - Settings dir file listing (looking for IPC sockets) - Startup resource existence checks (file vs directory) This is a diagnostic-only change to capture evidence for the next fix.

ROOT CAUSE (confirmed by diagnostics): ExTester's openResources uses 'code -r' CLI IPC which silently fails on Linux CI. When VS Code is launched by ChromeDriver/Selenium, the IPC socket for the CLI isn't set up. The code -r command runs, returns success, but does nothing. This was proven by: - VS Code title staying 'Visual Studio Code' before AND after openResources - Explorer state: EMPTY after openResources - No .sock files in settings dir FIX: Open workspaces via the command palette instead of code -r: - 'File: Open Workspace from File...' for .code-workspace files - 'File: Open Folder...' for directories - ExTester sets files.simpleDialog.enable=true so these show a simple text input instead of a native file picker - Type the full path and press Enter - Verify via title change and Explorer state

github-actions · 2026-03-06T23:48:30Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 504

…lette All VSBrowser.instance.openResources() calls use code -r IPC which silently fails on Linux CI. Replace with: 1. openDesignerViaExplorer: Use Ctrl+P Quick Open to open workflow.json, then re-focus Explorer to reveal it in the tree. 2. openFileInEditor: Same Ctrl+P Quick Open approach. 3. openOverviewPage: Same approach. 4. Conversion tests (4.8a-4.8e): Add openFolderInSession() helper that uses command palette 'File: Open Folder...' to open directories. ExTester startup resources also use code -r which doesn't work. 5. multipleDesigners: Same Ctrl+P approach for openDesignerViaExplorerRightClick.

github-actions · 2026-03-07T00:52:00Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 500

Two CI failures fixed: 1. openFolderInSession: 'element not interactable' because C# Dev Kit sign-in dialog blocks command palette input. Fix: aggressively dismiss all dialogs before each attempt, use Ctrl+Shift+P with '>' prefix for more reliable command execution. 2. switchToDesignerWebview: ExTester's WebView.switchToFrame() has ~5s default timeout. On CI, the webview's internal active-frame element takes longer to appear while the designer loads. Fix: retry switchToFrame up to 6 times (30s total) with 5s sleep between.

…me switching ExTester's WebView.switchToFrame() searches for *[id='active-frame'] inside the webview iframe, but VS Code 1.108.0 on Linux CI doesn't render that element. All 6 retry attempts time out after ~5s each. Fix: Replace with manual Selenium frame navigation: 1. Find visible iframe.webview.ready element 2. driver.switchTo().frame(outerIframe) 3. If #active-frame exists inside, switch into it too 4. If not, try first inner iframe or use outer directly 5. Verify webview content is accessible (#root or body) 6. Retry loop with 60s total timeout Applied to both designerHelpers.ts and designerActions.test.ts.

github-actions · 2026-03-09T19:49:56Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 500

…ssal Root cause: The 'Validating Runtime Dependency' notification downloads NodeJS, Functions Runtime, and .NET SDK sequentially within a single progress notification. Two problems: 1. dismissAllDialogs() was dismissing this notification during openWorkspaceFileInSession, killing in-progress downloads. Fix: Skip any dialog containing 'Validating Runtime Dependency' or 'Successfully installed'. 2. waitForDependencyValidation() only waited for the notification to disappear — which happens when NodeJS finishes (~2s) but before Functions Runtime finishes downloading (~200MB, ~60s). The test then tries to open the designer, but func binary doesn't exist yet so the design-time API can't start. Fix: After notification disappears, poll for func binary at ~/.azurelogicapps/dependencies/FuncCoreTools/func to actually exist on disk before returning. Log dependency folder contents on failure.

Root cause (confirmed from CI screenshots): The extension downloads func, dotnet, and node binaries to ~/.azurelogicapps/dependencies/ but doesn't set execute permission on Linux. The Output panel shows: /bin/sh: 1: .../FuncCoreTools/func: Permission denied The binary EXISTS on disk (our fs.existsSync check passes) but can't be executed. The extension then tries to re-download (~200MB) every session, and the designer can't open because the design-time API (func host start) fails. Fix: Run chmod -R +x on FuncCoreTools/, NodeJs/, and DotNetSDK/ directories in three places: 1. run-e2e.js prepareFreshSession() - between test phases 2. designerHelpers.ts waitForDependencyValidation() - before tests 3. designerActions.test.ts waitForDependencyValidation() - local copy

github-actions · 2026-03-09T23:42:51Z

❌ PR Validation Error

An error occurred while validating your PR. Please try again later or contact the maintainers.

Error: API request failed with status 500

Root cause: The extension validates dependencies (showing 'Validating Runtime Dependency' notifications for FuncCoreTools, NodeJS, DotNet, etc.) when a workspace is opened. The design-time API (func host start) won't start until ALL validations complete. Previously we only waited 4 seconds (PROJECT_RECOGNITION_WAIT) before opening the designer, so the React app had no API to connect to and never rendered. Fix: Add waitForExtensionValidationComplete() that: 1. Polls for 'Validating Runtime Dependency' notifications 2. Waits until no such notifications for 10s (stable period) 3. Then adds 3s buffer for design-time API to initialize 4. 120s overall timeout Applied in 3 places: - designerHelpers.ts openDesignerForEntry() (shared) - designerActions.test.ts openDesignerForEntry() (local copy) - multipleDesigners.test.ts after openWorkspaceFileInSession() Also added webview HTML diagnostics dump when designer content fails to load (Phase 2 timeout) for future debugging.

Root cause (confirmed from CI diagnostics): switchToDesignerWebview Phase 1 checks for '#root, body' after failing to find #active-frame. Since 'body' always exists in the outer webview bootstrap frame, it falsely sets switched=true. We end up trapped in the OUTER frame (41KB VS Code webview loader HTML with CSP meta tags and bootstrap script) instead of the INNER frame where the React designer app lives. Evidence from diagnostics dump: HTML length: 41148 Body children: 1 (just the bootstrap script tag) No #root, no .msla-designer-canvas, no React content Fix: Change '#root, body' selector to just '#root'. When no #active-frame or inner iframes are found, this correctly does NOT set switched=true, causing the retry loop to keep polling until the inner content frame appears. Also added DOM diagnostics logging every 15s while waiting. Applied in both designerHelpers.ts and designerActions.test.ts.

The previous commit changed '#root, body' to just '#root' to avoid a false positive. But CI diagnostics confirmed that #active-frame does NOT exist on VS Code 1.108.0 Linux — the webview bootstrap loads extension content directly in the outer frame (no nested iframe is ever created). The outer frame body has only: childNodes: [#text, SCRIPT, #text], iframes: 0 With #root-only, Phase 1 times out at 60s on every designer test, causing regressions in phases that previously passed: 4.1: 54→20 passing (before each timeout) 4.5: 2→0 passing (webview switch timeout) Revert to '#root, body' which correctly enters the outer frame. Phase 2/3 then handle waiting for the React app to mount.

Copilot AI review requested due to automatic review settings March 5, 2026 02:30

Copilot started reviewing on behalf of lambrianmsft March 5, 2026 02:32 View session

Copilot AI reviewed Mar 5, 2026

View reviewed changes

lambrianmsft force-pushed the lambrian/vscode_e2e_testing branch from 10cb7db to 672161e Compare March 5, 2026 02:47

lambrianmsft force-pushed the lambrian/vscode_e2e_testing branch from 672161e to 515f663 Compare March 5, 2026 02:48

lambrianmsft added 2 commits March 4, 2026 18:59

Merge main into lambrian/vscode_e2e_testing (resolve pnpm-lock.yaml c…

4a722e3

…onflict)

github-actions bot mentioned this pull request Mar 5, 2026

[repo-status] Daily Repo Status — March 5, 2026 #8885

Open

lambrianmsft added 2 commits March 5, 2026 17:02

github-actions bot mentioned this pull request Mar 6, 2026

[repo-status] Daily Repo Status — March 6, 2026 #8891

Open

This was referenced Mar 7, 2026

[repo-status] Daily Repo Status — March 7, 2026 #8898

Open

[repo-status] Daily Repo Status — March 8, 2026 #8899

Open

[repo-status] Daily Repo Status — March 9, 2026 #8902

Open

lambrianmsft added 2 commits March 9, 2026 11:42

lambrianmsft added 2 commits March 9, 2026 14:06

lambrianmsft added 3 commits March 9, 2026 20:19

github-actions bot mentioned this pull request Mar 10, 2026

[repo-status] Daily Repo Status — March 10, 2026 #8904

Open

		const functionNamespaceValid = functionNamespace.trim() !== '' && namespaceValidation.test(functionNamespace.trim());
		const functionNameValid = functionName.trim() !== '' && nameValidation.test(functionName.trim());

Conversation

lambrianmsft commented Mar 5, 2026

Commit Type

Risk Level

What & Why

Impact of Change

Test Plan

Contributors

Screenshots/Videos

Uh oh!

github-actions bot commented Mar 5, 2026

❌ PR Validation Error

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 5, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 5, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 5, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 5, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 6, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 6, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 6, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 6, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 6, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 6, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 7, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 9, 2026

❌ PR Validation Error

Uh oh!

github-actions bot commented Mar 9, 2026

❌ PR Validation Error