Skip to content

feat(DesignerV2): Overhauled workflow assistant to allow for in-editor workflow modifications#8896

Merged
rllyy97 merged 16 commits intomainfrom
riley/copilot-wf-modification
Mar 10, 2026
Merged

feat(DesignerV2): Overhauled workflow assistant to allow for in-editor workflow modifications#8896
rllyy97 merged 16 commits intomainfrom
riley/copilot-wf-modification

Conversation

@rllyy97
Copy link
Contributor

@rllyy97 rllyy97 commented Mar 6, 2026

Commit Type

  • feature - New functionality
  • fix - Bug fix
  • refactor - Code restructuring without behavior change
  • perf - Performance improvement
  • docs - Documentation update
  • test - Test-related changes
  • chore - Maintenance/tooling

Risk Level

  • Low - Minor changes, limited scope
  • Medium - Moderate changes, some user impact
  • High - Major changes, significant user/system impact

What & Why

Adds in-designer Copilot workflow editing. Users can describe workflow changes in natural language through the existing Copilot chatbot, and the assistant modifies the workflow definition directly.

Currently, the Copilot chatbot can only answer questions about Logic Apps. This PR extends it to actually modify the workflow: adding, updating, and removing actions, triggers, connections, parameters, and notes, all from a chat prompt. The assistant can also discover real connectors via tool-calling and produce ready-to-use action definitions from Swagger.

Key capabilities:

  • Natural language → workflow modification (add/edit/remove actions, notes, parameters, connections)
  • Connector discovery via LLM tool-calling (discover_connectors, get_connector_operations) that builds complete action templates from Swagger
  • SKU-aware system prompts (Standard vs Consumption differences in connection patterns, parameter formats)
  • Auto-apply or proposal mode — changes applied immediately or shown for user approval
  • Undo support — every applied change can be reverted
  • Structured change list with color-coded icons, connector branding, and clickable node links
  • Conversation history preserved across open/close of the chat drawer

Impact of Change

  • Users: Copilot chatbot now supports modifying workflows via natural language in addition to answering questions. Users see a structured change list with icons per change type (added/modified/removed), can click node names to focus them in the designer, and can undo applied changes. The greeting message updates to reflect the new capability.
  • Developers: New ICopilotWorkflowEditorService interface and BaseCopilotWorkflowEditorService base class following the existing service injection pattern. Supports both Chat Completions and Responses API, OpenAI and Azure OpenAI auth. CoPilotChatbot component has new props: enableWorkflowEditing, onWorkflowProposed, autoApply, getNodeVisuals, onNodeClick. Workflow model gains notes?: Record<string, Note>.
  • System: New files: copilotWorkflowEditor.ts (service interface), copilotWorkflowEditorPrompt.ts (system prompts), copilotWorkflowEditorTools.ts (tool definitions + Swagger-to-action template builder). Several chatbot UI components migrated from Fluent UI v8 to v9 patterns.

Test Plan

  • Unit tests added/updated
  • E2E tests added/updated
  • Manual testing completed
  • Tested in: Standalone designer (Standard + Consumption), VS Code extension

Manual testing performed:

  • Verified natural language prompts correctly add/modify/remove actions, notes, and parameters
  • Tested connector discovery tool-calling (search → Swagger resolution → action template insertion)
  • Tested undo/redo of applied changes
  • Verified auto-apply and proposal modes
  • Confirmed change list displays correct icons, colors, and clickable node links
  • Tested conversation history preservation across chat drawer open/close
  • Verified existing chatbot Q&A functionality is unaffected when workflow editing is disabled

Contributors

@rllyy97

Screenshots/Videos

wf_copilot

@github-actions
Copy link

github-actions bot commented Mar 6, 2026

🤖 AI PR Validation Report

PR Review Results

Thank you for your submission! Here's detailed feedback on your PR title and body compliance:

PR Title

  • Current: feat(DesignerV2): Overhauled workflow assistant to allow for in-editor workflow modifications
  • Issue: None — title is descriptive, follows conventional prefixing (feat) and indicates the area (DesignerV2) and intent (enable in-editor workflow modifications).
  • Recommendation: The title is good. If you want it even more concise you could use: feat(designer-v2): enable in-designer Copilot workflow edits but this is optional.

⚠️ Commit Type

  • Properly selected (feature) ✅
  • Note: Only one commit type is selected which is correct. If this PR also contains substantial test scaffolding and tooling changes you might optionally call those out in the body, but this is not required.

⚠️ Risk Level

  • Current: Medium (PR body) and label risk:medium on the PR ✅ (they match)
  • Assessment: I advise this PR should be labeled risk:high based on the diff.
    • Why: this change is very large (≈5.5k additions, 58 files changed) and touches many core areas: save/deploy artifact logic, draft publishing, Consumption vs Standard save paths, APIs (draft PUT endpoints), new persisted artifact files (notes, draft-notes), editor UI, chat UI, new services that call external LLM APIs, tool-calling support, and many tests. It introduces behavior that modifies persisted workflows (auto-apply proposals, draft saves) and new network interaction with LLM endpoints and connector discovery tools — all increase blast radius.
  • Recommendation: Update the Risk Level selection to High in the PR body and add the risk:high label to the PR so the label and body are aligned with the risk you should expect during review/merge/rollout.

What & Why

  • Current: Adds in-designer copilot workflow editing, connector discovery, undo, auto-apply/proposal modes, new services and UI props — well described in the PR body.
  • Issue: None missing, but please briefly call out the most sensitive runtime impacts explicitly (below) so reviewers know where to focus security/ops testing.
  • Recommendation: Add a short bullet in "What & Why" calling out:
    • The service-level behavior that will write draft artifacts and production artifacts (deployArtifacts signature change), and
    • The new required environment variables (VITE_COPILOT_EDITOR_API_KEY, VITE_COPILOT_EDITOR_ENDPOINT, etc.) and any secrets management guidance.

Impact of Change

  • Good detail present (Users, Developers, System).
  • Recommendation (clarifying additions): Expand Impact to explicitly call out:
    • Backwards compatibility: the new notes file/draft-notes changes and draft save flow may change artifact layout — note if older clients still work and how migration is handled.
    • Operational/infra: the new Copilot editor requires an API key and endpoint env vars — include required RBAC/network considerations (eg. allowlist ranges, token rotation). Mention whether logs or user telemetry are affected.
    • Security: mention that LLM API keys must not be checked into repo and should be provided via secure env var / key vault.

Test Plan

  • Current: Unit tests added/updated checked; manual testing completed; E2E left unchecked.
  • Assessment: The code diff contains many new unit tests and test suites (libs/chatbot tests, Copilot editor tests, designer-ui tests, and base service unit tests). Unit tests appear present and extensive.
  • Issue / Recommendation: Because this PR adds runtime workflow modifications, network requests to LLMs and connector-discovery tooling, and changes deploy/save logic across Standard and Consumption modes, you should:
    • Add or call out a plan for integration/E2E tests that validate end-to-end scenarios: proposing a workflow edit, auto-apply vs proposal mode, undo, saving drafts (both Standard and Consumption), and connector discovery tool-calling flow.
    • If E2E can't be provided now, add a strong rationale in the Test Plan explaining why not (e.g., waiting on infra or test harness), and provide a mitigation plan (manual test checklist + automated tests to follow).
    • Ensure unit tests cover parsing edge cases (JSON-with-comments, markdown code block extraction) — the diff does include parsing tests; call that out in the Test Plan so reviewers can easily see coverage areas.

⚠️ Contributors

  • Current: @rllyy97 listed
  • Assessment: OK but optional: consider listing additional contributors (design, PMs, security reviewer, QA) who aided requirements or manual testing. If there were no others, you can leave it as-is.
  • Recommendation: If designers / PMs / infra / security people reviewed or assisted, add them here so they get credited.

Screenshots/Videos

  • Current: One screenshot link present in the body ✅
  • Assessment: For a feature that also changes UI components and visual elements (change list UI, color-coded icons, clickable node links), one screenshot is a good start.
  • Recommendation: Consider adding a short GIF or additional screenshots that show (a) proposal card, (b) applied change + undo, (c) connector-discovery result and action insertion — this speeds review for UX/Designers.

Summary Table

Section Status Recommendation
Title Title is good; optional shorter variant suggested
Commit Type feature selected correctly
Risk Level ⚠️ Change to High and add risk:high label (see reasoning)
What & Why Add quick callout about env vars and sensitive save behavior
Impact of Change Expand to explicitly call out compatibility, ops, and security considerations
Test Plan Unit tests present. Add E2E/integration tests or rationale and a mitigation checklist
Contributors ⚠️ Add other contributors if applicable (PM/Design/QA/Security)
Screenshots/Videos Consider adding a short GIF showing workflow edit/apply/undo

Final recommendations (actionable):

  1. Update Risk Level to High in the PR template (select High) and add the repository label risk:high. This PR touches many core runtime and persistence paths, so label it high-risk.
  2. In the PR body Test Plan, either add E2E/integration tests (recommended) or include a short explanation why E2E cannot be added now and an explicit manual test checklist covering these critical flows:
    • Applying an auto-apply workflow change and verifying it persisted to production artifacts
    • Proposal-only flow (ensure proposal shows, approve, and reverts)
    • Undo flow (applied change undone)
    • Draft save for Consumption and Standard (ensure drafts stored in the new draft endpoints)
    • Connector discovery: discover_connectors → get_connector_operations → action template insertion
    • Security: verify API keys are read from env and not logged
  3. In "What & Why" and "Impact of Change", explicitly call out:
    • New env vars: VITE_COPILOT_EDITOR_API_KEY, VITE_COPILOT_EDITOR_ENDPOINT, VITE_COPILOT_EDITOR_MODEL/DEPLOYMENT and the requirement to store them securely
    • The deployArtifacts change: signature now accepts notesData and writes draft/notes files — mention backward compatibility and migration notes
    • Any expected operational steps (e.g., configure OpenAI/Azure OpenAI, network allow-listing)
  4. Add a brief Security/Privacy notes section or at least one paragraph in the PR body describing how LLM keys are protected and any telemetry/PII considerations for sending workflow content to LLMs.
  5. Ensure tests include integration coverage for the LLM response parsing edge cases (the code & tests include many such cases — highlight those tests in the Test Plan so reviewers see them quickly).
  6. Add or link to runbook/rollback instructions because this feature auto-applies edits (very high-impact). At minimum, indicate how to rollback (e.g., revert DB/store artifacts or restore from draft/prod files). If the app has an internal undo that is user-visible, mention how it interacts with persisted state.

If you make the recommended updates (especially changing the risk label to high and clarifying E2E / security items), this PR's title/body should be in good shape for a functional & security review. Thank you — this is a substantial, well-documented feature; the main ask is to acknowledge the higher risk and provide E2E/ops/security follow-up details so reviewers and release managers can plan accordingly.

Reviewer note: I recommended a higher risk level (High) because the code diff modifies persistence, network calls to LLM/provider, and introduces auto-apply behavior that modifies persisted workflows. Please update the PR labels and body to reflect that.


Last updated: Mon, 09 Mar 2026 20:31:04 GMT

@rllyy97 rllyy97 added the risk:medium Medium risk change with potential impact label Mar 6, 2026
@rllyy97 rllyy97 marked this pull request as ready for review March 6, 2026 21:45
Copilot AI review requested due to automatic review settings March 6, 2026 21:45
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an in-designer Copilot “workflow editing” capability, enabling natural-language prompts to propose/apply workflow definition changes (actions, parameters, connections, and notes) and presenting a structured change list with undo support in the chat UI.

Changes:

  • Introduces ICopilotWorkflowEditorService + base implementation, plus SKU-specific system prompts and connector discovery tool-calling helpers.
  • Updates chatbot UI/components (largely Fluent UI v9 patterns) to render structured workflow change lists, node links, and undo actions; preserves chat history while the drawer is hidden.
  • Adds workflow notes to the shared workflow model and wires notes/parameters handling through the Standalone designer draft/prod flows.

Reviewed changes

Copilot reviewed 52 out of 52 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
libs/logic-apps-shared/src/utils/src/lib/models/workflow.ts Adds shared Note type and notes field on Workflow.
libs/logic-apps-shared/src/designer-client-services/lib/copilotWorkflowEditor.ts Adds workflow editor service interface, change types, and service initializer singleton.
libs/logic-apps-shared/src/designer-client-services/lib/base/index.ts Re-exports new workflow editor base service, prompts, and tools.
libs/logic-apps-shared/src/designer-client-services/lib/base/copilotWorkflowEditorTools.ts Implements connector discovery tools + swagger-to-action template generation.
libs/logic-apps-shared/src/designer-client-services/lib/base/copilotWorkflowEditorPrompt.ts Adds SKU-aware system prompts and response format contract.
libs/logic-apps-shared/src/designer-client-services/lib/base/copilotWorkflowEditor.ts Implements OpenAI Chat Completions / Responses integration, tool-calling loop, and response parsing.
libs/logic-apps-shared/src/designer-client-services/lib/base/test/copilotWorkflowEditorTools.spec.ts Unit tests for tool definitions/execution and template building.
libs/logic-apps-shared/src/designer-client-services/lib/base/test/copilotWorkflowEditor.spec.ts Unit tests for editor service parsing, history, and API interaction behavior.
libs/logic-apps-shared/src/designer-client-services/lib/test/copilotWorkflowEditor.spec.ts Unit tests for service singleton init/guards and type shapes.
libs/logic-apps-shared/src/designer-client-services/index.ts Exposes copilotWorkflowEditor from the package entrypoint.
libs/designer/src/lib/ui/panel/agentChat/agentChat.tsx Updates agent chat input submission to accept value argument (no local text state).
libs/designer-v2/src/lib/ui/panel/runHistoryPanel/agentChatContent.tsx Same agent chat input submission refactor for v2 run history panel.
libs/designer-v2/src/lib/ui/panel/agentChat/agentChat.tsx Same agent chat input submission refactor for v2 agent chat panel.
libs/designer-v2/src/lib/core/index.ts Exports notes initialization action.
libs/designer-v2/src/lib/common/models/workflow.ts Switches Note type to shared export; re-exports Note.
libs/designer-ui/src/lib/styles.less Layout tweak (min-width: 0) for flex overflow behavior.
libs/designer-ui/src/lib/editor/codemirror/types.ts Adds noBorder prop to Monaco-compatible CodeMirror props.
libs/designer-ui/src/lib/editor/codemirror/CodeMirrorEditor.tsx Implements noBorder styling behavior.
libs/designer-ui/src/lib/copilotGetStarted/components/createNaturalLanguageToFlowInput.tsx Updates to new ChatInput ref/props and styling hookup.
libs/designer-ui/src/lib/chatbot/index.tsx Exports additional chatbot component types.
libs/designer-ui/src/lib/chatbot/feedbackHelper.tsx Migrates action types to ChatBubbleAction and Fluent v9 icon usage.
libs/designer-ui/src/lib/chatbot/components/thumbsReactionButton.tsx Migrates thumbs buttons to Fluent v9 Button/Tooltip + icon set.
libs/designer-ui/src/lib/chatbot/components/progressCardWithStopButton.tsx Migrates stop button UI to Fluent v9 Button with icon.
libs/designer-ui/src/lib/chatbot/components/conversationMessage.tsx Memoizes conversation message component and sets displayName.
libs/designer-ui/src/lib/chatbot/components/conversationItem.ts Extends conversation item types with workflow editing state, change list, and node click handler.
libs/designer-ui/src/lib/chatbot/components/connectionsSetupMessage.tsx Migrates class merging and icon usage to Fluent v9 patterns.
libs/designer-ui/src/lib/chatbot/components/chatInputBox.tsx Rebuilds ChatInput using Fluent v9 Textarea/Button; adds imperative handle ref.
libs/designer-ui/src/lib/chatbot/components/chatBubble.tsx Migrates footer actions/copy button to Fluent v9 controls; introduces ChatBubbleAction type.
libs/designer-ui/src/lib/chatbot/components/assistantReplyWithFlow.tsx Replaces flow diff preview with structured change list rendering + clickable node names and undo action wiring.
libs/designer-ui/src/lib/chatbot/components/assistantGreeting.tsx Updates greeting copy + suggestions; adds workflow-editing-enabled variant.
libs/designer-ui/src/lib/chatbot/components/assistantError.tsx Migrates footer actions typing to ChatBubbleAction.
libs/designer-ui/src/lib/chatbot/components/test/assistantReplyWithFlow.spec.tsx Adds tests for change list rendering, clickability, and undo behavior.
libs/designer-ui/src/lib/chatbot/components/test/assistantGreeting.spec.tsx Updates/extends greeting tests for editing-enabled content.
libs/designer-ui/src/lib/chatbot/components/test/snapshots/assistantGreeting.spec.tsx.snap Snapshot updates for greeting copy changes.
libs/designer-ui/src/lib/chatbot/chatbot.less CSS tweaks for new footer/actions layout.
libs/chatbot/src/lib/ui/styles.ts Updates header/protected badge styles to new layout expectations.
libs/chatbot/src/lib/ui/panelheader.tsx Refactors header to remove internal close button; adds “Protected” badge link.
libs/chatbot/src/lib/ui/test/panelheader.spec.tsx Updates tests for new header rendering and removed close handling.
libs/chatbot/src/lib/ui/CopilotChatbot.tsx Adds workflow editing mode (service-based), change enrichment, undo/apply logic, and open/close behavior.
libs/chatbot/src/lib/ui/ChatbotUi.tsx Migrates from v8 Panel to v9 InlineDrawer; localizes input state; adds drawer close button.
apps/vs-code-react/src/app/designer/CodeViewEditor/index.tsx Adjusts editor container sizing and enables noBorder.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/laDesignerV2.tsx Wires workflow editing chatbot into Standard designer, plus draft/prod params/notes selection and node focusing.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/laDesignerConsumptionV2.tsx Wires workflow editing chatbot into Consumption designer and initializes consumption prompt.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/laDesignerConsumption.tsx Updates chatbot mounting behavior and flex layout (non-v2 consumption).
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/laDesigner.tsx Wires workflow editing chatbot into Standard (non-v2) designer and adds parameter state updates.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/Services/WorkflowAndArtifacts.tsx Adds draft save for consumption and persists notes artifacts during deployment.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/Models/Workflow.ts Adds notes artifacts constants and draft artifacts typing updates.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/DesignerCommandBarV2.tsx Adds Assistant toolbar button and avoids callback URL refresh on auto-save.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/DesignerCommandBar.tsx Repositions Assistant command bar item to editor items.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/CodeViewV2.tsx Adjusts code view sizing and enables noBorder.
apps/Standalone/src/designer/app/AzureLogicAppsDesigner/CodeView.tsx Adjusts code view sizing, enables noBorder, and sets displayName.
Localize/lang/strings.json Adds new localized strings for workflow editing greeting/prompts and status messages.

@github-actions
Copy link

github-actions bot commented Mar 6, 2026

📊 Coverage Check

The following changed files need attention:

⚠️ libs/chatbot/src/lib/ui/CopilotChatbot.tsx - 32% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/chatbot/components/assistantError.tsx - 11% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/chatbot/components/chatBubble.tsx - 61% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/chatbot/components/chatInputBox.tsx - 23% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/chatbot/components/connectionsSetupMessage.tsx - 18% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/chatbot/components/conversationMessage.tsx - 16% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/chatbot/components/progressCardWithStopButton.tsx - 50% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/chatbot/feedbackHelper.tsx - 3% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/copilotGetStarted/components/createNaturalLanguageToFlowInput.tsx - 1% covered (needs improvement)
⚠️ libs/designer-ui/src/lib/editor/codemirror/CodeMirrorEditor.tsx - 70% covered (needs improvement)
⚠️ libs/designer-v2/src/lib/ui/panel/agentChat/agentChat.tsx - 67% covered (needs improvement)
⚠️ libs/designer-v2/src/lib/ui/panel/recommendation/browse/mcpToolWizard.tsx - 57% covered (needs improvement)
⚠️ libs/designer-v2/src/lib/ui/panel/runHistoryPanel/agentChatContent.tsx - 1% covered (needs improvement)
⚠️ libs/designer/src/lib/ui/panel/agentChat/agentChat.tsx - 76% covered (needs improvement)

Please add tests for the uncovered files before merging.

@rllyy97 rllyy97 merged commit 55400fe into main Mar 10, 2026
13 checks passed
@rllyy97 rllyy97 deleted the riley/copilot-wf-modification branch March 10, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-validated risk:medium Medium risk change with potential impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants