Add app-level dependency graph filtering for CI/CD build optimization#7111
Open
Groenbech96 wants to merge 20 commits intomainfrom
Open
Add app-level dependency graph filtering for CI/CD build optimization#7111Groenbech96 wants to merge 20 commits intomainfrom
Groenbech96 wants to merge 20 commits intomainfrom
Conversation
Introduces BuildOptimization.psm1 that builds a dependency graph from all 329 app.json files and filters appFolders/testFolders per project to only include affected apps. This reduces build times significantly for PRs that touch a small subset of apps (e.g., E-Document Core: 9 apps instead of ~55). Includes a small test change in E-Document Core to verify filtering in CI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Could not find a linked ADO work item. Please link one by using the pattern 'AB#' followed by the relevant work item number. You may use the 'Fixes' keyword to automatically resolve the work item when the pull request is merged. E.g. 'Fixes AB#1234' |
git writes progress info to stderr (e.g., "From https://..."), which PowerShell treats as a terminating error under $errorActionPreference=Stop. Switch to Continue around git calls and pipe stderr to Out-Null for fetch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shallow checkouts in CI don't have the base commit or parent history, so git diff fails. Use gh pr diff --name-only which queries the GitHub API and works regardless of checkout depth. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Will revert after verifying the filtering works in CI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Non-src files (workflows, build scripts) no longer trigger full build safety fallback — only unmapped files under src/ do - Normalize backslashes to forward slashes when matching project keys (AL-Go uses backslashes, our keys use forward slashes) - Use PSObject.Properties indexer for keys with special chars - Add grouped logging showing which apps are in scope for compile/test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows a clear list of which apps will be compiled and tested per project, distinguishing between filtered and full build projects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Revert all YAML changes (off-limits, managed by AL-Go). Add incrementalBuilds setting with mode: modifiedApps for compile filtering. Add test skip logic in RunTestsInBcContainer.ps1 using BuildOptimization module. - Compile filtering: AL-Go native incrementalBuilds reuses prebuilt .app files - Test filtering: Test-ShouldSkipTestApp skips unaffected test apps with caching - Add Get-ChangedFilesForCI for CI environment detection - Add [OutputType] attributes to fix PSScriptAnalyzer warnings - 31 Pester tests passing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove [SuppressMessageAttribute] that broke Pester's variable scoping in PowerShell 7 CI environment. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… builds This is temporary — will be reverted after verifying that incrementalBuilds and test skipping work correctly in CI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore build/* pattern. incrementalBuilds can't be smoke-tested from this PR because AL-Go forces full build when .github/*.json changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- YAML files point to Groenbech96/AL-Go fork with ignoreSettingsChanges - AL-Go-Settings.json: ignoreSettingsChanges=true, removed build/* from fullBuildPatterns - Will revert after verifying incremental builds work Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Our YAML changes match the workflow patterns. Remove temporarily for smoke test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore YAML files to main, restore fullBuildPatterns, remove ignoreSettingsChanges. incrementalBuilds with modifiedApps cannot help BCApps because System Application dependency cascade rebuilds all W1 apps on any System App change. Test skipping in RunTestsInBcContainer is the primary optimization. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ation-app-level-filtering
- Merged main (Shopify + DotNet Aliases changes now in baseline) - Rewrote BuildOptimization.psm1: 680 → 165 lines, removed V1 dead code, disk cache, upstream BFS, duplicate fullBuildPatterns check - Cleared fullBuildPatterns to allow testing (will restore) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| [void]$directlyChanged.Add($appId) | ||
| } elseif ($file.Replace('\', '/') -match '(^|/)src/') { | ||
| # Unmapped file under src/ — safety fallback to full build | ||
| return @($Graph.Keys) |
Check notice
Code scanning / PSScriptAnalyzer
The cmdlet 'Get-AffectedApps' returns an object of type 'System.Object[]' but this type is not declared in the OutputType attribute. Note
- Add PSDoc help comments to all 5 functions (PSScriptAnalyzer requirement) - Add [OutputType()] attributes to all functions - Remove accidentally committed .claude/settings.local.json Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Production-ready: only BuildOptimization module, RunTestsInBcContainer skip logic, incrementalBuilds setting, and tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Skip test execution for unaffected apps in CI. AL-Go runs tests for every app in a project even when only one app changed. This adds a check in
RunTestsInBcContainer.ps1that skips tests for apps outside the dependency graph of changed files.How it works
AL-Go calls
RunTestsInBcContainer.ps1once per test app with$parameters["appName"]. The new code:git diffagainst the base branchapp.jsonfiles (~329 apps)$true(skip)Safe defaults: skipping is disabled outside CI, on
workflow_dispatch, and whenBUILD_OPTIMIZATION_DISABLED=true.Changes
build/scripts/BuildOptimization.psm1build/scripts/RunTestsInBcContainer.ps1build/scripts/tests/BuildOptimization.Test.ps1.github/AL-Go-Settings.jsonincrementalBuildswithmodifiedApps(compile-side optimization from AL-Go)What this does NOT do
incrementalBuilds)