Skip to content

Add app-level dependency graph filtering for CI/CD build optimization#7111

Open
Groenbech96 wants to merge 20 commits intomainfrom
feature/build-optimization-app-level-filtering
Open

Add app-level dependency graph filtering for CI/CD build optimization#7111
Groenbech96 wants to merge 20 commits intomainfrom
feature/build-optimization-app-level-filtering

Conversation

@Groenbech96
Copy link
Contributor

@Groenbech96 Groenbech96 commented Mar 12, 2026

Intent

Skip test execution for unaffected apps in CI. AL-Go runs tests for every app in a project even when only one app changed. This adds a check in RunTestsInBcContainer.ps1 that skips tests for apps outside the dependency graph of changed files.

How it works

AL-Go calls RunTestsInBcContainer.ps1 once per test app with $parameters["appName"]. The new code:

  1. Gets changed files via git diff against the base branch
  2. Builds a dependency graph from all app.json files (~329 apps)
  3. Maps changed files to apps, then BFS downstream through dependents
  4. If the current test app is not in the affected set, returns $true (skip)

Safe defaults: skipping is disabled outside CI, on workflow_dispatch, and when BUILD_OPTIMIZATION_DISABLED=true.

Changes

File What
build/scripts/BuildOptimization.psm1 New module: dependency graph, affected app computation, skip decision (165 lines)
build/scripts/RunTestsInBcContainer.ps1 4 lines added: import module, check skip, early return
build/scripts/tests/BuildOptimization.Test.ps1 18 Pester 5 tests
.github/AL-Go-Settings.json Enable incrementalBuilds with modifiedApps (compile-side optimization from AL-Go)

What this does NOT do

  • Does not modify YAML workflows (managed by AL-Go)
  • Does not skip compilation (handled separately by AL-Go incrementalBuilds)

Introduces BuildOptimization.psm1 that builds a dependency graph from all
329 app.json files and filters appFolders/testFolders per project to only
include affected apps. This reduces build times significantly for PRs that
touch a small subset of apps (e.g., E-Document Core: 9 apps instead of ~55).

Includes a small test change in E-Document Core to verify filtering in CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Groenbech96 Groenbech96 requested review from a team as code owners March 12, 2026 15:01
@github-actions github-actions bot added Build: Automation Workflows and other setup in .github folder Build: scripts & configs Build scripts and configuration files AL: Apps (W1) Add-on apps for W1 labels Mar 12, 2026
BeforeAll {
Import-Module "$PSScriptRoot\..\BuildOptimization.psm1" -Force
$baseFolder = (Resolve-Path "$PSScriptRoot\..\..\..").Path
$graph = Get-AppDependencyGraph -BaseFolder $baseFolder

Check warning

Code scanning / PSScriptAnalyzer

The variable 'graph' is assigned but never used. Warning test

The variable 'graph' is assigned but never used.
@github-actions
Copy link

Could not find a linked ADO work item. Please link one by using the pattern 'AB#' followed by the relevant work item number. You may use the 'Fixes' keyword to automatically resolve the work item when the pull request is merged. E.g. 'Fixes AB#1234'

Magnus Hartvig Grønbech and others added 4 commits March 12, 2026 16:07
git writes progress info to stderr (e.g., "From https://..."), which
PowerShell treats as a terminating error under $errorActionPreference=Stop.
Switch to Continue around git calls and pipe stderr to Out-Null for fetch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shallow checkouts in CI don't have the base commit or parent history,
so git diff fails. Use gh pr diff --name-only which queries the GitHub
API and works regardless of checkout depth.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Will revert after verifying the filtering works in CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Non-src files (workflows, build scripts) no longer trigger full build
  safety fallback — only unmapped files under src/ do
- Normalize backslashes to forward slashes when matching project keys
  (AL-Go uses backslashes, our keys use forward slashes)
- Use PSObject.Properties indexer for keys with special chars
- Add grouped logging showing which apps are in scope for compile/test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Magnus Hartvig Grønbech and others added 5 commits March 12, 2026 16:28
Shows a clear list of which apps will be compiled and tested per project,
distinguishing between filtered and full build projects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Revert all YAML changes (off-limits, managed by AL-Go).
Add incrementalBuilds setting with mode: modifiedApps for compile filtering.
Add test skip logic in RunTestsInBcContainer.ps1 using BuildOptimization module.

- Compile filtering: AL-Go native incrementalBuilds reuses prebuilt .app files
- Test filtering: Test-ShouldSkipTestApp skips unaffected test apps with caching
- Add Get-ChangedFilesForCI for CI environment detection
- Add [OutputType] attributes to fix PSScriptAnalyzer warnings
- 31 Pester tests passing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Magnus Hartvig Grønbech and others added 8 commits March 13, 2026 15:12
Remove [SuppressMessageAttribute] that broke Pester's variable scoping
in PowerShell 7 CI environment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… builds

This is temporary — will be reverted after verifying that incrementalBuilds
and test skipping work correctly in CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore build/* pattern. incrementalBuilds can't be smoke-tested from
this PR because AL-Go forces full build when .github/*.json changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- YAML files point to Groenbech96/AL-Go fork with ignoreSettingsChanges
- AL-Go-Settings.json: ignoreSettingsChanges=true, removed build/* from fullBuildPatterns
- Will revert after verifying incremental builds work

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Our YAML changes match the workflow patterns. Remove temporarily for smoke test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore YAML files to main, restore fullBuildPatterns, remove
ignoreSettingsChanges. incrementalBuilds with modifiedApps cannot help
BCApps because System Application dependency cascade rebuilds all W1
apps on any System App change. Test skipping in RunTestsInBcContainer
is the primary optimization.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Merged main (Shopify + DotNet Aliases changes now in baseline)
- Rewrote BuildOptimization.psm1: 680 → 165 lines, removed V1 dead code,
  disk cache, upstream BFS, duplicate fullBuildPatterns check
- Cleared fullBuildPatterns to allow testing (will restore)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Groenbech96 Groenbech96 requested a review from a team as a code owner March 14, 2026 09:44
[void]$directlyChanged.Add($appId)
} elseif ($file.Replace('\', '/') -match '(^|/)src/') {
# Unmapped file under src/ — safety fallback to full build
return @($Graph.Keys)

Check notice

Code scanning / PSScriptAnalyzer

The cmdlet 'Get-AffectedApps' returns an object of type 'System.Object[]' but this type is not declared in the OutputType attribute. Note

The cmdlet 'Get-AffectedApps' returns an object of type 'System.Object[]' but this type is not declared in the OutputType attribute.
- Add PSDoc help comments to all 5 functions (PSScriptAnalyzer requirement)
- Add [OutputType()] attributes to all functions
- Remove accidentally committed .claude/settings.local.json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
}
}

return @($affected)

Check notice

Code scanning / PSScriptAnalyzer

The cmdlet 'Get-AffectedApps' returns an object of type 'System.Object[]' but this type is not declared in the OutputType attribute. Note

The cmdlet 'Get-AffectedApps' returns an object of type 'System.Object[]' but this type is not declared in the OutputType attribute.
Production-ready: only BuildOptimization module, RunTestsInBcContainer
skip logic, incrementalBuilds setting, and tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions github-actions bot removed the AL: Apps (W1) Add-on apps for W1 label Mar 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Build: Automation Workflows and other setup in .github folder Build: scripts & configs Build scripts and configuration files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant