Skip to content

Introduce MAL/LAL/Hierarchy V2 engine — replace Groovy DSL runtime with ANTLR4 + Javassist#13723

Open
wu-sheng wants to merge 63 commits intomasterfrom
groovy-replace
Open

Introduce MAL/LAL/Hierarchy V2 engine — replace Groovy DSL runtime with ANTLR4 + Javassist#13723
wu-sheng wants to merge 63 commits intomasterfrom
groovy-replace

Conversation

@wu-sheng
Copy link
Member

@wu-sheng wu-sheng commented Mar 3, 2026

Introduce MAL/LAL/Hierarchy V2 engine — replace Groovy DSL runtime with ANTLR4 + Javassist

  • This is a non-trivial feature. Design doc: docs/en/academy/dsl-compiler-design.md

  • Documentation updated to include this new feature.

  • Tests (UT, IT, E2E) are added to verify the new feature.

  • If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.

  • Update the CHANGES log.

What this PR does

Introduces the MAL/LAL/Hierarchy V2 engine — replacing the Groovy-based DSL runtime for MAL (Meter Analysis Language), LAL (Log Analysis Language), and Hierarchy matching rules with compile-time ANTLR4 parsing and Javassist bytecode generation — the same approach already used by the OAL V2 engine.

All three DSL compilers follow the same pipeline:

DSL string → ANTLR4 parse → Immutable AST → Javassist bytecode → Direct Java execution

Why

  • Remove Groovy runtime dependency (~7MB) from the OAP server classpath
  • Eliminate runtime interpretation — generated bytecode uses direct method calls with zero reflection at runtime
  • Thread-safe by design — all generated instances are stateless singletons; per-request state passed as parameters (no ThreadLocal, no mutable wrappers)
  • Fail-fast at boot — DSL compilation errors are caught during startup with file/line/column reporting, not at first log/metric arrival
  • Debugger-friendly — all generated methods include LocalVariableTable (LVT) entries with named variables

Architecture

DSL Compiled Interface Runtime Signature State Passing
MAL MalExpression SampleFamily run(Map<String, SampleFamily>) Parameter
LAL LalExpression void execute(FilterSpec, ExecutionContext) Parameter
Hierarchy BiFunction<Service, Service, Boolean> Boolean apply(Service, Service) Parameter

All v2 classes live under .v2. packages to avoid FQCN conflicts with v1 (Groovy) classes, which remain in test/script-cases/script-runtime-with-groovy/ for comparison testing.


Compiled Code Examples

MAL Example 1 — Simple aggregation

DSL: instance_jvm_cpu.sum(['service', 'instance'])

Generated run() method:

public SampleFamily run(Map samples) {
  SampleFamily sf;
  sf = ((SampleFamily) samples.getOrDefault("instance_jvm_cpu", SampleFamily.EMPTY));
  sf = sf.sum(new String[]{"service", "instance"});
  return sf;
}
MAL Example 2 — Tag closure (inlined as method on main class)

DSL: metric.tag({tags -> tags.service_name = 'APISIX::' + tags.skywalking_service})

Generated class (single .class file, no separate closure class):

// Closure body compiled as a method on the main class
public Map _tag_apply(Map tags) {
  tags.put("service_name", "APISIX::" + tags.get("skywalking_service"));
  return tags;
}

// _tag field holds a TagFunction instance created via LambdaMetafactory
// (same mechanism javac uses for lambdas — JIT can fully inline)
public SampleFamily run(Map samples) {
  SampleFamily sf;
  sf = ((SampleFamily) samples.getOrDefault("metric", SampleFamily.EMPTY));
  sf = sf.tag(this._tag);
  return sf;
}

Closures are compiled as methods on the main class. At class-load time, LambdaMetafactory wraps each method into a functional interface instance (e.g., TagFunction, ForEachFunction) — the JVM creates a hidden class internally with no .class file on disk. This is the same mechanism javac uses for lambda expressions.

MAL Example 3 — Regex match with ternary

DSL: metric.tag({ tags -> def matcher = (tags.metrics_name =~ /\.ssl\.certificate\.([^.]+)\.expiration/); tags.secret_name = matcher ? matcher[0][1] : "unknown" })

Generated _tag_apply() method:

public Map _tag_apply(Map tags) {
  String[][] matcher = MalRuntimeHelper.regexMatch(
      (String) tags.get("metrics_name"),
      "\\.ssl\\.certificate\\.([^.]+)\\.expiration");
  tags.put("secret_name", (((Object)(matcher)) != null ? (matcher[0][1]) : ("unknown")));
  return tags;
}

def type inferred as String[][] from =~ regex match. Ternary compiles to Java ternary with null-check on Object cast.


LAL Example 1 — JSON parser with extractor

DSL:

filter {
  json {}
  extractor {
    service parsed.service as String
    instance parsed.instance as String
  }
  sink {}
}

Generated class:

public void execute(FilterSpec filterSpec, ExecutionContext ctx) {
  LalRuntimeHelper h = new LalRuntimeHelper(ctx);
  filterSpec.json(ctx);
  if (!ctx.shouldAbort()) { _extractor(filterSpec.extractor(), h); }
  filterSpec.sink(ctx);
}

private void _extractor(ExtractorSpec _e, LalRuntimeHelper h) {
  _e.service(h.ctx(), h.toStr(h.mapVal("service")));
  _e.instance(h.ctx(), h.toStr(h.mapVal("instance")));
}

Single class, no closures. h.mapVal() accesses JSON parsed map. h.toStr() preserves null (unlike String.valueOf() which returns "null").

LAL Example 2 — Proto-based with extraLogType (Envoy ALS)

DSL:

filter {
  if (parsed?.response?.responseCode?.value as Integer < 400) { abort {} }
  extractor {
    if (parsed?.response?.responseCode) {
      tag 'status.code': parsed?.response?.responseCode?.value
    }
    tag 'response.flag': parsed?.commonProperties?.responseFlags
  }
  sink {}
}

Generated class (with extraLogType = HTTPAccessLogEntry):

public void execute(FilterSpec filterSpec, ExecutionContext ctx) {
  LalRuntimeHelper h = new LalRuntimeHelper(ctx);
  // Cast once, reuse as _p
  HTTPAccessLogEntry _p = (HTTPAccessLogEntry) h.ctx().extraLog();
  // Safe-nav chain cached in local variables
  HTTPResponseProperties _t0 = _p == null ? null : _p.getResponse();
  UInt32Value _t1 = _t0 == null ? null : _t0.getResponseCode();
  if (_t1 != null && _t1.getValue() < 400) { filterSpec.abort(ctx); }
  if (!ctx.shouldAbort()) { _extractor(filterSpec.extractor(), h); }
  filterSpec.sink(ctx);
}

private void _extractor(ExtractorSpec _e, LalRuntimeHelper h) {
  HTTPAccessLogEntry _p = (HTTPAccessLogEntry) h.ctx().extraLog();
  HTTPResponseProperties _t0 = _p == null ? null : _p.getResponse();
  UInt32Value _t1 = _t0 == null ? null : _t0.getResponseCode();
  if (_t1 != null) {
    _e.tag(h.ctx(), "status.code", h.toStr(Integer.valueOf(_t1.getValue())));
  }
  AccessLogCommon _t2 = _p == null ? null : _p.getCommonProperties();
  _e.tag(h.ctx(), "response.flag", h.toStr(_t2 == null ? null : _t2.getResponseFlags()));
}

Proto getter chains resolved via Java reflection at compile time — at runtime it's direct method calls. ?. safe navigation emits == null ? null : ternaries. Intermediate values cached in _tN local variables for readability and dedup.


Hierarchy Example 1 — Simple name match

DSL: { (u, l) -> u.name == l.name }

Generated class:

public Object apply(Object arg0, Object arg1) {
  Service u = (Service) arg0;
  Service l = (Service) arg1;
  return Boolean.valueOf(java.util.Objects.equals(u.getName(), l.getName()));
}
Hierarchy Example 2 — Block body with if/return

DSL: { (u, l) -> { if (l.shortName.lastIndexOf('.') > 0) { return u.shortName == l.shortName.substring(0, l.shortName.lastIndexOf('.')); } return false; } }

Generated class:

public Object apply(Object arg0, Object arg1) {
  Service u = (Service) arg0;
  Service l = (Service) arg1;
  if (l.getShortName().lastIndexOf(".") > 0) {
    return Boolean.valueOf(java.util.Objects.equals(
        u.getShortName(),
        l.getShortName().substring(0, l.getShortName().lastIndexOf("."))));
  }
  return Boolean.valueOf(false);
}

Property access → getter methods. ==Objects.equals(). Numeric > → direct operator.


v1 vs v2 Cross-Verification

The test/script-cases/script-runtime-with-groovy/ module runs every production DSL expression through both Groovy v1 and Javassist v2, then asserts identical results. Both v1 and v2 must pass — v1 failure also fail()s (no silent skip).

How it works
For each DSL expression in production YAML configs:
  1. Compile with v1 (Groovy)    → run with mock data → collect output
  2. Compile with v2 (Javassist) → run with same data → collect output
  3. Assert v1 output == v2 output field by field

v1 and v2 coexist in the same JVM via package isolation (*.dsl.* vs *.v2.dsl.*), each with its own ModuleManager mock.

What is compared
DSL Fields compared
MAL (metadata) samples (input metric names), scopeType, downsampling, isHistogram, scopeLabels, aggregationLabels
MAL (runtime) Output Sample[] arrays — label sets and values, after running both with identical SampleFamily input
LAL shouldAbort, shouldSave, log.service, log.serviceInstance, log.endpoint, log.layer, log.timestamp, log.tags
Hierarchy Boolean result for each (upper, lower) Service pair
Test data
  • MAL: 73 companion .data.yaml files provide realistic mock SampleFamily input per YAML config. For increase()/rate() expressions, the checker primes the counter window with initial data before the comparison run. Expressions without companion data fall back to auto-generated mock data.
  • LAL: Mock LogData protobuf built with service/instance/endpoint/timestamp/traceContext. Rules with extraLogType (e.g., envoy-als proto) use .input.data files with proto-json mock data and the LALSourceTypeProvider SPI to resolve the proto type per layer at compile time.
  • Hierarchy: Test pairs from .data.yaml with (upper, lower, expected) tuples per rule.
Verification counts
DSL Expressions Source
MAL expressions 1,229 73 YAML files across 4 directories
MAL filter closures 31 Separate MalFilterComparisonTest
LAL scripts 29 oap-cases + feature-cases
Hierarchy rules 4 rules × test pairs test-hierarchy-definition.data.yaml
Total ~1,290

Files changed

537 files, +40,436 / -1,480 (bulk from new test data files, POM version changes, and generated grammar sources)

wu-sheng and others added 29 commits February 28, 2026 14:10
Document the detailed implementation plan for eliminating Groovy from
OAP runtime via build-time transpilers (MAL/LAL) and v1/v2 module
split (hierarchy), based on Discussion #13716 and skywalking-graalvm-distro.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add MalExpression, MalFilter, LalExpression functional interfaces and
SampleFamilyFunctions (TagFunction, SampleFilter, ForEachFunction,
DecorateFunction, PropertiesExtractor). Add Java functional interface
overloads alongside existing Groovy Closure methods in SampleFamily,
FilterSpec, ExtractorSpec, and SinkSpec. Change InstanceEntityDescription
to use Function instead of Closure. All 129 existing tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hase 2)

Ports MalToJavaTranspiler from skywalking-graalvm-distro into a new
mal-transpiler analyzer submodule. The transpiler parses Groovy MAL
expressions/filters via AST at CONVERSION phase and emits equivalent
Java classes implementing MalExpression/MalFilter interfaces from Phase 1.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hase 3)

Introduces lal-transpiler module that parses LAL Groovy DSL scripts into
AST at Phases.CONVERSION and emits pure Java classes implementing
LalExpression. Handles filter/text/json/yaml/extractor/sink/abort blocks,
parsed property access, safe navigation, cast expressions, GString
interpolation, and SHA-256 deduplication. Makes MalToJavaTranspiler.escapeJava()
public for cross-module reuse. Includes 37 comprehensive tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ing (Phase 4)

Introduces meter-analyzer-v2 and log-analyzer-v2 modules that provide
same-FQCN replacement classes for DSL.java, Expression.java, and
FilterExpression.java. The v2 classes load transpiled MalExpression/
MalFilter/LalExpression implementations from META-INF manifests via
Class.forName() instead of Groovy GroovyShell/ExpandoMetaClass/
DelegatingScript. Uses maven-shade-plugin to overlay the upstream
Groovy-dependent classes. Includes 7 unit tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… (Phase 5)

Extract hierarchy matching rules from HierarchyDefinitionService into
pluggable HierarchyRuleProvider interface. Remove Groovy imports from
server-core by replacing Closure<Boolean> with BiFunction<Service,Service,Boolean>.

- hierarchy-v1: GroovyHierarchyRuleProvider (for CI checker only)
- hierarchy-v2: JavaHierarchyRuleProvider with 4 built-in rules + 12 tests
- HierarchyDefinitionService: add HierarchyRuleProvider interface, DefaultJavaRuleProvider
- HierarchyService: .getClosure().call() → .match()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ase 6)

Three checker modules verify v1 (Groovy) and v2 (transpiled Java) produce
identical results: hierarchy rules (22 tests), MAL expressions (1187 tests),
MAL filters (29 tests), and LAL scripts (10 tests). Zero behavioral
divergences found when both paths succeed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…L to on-the-fly compilation

- Merge mal-grammar + mal-compiler into meter-analyzer
- Merge lal-grammar + lal-compiler into log-analyzer
- Merge hierarchy-rule-grammar + hierarchy-rule-compiler into hierarchy
- Remove 6 standalone modules (3 grammar + 3 compiler)
- Update DSL.java to compile MAL expressions on-the-fly via MALClassGenerator
  instead of loading from non-existent manifest file
- Add varargs handling for tagEqual/tagNotEqual/tagMatch/tagNotMatch in
  generated Javassist code (wrap String args in new String[]{})
- Update test/script-compiler checker POMs to reference merged module names
- Update CLAUDE.md files with merged file structure and paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix MAL sample collection regression: skip downsampling() method
  arguments to prevent enum values (MAX, SUM, MIN) from being
  collected as sample names
- Fix MAL safe navigation (?.): parser now correctly propagates
  safeNav flag to chain segments; code generator uses local
  StringBuilder to avoid corrupting parent buffer
- Fix MAL filter grammar: add closureCondition alternatives to
  closureBody rule for bare conditions like { tags -> tags.x == 'v' }
- Fix MAL downsampling detection for bare identifiers parsed as
  ExprArgument wrapping MetricExpr
- Fix MAL sample ordering: use LinkedHashSet for consistent order
- Fix LAL tag() function call: add functionName rule allowing TAG
  token in functionInvocation for if(tag("LOG_KIND") == ...) patterns
- Fix LAL ProcessRegistry support: add PROCESS_REGISTRY to
  valueAccessPrimary grammar rule
- Fix LAL tag statement code generation: wrap single tag entries in
  Collections.singletonMap() since ExtractorSpec.tag() accepts Map
- Fix LAL makeComparison to handle CondFunctionCallContext properly
- Add debug logging to all three code generators (MAL, LAL, Hierarchy)
  showing AST and generated Java source at DEBUG level
- Add generateFilterSource() to MALClassGenerator for testing
- Add error handling unit tests with demo error comments for MAL (5),
  LAL (4), and Hierarchy (4) generators
- All 1248 checker tests pass: MAL 1187, Filter 29, LAL 10, Hierarchy 22

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ng all four DSL compilers (OAL, MAL, LAL, Hierarchy). Remove Groovy references from docs: LAL code blocks, hierarchy matching rule labels, and stale MeterProcessor comment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Provides a /run-e2e slash command with prerequisites (e2e CLI,
swctl, yq install instructions), rebuild detection, test execution,
and failure debugging workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, interpolated sampler IDs

Address five critical gaps in the LAL v2 compiler that broke shipped production rules:

1. tag("LOG_KIND") in conditions now emits tagValue() helper instead of null
2. Safe navigation (?.) for method calls emits safeCall() helper to prevent NPE
3. Metrics, slowSql, sampledTrace, sampler/rateLimit blocks generate proper
   sub-consumer classes with BindingAware wiring
4. else-if chains build nested IfBlock AST nodes instead of dropping
   intermediate branches
5. GString interpolation in rateLimit IDs (e.g. "${log.service}:${parsed.code}")
   parsed into InterpolationPart segments and emitted as string concatenation

Also fixes ProcessRegistry static calls to pass arguments through, and adds
comprehensive tests (55 total: 35 generator + 20 parser) covering all gaps
including production-like envoy-als, nginx, and k8s-service rule patterns.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…cripts and runtime comparison

- Rename test/script-compiler to test/script-cases/script-runtime-with-groovy
- Copy all shipped production configs into test/script-cases/scripts/ as test copies
  (MAL: test-otel-rules, test-meter-analyzer-config, test-log-mal-rules, test-envoy-metrics-rules;
   LAL: test-lal; Hierarchy: test-hierarchy-definition.yml)
- Update all checker tests to load from shared scripts/ directory
- Upgrade LAL checker from compile-only to full runtime execution comparison
  (v1 Groovy vs v2 ANTLR4+Javassist, comparing Binding state: service, layer, tags, abort/save)
- Update Maven coordinates and root pom module path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move benchmarks from the standalone oap-server/microbench module into
the src/test/ directories of the modules they actually test (server-core
and library-util). Drop AbstractMicrobenchmark base class in favor of
self-contained @test run() methods. Bump JMH 1.21 -> 1.37 and remove
the obsolete -XX:BiasedLockingStartupDelay=0 JVM flag (removed in JDK 18).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MAL compiler fixes (closes 38 previously failing expressions):
- Add ternary operator (?:) support in closures (grammar, AST, codegen)
- Fix valueEqual() and other primitive-double methods with numeric literal args
- Support double-paren argument syntax: sum((['cluster']))
- Handle NUMBER / SampleFamily via MalRuntimeHelper.divReverse() in v2 package
- Add variable declarations, map literals, forEach/instance closure types
- Add ProcessRegistry class references, improved safe navigation

LAL compiler fixes:
- Fix null-to-string conversion: use null-safe toStr() instead of String.valueOf()
- Add camelToSnake field name fallback for protobuf field access
- Add typed execute(FilterSpec, Binding) method signature
- Reorganize LAL test scripts into oap-cases/ and feature-cases/
- Add data-driven LALExpressionExecutionTest with 27 test cases

MAL checker enhancements:
- Add runtime execution comparison (mock SampleFamily data, execute both
  v1 and v2, compare output samples with labels and values)
- Handle increase()/rate() by priming CounterWindow with initial run
- Extract tagEqual patterns from expressions for matching mock data

All 1,187 MAL + 29 LAL + 22 hierarchy expressions now pass with zero gaps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…L typed signature

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Give v2 (ANTLR4+Javassist) classes distinct FQCNs from v1 (Groovy)
so both can coexist on the classpath without source duplication in
v1-with-groovy test modules.

Package mapping:
- MAL: meter.analyzer.* → meter.analyzer.v2.*
- LAL: log.analyzer.* → log.analyzer.v2.*
- Hierarchy: config.compiler.* → config.v2.compiler.*

Also: remove v2-only files (MalExpression, MalFilter, LalExpression)
from v1-with-groovy modules, add mal-v1-with-groovy dependency to
lal-v1-with-groovy, fix cross-version enum comparison by name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ime() scalar in MAL compiler

Add ANTLR4 lexer mode for regex literals (=~ /pattern/), def keyword with
type inference from initializer (String[][] for regex, String[] for split),
GString interpolation expansion, .size() to .length translation, decorate()
bean-mode closures, and time() as a scalar function in binary expressions.
Verified with 1,228 v1-v2 checker tests (1,197 MAL + 31 filter).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…v1-v2 checker data

Rewrite MAL run() code generation to use a single reassigned 'sf' variable
instead of multiple intermediate variables, producing cleaner decompiled output.
Add LocalVariableTable attribute so decompilers show 'samples' and 'sf' instead
of 'var1' and 'var2'. Integrate v2 compilers with runtime wiring, add checker
test data files, and clean up unused code across MAL/LAL/Hierarchy modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add LVT attribute to LAL execute() and consumer accept() methods, and to
Hierarchy apply() method, so decompilers show meaningful variable names
(filterSpec, binding, _t, u, l) instead of var0, var1, etc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 8 helper methods (getAt, toLong, toInt, toStr, toBool, isTruthy,
tagValue, safeCall) from being duplicated in every generated class via
addHelperMethods() to a shared LalRuntimeHelper in the rt package.
Generated code now calls LalRuntimeHelper.toStr() etc. via FQCN.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…th typed methods

Fix rateLimit() calls inside if-blocks within sampler generating empty bytecode
by handling the samplerContent grammar alternative in LALScriptParser.visitIfBody().

Replace generic LalRuntimeHelper.safeCall() and isTruthy() with specific typed
methods: isTrue() for Boolean conditions, isNotEmpty() for String non-emptiness,
toString() and trim() for null-safe navigation — making generated code explicit
about intended type semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…licitly, add LVT

- Merge consumer sub-classes into single generated class with private methods
- Remove BINDING ThreadLocal from AbstractSpec; all spec methods take ExecutionContext explicitly
- Delete BindingAware.java and Binding.java, replace with ExecutionContext
- Add abort guard before _extractor/_sink calls matching v1 Groovy behavior
- Add LocalVariableTable to all generated methods (execute, _extractor, _sink)
- Rename binding→ctx throughout for consistency
- Add extraLogType to envoy-als.yaml for compile-time proto resolution
- Remove all Consumer callback methods from spec files
- Add finalizeSink abort check in FilterSpec

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…es, filters)

Previously only run() had LVT. Now all generated methods have named locals
in debuggers/decompilers instead of var0/var1/var2:
- metadata(): this, _samples, _scopeLabels, _aggLabels, _pct
- tag/instance apply(Map): this, param name
- tag/instance apply(Object) bridge: this, o
- forEach accept(): this, element, tags
- decorate accept(): this, _arg, param name
- filter test(): this, param name

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…version 10.4.0-SNAPSHOT

Replace CI-friendly ${revision} with hardcoded 10.4.0-SNAPSHOT in all 104 POMs.
This eliminates persistent "Could not find artifact ...pom:${revision}" errors
when building individual modules without -am. Also removes flatten-maven-plugin
(no longer needed), updates release scripts to use versions:set, and wires
LALSourceTypeProvider SPI for envoy-als extraLog type resolution in tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…chains

Cache the extraLog cast in a _p local variable and break safe-nav chains
into sequential _tN locals instead of deeply nested ternaries. Repeated
access to the same chain prefix reuses existing variables (dedup).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove bind()/evaluate() two-phase pattern from DSL. The mutable
ExecutionContext field made DSL unsafe for concurrent use. Now
evaluate(ExecutionContext) takes ctx as a parameter, matching the
stateless pattern already used by MAL and Hierarchy v2 runtimes.

Update LogFilterListener to store per-request contexts in a list
and pass each to the corresponding DSL.evaluate(ctx) call.
Update LogTestQuery to use the new single-call API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wu-sheng wu-sheng added the core feature Core and important feature. Sometimes, break backwards compatibility. label Mar 3, 2026
wu-sheng added 4 commits March 4, 2026 13:16
…hod chaining

The eBPF network-profiling e2e test uses LAL patterns that the v2 parser
didn't support: string concatenation with +, parenthesized cast expressions
like (expr as String).split(":")[0].endsWith(".1"), and array indexing.

- Grammar: split valueAccess into valueAccessTerm with PLUS concat,
  add valueParen primary, segmentIndex segment, condParenGroup
- AST: add concatParts, parenInner/parenCast to ValueAccess, IndexSegment
- Parser: refactor visitValueAccess to handle terms, paren, index
- Codegen: add generateParenAccess, IndexSegment handling, concat codegen
- Runtime: add boolean overloads for isNotEmpty/isTrue (primitive returns)
- Test: add network-profiling-e2e.yaml checker case from e2e Helm override
- Add compile-time validation for decorate(): must follow service(),
  not instance()/endpoint(), and not with histogram metrics
- Add MeterEntity comparison to MAL v1-v2 checker: validates service,
  instance, and endpoint names match between v1 and v2
- Include scopeLabels in auto-generated mock data so entity names are
  realistic (not null/empty) during comparison
- Extract MALClosureCodegen and MALCodegenHelper from MALClassGenerator
- Extract LALBlockCodegen and LALCodegenHelper from LALClassGenerator
- Feed real .input.data into LalComparisonTest instead of synthetic LogData
- Add sampledTrace field comparison (traceId, serviceName, latency, etc.)
  between v1 and v2 paths in the checker
- Verify v2 dispatches trace via sourceReceiver.receive()
- Add missing .input.data entries for nginx-error-log,
  envoy-als/network-profiling-slow-trace, and network-profiling-e2e
- Add traceId/serviceName/serviceInstanceName/timestamp cases to
  LALExpressionExecutionTest.assertSampledTrace()
- Update all sampledTrace expect blocks with log-context fields
… fallback for NONE parser type

FilterSpec.json() and FilterSpec.yaml() now add LogData proto fields (service,
serviceInstance, endpoint, layer, timestamp) to the parsed map via putIfAbsent,
matching v1 Groovy Binding.Parsed.getAt() fallback behavior.

When a LAL rule has no parser and no extraLogType, the compiler now falls back
to LogData.Builder for parsed.* field access with compile-time reflection
validation, instead of failing with IllegalStateException.

Updated checker test with real e2e input (empty server_process.process_id) and
toRecord() assertions to verify v1-v2 parity for ProcessRegistry calls.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces the Groovy-based MAL/LAL/Hierarchy DSL runtime with an ANTLR4 parser + Javassist bytecode generation pipeline, aiming for faster execution, thread-safety, and fail-fast compilation at startup.

Changes:

  • Introduces v2 MAL DSL runtime types (Sample/SampleFamily helpers, metadata model, filter/expression APIs) and compiler runtime helpers.
  • Introduces v2 LAL engine components (SPI for extraLog type resolution, runtime listener pipeline, parsing/execution context, samplers).
  • Adds ANTLR grammars and Maven build plugin/runtime dependencies across affected modules, plus documentation updates.

Reviewed changes

Copilot reviewed 132 out of 557 changed files in this pull request and generated 14 comments.

Show a summary per file
File Description
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/tagOpt/Retag.java Adds Retag interface for tag optimization implementations.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/tagOpt/K8sRetagType.java Adds Kubernetes-based retag implementation using K8s metadata registry.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/registry/ProcessRegistry.java Adds registry helper to generate virtual process entities for DSL usage.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/counter/ID.java Adds counter key type for windowed counter calculations.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/counter/CounterWindow.java Adds shared counter window logic for increase/rate calculations in MAL.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/SampleFamilyFunctions.java Adds functional interfaces used as closure/functional parameters in MAL.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/SampleFamilyBuilder.java Adds builder helper for constructing SampleFamily with runtime context tweaks.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/Sample.java Adds Sample model with counter increase helpers calling CounterWindow.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/Result.java Adds parsing/execution result wrapper for MAL evaluation.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/MalFilter.java Adds functional interface for compiled MAL filter closures.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/MalExpression.java Adds compiled MAL expression interface (run + compile-time metadata).
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/FilterExpression.java Adds wrapper compiling/evaluating MAL filter closure against sample families.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/ExpressionParsingException.java Adds parsing-phase exception type for MAL v2.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/ExpressionMetadata.java Adds immutable compile-time metadata extracted from MAL AST.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/Expression.java Adds execution wrapper around compiled MalExpression, plus validation.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/EntityDescription/ServiceRelationEntityDescription.java Adds v2 entity description for service relations.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/EntityDescription/ServiceEntityDescription.java Adds v2 entity description for services.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/EntityDescription/ProcessRelationEntityDescription.java Adds v2 entity description for process relations.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/EntityDescription/ProcessEntityDescription.java Adds v2 entity description for processes.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/EntityDescription/InstanceEntityDescription.java Adds v2 entity description for instances with properties extractor hook.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/EntityDescription/EntityDescription.java Adds base interface for v2 entity descriptions.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/EntityDescription/EndpointEntityDescription.java Adds v2 entity description for endpoints.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/DownsamplingType.java Adds downsampling enum used in MAL metadata/codegen.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/DSL.java Adds MAL v2 DSL compile entrypoint (ANTLR4 + Javassist).
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/rt/MalRuntimeHelper.java Adds MAL runtime helper methods called by generated code.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/rt/MalExpressionPackageHolder.java Adds Javassist classloading anchor for generated MAL classes.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java Adds shared MAL codegen utilities/constants.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricRuleConfig.java Adds v2 metric rule config contract used by MetricConvert.
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricConvert.java Updates metric execution pipeline to v2 MAL compilation and evaluation.
oap-server/analyzer/meter-analyzer/src/main/antlr4/org/apache/skywalking/mal/rt/grammar/MALLexer.g4 Adds ANTLR lexer for MAL v2.
oap-server/analyzer/meter-analyzer/pom.xml Removes Groovy and adds ANTLR/Javassist + ANTLR Maven plugin.
oap-server/analyzer/meter-analyzer/CLAUDE.md Documents MAL compiler architecture and constraints.
oap-server/analyzer/log-analyzer/src/test/resources/META-INF/services/org.apache.skywalking.oap.log.analyzer.v2.spi.LALSourceTypeProvider Registers test SPI provider for LAL extraLog type resolution.
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/DSLV2Test.java Adds LAL v2 compilation smoke tests.
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/TestMeshLALSourceTypeProvider.java Adds test SPI implementation for mesh layer extraLog type.
oap-server/analyzer/log-analyzer/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleProvider Switches log-analyzer module provider to v2.
oap-server/analyzer/log-analyzer/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.library.module.ModuleDefine Switches log-analyzer module define to v2.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/spi/LALSourceTypeProvider.java Adds SPI for default per-layer extraLog Java type.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/TrafficSinkListener.java Adds sink listener to emit service/instance/endpoint meta from logs.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/RecordSinkListener.java Adds sink listener to persist logs with searchable tags + autocomplete.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogSinkListenerFactory.java Adds factory interface for sink listeners.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogSinkListener.java Adds sink listener interface for build+parse lifecycle.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogFilterListener.java Adds runtime listener executing compiled LAL DSL per log.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogAnalysisListenerFactory.java Adds analysis listener factory interface for per-layer listener creation.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/listener/LogAnalysisListener.java Adds analysis listener interface for build+parse lifecycle.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/analyzer/LogAnalyzerFactory.java Adds placeholder factory class for log analyzer (currently empty).
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/LogAnalyzerServiceImpl.java Adds v2 log analyzer service + listener manager.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/LogAnalyzer.java Adds v2 per-request log analysis orchestration.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/ILogAnalyzerService.java Adds v2 service interface for log analysis.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/log/ILogAnalysisListenerManager.java Adds v2 listener manager contract.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/LogAnalyzerModuleProvider.java Registers v2 provider and wires MAL metric converts + LAL listener at start.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/LogAnalyzerModuleConfig.java Adds v2 module config supporting LAL + log-MAL config loading.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/LALConfigs.java Adds LAL YAML loader for v2.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/provider/LALConfig.java Adds LAL rule config model (name/dsl/layer/extraLogType).
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/module/LogAnalyzerModule.java Adds v2 module define exposing ILogAnalyzerService.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/sink/sampler/Sampler.java Adds sampler abstraction for sink sampling control.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/sink/sampler/RateLimitingSampler.java Adds per-minute rate limiting sampler + reset handler.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/sink/sampler/PossibilitySampler.java Adds probability-based sampler implementation.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/sink/SinkSpec.java Adds sink spec with sampler + enforcer/dropper actions.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/sink/SamplerSpec.java Adds sampler spec with rate limiting sampling method.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/parser/YamlParserSpec.java Adds YAML parser spec using SnakeYAML SafeConstructor.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/parser/TextParserSpec.java Adds text regexp parser spec producing a Matcher parsed object.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/parser/JsonParserSpec.java Adds JSON parser spec with reusable ObjectMapper.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/parser/AbstractParserSpec.java Adds shared parser behavior including abort-on-failure flag.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/extractor/slowsql/SlowSqlSpec.java Adds slow SQL extractor spec writing into execution context.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/extractor/sampledtrace/SampledTraceSpec.java Adds sampled-trace extractor spec writing into execution context.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/AbstractSpec.java Adds base spec holding moduleManager + moduleConfig.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/LalExpression.java Adds compiled LAL expression interface.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/ExecutionContext.java Adds per-log mutable execution context passed into compiled LAL.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/DSL.java Adds LAL v2 DSL compile entrypoint + evaluate wrapper.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/rt/LalExpressionPackageHolder.java Adds Javassist classloading anchor for generated LAL classes.
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALCodegenHelper.java Adds shared LAL codegen utilities/constants.
oap-server/analyzer/log-analyzer/src/main/antlr4/org/apache/skywalking/lal/rt/grammar/LALLexer.g4 Adds ANTLR lexer for LAL v2.
oap-server/analyzer/log-analyzer/pom.xml Removes Groovy and adds ANTLR/Javassist + plugin + test dependency.
oap-server/analyzer/hierarchy/src/test/java/org/apache/skywalking/oap/server/core/config/v2/compiler/HierarchyRuleScriptParserTest.java Adds parser tests for hierarchy rule expressions.
oap-server/analyzer/hierarchy/src/test/java/org/apache/skywalking/oap/server/core/config/v2/compiler/HierarchyRuleClassGeneratorTest.java Adds codegen tests for compiled hierarchy rule matchers.
oap-server/analyzer/hierarchy/src/main/resources/META-INF/services/org.apache.skywalking.oap.server.core.config.HierarchyDefinitionService$HierarchyRuleProvider Registers compiled hierarchy rule provider via SPI.
oap-server/analyzer/hierarchy/src/main/java/org/apache/skywalking/oap/server/core/config/v2/compiler/hierarchy/rule/rt/HierarchyRulePackageHolder.java Adds Javassist classloading anchor for hierarchy generated classes.
oap-server/analyzer/hierarchy/src/main/java/org/apache/skywalking/oap/server/core/config/v2/compiler/HierarchyRuleModel.java Adds immutable AST model for hierarchy rule expressions.
oap-server/analyzer/hierarchy/src/main/java/org/apache/skywalking/oap/server/core/config/v2/compiler/CompiledHierarchyRuleProvider.java Adds SPI provider compiling hierarchy rules using ANTLR4 + Javassist.
oap-server/analyzer/hierarchy/src/main/antlr4/org/apache/skywalking/hierarchy/rt/grammar/HierarchyRuleParser.g4 Adds hierarchy rule parser grammar.
oap-server/analyzer/hierarchy/src/main/antlr4/org/apache/skywalking/hierarchy/rt/grammar/HierarchyRuleLexer.g4 Adds hierarchy rule lexer grammar.
oap-server/analyzer/hierarchy/pom.xml Adds hierarchy module POM with ANTLR/Javassist + plugin.
oap-server/analyzer/hierarchy/CLAUDE.md Documents hierarchy rule compiler architecture.
oap-server/analyzer/event-analyzer/pom.xml Updates module parent version reference.
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/meter/process/SampleBuilder.java Switches agent analyzer to MAL v2 Sample class.
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/meter/process/MeterProcessor.java Switches agent analyzer to v2 MetricConvert/Sample/SampleFamilyBuilder and updates comment.
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/meter/process/MeterProcessService.java Switches agent analyzer to v2 MetricConvert.
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/meter/config/MeterConfig.java Switches MeterConfig to v2 MetricRuleConfig and drops initExp field.
oap-server/analyzer/agent-analyzer/pom.xml Updates module parent version reference.
oap-server/ai-pipeline/pom.xml Updates module parent version reference.
oap-server-bom/pom.xml Updates module parent version reference.
docs/menu.yml Adds menu entry for DSL compiler design doc.
docs/en/concepts-and-designs/service-hierarchy-configuration.md Updates hierarchy docs to remove Groovy mention and describe expressions.
docs/en/concepts-and-designs/lal.md Updates LAL docs code fences and removes stray import.
docs/en/changes/changes.md Adds change log entry (currently about Maven revision/flatten removal).
dist-material/release-docs/LICENSE Removes Groovy dependency entry (reflecting removal of Groovy runtime).
apm-webapp/pom.xml Updates module parent version reference.
apm-protocol/pom.xml Updates module parent version reference.
apm-protocol/apm-network/pom.xml Updates module parent version reference.
apm-dist/pom.xml Updates module parent version reference.
.github/workflows/skywalking.yaml Removes flatten plugin usage from CI build steps.
.claude/skills/test/SKILL.md Adds contributor skill doc for running tests.
.claude/skills/run-e2e/SKILL.md Adds contributor skill doc for running E2E locally.
.claude/skills/license/SKILL.md Adds contributor skill doc for license checks.
.claude/skills/gh-pull-request/SKILL.md Adds contributor skill doc for PR workflow.
.claude/skills/compile/SKILL.md Adds contributor skill doc for compile/checkstyle/javadoc.
.claude/skills/ci-e2e-debug/SKILL.md Adds contributor skill doc for CI E2E artifact debugging.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

wu-sheng and others added 2 commits March 4, 2026 21:31
…/Hierarchy

- MAL/LAL/Hierarchy generators auto-set classOutputDir when SW_OAL_ENGINE_DEBUG
  env var is set, dumping generated .class files to mal-rt/, lal-rt/, hierarchy-rt/
- Add debug log in Analyzer.analyse() with metric name, generated class, and input samples
- Add debug log in DSL.evaluate() with rule name, generated class, and log data summary
- Add debug log in MatchingRule.match() with rule name, generated class, and service names
- LAL DSL.of() now accepts ruleName and sets classNameHint for meaningful class names
- Update CLAUDE.md docs for all three modules with Debug Output section
… and imports, update changelog

- Rename `EntityDescription` package to lowercase `entity` (Java convention)
- Fix grammar: "should be invoke" → "should be invoked" in Expression.java
- Replace `io.netty.util.internal.ThreadLocalRandom` with `java.util.concurrent.ThreadLocalRandom`
- Fix hierarchy CLAUDE.md package paths to include `.v2.` segment
- Add MAL/LAL/Hierarchy V2 engine changelog entry with highlights

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wu-sheng wu-sheng changed the title Replace Groovy DSL runtime with ANTLR4 + Javassist for MAL, LAL, and Hierarchy Introduce MAL/LAL/Hierarchy V2 engine — replace Groovy DSL runtime with ANTLR4 + Javassist Mar 4, 2026
wu-sheng and others added 18 commits March 4, 2026 23:51
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MAL execute ~4.9x, LAL compile ~39x / execute ~2.8x, Hierarchy execute ~2.6x
faster than Groovy v1.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ile skill

- Extract MAL codegen utility methods into MALCodegenHelper
- Add flatten:flatten to checkstyle command in compile skill
- Add contributing guide doc for Claude Code skills
- Add generate-classes skill

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ve failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…-plugin is restored

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…or new test modules

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…eactor modules

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…to ~6.8x

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 1: Fix MalInputDataGenerator to track per-rule tagEqual/tagMatch
variants, generating multiple sample variants per metric for complete
coverage. Handles tagNotEqual/tagNotMatch labels in input samples.

Phase 2: Add MalExpectedDataGenerator that runs v1 (Groovy) MAL
expressions and captures output (entities, samples, values) as rich
expected sections in .data.yaml files. Uses Mockito mockStatic for
K8s metadata mocking.

Phase 3: Enhance MalComparisonTest with hard assertions on expected
entities (scope/service/instance/endpoint/layer) and samples (labels/
values). EMPTY is a hard failure when rich expected exists. Duplicate
rule names disambiguated with _2/_3 suffix. v1 runtime errors fail
instead of silently skipping.

Phase 4: Add expected validation to LalComparisonTest for save, abort,
service, instance, endpoint, layer, tags, timestamp, and sampledTrace
fields. Fix enum comparison for reason/detectPoint fields.

All 1301 tests pass (1233 MAL + 35 LAL + 1 generator + 32 hierarchy).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PowerMock is a dead project. The only usage was powermock-reflect's Whitebox
class for setInternalState/getInternalState/invokeMethod — a thin wrapper
around java.lang.reflect. Replace with a project-owned ReflectUtil in the
server-testing module and add server-testing as test dependency to all 17
modules that previously relied on powermock-reflect.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…put data mock principles

- Copy vm.yaml (telegraf) and agent.yaml (zabbix) to test script directories
- Handle both 'metricsRules' and 'metrics' YAML keys (zabbix uses 'metrics')
- Handle numeric YAML keys (zabbix labels like '1', '2') via String.valueOf()
- Generate .data.yaml with proper label variants (e.g., cpu-total + cpu0 for tagEqual/tagNotEqual)
- Add CLAUDE.md documentation for input data mock principles in MAL, LAL, hierarchy modules
- Create CLAUDE.md for the checker test module

Total: 1268 MAL + 35 LAL + 31 filter = 1336 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… LAL rule

Cover all branches in the k8s-service.yaml network-profiling-slow-trace rule:
- Virtual local process (process_id empty, local=true)
- Virtual remote process (process_id empty, local=false)
- HTTP without SSL (componentId 49)
- TCP with SSL (componentId 130)
- Default component (componentId 110)
- LOG_KIND false path (no sampledTrace extraction)

Also align v1/v2 ProcessRegistry mock return values so v1-v2 comparison
works correctly for virtual process branches.

LAL tests: 35 → 39 (+4 new entries). Total: 1340 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ssRegistry dependencies

- Change inputData type from Map<String, Map> to Map<String, Object> to prevent
  ClassCastException when YAML values are Lists (multi-entry input data)
- Add instanceof List handling for multi-entry input data per rule
- Add @BeforeAll/@afterall mockStatic for K8sInfoRegistry and MetricsStreamProcessor
  (required by production ProcessRegistry for virtual process ID generation)
- Remove sampledTrace.processId/destProcessId from virtual process entries in
  envoy-als.input.data and k8s-service.input.data — values depend on ProcessRegistry
  implementation (mock vs production), validated via v1-v2 comparison in checker test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…data.yaml files

Add extractEntityFunctionLabels() to parse service/instance/endpoint/process function
arguments like instance(['host_name'], ['service_instance_id'], Layer.MYSQL) and ensure
these labels appear in all input samples. Without entity labels, scope/service/instance
extraction produces incorrect results.

Regenerated .data.yaml files for rules with entity function labels.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… PowerMock changelog

getInternalState() returns generic <T>, and when passed directly to setInternalState(),
Java resolves the overload to setInternalState(Class<?>, ...) instead of
setInternalState(Object, ...), causing ClassCastException. Fix by storing the result
in a local Object variable first.

Update changelog to reflect full PowerMock removal from all modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts:
#	docs/en/changes/changes.md
…th, regenerate expected data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wu-sheng
Copy link
Member Author

wu-sheng commented Mar 5, 2026

Gemini Code Review — V2 Engine Migration (Groovy Replacement)

I have completed a comprehensive review of the changes in the groovy-replace branch. The migration from Groovy-based V1 to the ANTLR4/Javassist-based V2 engine is architecturally sound and significantly improves the robustness of the OAP analyzer.

Key Technical Highlights:

  • Concurrency & Thread Safety: The removal of ThreadLocal in favor of explicit context passing (ExecutionContext, RunningContext) is a major improvement. The generated classes (MalExpr, LalExpr) are stateless and thread-safe, ensuring predictable behavior under high concurrency.
  • Feature Parity & Validation: Strong parity is verified via MalComparisonTest and LalComparisonTest. The latest fix to fail explicitly on V1 runtime errors during data generation ensures the integrity of the test expectations.
  • Performance Optimization: The transition to compiled bytecode (via LALClassGenerator and MALClassGenerator) eliminates the overhead of dynamic script evaluation, providing a more performant execution path for LAL, MAL, and Hierarchy rules.

The migration successfully modernizes the DSL engine while maintaining strict compatibility with existing rule configurations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

complexity:high Relate to multiple(>4) components of SkyWalking core feature Core and important feature. Sometimes, break backwards compatibility. feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants