Skip to content

Fix interpreter package scoping, array interpolation, and scalar context#199

Merged
fglock merged 8 commits intomasterfrom
feature/interpreter-array-operators
Feb 14, 2026
Merged

Fix interpreter package scoping, array interpolation, and scalar context#199
fglock merged 8 commits intomasterfrom
feature/interpreter-array-operators

Conversation

@fglock
Copy link
Owner

@fglock fglock commented Feb 14, 2026

Summary

This PR fixes three critical bugs in the interpreter that were preventing proper package scoping, array interpolation in strings, and scalar context handling.

1. Package Scoping for our Variables (a9edd88)

Problem: our @ISA = (...) in non-main packages created @main::ISA instead of @Obj2::ISA, breaking method inheritance.

Solution: Updated the "package" operator handler to extract the package name from the AST and update the currentPackage field, mirroring how the compiled backend uses the symbol table.

Impact:

  • ✅ Method inheritance via @ISA now works in all packages
  • ✅ Qualified access @Obj2::X now sees values from unqualified @X
  • ✅ Object tests fully pass (10/10 including inheritance tests)

2. Array Interpolation in Strings (47c4901)

Problem: Array interpolation like "@array" was converting arrays to scalar count instead of space-separated elements.

  • Example: my @x=(1,2,3); "@x" produced "3" instead of "1 2 3"

Root Cause: The parser converts "@array" into BinaryOperatorNode(join, $", @array). The interpreter compiled both operands in the current context (SCALAR), causing the array to be converted to its size instead of being expanded.

Solution: Added special handling for the "join" operator before general operand compilation, explicitly setting contexts:

  • Left operand (separator $"): SCALAR context
  • Right operand (array): LIST context

Impact:

  • ✅ Array interpolation in strings works correctly
  • ✅ Sort tests fully pass (5/5) - were failing because Test::More uses "@sorted"

3. Scalar Context Handling (29189e1)

Problem: scalar(@array) and keys %hash returned joined strings instead of counts.

  • Example: scalar(@x) where @x=(1,2,3) returned "123" instead of 3
  • Test::More is(keys %h, 0, "msg") received wrong argument types

Root Causes:

  1. scalar() operator set context but didn't emit ARRAY_SIZE opcode
  2. ListNode always created RuntimeList regardless of context
  3. Parser generates scalar(@x) as OperatorNode(scalar, ListNode(@x))
  4. ListNode wrapped array in list, then scalar() got size of wrapper (1) not array (3)

Solution:

  1. scalar() operator: Explicitly emit ARRAY_SIZE after evaluating operand
  2. ListNode in SCALAR context: Match compiled backend behavior
    • Evaluate all elements except last for side effects (discard results)
    • Return only the last element's value (not wrapped in RuntimeList)

Impact:

  • scalar(@array) returns array count
  • keys %hash in scalar context returns correct count
  • ✅ Test::More argument handling works correctly
  • ✅ All demo.t tests now pass (9/9)

Changes

  • BytecodeCompiler.java: Package operator now updates currentPackage
  • BytecodeCompiler.java: Special handling for "join" operator contexts
  • BytecodeCompiler.java: Fixed scalar() operator to emit ARRAY_SIZE
  • BytecodeCompiler.java: Fixed ListNode to handle scalar context properly
  • BytecodeInterpreter.java: Added CALL_METHOD opcode support
  • InterpretedCode.java: Added disassembler entries for global arrays/hashes

Testing

make test-unit          # All pass ✅
./jperl --interpreter src/test/resources/unit/demo.t
  ok 1 - Variable assignment and modification
  ok 2 - List assignment in scalar context
  ok 3 - List assignment with lvalue array and hash
  ok 4 - Basic syntax tests
  ok 5 - Splice tests
  ok 6 - Map tests
  ok 7 - Grep tests
  ok 8 - Sort tests
  ok 9 - Object tests
  1..9                  # All tests pass ✅

Specific test cases verified:

  • our @ISA = ("Parent") in non-main packages ✅
  • Method inheritance via @ISA
  • Array interpolation: "@array"
  • scalar(@array) returns count ✅
  • keys %hash in scalar context ✅
  • Test::More is() with array/hash operations ✅

🤖 Generated with Claude Code

fglock and others added 8 commits February 14, 2026 10:59
Two minor test failures:
- Splice scalar context returns list instead of last element
- Sort without block doesn't apply default cmp comparison

Plus done_testing() framework error to investigate.

Overall: 50+ tests passing, excellent progress.
- Added scalar auto-conversion to all arithmetic operators (ADD, SUB, MUL, DIV, MOD, POW)
- Fixed package-qualified variable access ($main::a) to remove sigil before lookup
- This fixes sort without block, which uses auto-generated { $main::a cmp $main::b }
- Now matches codegen behavior: GlobalVariable.getGlobalVariable("main::a")
- Updated remaining issues: 8/9 subtests passing, sort now fully working

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add note about switching to interpreter for "Method too large" errors
- Add note about eval-STRING optimization with interpreter mode
- Update GraalVM docs path and note interpreter mode compatibility
- Interpreter can compile faster for dynamic code scenarios

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Added iterator support to bytecode interpreter, eliminating range
materialization and matching compiler's efficient approach.

New opcodes (106-108):
- ITERATOR_CREATE: rd = rs.iterator()
- ITERATOR_HAS_NEXT: rd = iterator.hasNext()
- ITERATOR_NEXT: rd = iterator.next()

Performance results (50M element range loop):
- Before: 2.74 seconds (5.1x slower than Perl 5)
- After: 1.02 seconds (1.9x slower than Perl 5)
- Speedup: 2.68x faster

Root cause fixed:
- OLD: For1Node converted ranges to arrays, materializing all elements
- NEW: For1Node uses iterator pattern, one element at a time
- Memory: O(N) → O(1) for range-based loops

All demo.t tests still pass (8/9 subtests).

Files modified:
- Opcodes.java: Added iterator opcodes
- BytecodeInterpreter.java: Implemented iterator operations
- BytecodeCompiler.java: Rewrote For1Node to use iterators
- InterpretedCode.java: Added disassembler support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
In Perl, splice in scalar context returns the last element removed,
not the entire list. Added context tracking to SLOWOP_SPLICE.

Changes:
- BytecodeCompiler: Emit currentCallContext after SLOWOP_SPLICE args
- SlowOpcodeHandler: Read context and return last element in SCALAR context
- Returns undef if no elements were removed (empty list)
- InterpretedCode: Update disassembler to show context parameter

Test results:
- Before: splice returned '97' (concatenated list)
- After: splice returns '7' (last element)
- ALL 9 demo.t subtests now pass! (60+ individual tests)

Fixes demo.t line 185: is("$a", "7", "splice removed elements")

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit fixes multiple interpreter issues related to package handling:

1. **Package operator now updates currentPackage** (BytecodeCompiler.java:3272)
   - Extract package name from IdentifierNode operand
   - Update currentPackage field for subsequent variable declarations
   - Fixes `our @isa = (...)` in non-main packages for inheritance

2. **Add CALL_METHOD opcode support** (BytecodeInterpreter.java)
   - Implement method call dispatch via RuntimeCode.call()
   - Support method resolution with @isa inheritance
   - Handle method calls in scalar/list context

3. **Add disassembler support for array/hash globals** (InterpretedCode.java)
   - LOAD_GLOBAL_ARRAY: displays `@Package::var`
   - LOAD_GLOBAL_HASH: displays `%Package::var`
   - CALL_METHOD: displays method call details

4. **Fix scalar context for our declarations** (BytecodeCompiler.java:663)
   - Treat `our $x = expr` like `my $x = expr` for RHS context
   - Use SET_SCALAR instead of MOVE to preserve aliasing

5. **Add scalar() operator support** (BytecodeCompiler.java:2959)
   - Force scalar context evaluation
   - Required for scalar(@array) and similar expressions

**Impact:**
- Method inheritance via @isa now works in interpreter mode
- `our @X` in package Obj creates `@Obj::X`, not `@main::X`
- Object tests in demo.t now fully pass (10/10)

**Testing:**
- All unit tests pass (make test-unit)
- Object tests fully pass including inheritance
- Verified with disassembly output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
**Problem:**
Array interpolation in strings like `"@array"` was converting arrays
to scalar count instead of expanding elements. For example:
  my @x = (1,2,3); my $s = "@x"  # Got "3", expected "1 2 3"

**Root Cause:**
The parser converts `"@array"` into BinaryOperatorNode(join, $", @array).
The interpreter's BinaryOperatorNode visitor compiled both operands in
the current context before checking which operator it was. Since the
assignment `my $s = "@x"` is scalar context, the `@x` operand was
evaluated in SCALAR context, which emits ARRAY_SIZE instead of returning
the array itself.

**Solution:**
Handle "join" operator specially before general operand compilation,
explicitly setting contexts as the compiler backend does:
- Left operand (separator): SCALAR context
- Right operand (array/list): LIST context

This ensures arrays are expanded (list context) rather than converted
to their size (scalar context).

**Changes:**
- BytecodeCompiler.java:1839-1861: Add special handling for "join"
- BytecodeCompiler.java:1995-2004: Remove duplicate "join" case from switch

**Impact:**
- Array interpolation in strings now works correctly
- Sort tests now fully pass (were failing due to Test::More using "@sorted")
- Test 8 "Sort tests" in demo.t: 5/5 pass (was 0/5)

**Testing:**
- All unit tests pass (make test-unit)
- Verified disassembly shows correct bytecode:
  Before: ARRAY_SIZE r10 = size(r3); JOIN r11 = join(r9, r10)
  After:  JOIN r10 = join(r9, r3)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
**Problem:**
The `scalar()` operator and ListNode in scalar context were incorrectly
converting arrays/lists to joined strings instead of counts:
  my @x = (1,2,3); scalar(@x)  # Got "123", expected 3
  keys %hash  # in Test::More is() got wrong argument order

**Root Causes:**
1. scalar() operator set context but didn't emit ARRAY_SIZE opcode
2. ListNode always created RuntimeList regardless of context
3. Parser generates scalar(@x) as OperatorNode(scalar, ListNode(@x))
4. ListNode wrapped array in list, then scalar() got size of wrapper (1)

**Solution:**
1. **scalar() operator**: Explicitly emit ARRAY_SIZE after evaluating operand
   - Converts arrays/hashes to size, passes through scalars

2. **ListNode in SCALAR context**: Match compiled backend behavior
   - Evaluate all elements except last for side effects (discard results)
   - Return only the last element's value
   - Do NOT wrap in RuntimeList
   - This allows scalar(@x) to properly get @x's size (3) not wrapper size (1)

3. **ListNode in LIST context**: Keep existing behavior
   - Create RuntimeList with all elements
   - Evaluate elements in LIST context for proper flattening

**Changes:**
- BytecodeCompiler.java:2975-3000: Fix scalar() to emit ARRAY_SIZE
- BytecodeCompiler.java:5128-5170: Add scalar context handling to ListNode

**Impact:**
- scalar(@array) now returns array count (3) not joined elements ("123")
- keys %hash in scalar context returns correct count
- Test::More is() receives correct argument types/order
- All demo.t tests now pass (9/9)

**Testing:**
- scalar(@x) where @x=(1,2,3) returns 3 ✓
- keys %empty_hash returns 0 ✓
- Test::More is(keys %h, 0, "msg") works correctly ✓
- All unit tests pass (make test-unit) ✓

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@fglock fglock merged commit 8592d79 into master Feb 14, 2026
2 checks passed
@fglock fglock deleted the feature/interpreter-array-operators branch February 14, 2026 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant