Skip to content

staged changes#472

Closed
junrushao wants to merge 14 commits intoapache:mainfrom
junrushao:main-staged
Closed

staged changes#472
junrushao wants to merge 14 commits intoapache:mainfrom
junrushao:main-staged

Conversation

@junrushao
Copy link
Member

DO NOT MERGE

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @junrushao, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the FFI object system by introducing robust recursive comparison and hashing capabilities, which are crucial for deep equality checks and deterministic hashing across complex, potentially cyclic, data structures. It also refines the Python binding layer by optimizing object memory usage with __slots__, enforcing proper object construction, and improving the flexibility of tvm_ffi.Function initialization. Additionally, the changes include support for a new hardware device type and minor but important fixes to object representation for better debugging and consistency. This is a staged set of changes and should not be merged yet.

Highlights

  • Recursive Comparison and Hashing: Introduced new C++ implementations for deep equality, ordering comparisons (RecursiveEq, RecursiveLt, etc.), and deterministic hashing (RecursiveHash) for FFI objects, supporting primitives, containers, and reflected objects with cycle detection and NaN handling.
  • Reflection Flags for Comparison/Hashing: Added new FFI field flags and corresponding refl::Compare(false) and refl::Hash(false) traits to allow specific object fields to be excluded from recursive comparison and hashing.
  • Python Object System Refinements: Refactored the Python FFI object hierarchy by introducing CObject as a Cython base class, making Object inherit from it with __slots__ for efficiency, and disabling direct instantiation of base FFI objects. Also added an __init__ method to tvm_ffi.Function for direct wrapping of Python callables.
  • Deep Copy Cycle Handling: Enhanced the ObjectDeepCopier to correctly manage cyclic references within immutable containers (like Array and Map) that contain mutable children (List and Dict), ensuring proper deep copying.
  • New Device Type and Improved Repr: Added kDLMAIA as a new device type and improved the ReprPrint utility to correctly escape special characters in strings and represent the 'trn' device name accurately.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • CMakeLists.txt
    • Added new source files for recursive comparison and hashing to the build configuration.
  • docs/conf.py
    • Updated Sphinx documentation generation to correctly handle the CObject and Object class renaming.
    • Added new FFI methods to be hidden from the generated documentation.
  • include/tvm/ffi/c_api.h
    • Defined new bitmask flags (kTVMFFIFieldFlagBitMaskCompareOff, kTVMFFIFieldFlagBitMaskHashOff) to control field participation in recursive comparison and hashing.
  • include/tvm/ffi/reflection/registry.h
    • Added refl::Compare and refl::Hash InfoTrait classes to apply the new comparison and hashing flags to field information.
  • python/tvm_ffi/_ffi_api.py
    • Exposed new FFI functions for recursive comparison (RecursiveEq, RecursiveGe, RecursiveGt, RecursiveLe, RecursiveLt) and hashing (RecursiveHash).
  • python/tvm_ffi/access_path.py
    • Removed a redundant super().__init__() call in AccessPath.__init__.
  • python/tvm_ffi/core.pyi
    • Renamed the Object class to CObject and introduced a new Object class inheriting from CObject for type hinting.
    • Added an __init__ method to the Function class for direct callable wrapping.
  • python/tvm_ffi/cython/base.pxi
    • Added __slots__ = () to ByteArrayArg for memory efficiency.
  • python/tvm_ffi/cython/device.pxi
    • Added kDLMAIA to the DLDeviceType enum and updated device type mappings.
    • Added __slots__ = () to the Device class for memory efficiency.
  • python/tvm_ffi/cython/dtype.pxi
    • Added __slots__ = () to the DataType class for memory efficiency.
  • python/tvm_ffi/cython/error.pxi
    • Changed the base class of Error from Object to CObject.
    • Added __slots__ = () to the Error class.
    • Updated chandle casting to CObject in error handling logic.
  • python/tvm_ffi/cython/function.pxi
    • Updated type casting from Object to CObject across various FFI argument setters and function creation.
    • Added an __init__ method to the Function class to allow direct wrapping of Python callables.
  • python/tvm_ffi/cython/object.pxi
    • Refactored the Object class into CObject (Cython base) and Object (Python base with __slots__ and metaclass).
    • Disabled direct instantiation of CObject to enforce FFI constructors.
    • Updated internal object handling to reflect the CObject and Object split.
  • python/tvm_ffi/cython/string.pxi
    • Updated _string_obj_get_py_str and _bytes_obj_get_py_bytes to cast to CObject.
  • python/tvm_ffi/cython/tensor.pxi
    • Updated _shape_obj_get_py_tuple and the Tensor base class to CObject.
    • Added __slots__ = () to Tensor and DLTensorTestWrapper for memory efficiency.
  • python/tvm_ffi/cython/type_info.pxi
    • Updated FieldGetter and FieldSetter to accept CObject for field access.
  • python/tvm_ffi/module.py
    • Added _tvm_ffi_attr_cache to Module.__slots__.
    • Improved __getattr__ caching logic for module functions.
  • python/tvm_ffi/registry.py
    • Included ffi.Shape in the list of container types for copy methods.
  • python/tvm_ffi/testing/init.py
    • Imported new testing classes TestCompare and TestHash.
  • python/tvm_ffi/testing/testing.py
    • Added __test__ = False to TestIntPair.
    • Defined new TestCompare and TestHash object classes with specific fields marked for comparison/hashing exclusion.
  • src/ffi/extra/deep_copy.cc
    • Added unordered_set for in_progress_ tracking to manage objects currently being copied.
    • Implemented FixupDeferredReferences to handle cyclic references involving immutable containers and mutable children during deep copy.
  • src/ffi/extra/recursive_compare.cc
    • Added new file implementing RecursiveComparer class and FFI functions for deep equality and ordering comparisons, including NaN handling and cycle detection.
  • src/ffi/extra/recursive_hash.cc
    • Added new file implementing RecursiveHasher class and FFI function for deterministic recursive hashing, including NaN handling, cycle detection, and order-independent map hashing.
  • src/ffi/extra/repr_print.cc
    • Updated ReprPrinter to correctly escape strings and prevent duplicate field names in generic object representation.
    • Ensured correct representation of the 'trn' device name.
  • src/ffi/testing/testing.cc
    • Defined C++ classes TestCompareObj and TestHashObj and registered their FFI object info, including refl::Compare(false) and refl::Hash(false) for specific fields.
  • tests/python/test_copy.py
    • Added new test cases for deep copy cycle preservation with immutable root containers.
    • Added tests for Python deepcopy consistency for Shape objects.
  • tests/python/test_function.py
    • Added new test cases for tvm_ffi.Function initialization from Python callables and existing Function objects.
    • Included error handling tests for moved-from function objects and non-callable inputs.
  • tests/python/test_object.py
    • Added new test cases to verify that direct instantiation of Object and unregistered subclasses is disabled.
    • Added tests to check __slots__ behavior for object subclasses.
  • tests/python/test_recursive_compare.py
    • Added new file with extensive unit tests for RecursiveEq, RecursiveLt, RecursiveLe, RecursiveGt, RecursiveGe covering primitives, containers, reflected objects, NaN handling, type mismatches, CompareOff flag, and cycle detection.
  • tests/python/test_recursive_hash.py
    • Added new file with extensive unit tests for RecursiveHash covering primitives, containers, reflected objects, NaN handling, HashOff flag, order independence for maps, cycle detection, and consistency with RecursiveEq.
  • tests/python/test_repr.py
    • Added new test cases for string literal round-tripping with special characters.
    • Added tests for container literal round-tripping with special strings.
    • Verified correct representation of the 'trn' device name.
    • Ensured inherited fields do not appear twice in generic repr for unregistered objects.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request introduces recursive comparison and hashing for FFI objects, improves deep copy to handle cycles involving immutable containers, and enhances the Python API with better slots enforcement and function caching. The additions are well-tested and follow the existing architecture. I have identified a few issues in the deep copy fixup logic (specifically missing key fixups in Dicts and potential Copy-On-Write issues) and some potential null pointer dereferences in the recursive comparison/hashing implementations.

junrushao and others added 13 commits February 26, 2026 11:51
Enable creating `tvm_ffi.Function` directly from a Python callable or
an existing Function instance via its constructor, e.g.
`tvm_ffi.Function(add)`.
Previously, calling `Object()` or constructing an unregistered subclass
without `__init__` would silently succeed and produce an object with a
NULL handle. Add `Object.__init__` that raises TypeError to catch this
misuse early. Remove the now-redundant `super().__init__()` call in
AccessPath.__init__.
…s, deduplicate fields (#31)

- Replace custom FormatString with EscapeString from string.h for proper
  string escaping in repr output (handles \b, \f, \uXXXX for control chars)
- Add missing kDLMAIA and kDLTrn entries to DeviceTypeName() in repr_print.cc
- Fix DLDeviceType enum in Python device.pxi: add kDLMAIA=17, correct kDLTrn=18
- Deduplicate inherited fields in GenericRepr using seen_names set to prevent
  ForEachFieldInfo from emitting parent fields redeclared in child classes
- Add tests for string escaping, device repr, and duplicate field suppression
…eepcopy (#32)

- Add in_progress_ tracking for Array/Map in ObjectDeepCopier to detect
  back-references during construction of immutable containers
- When Resolve() encounters an in-progress object, return original as
  placeholder and defer fixup until all copies are built
- FixupDeferredReferences() scans copied List/Dict containers, replacing
  stale original references with their copies from copy_map_
- Add "ffi.Shape" to deepcopy-supported types in registry.py (Shape is
  immutable, ffi.DeepCopy returns it as-is)
- Add tests for cycle preservation with Array/Map roots through List/Dict
  children, Map key cycles, shared identity in cycles, and Shape deepcopy
Add RecursiveEq/Ne/Lt/Le/Gt/Ge functions that perform deep structural
comparison of FFI values using the reflection system.

Key design decisions:
- Three-way comparison (CompareAny returns -1/0/+1) with eq_only_ mode
  that avoids unnecessary ordering on unorderable types
- Pointer identity fast path: same-object comparisons return 0 immediately
- Compare(false) field flag to exclude fields from comparison
- Type index comparison as tiebreaker for cross-type ordering
- Maps/Dicts: equality checks key-by-key; ordering raises TypeError for
  unequal maps (no natural total order on maps)
- Recursion depth guard (kMaxDepth=128) to detect cyclic structures
- Safe int64 comparison using explicit operators (no subtraction overflow)

Implementation:
- src/ffi/extra/recursive_compare.cc: core comparison engine
- include/tvm/ffi/c_api.h: kTVMFFIFieldFlagBitMaskCompareOff flag
- include/tvm/ffi/reflection/registry.h: Compare() attribute builder
- CMakeLists.txt: add recursive_compare.cc to build
- Python API: RecursiveEq/Ne/Lt/Le/Gt/Ge in _ffi_api.py
- Testing: TestCompare object with Compare(false) field, 91 test cases
  covering primitives, containers, objects, edge cases, and cycles
…in wheel builds (#34)

The _run_recursive_compare_subprocess helper set PYTHONPATH to the source
tree, which in cibuildwheel environments resolves to uncompiled sources
(missing core.cpython-*.so). This caused `from . import core` in
registry.py to fail with a circular import error.

Replace the two subprocess-based tests (test_cyclic_list_raises,
test_cyclic_dict_raises) with direct in-process pytest.raises equivalents.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add `ffi.RecursiveHash` — the hash companion to `RecursiveEq`. This enables
using FFI objects as dictionary keys and in sets via consistent hashing.

Key design decisions:
- Consistent with RecursiveEq: equal objects always produce equal hashes
- Stack-based cycle detection with RAII guard (on_stack_ set)
- Memoization of computed object hashes to avoid exponential DAG traversal
- Sort-based order-independent map/dict hashing with full avalanche mixing
- Index-mixed sequence hashing for better collision resistance
- NaN/signed-zero canonicalization matching RecursiveEq semantics
- New `kTVMFFIFieldFlagBitMaskHashOff` flag and `Hash` InfoTrait for
  per-field opt-out; fields with CompareOff are also excluded from hash
#36)

## Summary

- Add C++ `Init(true/false)` and `KwOnly(true/false)` per-field InfoTraits with C ABI field-flag bitmasks: `InitOff` (bit 9), `KwOnly` (bit 10), `HasDefault` (bit 1)
- Auto-generate packed `__ffi_init__` when `ObjectDef` has no explicit `refl::init<Args...>()`
- Pre-compute field analysis in `AutoInitInfo` struct (shared via `shared_ptr`), eliminating per-call `ForEachFieldInfo` traversal; implementation in `src/ffi/auto_init.cc`, declaration-only header
- KWARGS sentinel resolved eagerly via direct C++ call (`ffi::GetKwargsObject` through `object_internal.h`), eliminating `std::call_once` and global function registry lookup
- Keyword arg matching uses `unordered_map<string_view, size_t>` for O(1) average lookup instead of O(n) linear scan over `init_indices`
- Type key and field names stored as `string_view` into stable reflection data, avoiding repeated `TypeIndexToTypeKey` calls
- Python `TypeField` exposes `c_init`, `c_kw_only`, `c_has_default` directly from C ABI bitmasks
- Python `__init__` is a trivial adapter (`_make_init` + `_make_init_signature`): packs `*args` + KWARGS sentinel + `**kwargs`, delegates all validation to C++
- C++ tests use `std::exception` catch for RTTI compatibility across `-fvisibility=hidden` shared library boundary

## Test plan

- [x] 56 Python tests in `test_init_kw_only.py` (metadata bitmask, signatures, construction, errors, inheritance, low-level FFI)
- [x] 12 C++ tests in `test_reflection.cc` (auto-init positional, KWARGS, defaults, error paths)
- [x] All 807 Python tests pass (22 skipped, 1 xfailed)
- [x] All 342 C++ tests pass
- [x] All pre-commit hooks pass
- [x] clang-tidy clean (exit 0)
…mpare__ hooks (#37)

Replace system-stack recursion in RecursiveHash and RecursiveCompare with
explicit heap-allocated frame stacks, and add per-type customization hooks
following the established __ffi_repr__ pattern.

Architecture:
- RecursiveHasher and RecursiveComparer use std::vector<Frame> instead of
  system recursion, raising the effective depth limit from 128 to 1M frames
  and eliminating stack overflow risk on deep object graphs.
- Type attributes __ffi_hash__, __ffi_eq__, __ffi_compare__ registered via
  TypeAttrDef<T> and looked up through TypeAttrColumn at dispatch time.
- Hook callbacks receive a recursive helper (fn_hash / fn_eq / fn_cmp) that
  isolates the explicit stack via swap-save-restore while sharing memo and
  cycle-detection state across nested invocations.
- The three hooks form a consistency contract:

    RecursiveEq(a, b)  =>  RecursiveHash(a) == RecursiveHash(b)

  This invariant is enforced at two levels:
  1. RecursiveHash raises ValueError if a reflected type registers
     __ffi_eq__ or __ffi_compare__ without a corresponding __ffi_hash__,
     because custom equality that ignores fields would make the default
     field-by-field hash inconsistent.
  2. RecursiveCompare raises ValueError on cyclic object graphs instead of
     treating cycles as equal, because the hasher cannot produce matching
     hashes for structurally different cyclic topologies.

- Ordering dispatch hierarchy in RecursiveCompare:
  - eq_only mode: prefers __ffi_eq__, falls back to __ffi_compare__, then
    reflection.
  - ordering mode (Lt/Le/Gt/Ge): uses __ffi_compare__ only; if absent,
    falls back to field-by-field reflection. An __ffi_eq__-only type will
    NOT have its eq semantics carried into ordering — register
    __ffi_compare__ for consistent ordering behavior.

Public Interfaces:
- New type_attr constants in reflection::type_attr namespace:
  kHash ("__ffi_hash__"), kEq ("__ffi_eq__"), kCompare ("__ffi_compare__").
- Hook signatures:
  __ffi_hash__:    (Object*, Function fn_hash) -> int64_t
  __ffi_eq__:      (Object*, Object*, Function fn_eq) -> bool
  __ffi_compare__: (Object*, Object*, Function fn_cmp) -> int32_t
- New test fixtures: TestCustomHash (__ffi_hash__ only),
  TestCustomCompare (__ffi_hash__ + __ffi_eq__ + __ffi_compare__),
  TestEqWithoutHash (__ffi_eq__ only, no __ffi_hash__ — exercises guard).

UI/UX:
- none

Behavioral Changes:
- Deep object graphs (depth > 128) no longer raise in RecursiveHash or
  RecursiveCompare; they succeed up to 1M heap-allocated frames.
- Cyclic structures now raise ValueError in RecursiveEq/Compare instead
  of the previous behavior (ValueError in hash, implicit equal in compare).
- Types registering __ffi_eq__ or __ffi_compare__ without __ffi_hash__
  now raise ValueError when hashed, instead of silently producing
  inconsistent hashes.
- All existing non-boundary semantics preserved: POD, String/Bytes
  cross-variant, NaN, signed-zero, map order-independence, memoization.

Docs:
- No doc updates; hook patterns mirror __ffi_repr__ (already documented).
  Gap: docs/concepts/ should describe the new hooks and the Eq=>Hash
  consistency contract.

Tests:
- Executed: uv run pytest tests/python/ → 961 passed, 22 skipped, 1 xfailed
- Executed: uv run pre-commit run --all-files → all 27 hooks passed
- New tests cover: custom hook dispatch (hash/eq/compare), Eq=>Hash
  consistency across parametrized keys and container wrappers, eq-without-
  hash guard (direct and nested), cyclic structure rejection, eq-only vs
  ordering divergence, and ordering consistency for __ffi_compare__ types.

Untested Edge Cases:
- Custom hooks creating cyclic callback chains (low risk: on_stack_
  tracking shared across hook invocations prevents infinite loops).
- Extremely deep hook nesting (hook A triggers hook B via fn_hash);
  bounded by kMaxStackDepth per save/restore isolation level.

BREAKING CHANGE: RecursiveEq on two distinct cyclic Lists/Dicts now raises
ValueError instead of returning True. RecursiveHash on types with
__ffi_eq__/__ffi_compare__ but no __ffi_hash__ now raises ValueError.
…ase InfoTraits (#38)

Architecture:
- Replace four separate C++ source files (deep_copy.cc, repr_print.cc,
  recursive_hash.cc, recursive_compare.cc) with a single dataclass.cc
  built on a shared CRTP base class ObjectGraphDFS<Derived, FrameT, ResultT>.
- Move auto_init.cc logic into object.cc behind a new C API function
  TVMFFITypeInfoAddInit(type_index). AutoRegisterInit in registry.h now
  delegates to this C API, eliminating the need for auto_init.h/.cc.
- Move GetMissingObject/GetKwargsObject definitions and their global
  function registrations from container.cc to object.cc, centralizing
  sentinel object management.
- Unify refl::Init (InfoTrait) and refl::init<Args...> (constructor
  helper) into a single init template with an explicit init<>
  specialization that inherits InfoTrait. CTAD deduction guide
  init(bool)->init<> enables init(false) syntax.
- Rename reflection InfoTrait classes to lowercase Python-style
  identifiers: Repr->repr, KwOnly->kw_only, Compare->compare,
  Hash->hash. DefaultValue and DefaultFactory retain their names for
  backward compatibility; lowercase subclass aliases default_ and
  default_factory are added and preferred in new code.
- Fix use-after-forward in ObjectDef constructor: MaybeSuppressAutoInit
  now runs before RegisterExtraInfo to avoid reading forwarded args.

Public Interfaces:
- Header removed: include/tvm/ffi/extra/deep_copy.h replaced by
  include/tvm/ffi/extra/dataclass.h with expanded API surface.
- Header removed: include/tvm/ffi/reflection/auto_init.h (internal only).
- C API addition: TVMFFITypeInfoAddInit(int32_t type_index) in c_api.h.
- Global function rename: ffi.MakeAutoInit -> ffi.MakeInit.
- C++ API: ReprPrint, RecursiveHash, RecursiveEq, RecursiveLt/Le/Gt/Ge
  now have public C++ declarations in dataclass.h.
- InfoTrait renames (all in tvm::ffi::reflection namespace):
  Init -> init<> (via CTAD), Repr -> repr, KwOnly -> kw_only,
  Compare -> compare, Hash -> hash.
- New lowercase aliases (backward-compatible, preferred in new code):
  default_ (subclass of DefaultValue), default_factory (subclass of
  DefaultFactory).

UI/UX:
- none

Behavioral Changes:
- Built-in __ffi_repr__ type attribute hooks for String, Bytes, Tensor,
  Shape, Array, List, Map, Dict are no longer registered. Repr for these
  types is handled inline. Custom __ffi_repr__ hooks on containers still
  work; hooks on String/Bytes/Tensor/Shape are effectively bypassed.
- FixupDeferredReferences now also fixes up reflected object fields
  (FixupObject) and Dict keys, improving correctness for cycles through
  immutable containers.
- AutoRegisterInit now calls TVMFFITypeInfoAddInit C API instead of
  directly invoking MakeAutoInit; MakeInit is TU-local in object.cc.

Bugfixes included in this refactor:
- Fix use-after-move in CRTP RunLoop: FeedChild(f, std::move(r)) followed
  by OnTerminate(std::move(r)) read a moved-from value.
- Fix FixupDict to fix up both keys and values.
- Remove dead kTVMFFIStr case in RecursiveComparer::PushPairFrame.
- Remove fragile last_result_ coupling in ReprPrinter.
- Add missing <cstring> include for std::memcpy.
- Fix use-after-forward in ObjectDef constructor (clang-tidy
  bugprone-use-after-move).
- Fix Doxygen @param mismatch for kw_only constructor.

Doc-build fixes:
- Exclude default_ and default_factory from Doxygen EXCLUDE_SYMBOLS to
  prevent exhale generating RST files where trailing underscores are
  misinterpreted as RST hyperlink references.
- Replace :cpp:class:`tvm::ffi::reflection::init` with ``init<Args...>()``
  in export_func_cls.rst to avoid Sphinx warning where the CTAD deduction
  guide causes Doxygen to register init as both struct and function.

Docs:
- New guide: docs/guides/dataclass_reflection.rst — comprehensive tutorial
  covering auto-init, field traits (init, kw_only, default_, repr, hash,
  compare), dataclass operations (deep copy, repr, hash, comparison),
  custom hooks via TypeAttrDef, Python c_class decorator, and inheritance.
- docs/concepts/object_and_class.rst updated with refl::default_().
- DeepCopy() docstring corrected: Shape listed as immutable leaf,
  Dict listed as deep-copied container.
- docs/index.rst toctree updated to include new guide.
- docs/conf.py Doxygen EXCLUDE_SYMBOLS updated.

Tests:
- Executed: uv run pytest -vvs tests/python (970 passed, 22 skipped, 1 xfailed)
- C++ library build: cmake --build (clean)
- Pre-commit: all 27 hooks pass (including rstcheck on new guide).
- C++ unit tests not run (GoogleTest FetchContent fails in worktree; same
  code compiles via Cython wheel build which exercises all paths).

Untested Edge Cases:
- Deep copy of Tensor inside a container (matches old behavior but
  lacks explicit test coverage).
- CTAD edge case: init<bool> vs init(bool) — deduction guide ensures
  init(bool) always resolves to init<>, but no C++ unit test for this
  specific CTAD path (covered by Python tests that exercise init(false)
  from C++ testing objects).
- Sphinx doc build not run locally (requires BUILD_CPP_DOCS=1 and Doxygen);
  EXCLUDE_SYMBOLS fix validated by reading exhale/Doxygen config logic.

BREAKING CHANGE: include/tvm/ffi/extra/deep_copy.h removed; use
include/tvm/ffi/extra/dataclass.h. include/tvm/ffi/reflection/auto_init.h
removed. InfoTrait renames in tvm::ffi::reflection: Init->init,
Repr->repr, KwOnly->kw_only, Compare->compare, Hash->hash.
DefaultValue/DefaultFactory unchanged; prefer default_/default_factory.
Migration: update refl::Init(false) to refl::init(false), etc.
…th opt-in flags (#39)

## Summary

- Rewrite `@c_class` decorator as a thin wrapper around `register_object` + `_install_dataclass_dunders` with opt-in `init`, `eq`, `order`, `unsafe_hash`, `slots` flags
- Move `__init__` installation from `register_object` into `c_class(init=True)` — `register_object` no longer auto-attaches `__init__`
- Unified `_install_init(cls, enabled=bool)`: installs real `__init__` when enabled and C++ `__ffi_init__` exists, or a `TypeError` guard with actionable message otherwise
- `_ObjectSlotsMeta` accepts `slots=False` kwarg — `class Foo(Object, slots=False):` opts out of `__slots__` injection
- `c_class(slots=True)` validates consistency between decorator and metaclass `slots` kwarg; mismatch raises `TypeError`
- Migrate all 22 test classes in `testing.py` from `@register_object` to `@c_class` with stubgen marker blocks
- Extend `tvm-ffi-stubgen` to generate `__init__` method stubs from C++ reflection metadata (both auto-init and non-auto-init types)
- Migrate C++ test types from manual `refl::init<>()` to auto-init with per-field traits
- Suppress auto-init for `AccessStepObj`/`AccessPathObj` via `refl::init(false)` (factory-method types)
- Simplify `_deepcopy_supported` to import `_ffi_api.DeepCopy` directly, removing `functools.lru_cache` wrapper
- Fix pre-existing stubgen crash on auto-init types whose `type_schema` metadata was a dict instead of a JSON string

## Architecture

**c_class simplification**: `c_class(type_key, *, init=True, eq=True, order=False, unsafe_hash=False, slots=True)` now calls `register_object(type_key)` then `_install_dataclass_dunders(cls, ...)`. Deleted `field.py` (169 LOC), `_utils.py` (210 LOC).

**init flag with unified _install_init**: `_install_init(cls, enabled=bool)` reads `__tvm_ffi_type_info__` to detect `__ffi_init__` presence and `auto_init` metadata. When `enabled=True` it installs the appropriate `__init__`; when `enabled=True` but no `__ffi_init__` exists, or when `enabled=False`, it installs a guard raising `TypeError` with actionable guidance (suggesting `refl::init()` or a factory method). Called unconditionally by `_install_dataclass_dunders`.

**Opt-in dunder flags**: `_install_dataclass_dunders` moved to `registry.py` with four flags:
- `init=True` (default) → installs `__init__` from C++ reflection metadata
- `eq=True` (default) → installs `__eq__`, `__ne__`
- `order=False` → installs `__lt__`, `__le__`, `__gt__`, `__ge__` when True
- `unsafe_hash=False` → installs `__hash__` when True

**Slots control**: `_ObjectSlotsMeta` accepts `*, slots: bool = True` in both `__new__` and `__init__`. When `slots=False`, the metaclass skips `__slots__ = ()` injection. The `__init__` override consumes the kwarg for Python 3.14 compatibility. `c_class(slots=True)` validates that the class header matches — mismatch raises `TypeError`.

`@dataclass_transform(eq_default=True, order_default=False)` makes type checkers aware of the defaults per PEP 681.

**Stubgen `__init__` generation**: `ObjectInfo` walks the parent chain to collect init-eligible fields. Two codepaths: `_gen_auto_init` (named params, `*` kw_only separator, defaults) and `_gen_c_init` (positional-only from `__c_ffi_init__`).

## Test plan

- [x] `uv run pytest -vvs tests/python/` — 148 tests in touched files pass
- [x] `uv run pre-commit run --all-files` — all 27 hooks pass
- [x] 4 new slots validation tests (default ok, slots=False ok, true mismatch, false mismatch)
- [ ] C++ tests (`ctest`) — not re-run (no header/API changes)
- [ ] Rust tests — not re-run (no Rust-visible changes)

## Breaking changes

- `c_class` no longer accepts old `init`/`kw_only` parameters (Python-side field metadata)
- `field()`, `Field`, `KW_ONLY`, `MISSING` removed from `tvm_ffi.dataclasses`
- `c_class`-decorated classes must explicitly inherit from `Object`
- `c_class` no longer installs `__hash__` or ordering operators by default — pass `unsafe_hash=True` and/or `order=True` to opt in
- `register_object` no longer installs `__init__` — use `c_class` (which defaults to `init=True`) for automatic `__init__` from C++ reflection
- `c_class(init=True)` without C++ `__ffi_init__` now raises `TypeError` (previously silently inherited `object.__init__`)
- `c_class(init=False)` installs a guard raising `TypeError` with "cannot be constructed directly"

**Migration**: Move field defaults, init exclusions, and kw_only markers from Python `field()` calls to C++ `ObjectDef` per-field traits; add `Object` as explicit base class; add `unsafe_hash=True`/`order=True` where hash/ordering behavior was expected; switch from `@register_object` to `@c_class` if auto `__init__` is needed. Use `class Foo(Object, slots=False):` to opt out of `__slots__` injection.
…n in c_class (#40)

## Summary

- Consolidate `__object_repr__` into a public `object_repr()` function that uses `_ffi_api.ReprPrint` directly, replacing the hand-rolled lazy-caching pattern with `_get_global_func`.
- `CObject.__repr__` now delegates to `object_repr(self)` — single source of truth for repr logic.
- Remove four redundant `__repr__` overrides from `Array`, `List`, `Map`, `Dict` in `container.py` that duplicated inherited `CObject.__repr__`.
- Add `repr=True` flag to `@c_class`: when enabled (default) and no explicit `__repr__` exists, installs `object_repr` directly on the class.
- Remove parameter defaults from `_install_dataclass_dunders` (callers must be explicit).
- **Breaking**: Change `c_class` `eq` default from `True` to `False` — structural equality is now opt-in. Updated `_TestCxxClassDerived` to pass `eq=True` explicitly.

## Test plan

- [x] `uv run pytest -vvs tests/python -x` — 993 passed, 14 skipped, 1 xfailed
- [ ] Verify `repr=False` disables repr installation (not covered by existing tests; low risk)
}
case TypeIndex::kTVMFFIFloat: {
double v = data->v_float64;
uint64_t bits;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

union based reinterepreti is perhaps better here

…lection header (#41)

## Summary

- Remove the `TVMFFITypeInfoAddInit` C ABI function from `c_api.h`
- Move `AutoInitInfo`, `MakeInit`, and `RegisterAutoInit` from `src/ffi/object.cc` into a new public header `include/tvm/ffi/reflection/init.h` as inline C++ functions
- `registry.h` now calls `RegisterAutoInit()` directly instead of going through the C ABI
- Register `ffi.MakeInit` as a global function in `dataclass.cc` for cross-language access
- KWARGS sentinel is resolved via the global function registry (`ffi.GetKwargsObject`) instead of a direct C++ call, keeping `GetKwargsObject` as a private implementation detail

## Architecture

- `init.h` uses literal `"__ffi_init__"` instead of `type_attr::kInit` to avoid a circular include dependency (`registry.h` → `init.h` → `registry.h`)
- `MakeInit` is now a public inline function callable from any C++ translation unit that includes the header

## Test plan

- [x] C++ tests: 342 passed
- [x] Python tests: 985 passed, 22 skipped, 1 xfailed

## Breaking change

`TVMFFITypeInfoAddInit` removed from C ABI. Use `tvm::ffi::reflection::RegisterAutoInit` (C++) or `ffi.MakeInit` (cross-language) instead.
@junrushao
Copy link
Member Author

moved to #477

@junrushao junrushao closed this Feb 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants