Proposal: Extensible hooks and plugin support

[araujof]

Summary

Introduces a design specification for an extensible hook system and plugin framework for Mellea. The hook system provides extension points across the framework's execution lifecycle — from session initialization through component execution, LLM generation, validation, sampling, and cleanup — enabling policy enforcement, observability, customization, and integrations without modifying core library code.

Specification Outline

• Plugin framework design: Async hook API with typed payloads, read-only context, and result objects that can modify payloads or block execution. Plugins register via convention-based naming (on_<hook_name>), programmatic registration, or YAML configuration.
• Hook points organized into categories that reflect Mellea's abstraction boundaries.
• Execution model: Blocking (default) and fire-and-forget modes, priority-based ordering, and enforcement modes.
• Concurrency semantics: Documents how Python's cooperative async model keeps hook execution deterministic when awaited, and where race conditions can arise.
• Observability integration: Describes how hooks compose with OTel-instrumented LLM client libraries.
• Error handling: Plugin error isolation, circuit breaking, configurable timeouts.
• Example plugins: Content policy, audit logging, token budget enforcement, and generative slot profiling.

Specification document

Implementation Plan

The implementation plan defines how Mellea's extensibility hook system will be built on top of the ContextForge plugin framework as an optional dependency. The primary developer API uses @hook and @plugin decorators plus a register() function for programmatic registration, with YAML configuration supported as a secondary deployment-time mechanism. Plugins can be scoped globally or per-session (via start_session(plugins=[...])), and a PluginSet class enables composable grouping. The central invoke_hook() dispatcher includes three layers of zero-overhead guards (_plugins_enabled flag, has_hooks_for() check, and early return) to ensure no performance impact when plugins are not configured. Hook call sites are added across functional.py, session.py, sampling/base.py, backend.py, context.py, and the OpenAI/HuggingFace adapter backends, with a generate_from_context_with_hooks() wrapper on the Backend ABC to avoid modifying individual backend
implementations.

Implementation plan

Review plan

• Review spec for completeness against current Mellea internals
• Validate hook trigger points map to actual code paths
• Gather team feedback on hook surface area (too many? too few? missing categories?)

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 Enforce conventional commit

This rule is failing.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

[araujof]

@nrfulton I added an implementation plan document to detail the relevant components of the ContextForge Plugin Framework and required changes to Mellea to introduce the hook system.

[araujof]

@jakelorocco @HendrikStrobelt I updated the hook system specification and implementation plan with functional-first, programmatic API as per our recent discussions.