Add dynamic MCP tools: list_indexes, index_repo, delete_index

[richhankins]

Overview

This PR adds dynamic index discovery to the context-connectors MCP server, enabling agents to see new indexes as they're added by users without restarting the MCP server.

Two Modes

Mode	CLI	Description
Fixed	ctxc mcp stdio -i pytorch -i react	Static index list from CLI args
Discovery	ctxc mcp stdio --discovery	Dynamic index list from filesystem

Tool Availability

Tool	Fixed Mode	Discovery Mode
search	✅ with enum	✅ no enum
list_files	✅ with enum	✅ no enum
read_file	✅ with enum	✅ no enum
list_indexes	✅ (returns static list)	✅ (required for discovery)
index_repo	❌	❌
delete_index	❌	❌

Note: In Discovery mode, users manage indexes via CLI (ctxc index ... commands), not via MCP tools.

Key Changes

1. Dynamic MultiIndexRunner

• Removed readonly from indexNames and indexes
• Added refreshIndexList() method to reload from store
• Added invalidateClient(name) to clear cached SearchClient
• Allowed empty index list on startup

2. list_indexes Tool

• New MCP tool to list available indexes with metadata
• Available in both Fixed and Discovery modes
• Returns index name, source, and last sync time

3. Mode-Based Tool Descriptions

• Fixed mode: Includes enum in tool schemas for zero-tool-call discovery
• Discovery mode: No enum, agent must call list_indexes
• Prevents stale enum values in discovery mode

4. CLI Flag

• Added --discovery flag for explicit mode selection
• Default (no flags) uses FilesystemStore in discovery mode
• -i flags use CompositeStoreReader in fixed mode

Benefits

1. User controls lifecycle - No accidental index creation/deletion by agents
2. No concurrency issues - Only user writes to the store via CLI
3. Simpler implementation - No write operations in MCP server
4. Incremental improvement - Minimal changes to existing MCP configs
5. Clear separation - Agent discovers, user manages

Example Workflow

Agent: "I need React Query docs to answer this question"
Agent: → list_indexes() → Not found
Agent: "I don't have React Query indexed. Would you like me to help you index it?"

User: "Yes"
User: $ ctxc index github TanStack/query

Agent: → list_indexes() → Now sees "tanstack-query"
Agent: → search("tanstack-query", "...")

Testing

• ✅ All 163 tests pass
• ✅ TypeScript compiles cleanly
• ✅ Manual testing via CLI and MCP tools
• ✅ Verified both Fixed and Discovery modes

Future Work

Discovery mode is an incremental step. A future PR can add full agent-managed mode with:

• index_repo and delete_index tools
• Writable FilesystemStore
• Optional LayeredStore for remote + local indexes
• Concurrency handling (lock files)

[richhankins]

augment review

[augment-app-staging]

🤖 Augment PR Summary

Summary: This PR adds a read-only “discovery mode” to the MCP server so agents can dynamically discover available indexes at runtime via a new list_indexes tool.

Changes:

• Adds a --discovery flag to ctxc mcp stdio and ctxc mcp http, and wires a discovery boolean through to the MCP server config.
• Introduces list_indexes MCP tool and adjusts tool descriptions/schemas: fixed mode includes an enum of index names, discovery mode omits it and points clients to list_indexes.
• Extends MultiIndexRunner with refreshIndexList() (dynamic list in discovery mode) and cache pruning, plus invalidateClient().
• Adds ReadOnlyLayeredStore to merge local filesystem indexes with remote composite indexes for discovery workflows.
• Adds unit tests covering list_indexes presence and fixed-vs-discovery schema behavior, plus runner refresh behavior.

Technical Notes: Discovery mode is intentionally read-only; index creation/deletion remains a CLI responsibility (ctxc index ...) while the MCP server focuses on search and file read tooling.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augment-app-staging]

Review completed. 4 suggestions posted.

Comment augment review to trigger a new review at any time.

[augment-app-staging]

This branch sets up discovery mode without requiring any indexes, but MultiIndexRunner.create() still throws when the store has zero valid indexes, so ctxc mcp ... can still fail to start on a fresh install. If discovery mode is intended to allow “start empty and index later”, consider aligning runner/server startup behavior with that expectation.

Severity: high

Other Locations

• src/bin/cmd-mcp.ts:106

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augment-app-staging]

list_indexes is handled before the surrounding try/catch, so failures in refreshIndexList() (e.g., store I/O errors) may escape and fail the MCP request differently than other tools. Consider applying the same error handling behavior as the search/list_files/read_file branches.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augment-app-staging]

This response says “Use index_repo to create one”, but this server doesn’t expose index_repo (and the PR description says index management is via CLI in discovery mode). Consider pointing users to the appropriate ctxc index ... CLI workflow instead to avoid confusion.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augment-app-staging]

The discovery docstring says “write tools available” when false, but this server doesn’t currently expose any write tools (e.g., index_repo/delete_index). Consider updating the comment to match actual tool availability to prevent misleading integrators.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[richhankins]

augment review

All 4 review comments from the previous review have been addressed:

1. ✅ Empty indexes now allowed in discovery mode
2. ✅ list_indexes handler wrapped in try/catch
3. ✅ Message now says "Use ctxc index CLI" instead of index_repo
4. ✅ Docstring updated to reflect actual behavior (no write tools)

[augment-app-staging]

Review completed. 2 suggestions posted.

Comment augment review to trigger a new review at any time.

[augment-app-staging]

refreshIndexList() overwrites this.indexNames from store.list(), which can override an initially configured indexNames allowlist (fixed mode) and unintentionally widen what getClient() will accept after a refresh. If fixed mode is meant to stay static, consider keeping the originally requested list separate from the refreshed/discovered list.

Severity: medium

Other Locations

• src/clients/mcp-server.ts:300

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augment-app-staging]

The new list_indexes tool + the discovery-vs-fixed behavior (no enum vs enum for index_name) doesn’t appear to be covered by existing MCP server tests; a focused unit test here could help prevent regressions.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[richhankins]

augment review

Changes since last review:

1. ✅ Fixed refreshIndexList() allowlist bypass (medium severity) - In fixed mode, refreshIndexList() is now a complete no-op. The index list is set at startup and never changes, even if indexes are added or deleted from the store.

2. ✅ Fixed mode is completely static - Calling list_indexes in fixed mode always returns exactly the indexes specified via -i flags, regardless of store state.

[augment-app-staging]

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.

[augment-app-staging]

MultiIndexRunner.create() now allows returning an empty validIndexNames even when config.indexNames was explicitly provided (fixed mode), which can make the server start “successfully” but reject all tool calls. Consider whether fixed mode should still fail fast when none of the requested indexes can be loaded (e.g., due to corruption/partial state).

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augment-app-staging]

In discovery mode, refreshIndexList() updates indexNames/indexes but doesn’t prune or invalidate entries in clientCache, so a re-indexed name could keep serving stale data (and removed indexes can remain resident in memory). Consider whether refresh should reconcile the cache with the refreshed list/metadata.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augment-app-staging]

SearchClientConfig.clientUserAgent is still accepted/stored, but it’s no longer passed into DirectContext.import(), so the option appears to be a no-op while docs say it’s “sent to the Augment API”. Consider either wiring it through again (if supported) or removing/updating the option/docs to avoid misleading callers.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[richhankins]

augment review

[augment-app-staging]

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.

[augment-app-staging]

--discovery is currently only honored when --index is also provided; ctxc mcp stdio --discovery (without -i) falls through to the “No flags” branch and sets discovery = false, so discovery mode never activates.

Severity: high

Other Locations

• src/bin/cmd-mcp.ts:98

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augment-app-staging]

This test mocks store.list() to reject before calling createMCPServer(), but createMCPServer()/MultiIndexRunner.create() calls store.list() during startup, so the server will likely fail to construct and the test won’t actually exercise the list_indexes error handling path.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

[augment-app-staging]

PR title mentions index_repo/delete_index, but this server only exposes list_indexes (and the PR description says write tools are unavailable). Consider aligning the PR title (and any related docs) with the actual tool set to avoid confusion for integrators.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}