NIFI-15355 Connector Configuration Repository

[kevdoran]

Summary

NIFI-15355

Motivation

Connector working configuration currently lives only in memory and is persisted to flow.json.gz. There is no way for an external system to manage or override it. This PR introduces ConnectorConfigurationProvider, a framework extension point that allows Connector name and working flow configuration to be persisted in and loaded from an external store.

Architecture

ConnectorConfigurationProvider is a nullable collaborator of StandardConnectorRepository. When absent (the default), all behavior is unchanged. When present, the repository calls it on every read and write:

graph TB
    subgraph "REST / Service / DAO Layers"
        CR["ConnectorResource"]
        SF["NiFiServiceFacade"]
        DAO["ConnectorDAO"]
    end

    subgraph "Controller Layer"
        FC["FlowController"]
        FM["FlowManager"]
    end

    subgraph "Repository Layer"
        REPO["ConnectorRepository (StandardConnectorRepository)"]
        ASSETREPO["ConnectorAssetRepository"]
        CCP["🆕 ConnectorConfigurationProvider (nullable, NAR-discovered)"]
    end

    subgraph "In-Memory"
        CN["ConnectorNode"]
        AFC["activeFlowContext"]
        WFC["workingFlowContext"]
    end

    subgraph "Persistence"
        FLOW["flow.json.gz"]
        EXT["🆕 External Store (database, etc.)"]
    end

    CR --> SF --> DAO --> REPO
    DAO --> FM --> CN
    REPO --> CN
    REPO --> ASSETREPO
    REPO -->|"load / save / discard / delete / verifyCreate"| CCP
    CCP --> EXT
    CN --> AFC
    CN --> WFC
    SF -.->|"saveFlow()"| FLOW

Loading

Read operations (getConnector, getConnectors, addConnector, applyUpdate) call provider.load() and override the in-memory name and working flow configuration with the provider's values.

Write operations (configureConnector, discardWorkingConfiguration, updateConnector) sync with the provider before modifying in-memory state. If the provider throws, the ConnectorNode is unchanged.

Cluster sync (inheritConfiguration) does not interact with the provider, as the design is that cluster operations such as a new node joining the cluster should always inherit the current, local connector state.

Provider failures propagate as ConnectorConfigurationProviderException rather than being silently swallowed, giving implementations control over failure semantics.

New Types (nifi-framework-api)

• ConnectorConfigurationProvider -- Extension point interface: initialize, load, save, discard, delete, verifyCreate
• ConnectorConfigurationProviderInitializationContext -- Exposes Map<String, String> properties (following the established NiFi initialization context pattern)
• ConnectorWorkingConfiguration -- Mutable POJO carrying name and List<VersionedConfigurationStep> workingFlowConfiguration
• ConnectorConfigurationProviderException -- RuntimeException for provider failures

Other Changes

• Centralized ConnectorNode metadata updates -- All name modifications now go through ConnectorRepository.updateConnector() instead of calling connectorNode.setName() directly. Updated call sites: StandardNiFiServiceFacade (via new ConnectorDAO.updateConnector()), VersionedFlowSynchronizer.
• Centralized create verification -- ConnectorRepository.verifyCreate() checks for duplicate IDs and delegates to provider.verifyCreate(). StandardConnectorDAO.verifyCreate() now delegates to the repository.
• Removed redundant addConnector() -- StandardConnectorDAO.createConnector() no longer calls addConnector() since FlowManager.createConnector() already does this.

Configuration

nifi.components.connectors.configuration.provider.implementation=com.example.MyProvider
nifi.components.connectors.configuration.provider.db.url=jdbc:postgresql://localhost/nifi

Testing

25 unit tests in TestStandardConnectorRepository covering read overrides, write-through atomicity, partial merge semantics, lifecycle delegation, exception propagation, cluster sync exclusion, and verifyCreate / addConnector atomicity. Updated StandardConnectorDAOTest for verifyCreate delegation.

Files Changed

17 files, +975 / -30. Four new types in nifi-framework-api, one new implementation in nifi-framework-core, twelve modified files across nifi-framework-core, nifi-framework-core-api, nifi-web-api, and nifi-properties.

[bbende]

Minor - would consider updating the default nifi.properties to include these

[kevdoran]

I added it. I left it blank by default (with a mvn/env override) because there is no standard implementation by default, it is a nullable/optional collaborator.

[sfc-gh-mpayne]

Should avoid the fully-qualified class name :)

[kevdoran]

Thanks all. I've addressed the feedback and pushed updates to the PR

[bbende]

+1 Looks good