improve: Prevent double plugin discovery during initialization

[finxo]

Summary

The current initialization process triggers plugin discovery twice, leading to unnecessary overhead and slower startup times. This occurs because PluginRegistry.__init__ calls discover(), and subsequently, TitanConfig.load() calls registry.reset(), which triggers discover() again.

Description

During application startup, the plugin registry is initialized, which includes a discovery phase to find all available plugins. However, the configuration loading process, which happens shortly after, resets the registry and re-runs the entire discovery process. This double-loading is inefficient and adds unnecessary latency to the startup sequence.

Objectives

• To eliminate the redundant plugin discovery call during initialization.
• To improve application startup performance.
• To simplify the initialization sequence and make it more efficient.

Current Behavior

1. PluginRegistry.__init__ is called, which in turn calls discover() to load all plugins.
2. TitanConfig.load() is called as part of the startup sequence.
3. Inside TitanConfig.load(), registry.reset() is invoked.
4. registry.reset() calls discover() again, causing all plugins to be re-discovered and re-loaded unnecessarily.

Expected Behavior

The plugin discovery process (discover()) should be executed only once during the application's initialization sequence. The registry should be populated once and then used without being reset immediately after the initial load.

Proposed Solution

The proposed solution is to remove the registry.reset() call from within TitanConfig.load(). Since the plugins have already been discovered during the PluginRegistry's instantiation, this call is redundant for the initial load. This change requires careful analysis of the potential side effects, particularly concerning wizard workflows and error state management.

Implementation Phases

Phase 1: Analysis and Investigation

• Thoroughly trace all execution paths within the application's wizards that might call TitanConfig.load() or initialize_plugins() directly or indirectly.
• Analyze the role of reset() in clearing the _failed_plugins list and determine how to manage this state if reset() is removed from the load() method.

Phase 2: Implementation

• Remove the registry.reset() call from TitanConfig.load().
• Implement an alternative mechanism to clear _failed_plugins if it is determined to be necessary for specific workflows (e.g., wizards that re-initialize plugins).

Phase 3: Validation and Testing

• Update unit and integration tests to cover the new initialization logic.
• Perform extensive manual testing on all wizard flows to ensure plugin states remain consistent across different operations.
• Verify that startup time is improved and no regressions are introduced.

Tasks

• Investigate wizard code paths that call load() or initialize_plugins().
• Analyze the impact of removing reset() on the _failed_plugins list management.
• Remove the registry.reset() call in TitanConfig.load().
• Refactor plugin state management if necessary to accommodate the change.
• Update existing tests to reflect the new initialization flow.
• Add new tests to prevent regressions related to plugin state.
• Manually execute and validate all wizard workflows.

Acceptance Criteria

• Plugin discovery is performed only once during standard application startup.
• All wizard flows function correctly without inconsistent plugin states.
• The _failed_plugins list is correctly managed to prevent the accumulation of errors from previous sessions or calls.
• No functional regressions are introduced.
• Application startup time is measurably improved.

Testing Strategy

Unit Tests:

• Verify that TitanConfig.load() no longer calls registry.reset().
• Test the plugin registry's state after initialization to confirm it's populated correctly.

Integration Tests:

• Test workflows that involve multiple plugin interactions to ensure state consistency.

Manual Testing:

• Execute every wizard from start to finish.
• Attempt to trigger edge cases where load() might be called multiple times to see if the plugin state remains stable.
• Induce plugin loading failures and verify that the _failed_plugins state is handled correctly in subsequent operations.

Considerations

This change is moderately risky due to its impact on the core initialization sequence. Key considerations are:

• Indirect load() calls: Some wizard paths might indirectly call load() after the initial startup. Removing reset() could leave plugins in an inconsistent state between these calls if not handled carefully.
• _failed_plugins state: The reset() method is currently responsible for clearing the _failed_plugins list. Removing it means this list could persist, potentially accumulating errors from previous sessions or wizard runs. An alternative mechanism for clearing this state at appropriate times must be considered.