-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Add runtime migration creation and application #37415
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
thromel
wants to merge
14
commits into
dotnet:main
Choose a base branch
from
thromel:feature/runtime-migrations
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implements dotnet#37342: Allow to create a migration and apply it without recompiling. This adds support for creating and applying migrations at runtime using Roslyn compilation, enabling scenarios like .NET Aspire and containerized applications where recompilation isn't possible. ## New Features ### CLI ```bash dotnet ef database update --add MigrationName [--output-dir <DIR>] [--namespace <NS>] ``` ### Programmatic API ```csharp var result = context.Database.CreateAndApplyMigration("MigrationName"); // or with options var result = context.Database.CreateAndApplyMigration("MigrationName", new RuntimeMigrationOptions { PersistToDisk = true, OutputDirectory = "Migrations", DryRun = false }); ``` ## Architecture - `IMigrationCompiler` / `CSharpMigrationCompiler`: Roslyn-based compilation of scaffolded migrations - `IDynamicMigrationsAssembly` / `DynamicMigrationsAssembly`: Decorator that merges runtime-compiled migrations with static ones - `IRuntimeMigrationService` / `RuntimeMigrationService`: Orchestrates scaffold -> compile -> register -> apply flow - `RuntimeMigrationOptions`: Configuration for persistence, output directory, namespace, dry-run - `RuntimeMigrationResult`: Contains migration ID, applied status, SQL commands, file paths ## Limitations - Requires dynamic code generation (incompatible with NativeAOT) - C# only (no VB.NET/F# support initially)
- Test error handling when no model changes detected - Test error handling for invalid context type - Uses OperationExecutor pattern consistent with AddMigration tests
…iring Integration tests: - Can_create_and_apply_initial_migration - Can_create_and_apply_initial_migration_async - Can_create_migration_with_dry_run - CreateAndApplyMigration_generates_valid_sql_commands - CreateAndApplyMigration_throws_when_no_pending_changes - Can_create_migration_with_custom_namespace - Can_check_for_pending_model_changes - Applied_migration_appears_in_migration_history Service wiring fixes: - Create DynamicMigrationsAssembly in AddDbContextDesignTimeServices to properly wrap context's IMigrationsAssembly - Register DynamicMigrationsAssembly as both IDynamicMigrationsAssembly and IMigrationsAssembly - Add IDesignTimeModel registration from context RuntimeMigrationService changes: - Apply migrations directly instead of using IMigrator.Migrate() to avoid migration lookup issues - Add ApplyMigration and ApplyMigrationAsync methods that execute commands directly - Inject additional services: IMigrationCommandExecutor, IRelationalConnection, IRawSqlCommandBuilder, IRelationalCommandDiagnosticsLogger
Add RuntimeMigrationSqlServerTest.cs with 7 integration tests for SQL Server: - Can_create_and_apply_initial_migration - Can_create_and_apply_initial_migration_async - Can_create_migration_with_dry_run - CreateAndApplyMigration_generates_valid_sql_commands - Can_check_for_pending_model_changes - Applied_migration_appears_in_migration_history - Can_create_migration_with_custom_namespace Tests are marked with [SqlServerCondition] and will be skipped on platforms without SQL Server configured.
a15611a to
9a35a9b
Compare
The dotnet-ef project includes *.Configure.cs files from the ef project, but uses its own Resources class. When DatabaseUpdateCommand.Configure.cs references Resources.DatabaseUpdateAddDescription, it needs to be present in both the ef and dotnet-ef Resources.
The previous implementation called context.GetService<IMigrationsAssembly>() immediately during service registration. This failed for contexts that don't have a database provider configured (like some test contexts used in DbContextOperationsTest). Changed to lazy initialization so the migrations assembly is only retrieved when actually needed, matching the original pattern for other services.
In dry run mode after EnsureDeleted(), the database doesn't exist, so we can't query INFORMATION_SCHEMA.TABLES. The result.Applied=false assertion is sufficient to verify dry run behavior.
62018f1 to
5c41f2f
Compare
In Helix distributed testing, PIPELINE_WORKSPACE is not set. Add HELIX_CORRELATION_PAYLOAD check so tests marked with [SqlServerCondition(IsNotCI)] are properly skipped in Helix.
Contributor
Author
Note on SQL Server Integration TestsThe Why these tests are skipped in CI:
Test coverage is still maintained:
This is consistent with how other complex migration infrastructure tests handle CI limitations. |
New SQLite integration tests: - Can_apply_multiple_sequential_migrations: Apply 2 migrations in sequence - Can_create_migration_with_indexes_and_foreign_keys: Complex model with FK/indexes - Generated_migration_file_has_correct_structure: Verify file contents - Down_migration_reverses_changes: Verify rollback works New CLI/OperationExecutor tests: - CreateAndApplyMigration_errors_for_bad_names: Invalid migration name handling - CreateAndApplyMigration_errors_for_empty_name: Empty name validation
- Add RevertMigration and RevertMigrationAsync methods to IRuntimeMigrationService interface - Implement revert functionality in RuntimeMigrationService that: - Tracks applied dynamic migrations - Executes Down operations to undo changes - Removes migration from history table - Add error handling for missing/not-found migrations - Fix tests to properly verify actual rollback behavior instead of just checking generated code - Add tests for error cases and async revert
- Add partial failure tests (table conflict, external interference) - Add error recovery tests (revert after manual drop) - Add dry run safety verification - Add migration tracking and result verification tests - Add edge case tests for special characters and null paths - Total: 26 SQLite functional tests, 20 design tests (46 total)
Testing for transient CI failures.
Add validation for invalid file name characters and context name conflicts to the CreateAndApplyMigration operation, matching the existing validation in the AddMigration operation. This fixes the Windows CI test failure where tests for bad migration names were expecting proper error handling.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Implements #37342: Allow to create a migration and apply it without recompiling.
This adds support for creating and applying migrations at runtime using Roslyn compilation, enabling scenarios like .NET Aspire and containerized applications where recompilation isn't possible.
New Features
CLI:
Programmatic API:
Architecture
IMigrationCompiler/CSharpMigrationCompilerIDynamicMigrationsAssembly/DynamicMigrationsAssemblyIRuntimeMigrationService/RuntimeMigrationServiceRuntimeMigrationOptionsRuntimeMigrationResultLimitations
[RequiresDynamicCode]Test plan
CSharpMigrationCompiler(5 tests)DynamicMigrationsAssembly(12 tests)Fixes #37342