When is the migration executed?

[bytepoets-jhe]

I am currently working on a major migration.
Now I have the problem that while debugging through the migration, I get an exception.
The exception happens in a mapper that tries to read from the db before the migration is finished.
This means that the UI is created before the migration is finished.
I checked that I didn't miss any awaits.
I thought that the migration is done while creating the db so it should not start creating UI before the db is ready (which is implemented with getIt dependency injection and awaited).
Is there anything else I could have missed here?

We had a small migration before where we didn't have this issue.
I migrated to stepByStep migrations before the new migrations, maybe there is a bug here?

[simolus3]

The exception happens in a mapper that tries to read from the db before the migration is finished.

That does sound like a bug in drift. As you've said, migrations should run before anything else is able to use the database.

To help debug this, you could enable logStatements: true on the NativeDatabase and also add print statements at the start and end of the migration? Maybe that helps reveal where it's going wrong (or if there's a particular migration step where drift erroneously allows other statements to use the database). But yeah, if you've awaited everything in the migration then there's nothing you've done wrong and this is likely a problem in drift. Let's see if the logs bring anything up, if not I'll have to think about more ways to reproduce and debug this.

[bytepoets-jhe]

This shows the same error.
I have 4 selects (awaited), the I drop 7 tables (awaited).
Between table dropping 6 and 7 I already get a log from dio that my first API call was sent, which is triggered when the UI is built.

I/flutter (31398): migrating to version 3
I/flutter (31398): Drift: Sent SELECT * from table1 with args []
I/flutter (31398): Drift: Sent SELECT * from table2 with args []
I/flutter (31398): Drift: Sent SELECT * from table3 with args []
I/flutter (31398): Drift: Sent SELECT * from table4 with args []
I/flutter (31398): Drift: Sent DROP TABLE IF EXISTS "table4"; with args []
I/flutter (31398): Drift: Sent DROP TABLE IF EXISTS "table3"; with args []
I/flutter (31398): Drift: Sent DROP TABLE IF EXISTS "table2"; with args []
I/flutter (31398): Drift: Sent DROP TABLE IF EXISTS "table1"; with args []
I/flutter (31398): Drift: Sent DROP TABLE IF EXISTS "table7"; with args []
I/flutter (31398): Drift: Sent DROP TABLE IF EXISTS "table6"; with args []
//Here a API call is logged
I/flutter (31398): Drift: Sent DROP TABLE IF EXISTS "table5"; with args []

I went back to not using stepByStep migrations and I get the same error unfortunately.

This is my code (changed some names and variables to hide implementation details)

@override
  MigrationStrategy get migration => MigrationStrategy(onCreate: (Migrator m) {
        return m.createAll();
      }, onUpgrade: (Migrator m, int from, int to) async {
        if (from < 2) {
          await migrateToV2(m);
        }
        if (from < 3) {
          await migrateToV3(m);
        }
      });

Future<void> migrateToV3(Migrator m) async {
    print('migrating to version 3');
    final test4 =
        (await customSelect('SELECT * from table1', readsFrom: {table1}).get()).map((e) => e.data).toList();
    final test5 = (await customSelect('SELECT * from table2', readsFrom: {table2}).get()).map((e) => e.data).toList();
    final test6 = (await customSelect('SELECT * from table3', readsFrom: {table3}).get())
        .map((e) => e.data)
        .toList();
    final test7 = (await customSelect('SELECT * from table4', readsFrom: {table4}).get())
        .map((e) => e.data)
        .toList();

    await m.drop(table4);
    await m.drop(table3);
    await m.drop(table2);
    await m.drop(table1);

    await m.drop(table7);
    await m.drop(table6);
    await m.drop(table5);

    await m.createAll();

   //Doing stuff with the data from selects and inserting again

[simolus3]

Thanks for the follow-up! The migration code looks correct to me.

I thought that the migration is done while creating the db so it should not start creating UI before the db is ready (which is implemented with getIt dependency injection and awaited).

I'm not sure what you mean with "start creating the UI" here. But to clarify, drift will let you use the database before initializing it, it just doesn't run the statements until the migration ran. The first statement running against the database will open it, other statements are enqueued. Do you see any Drift: Sent calls that would point to UI activity before the migration is done?

If you want to explicitly wait for the database to be ready before injecting it into other components, you can do something like

registerAsyncSingleton(()  async {
  final database = MyDatabase();
  await database.customStatement('select 1'); // this will open the database and run migrations
  return database;
});

Most of the time that shouldn't be necessary though, so I'm curious to see what exception you got.

[bytepoets-jhe]

I will try that, thank you!
The Ui is clearly already created when the migration is running.
The exception is happening in my mapper, because it tries to access the database and still gets a int instead of a string, which is what I changed in my migration. So the code tries to do a select, but the data in the db is not yet migrated, so the datatype in there is not what it is expecting.

[bytepoets-jhe]

The custom statement did work, so the UI is not being loaded yet, but I get the same error.
So it does not have to do with the select statement which is triggered by the UI.
Here is the stacktrace:

#0      DriftCommunication.request (package:drift/src/remote/communication.dart:113:66)
#1      _ServerDbUser.beforeOpen (package:drift/src/remote/server_impl.dart:285:24)
#2      DelegatedDatabase._runMigrations (package:drift/src/runtime/executor/helpers/engines.dart:458:16)
<asynchronous suspension>
#3      DelegatedDatabase.ensureOpen.<anonymous closure> (package:drift/src/runtime/executor/helpers/engines.dart:426:7)
<asynchronous suspension>
#4      ServerImplementation._handleEnsureOpen (package:drift/src/remote/server_impl.dart:137:12)
<asynchronous suspension>
#5      DriftCommunication.setRequestHandler.<anonymous closure> (package:drift/src/remote/communication.dart:165:20)
<asynchronous suspension>

[bytepoets-jhe]

Thank you for your work on drift db and your help here!
The migrations running before the first select threw me off and the bug was actually in my migration code and is fixed now.