convex_migrations.mdc

---
description: Guidelines for performing data migrations on convex
globs: convex/**
alwaysApply: false
---
Data migrations are really easy on Convex. 
# Adding a new field
Here's the general flow for adding a new field. Let's say we have a messages table
```ts
// convex/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";
export default defineSchema({
  messages: defineTable({
    body: v.string(),
  }),
});
```
and we'd like to add an author field. 
## Adding the field in schema as optional
First, edit the `convex/schema.ts` field to add an *optional* field for the author. By making the field optional, all 
of the existing data in the Convex deployment will continue to pass schema validation.
```ts
// convex/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";
export default defineSchema({
  messages: defineTable({
    author: v.optional(v.string()),
    body: v.string(),
  }),
});
```
Then, push the new schema and ensure that schema validation passes.
```bash
npx convex dev --once
```
Once the new schema is deployed, we'll need to populate the new column. There are two steps to doing this.
## Ensuring the field is populated for new rows
First, update all of the code in the `convex/` folder to insert the new `author` field. This will ensure that new rows
inserted into the system have the new field populated. You can find all of the insertion points in a few ways:
- Grep for `ctx.db.insert` and the table name ("messages" in this case): `ctx.db.insert("messages", ...)`
- Temporarily remove the `v.optional()` annotation on the new field and run TypeScript with `npx tsc -noEmit -p convex/`, and
  find all the type errors where the required field isn't being provided. After finding the callsites via the type errors, restore
  the `v.optional()` annotation.
After updating the code to populate the new field, run `npx convex dev --once` to deploy the new version of the code.
You MUST do this step before running the migration so we're sure that the migration doesn't miss any concurrent writes.
## Backfilling the new field for existing rows
Second, we'll need to backfill this new field for all existing rows. We can use the Convex migrations component for helping
us do this.
First, ensure that `@convex-dev/migrations` is installed in the project's `package.json`. For NPM, run the following command:
```bash
npm install @convex-dev/migrations
```
Then, we need to install the migrations component within the Convex app within the `convex/convex.config.ts` file. Be sure
to leave existing components (`exampleComponent` here) installed.
```ts
// convex/convex.config.ts
import { defineApp } from "convex/server";
import exampleComponent from "exampleComponent/convex.config";
// Import the migration component.
import migrations from "@convex-dev/migrations/convex.config";
const app = defineApp();
// Keep the existing `exampleComponent` intact.
app.use(exampleComponent);
// Install the migration component.
app.use(migrations);
export default app;
```
After installing the migrations component, you will need to rerun
`npx convex dev --once` to deploy the component and update codegen
to include migrations in the `components` TypeScript type.
Next, create a new file at `convex/migrations.ts`
```ts
import { Migrations } from "@convex-dev/migrations";
import { components } from "./_generated/api.js";
import { DataModel } from "./_generated/dataModel.js";
```
After creating the `migrations.ts` file, run `npx convex dev --once`
to update codegen to include the functions in the `migrations.ts` file
in the `api` and `internal` types.
Then, we can define a *migration* that updates each row in the database. The migrations component will handle running this
across all of the existing rows (even if there are a ton of them). Let's say in our example that we want to just fill in 
"Unknown" as the author for existing messages.
```ts
// convex/migrations.ts
import { Migrations } from "@convex-dev/migrations";
import { components } from "./_generated/api.js";
import { DataModel } from "./_generated/dataModel.js";
import { internal } from "./_generated/api.js";
export const migrations = new Migrations<DataModel>(components.migrations);
export const run = migrations.runner();
export const setUnknownAuthor = migrations.define({
  table: "messages",
  migrateOne: async (ctx, doc) => {
    if (doc.author === undefined) {
      await ctx.db.patch(doc._id, { author: "Unknown" });
    }
  },
});
// Use `internal.migrations.setUnknownAuthor` to point to the `setUnknownAuthor`
// migration defined above. Do NOT directly reference it as `setUnknownAuthor`.
export const runSetUnknownAuthor = migrations.runner(internal.migrations.setUnknownAuthor);
```
Deploy the migration to Convex with `npx convex dev --once`.
After successfully deploying, kick off the migration with the CLI command `npx convex run migrations:setUnknownAuthor`. Once it's kicked off, you can get the deployment's migration status with this CLI command:
```bash
npx convex run --component migrations lib:getStatus
```
## Cleaning up the schema
After all of the previous steps, every row in the database should have the new field. Verify this by updating the schema
to remove the `v.optional()` annotation, and confirm that deploying the new schema succeeds.