Convex rules (optimized)

_convex.mdc

---
description: Guidelines and best practices for building Convex projects with Convex Ents, including database schema design, queries, mutations, and real-world examples
globs: **/*.ts,**/*.tsx
alwaysApply: false
---

# Convex guidelines (with Convex Ents)

## Authentication & Functions

**Edit:** `convex/functions.ts`, `convex/*.ts` files

### Function wrappers (NEVER use raw query/mutation/action)

```typescript
import {
  createAuthQuery,
  createPublicQuery,
  createAuthMutation,
} from './functions';
import { z } from 'zod';
import { zid } from 'convex-helpers/server/zod';

// Public query - auth optional (ctx.userId available if authenticated)
export const getItem = createPublicQuery()({
  args: { id: zid('items') },
  returns: z.object({ name: z.string() }).nullable(),
  handler: async (ctx, args) => {
    return await ctx.table('items').get(args.id);
  },
});

// Auth query - requires authentication (ctx.userId guaranteed)
export const myItems = createAuthQuery()({
  args: {},
  returns: z.array(z.object({ id: zid('items'), name: z.string() })),
  handler: async (ctx) => {
    return await ctx
      .table('items')
      .query()
      .withIndex('userId', (q) => q.eq('userId', ctx.userId))
      .collect();
  },
});

// Auth mutation with rate limiting
export const createItem = createAuthMutation({
  rateLimit: 'character/create', // Uses tier limits (free/premium)
})({
  args: { name: z.string().min(1).max(100) },
  returns: zid('items'),
  handler: async (ctx, args) => {
    return await ctx.table('items').insert({
      name: args.name,
      userId: ctx.userId,
    });
  },
});

// Admin-only function
export const adminData = createAuthQuery({ role: 'ADMIN' })({
  returns: z.array(z.object({ id: zid('users'), email: z.string() })),
  handler: async (ctx) => ctx.table('users').query().collect(),
});
```

**Always throw ConvexError instead of Error:** `throw new ConvexError({ code: 'UNAUTHENTICATED', message: 'Not authenticated' });`

### Rate Limiting

**Edit:** `convex/helpers/rateLimiter.ts`

Rate limits are defined in `convex/helpers/rateLimiter.ts` and automatically applied when specified in mutations:

```typescript
// Define rate limits in rateLimiter.ts
export const rateLimiter = new RateLimiter(components.rateLimiter, {
  'comment/create:free': { kind: 'fixed window', period: MINUTE, rate: 10 },
  'comment/create:premium': { kind: 'fixed window', period: MINUTE, rate: 30 },
  // ... other limits
});

// Use in mutations
export const createComment = createAuthMutation({
  rateLimit: 'comment/create', // Automatically adds :free/:premium suffix based on user tier
})({
  // ... mutation implementation
});
```

### Internal functions

**Edit:** `convex/*.ts` files

```typescript
import {
  createInternalQuery,
  createInternalMutation,
  createInternalAction,
} from './functions';

// Internal functions also use Zod
export const processData = createInternalQuery()({
  args: { id: zid('items') },
  returns: z.null(),
  handler: async (ctx, args) => {
    // Only callable by other Convex functions
    return null;
  },
});
```

### HTTP endpoints

**Edit:** `convex/http.ts`

For HTTP endpoints and webhooks, see [convex-http.mdc](mdc:.cursor/rules/convex-http.mdc).

### Validators

**Edit:** `convex/schema.ts` (schema files ONLY, we use zod for args/returns)

```typescript
import { defineEnt, defineEntSchema, getEntDefinitions } from 'convex-ents';
import { v } from 'convex/values';

// Schema validators (v.) - ONLY in schema.ts
const schema = defineEntSchema(
  {
    users: defineEnt({
      name: v.string(),
      age: v.number(),
      role: v.union(v.literal('user'), v.literal('admin')),
    })
      .index('name', ['name'])
      .field('email', v.string(), { unique: true }), // unique field example

    // Relationship example
    userFollows: defineEnt({
      followingId: v.id('users'),
      followerId: v.id('users'),
    })
      .index('following_follower', ['followingId', 'followerId'])
      .edge('following', { to: 'users', field: 'followingId' })
      .edge('follower', { to: 'users', field: 'followerId' }),
  },
  {
    schemaValidation: true,
  }
);

export default schema;

// Export ent definitions for use throughout the app
export const entDefinitions = getEntDefinitions(schema);
```

**Quick Reference:**

- `v.id('table')` - Document IDs
- `v.string()`, `v.number()`, `v.boolean()` - Basic types
- `v.null()` - Use instead of undefined
- `v.array(v.string())` - Arrays (max 8192)
- `v.object({...})` - Objects (max 1024 fields)
- `v.record(v.string(), v.number())` - Dynamic keys
- `v.union(...)`, `v.literal(...)` - Union types
- `v.optional(...)` - Optional fields
- `v.int64()` - BigInt support

### Zod validators (Functions)

**Edit:** `convex/*.ts` files (all function files)

```typescript
import { createAuthMutation } from './functions';
import { z } from 'zod';
import { zid } from 'convex-helpers/server/zod';

// Function validators (z.) - ALL functions
export const createUser = createAuthMutation()({
  args: {
    // CRITICAL: Always use zid() for IDs, not z.string()
    userId: zid('users'),
    name: z.string().min(1).max(100),
    email: z.string().email(),
    age: z.number().positive().int(),
    tags: z.array(z.string()).max(10),
    status: z.enum(['draft', 'published']),
    bio: z.string().optional(), // Use .optional(), not .nullable()
  },
  returns: zid('users'),
  handler: async (ctx, args) => {
    /* ... */
  },
});
```

**Quick Reference:**

- `zid('table')` - ALWAYS for document IDs
- `z.string().email()/.url()/.min()/.max()` - String validation
- `z.number().positive()/.int()/.min()/.max()` - Number validation
- `z.boolean().default(true)` - Booleans with defaults
- `z.array(z.string()).max(10)` - Arrays
- `z.object({...}).strict()` - Objects
- `z.enum(['a', 'b'])` - Enums
- `z.union([z.string(), z.number()])` - Unions
- `.optional()` - Optional fields (NOT .nullable())
- `z.record(z.string(), z.number())` - Dynamic keys

### Null vs undefined

```typescript
// ✅ CORRECT patterns
email: z.string().optional(); // Optional field
return null; // Functions return null
updates.email = args.email || undefined; // Patch: undefined removes field

// ❌ WRONG patterns
email: z.string().nullable(); // Don't use nullable for optional
return undefined; // Functions can't return undefined
```

### Document IDs

- **Function args/returns:** ALWAYS use `zid('tableName')`, never `z.string()`
- **TypeScript:** Use `Id<'tableName'>` type
- **External IDs:** Cast with `as Id<'tasks'>` when needed

### Function types summary

**Public functions** (exposed to internet):

- `createPublicQuery()` - auth optional
- `createAuthQuery()` - requires auth
- `createPublicMutation()` - auth optional
- `createAuthMutation()` - requires auth
- `createPublicPaginatedQuery()` - with pagination
- `createAuthPaginatedQuery()` - auth + pagination
- `createAction()` - public actions

**Internal functions** (only callable by Convex, NOT even by NextJS server):

- `createInternalQuery/Mutation/Action()`

**Rules:**

- CRITICAL: NEVER use raw `query`, `mutation`, `action` from `./_generated/server`
- ALWAYS include `args` and `returns` validators
- Use `returns: z.null()` when no return value

### Function calling

```typescript
// From query/mutation/action
ctx.runQuery(api.example.getData, args);
ctx.runMutation(api.example.update, args);
ctx.runAction(internal.example.process, args);

// Same-file calls need type annotation
const result: string = await ctx.runQuery(api.example.f, { name: 'Bob' });
```

**Rules:**

- Use FunctionReference (api/internal), not function directly
- Minimize action→query/mutation calls (race conditions)
- Share code via helper functions instead of action chains

### Function references

**Import:** `import { api, internal } from './_generated/api'`

- **Public:** `api.filename.functionName` (e.g., `api.users.create`)
- **Internal:** `internal.filename.functionName` (e.g., `internal.users.process`)
- **Nested:** `api.folder.file.function` (e.g., `api.messages.access.check`)

### File organization

**Edit:** `convex/` directory

- File-based routing: `convex/users.ts` → `api.users.*`
- Group by feature: `users.ts`, `messages.ts`, `auth.ts`
- Use internal functions for sensitive operations

### Pagination

- Use `paginationOptsValidator` from Convex (not Zod) for pagination:

```ts
import { z } from 'zod';
import { createAuthQuery } from './functions';
import { paginationOptsValidator } from 'convex/server';

export const listWithExtraArg = createAuthQuery()({
  args: {
    paginationOpts: paginationOptsValidator,
    author: z.string(),
  },
  handler: async (ctx, args) => {
    return await ctx
      .table('messages')
      .query()
      .withIndex('author', (q) => q.eq('author', args.author))
      .order('desc')
      .paginate(args.paginationOpts);
  },
});
```

Note: `paginationOpts` is an object with the following properties:

- `numItems`: the maximum number of documents to return
- `cursor`: the cursor to use to fetch the next page of documents (string | null)
- A query that ends in `.paginate()` returns an object with:
  - `page`: Array of documents fetched
  - `isDone`: Boolean indicating if this is the last page
  - `continueCursor`: String cursor for the next page

### Paginated Queries

**OPTIMIZATION:** Always use pagination for user-facing lists that could grow large. See [convex-optimize.mdc](mdc:.cursor/rules/convex-optimize.mdc).

- Use `createPublicPaginatedQuery()` or `createAuthPaginatedQuery()` (paginationOpts auto-injected):

```typescript
// Basic
export const list = createPublicPaginatedQuery()({
  handler: async (ctx, args) => {
    return await ctx
      .table('messages')
      .query()
      .order('desc')
      .paginate(args.paginationOpts);
  },
});

// With args + transform
export const listByAuthor = createPublicPaginatedQuery()({
  args: { author: z.string() },
  handler: async (ctx, args) => {
    const results = await ctx
      .table('messages')
      .query()
      .withIndex('author', (q) => q.eq('author', args.author))
      .order('desc')
      .paginate(args.paginationOpts);

    // Batch fetch users by IDs
    const userIds = results.page.map((m) => m.userId);
    const users = await Promise.all(
      userIds.map((id) => ctx.table('users').get(id))
    );

    return {
      ...results,
      page: results.page.map((msg) => ({
        ...msg,
        user: users.find((u) => u?._id === msg.userId) || null,
      })),
    };
  },
});

// For complex filters with pagination, use streams
// See [convex-streams.mdc](mdc:.cursor/rules/convex-streams.mdc) for consistent page sizes
```

- **React hooks**: `usePublicPaginatedQuery`, `useAuthPaginatedQuery` (NEVER use `usePaginatedQuery` directly)

```typescript
import { usePublicPaginatedQuery, useAuthPaginatedQuery } from '@/lib/convex/hooks';

// Public paginated query
const { pages, hasNextPage, isLoading, isFetchingNextPage, fetchNextPage } = usePublicPaginatedQuery(
  api.messages.list,
  { author: 'alice' }, // args (excluding paginationOpts)
  { initialNumItems: 10 }
);

// Authenticated paginated query (skips if not authenticated)
const { pages, hasNextPage, isLoading, isFetchingNextPage, fetchNextPage } = useAuthPaginatedQuery(
  api.userFollows.getFollowers,
  { username },
  { initialNumItems: 20 }
);

// UI example
<button onClick={() => fetchNextPage(5)} disabled={!hasNextPage}>
  Load More
</button>
```

**Return values**:

- `pages`: Array of all loaded items (accumulates across all pages)
- `hasNextPage`: Boolean indicating if more pages can be loaded
- `isLoading`: Boolean indicating if the first page is loading
- `isFetchingNextPage`: Boolean indicating if a subsequent page is loading
- `fetchNextPage(numItems)`: Function to load the next page with specified number of items

**Notes**:

- `pages` accumulates all items from all loaded pages
- `fetchNextPage(n)` only works when `hasNextPage === true`
- `useAuthPaginatedQuery` returns empty pages array if not authenticated

## Validator summary

- **Schema files (`convex/schema.ts`):** Use `v.` validators ONLY
- **Function files (`convex/*.ts`):** Use `z.` validators ONLY
- **Document IDs:** `zid('table')` in functions, `v.id('table')` in schemas
- **Records:** `z.record()` in functions, `v.record()` in schemas

## Schema guidelines

**Edit:** `convex/schema.ts`

```typescript
import { defineEnt, defineEntSchema, getEntDefinitions } from 'convex-ents';
import { v } from 'convex/values';

const schema = defineEntSchema(
  {
    users: defineEnt({
      name: v.string(),
    })
      .field('email', v.string(), { unique: true }) // Unique field
      .field('role', v.string(), { default: 'user' }) // Default value
      .index('name_email', ['name', 'email']) // Compound index
      .edge('profile', { ref: true }) // 1:1 edge (optional side)
      .edges('messages', { ref: true }), // 1:many edge ("one" side)

    profiles: defineEnt({
      bio: v.string(),
    }).edge('user'), // 1:1 edge (required side)

    messages: defineEnt({
      text: v.string(),
    })
      .index('userId', ['userId'])
      .edge('user') // 1:many edge ("many" side)
      .edge('author', { field: 'authorId', optional: true }) // Optional edge
      .edges('tags'), // many:many edge

    tags: defineEnt({
      name: v.string(),
    }).edges('messages'), // many:many edge
  },
  {
    schemaValidation: true,
  }
);

export default schema;
export const entDefinitions = getEntDefinitions(schema);
```

**Field options:**

- `.field('email', v.string(), { unique: true })` - Unique constraint
- `.field('type', v.string(), { default: 'text' })` - Default value

**Edge patterns:**

- 1:1: `.edge('profile', { ref: true })` on optional side
- 1:many: `.edges('messages', { ref: true })` on "one" side, `.edge('user')` on "many" side
- many:many: `.edges('tags')` on both sides
- Optional: `.edge('user', { optional: true })`

**System fields:** `_id: v.id('table')`, `_creationTime: v.number()`  
**Index naming:** `field1_field2` for compound indexes  
**OPTIMIZATION:** Create indexes for all equality filters - see [convex-optimize.mdc](mdc:.cursor/rules/convex-optimize.mdc)

## Import guidelines

**From Next.js (`src/`):**

```typescript
import { api, internal } from '@convex/_generated/api';
import { Id, Doc } from '@convex/_generated/dataModel';
import type { Ent, EntWriter } from '@convex/shared/types'; // Ent types
```

**From Convex (`convex/`):**

```typescript
import { api, internal } from './_generated/api';
import { Id, Doc } from './_generated/dataModel';
import { entDefinitions } from './schema'; // Import ent definitions
import { entsTableFactory } from 'convex-ents'; // For function context
```

## React Integration guidelines

### Query hooks

**Edit:** `src/components/**/*.tsx`

```typescript
import { usePublicQuery, useAuthQuery } from '@/lib/convex/hooks';

// Public query (auth optional) - ALWAYS pass {} for no args
const { data, isPending } = usePublicQuery(api.items.list, {});
const { data: item } = usePublicQuery(api.items.get, { id: itemId });

// Auth query (skips if not authenticated)
const { data: profile } = useAuthQuery(api.user.getProfile, {});

// Skip conditionally
const { data } = usePublicQuery(
  api.items.get,
  itemId ? { id: itemId } : 'skip'
);

// Loading state
if (isPending) return <Skeleton />;
```

**Never use** `useQuery` or `usePaginatedQuery` directly

### Skeleton Loading with Convex

- **ALWAYS use `WithSkeleton` + `useMockQuery`** for consistent skeleton UI:

```typescript
import { WithSkeleton } from '@/components/ui/skeleton';
import { useMockQuery } from '@/hooks/useLoading';

// Basic usage
const characterQuery = useAuthQuery(api.character.list, {});
const { data: { characters }, isLoading } = useMockQuery(characterQuery, {
  characters: [
    { id: '1', name: 'Character 1', description: 'Description text' },
    { id: '2', name: 'Character 2', description: 'Description text' },
  ],
});

return (
  <div>
    {characters.map((char, index) => (
      <WithSkeleton key={index} isLoading={isLoading}>
        <Card>
          <h3>{char.name}</h3>
          <p>{char.description}</p>
        </Card>
      </WithSkeleton>
    ))}
  </div>
);
```

**Rules**:

- NEVER use random values in mock data (causes hydration errors)
- Keep mock data properties alphabetically ordered
- Use static, predictable mock data matching expected structure
- Use index as key for mock data items when mapping

### Mutations

**Edit:** `src/components/**/*.tsx`

```typescript
import { usePublicMutation, useAuthMutation } from '@/lib/convex/hooks';

// Don't destructure to avoid naming conflicts
const createItem = usePublicMutation(api.items.create);
const updateSettings = useAuthMutation(api.user.updateSettings);

// Option 1: toast.promise (with loading state)
toast.promise(updateSettings.mutateAsync({ name: 'New' }), {
  loading: 'Updating...',
  success: 'Updated!',
  error: (e) => e.data?.message ?? 'Failed',
});

// Option 2: callbacks (no loading toast)
const updateSettings = useAuthMutation(api.user.updateSettings, {
  onSuccess: () => toast.success('Updated!'),
  onError: () => toast.error('Failed'),
});

<Button disabled={updateSettings.isPending}>Save</Button>
```

### Other hooks

```typescript
// Actions
import { useAction } from 'convex/react';
const generateReport = useAction(api.reports.generate);

// One-off queries
import { useConvex } from 'convex/react';
const convex = useConvex();
await convex.query(api.items.validate, { id });

// Auth hooks
import { useIsAuth, useCurrentUser } from '@/lib/convex/hooks';
const isAuth = useIsAuth();
const { isLoading, ...user } = useCurrentUser();
```

### Server-side auth (RSC)

```typescript
import {
  getSessionToken,
  getSessionUser,
  isAuth,
  isUnauth,
  fetchAuthQuery,
  fetchAuthQueryOrThrow,
} from '@/lib/convex/server';

// Get session token/user in server components
const token = await getSessionToken(); // Returns string | null
const user = await getSessionUser(); // Returns SessionUser & { token } | null

// Check auth status
if (await isAuth()) {
  /* authenticated */
}
if (await isUnauth()) {
  /* not authenticated */
}

// Fetch authenticated queries in server components
const data = await fetchAuthQuery(api.user.getData, { id: userId });
// Returns null if not authenticated

const data = await fetchAuthQueryOrThrow(api.user.getData, { id: userId });
// Throws error if not authenticated
```

### Auth guards

**Server-side guards (RSC)**:

```typescript
import { authGuard, adminGuard, superAdminGuard } from '@/lib/convex/rsc';

// In server components/pages
await authGuard(); // Redirects to login if not authenticated
await adminGuard(); // Returns 404 if not admin
await superAdminGuard(); // Returns 404 if not superadmin
```

**Client-side guards (hooks)**:

```typescript
import { useAuthGuard, usePremiumGuard } from '@/lib/convex/hooks';

// Auth guard - shows login modal if not authenticated
const authGuard = useAuthGuard();
if (authGuard(() => console.info('authenticated'))) return; // Blocked

// Premium guards - shows subscription modal if not premium
const premiumGuard = usePremiumGuard();
if (premiumGuard()) return; // Blocked
```

### Relationship patterns with Ents

**Edit:** `convex/*.ts` files

```typescript
import { asyncMap } from 'convex-helpers';

// Get entity by ID
const user = await ctx.table('users').get(userId);
const userX = await ctx.table('users').getX(userId); // Throws if not found

// Get by unique field or index
const userByEmail = await ctx.table('users').get('email', '[email protected]');
const userByEmailX = await ctx.table('users').getX('email', '[email protected]');
const follow = await ctx
  .table('userFollows')
  .get('following_follower', userId1, userId2);

// Batch operations
const users = await ctx.table('users').getMany([id1, id2, id3]); // (Doc | null)[]
const usersX = await ctx.table('users').getManyX([id1, id2, id3]); // Doc[] or throws

// Unique document
const settings = await ctx.table('settings').unique(); // null if 0, error if 2+
const settingsX = await ctx.table('settings').uniqueX(); // throws if not exactly 1

// Edge traversal
const profile = await user.edge('profile'); // may return null for optional
const profileX = await user.edgeX('profile'); // throws if missing
const messages = await user.edge('messages'); // returns array
const messages = await user.edge('messages').take(5); // limit results

// Paginated edge traversal
const followers = await user
  .edge('followers')
  .order('desc')
  .paginate(args.paginationOpts);

// Query with filters
const activePosts = await ctx
  .table('posts')
  .query()
  .withIndex('authorId', (q) => q.eq('authorId', authorId))
  .filter((q) => q.eq(q.field('status'), 'active'))
  .take(50);

// Map patterns
const ids = results.map((r) => r.id); // ✅ Sync: use regular map
const enriched = await asyncMap(results, async (r) => {
  // ✅ Async: use asyncMap
  const user = await ctx.table('users').get(r.userId);
  return { ...r, user };
});

// Count patterns - AVOID fetching all just to count
// ❌ INEFFICIENT: const count = (await ctx.table('posts').query().collect()).length;
// ✅ EFFICIENT: Use aggregates for counts - see [convex-aggregate.mdc](mdc:.cursor/rules/convex-aggregate.mdc)
```

**Relationship patterns:**

```typescript
// One-to-many (scalable)
defineTable({ authorId: v.id('users') }).index('authorId', ['authorId']);

// Many-to-many (join table)
defineTable({
  userId: v.id('users'),
  channelId: v.id('channels'),
})
  .index('userId', ['userId'])
  .index('channelId', ['channelId'])
  .index('user_channel', ['userId', 'channelId']);
```

**Performance:** N+1 is fast (~1ms/doc) | Arrays max 10 items | Use indexes  
**Count Operations:** For counting documents, use [convex-aggregate.mdc](mdc:.cursor/rules/convex-aggregate.mdc) instead of `.length` on collections

## Typescript guidelines

### Core type patterns

```typescript
// Document IDs
type UserId = Id<'users'>; // Use Id<T> for document IDs
const userId = localStorage.getItem('id') as Id<'users'>; // Cast external IDs

// Infer complex types (NEVER manually type)
const mainCharacter = await (async () => {
  if (!condition) return null;
  return { id: char._id, image: char.image ?? null }; // Auto-inferred
})();

// Record with IDs
const idToUsername: Record<Id<'users'>, string> = {};

// Arrays and unions
const array: Array<string> = ['a', 'b'];
const status = 'active' as const; // Use 'as const' for literals
```

### Helper types

```typescript
// Auth-aware context (from convex/functions.ts), NEVER use ctx: any
AuthQueryCtx, PublicQueryCtx, AuthMutationCtx, PublicMutationCtx;

// Return type inference (client-side)
import { FunctionReturnType } from 'convex/server';
type MyData = FunctionReturnType<typeof api.users.get>;

// ❌ NEVER use: Awaited<ReturnType<typeof api.function>>
```

**Rules:** No `any` | No manual complex types | Strict ID types

## Full text search guidelines

**Edit:** `convex/schema.ts` for search indexes

### Define search indexes

```typescript
// convex/schema.ts
export default defineSchema({
  messages: defineTable({
    body: v.string(),
    channel: v.string(),
    userId: v.id('users'),
  }).searchIndex('search_body', {
    searchField: 'body',
    filterFields: ['channel', 'userId'], // Fast equality filters
  }),
});
```

### Run search queries

```typescript
// Basic search with filters
const messages = await ctx
  .table('messages')
  .query()
  .withSearchIndex(
    'search_body',
    (q) =>
      q
        .search('body', 'hello world') // Full-text search
        .eq('channel', '#general') // Filter field
        .eq('userId', userId) // Multiple filters
  )
  .take(10); // Always limit results

// Search with post-filtering (less efficient)
const recent = await ctx
  .table('messages')
  .query()
  .withSearchIndex('search_body', (q) => q.search('body', 'hello'))
  .filter((q) => q.gt(q.field('_creationTime'), Date.now() - 3600000))
  .take(10);
```

For advanced patterns, see [convex-search.mdc](mdc:.cursor/rules/convex-search.mdc).

## Query guidelines

- **CRITICAL: NEVER fetch ALL documents when you only need a subset**:
  - Use `.take(n)` to limit results: `ctx.table('table').query().take(100)` (default is 100 if not specified)
  - Use `.first()` for single document: `ctx.table('table').query().first()`
  - Use `.get()` for fetching by ID or unique field: `ctx.table('table').get(id)` or `ctx.table('table').get('email', email)`
  - Use `.paginate()` for user-facing lists: see [convex-optimize.mdc](mdc:.cursor/rules/convex-optimize.mdc)
  - Only use `.collect()` when you truly need ALL documents (rare, <1000 docs)
  - **For counts:** Use [convex-aggregate.mdc](mdc:.cursor/rules/convex-aggregate.mdc) instead of `.collect().length`
- **OPTIMIZATION:** Always use indexes first for filtering:
  - **Use indexes:** Define an index in schema and use `withIndex` for equality filters
  - **Get by index:** Use `.get('indexName', value)` for unique lookups
  - **Simple cases:** Use built-in `.filter()` for simple field comparisons after indexing
  - **Complex + Paginated:** Use streams from `convex-helpers/server/stream` for consistent page sizes (streams require `ctx.db` in first param only)
  - See [convex-optimize.mdc](mdc:.cursor/rules/convex-optimize.mdc) and [convex-streams.mdc](mdc:.cursor/rules/convex-streams.mdc)
- Ents queries support chaining: `.query().withIndex().filter().take()`
- Use `.unique()` to get a single document from a query. This method will throw an error if there are multiple documents that match the query.
- When using async iteration, don't use `.collect()` or `.take(n)` on the result of a query. Instead, use the `for await (const row of query)` syntax.

### Query Filtering Guidelines

**⚠️ CRITICAL: Use the Right Tool for Filtering**

1. **For simple filters (paginated or not):** Use built-in `.filter()` - maintains full page sizes!
2. **For complex filters WITHOUT pagination:** Use `filter` helper from `convex-helpers/server/filter`
3. **For complex filters WITH pagination:** Use streams from `convex-helpers/server/stream`

#### Simple Filtering (With or Without Pagination)

```typescript
// ✅ GOOD: Built-in .filter() for simple cases
const activeUsers = await ctx
  .table('users')
  .query()
  .withIndex('status', (q) => q.eq('status', 'active'))
  .filter((q) => q.gt(q.field('lastSeen'), Date.now() - 3600000))
  .take(10);

// Built-in .filter() supports:
// - eq, neq, gt, gte, lt, lte
// - and, or, not
// - Field comparisons only
```

#### Complex Filtering (Arrays, Strings, Async)

**Problem:** Built-in `.filter()` is limited to simple field comparisons.

**Solutions:**

**1. Without Pagination - Use Filter Helper:**

```typescript
import { filter } from 'convex-helpers/server/filter';

// ✅ GOOD: Filter helper for complex filters without pagination
const featuredPosts = await filter(
  ctx
    .table('posts')
    .query()
    .withIndex('author', (q) => q.eq('author', authorId)),
  (post) => post.tags.includes('featured') && post.views > 100
).take(10);

// Works with: .first(), .unique(), .take(), .collect()
// DON'T use with: .paginate() (causes variable page sizes)
```

**2. With Pagination - Use Streams:**

```typescript
import { stream } from 'convex-helpers/server/stream';
import schema from './schema';

// ✅ BEST: Streams filter BEFORE pagination - consistent page sizes!
export const searchCharacters = createPublicPaginatedQuery()({
  args: {
    category: z.string().optional(),
    tags: z.array(z.string()).optional(),
  },
  handler: async (ctx, args) => {
    return await stream(ctx.db, schema) // ⚠️ Stream requires ctx.db here ONLY
      .query('characters')
      .withIndex('private', (q) => q.eq('private', false))
      .filterWith(async (char) => {
        // Full TypeScript power: arrays, strings, async lookups
        if (args.category && !char.categories?.includes(args.category)) {
          return false;
        }
        if (args.tags && !args.tags.some((tag) => char.tags?.includes(tag))) {
          return false;
        }
        // Inside stream operations, use ctx.table()!
        const author = await ctx.table('users').get(char.userId);
        return author && !author.isBanned;
      })
      .paginate(args.paginationOpts); // Always returns requested items!
  },
});
```

**When to Use Each:**

| Use Case                             | Tool                 | Why                            |
| ------------------------------------ | -------------------- | ------------------------------ |
| Simple field comparisons             | Built-in `.filter()` | Maintains full page sizes      |
| Complex filters + `.take()/.first()` | `filter` helper      | Full TypeScript, no pagination |
| Complex filters + `.paginate()`      | Streams              | Consistent page sizes          |
| Array/string operations + paginate   | Streams              | Filter before pagination       |
| Async lookups in filter              | `filter` or Streams  | Both support async             |

For complete stream patterns including unions, joins, and more, see [convex-streams.mdc](mdc:.cursor/rules/convex-streams.mdc).

## 🚨 CRITICAL: Never Use ctx.db

**FORBIDDEN:** `ctx.db` is banned in all Convex functions. Always use `ctx.table()` for database operations.

**ONLY EXCEPTION:** Streams from `convex-helpers/server/stream` still require `ctx.db` in the first parameter only:

```typescript
// ❌ NEVER: ctx.db.query('users')
// ✅ ALWAYS: ctx.table('users').query()

// ⚠️ Exception: Stream initialization ONLY
import { stream } from 'convex-helpers/server/stream';
// Stream requires ctx.db in first parameter
const myStream = stream(ctx.db, schema).query('posts');

// BUT: Inside stream operations, you CAN and SHOULD use ctx.table()!
.filterWith(async (post) => {
  // ✅ Use ctx.table() inside stream operations
  const author = await ctx.table('users').get(post.authorId);
  return author && !author.isBanned;
})
```

### Ordering

- By default Convex always returns documents in ascending `_creationTime` order.
- You can use `.order('asc')` or `.order('desc')` to pick whether a query is in ascending or descending order. If the order isn't specified, it defaults to ascending.
- Document queries that use indexes will be ordered based on the columns in the index and can avoid slow table scans.

## Mutation guidelines

- Use `.replace()` to fully replace an existing document: `await ctx.table('users').getX(id).replace(data)`
- Use `.patch()` to shallow merge updates: `await ctx.table('users').getX(id).patch({ name: 'New' })`
- Use `.delete()` to remove a document: `await ctx.table('users').getX(id).delete()`
- Use `.insert()` to create new documents: `await ctx.table('users').insert(data)`
- `.getX()` throws if document not found, `.get()` returns null
- Isolate frequently-changing data (timestamps, counters) in separate tables - see [convex-optimize.mdc](mdc:.cursor/rules/convex-optimize.mdc)

### Writing with Ents

```typescript
// Insert and get entity
const task = await ctx.table('tasks').insert({ text: 'Build feature' }).get();

// Insert with 1:1 or 1:many edges
await ctx.table('messages').insert({
  text: 'Hello world',
  userId, // Creates edge to user
});

// Insert with many:many edges
await ctx.table('messages').insert({
  text: 'Tagged message',
  tags: [tagId1, tagId2], // Creates edges to tags
});

// Update many:many edges
await ctx
  .table('messages')
  .getX(messageId)
  .patch({
    tags: { add: [tagId3], remove: [tagId1] },
  });

// Replace many:many edges (replaces all)
await ctx
  .table('messages')
  .getX(messageId)
  .replace({
    text: 'Updated text',
    tags: [tagId2, tagId3], // Replaces all tag edges
  });

// Edge mutations
await ctx
  .table('users')
  .getX(userId)
  .edgeX('profile')
  .patch({ bio: 'New bio' });
```

### Bulk operations

- **OPTIMIZATION: Loop inserts/updates are efficient** - Convex batches all database changes in a single transaction, no need for `Promise.all()`
- **Limit bulk operations:** Process in batches of 100-1000 to avoid timeout

  ```typescript
  // ✅ Efficient - all inserts execute in one transaction
  for (const item of items) {
    await ctx.table('table').insert(item);
  }

  // ❌ Unnecessary - adds complexity without performance benefit
  await Promise.all(items.map((item) => ctx.table('table').insert(item)));

  // ✅ For large datasets, process in batches
  const batchSize = 500;
  for (let i = 0; i < items.length; i += batchSize) {
    const batch = items.slice(i, i + batchSize);
    for (const item of batch) {
      await ctx.table('table').insert(item);
    }
  }

  // ✅ Bulk delete example
  const toDelete = await ctx
    .table('posts')
    .query()
    .withIndex('authorId', (q) => q.eq('authorId', authorId))
    .collect();

  for (const post of toDelete) {
    await post.delete(); // Entity method
  }
  ```

## Action guidelines

- Always add `"use node";` to the top of files containing actions that use Node.js built-in modules.
- Never use `ctx.table` or `ctx.db` inside of an action. Actions don't have access to the database.
- Use `createAction()` for public actions and `createInternalAction()` for internal actions:

```ts
'use node';

import { createAction } from './functions';
import { z } from 'zod';

// Public action
export const exampleAction = createAction()({
  args: {},
  returns: z.null(),
  handler: async (ctx, args) => {
    console.info('This action does not return anything');
    return null;
  },
});
```

## Scheduling guidelines

For cron jobs and scheduling patterns, see [convex-scheduling.mdc](mdc:.cursor/rules/convex-scheduling.mdc).

## Examples

For complete examples including chat applications and other patterns, see [convex-examples.mdc](mdc:.cursor/rules/convex-examples.mdc).

## Deletion Behaviors

**Edit:** `convex/schema.ts`

Convex Ents automatically handles cascading deletes. **Hard deletion is now the default behavior** - when you delete an ent, related ents are automatically hard deleted based on edge relationships:

```typescript
defineEnt({
  name: v.string(),
})
  .deletion('soft') // Soft delete with deletionTime field
  .deletion('scheduled', { delayMs: 24 * 60 * 60 * 1000 }) // Delete after 24h
  .edge('profile') // Profile is automatically deleted when user is deleted (default: hard)
  .edge('files', { to: '_storage' }); // Files are automatically deleted (default: hard)

// Soft delete operations
await ctx.table('users').getX(userId).delete(); // Sets deletionTime
await ctx.table('users').getX(userId).patch({ deletionTime: undefined }); // Undelete
```

For detailed deletion patterns and rules, see [convex-ents.mdc](mdc:.cursor/rules/convex-ents.mdc).

## Triggers

**Edit:** `convex/triggers.ts`, `convex/functions.ts`

Triggers automatically run code when data changes. For complete trigger documentation, see [convex-trigger.mdc](mdc:.cursor/rules/convex-trigger.mdc).

```typescript
// Setup in functions.ts
import { Triggers } from 'convex-helpers/server/triggers';
export const triggers = new Triggers<DataModel>();
import './triggers'; // Import trigger registrations

// Common patterns in triggers.ts
// Aggregate maintenance - ALWAYS use .trigger()
triggers.register('characterWorks', aggregateCharacterWorks.trigger());

// Data validation
triggers.register('users', async (ctx, change) => {
  if (change.newDoc && !change.newDoc.email.includes('@')) {
    throw new Error('Invalid email');
  }
});
```

## Convex Aggregate

**Edit:** `convex/aggregates.ts`

**CRITICAL for Scale:** Use aggregates instead of `.collect().length` for counts. O(log n) vs O(n) performance.

**🚨 CRITICAL: ALWAYS use triggers to keep aggregates in sync. NEVER update aggregates manually in mutations.**

**⚠️ IMPORTANT: All aggregate components must be registered in `convex/convex.config.ts` before they can be used.**

For complete aggregate documentation, see [convex-aggregate.mdc](mdc:.cursor/rules/convex-aggregate.mdc).

```typescript
// ❌ INEFFICIENT: Fetches all documents just to count
const count = (await ctx.table('scores').query().collect()).length;

// ✅ EFFICIENT: O(log n) count operation with triggers
const leaderboard = new TableAggregate<...>(components.leaderboard, {
  namespace: (doc) => doc.gameId,
  sortKey: (doc) => doc.score,
});

// STEP 1: Register component in convex/convex.config.ts
// app.use(aggregate, { name: 'leaderboard' });

// STEP 2: Set up automatic aggregate updates in triggers.ts
triggers.register('scores', leaderboard.trigger());

// STEP 3: In mutations - Just do normal operations, triggers handle aggregates!
await ctx.table('scores').insert(data); // That's it! Aggregate updates automatically
await ctx.table('scores').getX(scoreId).delete(); // Triggers fire automatically!

// STEP 4: Read O(log n) operations
const count = await leaderboard.count(ctx, { namespace: gameId });
const rank = await leaderboard.indexOf(ctx, score);

// ❌ NEVER do manual updates
await leaderboard.insert(ctx, doc); // WRONG!
await leaderboard.delete(ctx, doc); // WRONG!
```

## Shared Code Organization

- **Shared helpers**: ALWAYS move shared code from `src/` to `convex/shared/` instead of duplicating
- **Model exception**: `[modelName]Shared.ts` can be next to `[modelName].ts`, not in `convex/shared/` folder
- **Import paths**:
  - From Convex: `import { helper } from './shared/filename'` or `import { helper } from './characterShared'`
  - From Next.js: `import { helper } from '@convex/shared/filename'` or `import { helper } from '@convex/characterShared'`

## See Also

For specific patterns and detailed examples:

- [convex-aggregate.mdc](mdc:.cursor/rules/convex-aggregate.mdc) - Efficient aggregation operations
- [convex-examples.mdc](mdc:.cursor/rules/convex-examples.mdc) - Complete application examples
- [convex-scheduling.mdc](mdc:.cursor/rules/convex-scheduling.mdc) - Cron jobs and scheduled functions
- [convex-http.mdc](mdc:.cursor/rules/convex-http.mdc) - HTTP endpoints and webhooks
- [convex-search.mdc](mdc:.cursor/rules/convex-search.mdc) - Full-text search implementation
- [convex-optimize.mdc](mdc:.cursor/rules/convex-optimize.mdc) - Query optimization for scale
- [convex-trigger.mdc](mdc:.cursor/rules/convex-trigger.mdc) - Database triggers and cascade operations
- [convex-doc.mdc](mdc:.cursor/rules/convex-doc.mdc) - Documentation maintenance guidelines
- [convex-streams.mdc](mdc:.cursor/rules/convex-streams.mdc) - Advanced query patterns with streams

convex-ents.mdc

---
description:
globs:
alwaysApply: false
---

# Convex Ents

Convex Ents are an ergonomic layer on top of the [Convex](https://convex.dev)
built-in [`ctx.db`](https://docs.convex.dev/database) API for reading from and
writing to the database.

Convex Ents:

1. Build upon the relational capabilities of the database to provide an easier
   way to query related documents.
2. Allow defining default values for easier document shape evolution.
3. Simplify backend code by collocating common authorization rules in a single
   place.
4. And more!

Convex Ents provide similar capabilities to
[Prisma ORM](https://www.prisma.io/client), but target only Convex, and use
neither proprietary schema language nor SQL concepts nor code generation. Convex
Ents are a pure TypeScript/JavaScript library.

## Examples

Check out [these code snippets](https://labs.convex.dev/convex-vs-prisma) for
comparison of Prisma, Convex and Convex Ents.

[SaaS Starter](https://github.com/xixixao/saas-starter) is a full project
template built out using Convex Ents.

<Tabs items={["Example query", "mutation", "schema"]}>

<Tabs.Tab>

```ts filename="convex/teams.ts"
export const listTeamInvites = query({
  args: { teamId: v.id('teams') },
  async handler(ctx, { teamId }) {
    return await ctx
      .table('teams')
      .getX(teamId)
      .edge('invites')
      .map(async (invite) => ({
        _id: invite._id,
        email: invite.email,
        role: (await invite.edge('role')).name,
      })); // `{ _id: Id<"invites">, email: string, role: string }[]`
  },
});
```

</Tabs.Tab>
<Tabs.Tab>

```ts filename="convex/teams.ts"
export const acceptInvite = mutation({
  args: { inviteId: v.id('invites') },
  async handler(ctx, { inviteId }) {
    const invite = await ctx.table('invites').getX(inviteId);
    await ctx.table('members').insert({
      teamId: invite.teamId,
      userId: ctx.viewerId,
      roleId: invite.roleId,
    });
    await invite.delete();
    return (await invite.edge('team')).slug;
  },
});
```

</Tabs.Tab>
<Tabs.Tab>

```ts filename="convex/schema.ts"
const schema = defineEntSchema({
  teams: defineEnt({
    name: v.string(),
  })
    .field('slug', v.string(), { unique: true })
    .edges('members', { ref: true })
    .edges('invites', { ref: true }),

  members: defineEnt({}).edge('team').edge('user').edge('role'),

  invites: defineEnt({})
    .field('email', v.string(), { unique: true })
    .edge('team')
    .edge('role'),

  roles: defineEnt({
    isDefault: v.boolean(),
  }).field('name', v.union(v.literal('Admin'), v.literal('Member')), {
    unique: true,
  }),

  users: defineEnt({}).edges('members', { ref: true }),
});
```

</Tabs.Tab>
</Tabs>

## I'm intrigued, what now?

Read the [Ent Schema](/schema) page to understand the improved data modeling
that Convex Ents enable, and the [Reading Ents](/read) and
[Writing Ents](/write) to see the more powerful interface to the database.

If you're sold, head over to [Setup](/setup) to get started.

### Create a schema

Create your schema file, for example:

```ts filename="convex/schema.ts"
import { v } from 'convex/values';
import { defineEnt, defineEntSchema, getEntDefinitions } from 'convex-ents';

const schema = defineEntSchema({
  messages: defineEnt({
    text: v.string(),
  })
    .edge('user')
    .edges('tags'),

  users: defineEnt({
    name: v.string(),
  }).edges('messages', { ref: true }),

  tags: defineEnt({
    name: v.string(),
  }).edges('messages'),
});

export default schema;

export const entDefinitions = getEntDefinitions(schema);
```

### Modify your schema

Update your schema file. These changes will have no effect on your existing code
using `ctx.db`.

- Replace `defineSchema` with `defineEntSchema`
- Replace all `defineTable` with `defineEnt`
- Add `entDefinitions` as shown

Example changes:

```diff filename="convex/schema.ts"
- import { defineSchema, defineTable } from "convex/server";
+ import { defineEnt, defineEntSchema, getEntDefinitions } from "convex-ents";
import { v } from "convex/values";

- const schema = defineSchema({
+ const schema = defineEntSchema({
-  messages: defineTable({
+  messages: defineEnt({
    text: v.string(),
  }),
-  users: defineTable({
+  users: defineEnt({
    name: v.string(),
  }),
});

export default schema;

+ export const entDefinitions = getEntDefinitions(schema);
```

For more details on declaring the schema see [Ent Schema](/schema).

### Add helper types

Add a `types.ts` file with the following contents:

<Aside title="Click to show">

```ts filename="convex/types.ts"
import { GenericEnt, GenericEntWriter } from 'convex-ents';
import { CustomCtx } from 'convex-helpers/server/customFunctions';
import { TableNames } from './_generated/dataModel';
import { mutation, query } from './functions';
import { entDefinitions } from './schema';

export type QueryCtx = CustomCtx<typeof query>;
export type MutationCtx = CustomCtx<typeof mutation>;

export type Ent<TableName extends TableNames> = GenericEnt<
  typeof entDefinitions,
  TableName
>;
export type EntWriter<TableName extends TableNames> = GenericEntWriter<
  typeof entDefinitions,
  TableName
>;
```

</Aside>

### Use custom functions to read and write ents

You can now replace function constructors from `_generated` with the custom ones
you just set up. This will have no impact on your existing functions:

```diff filename="convex/messages.ts"
import { v } from "convex/values";
- import { mutation, query } from "./_generated/server";
+ import { mutation, query } from "./functions";

export const list = query({
  args: {},
  handler: async (ctx) => {
    return await ctx.db.query("messages");

+    // You can now use ctx.table as well:
+    return await ctx.table("messages");
  },
});
```

Similarly replace `QueryCtx` and `MutationCtx` with imports from `./types`.

You can now use `ctx.table` for new code and to replace existing code.

1. The `table` API comes with an [additional level of security](/read#security)
2. The `table` API preserves invariants, such as:
   - fields having unique values
   - 1:1 edges being unique on each end of the edge
   - deleting ents deletes corresponding edges

# Ent Schema

Ents (short for entity) are Convex documents, which allow explicitly declaring
edges (relationships) to other documents. The simplest ent has no fields besides
the built-in `_id` and `_creationTime` fields, and no declared edges. Ents can
contain all the same field types Convex documents can. Ents are stored in tables
in the Convex database.

Unlike bare documents, ents require a schema. Here's a minimal example of the
`convex/schema.ts` file using Ents:

```ts filename="convex/schema.ts"
import { v } from 'convex/values';
import { defineEnt, defineEntSchema, getEntDefinitions } from 'convex-ents';

const schema = defineEntSchema({
  messages: defineEnt({
    text: v.string(),
  }).edge('user'),

  users: defineEnt({
    name: v.string(),
  }).edges('messages', { ref: true }),
});

export default schema;

export const entDefinitions = getEntDefinitions(schema);
```

Compared to a [vanilla schema file](https://docs.convex.dev/database/schemas):

- `defineEntSchema` replaces `defineSchema` from `convex/server`
- `defineEnt` replaces `defineTable` from `convex/server`
- Besides exporting the `schema`, which is used by Convex for schema validation,
  you also export `entDefinitions`, which include the runtime information needed
  to enable retrieving Ents via edges and other features.

## Fields

An Ent field is a field in its backing document. Some types of [edges](#edges)
add additional fields not directly specified in the schema.

Ents add field configurations beyond vanilla Convex.

### Indexed fields

`defineEnt` provides a shortcut for declaring a field and a simple index over
the field. Indexes allow efficient point and range lookups and efficient sorting
of the results by the indexed field. The following schema:

```ts
defineEntSchema({
  users: defineEnt({}).field('email', v.string(), { index: true }),
});
```

declares that "users" ents have one field, `"email"` of type `string`, and one
index called `"email"` over the `"email"` field. It is exactly equivalent to the
following schema:

```ts
defineEntSchema({
  users: defineEnt({
    email: v.string(),
  }).index('email', ['email']),
});
```

### Unique fields

Similar to an indexed field, you can declare that a field must be unique in the
table. This comes at a cost: On every write to the backing table, it must be
checked for an existing document with the same field value. Every unique field
is also an indexed field (as the index is used for an efficient lookup).

```ts
defineEntSchema({
  users: defineEnt({}).field('email', v.string(), { unique: true }),
});
```

### Field defaults

When evolving a schema, especially in production, the simplest way to modify the
shape of documents in the database is to add an optional field. Having an
optional field means that your code either always has to handle the "missing"
value case (the value is `undefined`), or you need to perform a careful
migration to backfill all the documents and set the field value.

Ents simplify this shape evolution by allowing to specify a default for a field.
The following schema:

```ts
defineEntSchema({
  posts: defineEnt({}).field(
    'contentType',
    v.union(v.literal('text'), v.literal('video')),
    { default: 'text' }
  ),
});
```

declares that "posts" ents have one **required** field `"contentType"`,
containing a string, either `"text"` or `"video"`. When the value is missing
from the backing document, the default value `"text"` is returned. Without
specifying the default value, the schema could look like:

```ts
defineEntSchema({
  posts: defineEnt({
    contentType: v.optional(v.union(v.literal('text'), v.literal('video'))),
  }),
});
```

but for this schema the `contentType` field missing must be handled by the code
reading the ent.

### Adding fields to all union variants

If you use a [union](https://docs.convex.dev/database/schemas#unions) as an
ent's schema, you can add a field to all the variants:

```ts
defineEntSchema({
  posts: defineEnt(
    v.union(
      v.object({
        type: v.literal('text'),
        content: v.string(),
      }),
      v.object({
        type: v.literal('video'),
        link: v.string(),
      })
    )
  ).field('author', v.id('users')),
});
```

adds an `author` field to both text and video posts.

## Edges

An edge is a representation of some business logic modeled by the database. Some
examples are:

- User A liking post X
- User B authoring post Y
- User C is friends with user D
- Folder F is a child of folder G

Every edge has two "ends", each being an ent. Those ents can be stored in the
same or in 2 different tables. Edges can represent symmetrical relationships.
The "friends with" edge is an example. If user C is friends with user D, then
user D is friends with user C. Symmetrical edge only make sense if both ends of
the edge point to Ents in the same table. Edges which are not symmetrical have
two names, one for each direction. For the user liking a post example, one
direction can be called "likedPosts" (from users to posts), the other "likers"
(from posts to users).

Edges can also declare how many ents can be connected through the same edge. For
each end, there can be 0, 1 or many ents connected to the same ent on the other
side of the edge. For example, a user (represented by an ent) has a profile
(represented by an ent). In this case we call this a 1:1 edge. If we ask "how
many profiles does a user X have?" the answer is always 1. Similarly there can
be 1:many edges, such as a user with the messages they authored, when each
message has only a single author; and many:many edges, such as messages to tags,
where each message can have many tags, and each tag can be attached to many
messages.

Now that you understand all the properties of edges, here's how you can declare
them: Edges are always declared on the ents that constitute its ends. Let's take
the example of users authoring messages:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edges('messages', { ref: true }),
  messages: defineEnt({
    text: v.string(),
  }).edge('user'),
});
```

In this example, the edge between "users" and "messages" is 1:many, each user
can have many associated messages, but each message has only a single associated
user. The syntax is explained below.

### Understanding how edges are stored

To further understand the ways in which edges can be declared, we need to
understand the difference in how they are stored. There are two ways edges are
stored in the database:

1. _Field edges_ are stored as a single foreign key column in one of the two
   connected tables. All 1:1 and 1:many edges are field edges.
2. _Table edges_ are stored as documents in a separate table. All many:many
   edges are table edges.

In the example above the edge is stored as an `Id<"users>` on the "messages"
document.

### 1:1 edges

1:1 edges are in a way a special case of 1:many edges. In Convex, one end of the
edge must be optional, because there is no way to "allocate" IDs before
documents are created (see
[circular references](https://docs.convex.dev/database/schemas#circular-references)).
Here's a basic example of a 1:1 edge, defined for each ent using the `edge`
(singular) method:

<Graphic src={edge} dark={edgeDark} alt="1:1 edges pictogram" />

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edge('profile', { ref: true }),
  profiles: defineEnt({
    bio: v.string(),
  }).edge('user'),
});
```

In this case, each user can have 1 profile, and each profile must have 1
associated user. This is a field edge stored on the "profiles" table as a
foreign key. The "users" table's documents do not store the edge, because the
`ref: true` option specifies that the edge a "refers" to the field on the other
end of the edge.

The syntax shown is actually a shortcut for the following declaration:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edge('profile', { to: 'profiles', ref: 'userId' }),
  profiles: defineEnt({
    bio: v.string(),
  }).edge('user', { to: 'users', field: 'userId' }),
});
```

The available options are:

- `to` is the table storing ents on the other end of the edge. It defaults to
  edge name suffixed with `s` (edge `profile` -> table `"profiles"`). You'll
  want to specify it when this simple pluralization doesn't work (like edge
  `category` and table `"categories"`).
- `ref` signifies that the edge is stored in a field on the other ent. It can
  either be the literal `true`, or the actual field's name. You must specify the
  name when you want to have another field edge between the same pair of tables.
- `field` is the name of the field that stores the foreign key. It defaults to
  the edge name suffixed with `Id` (edge `user` -> field `userId`).

The edge names are used when querying the edges, but they are not stored in the
database (the field name is, as part of each document that stores its value).

#### Optional 1:1 edges

You can make the field storing the edge optional with the `optional` option.

Shortcut syntax:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edge('profile', { ref: true }),
  profiles: defineEnt({
    bio: v.string(),
  }).edge('user', { field: 'userId', optional: true }),
});
```

You must specify the `field` name when using `optional`.

Fully specified:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edge('profile', { to: 'profiles', ref: 'userId' }),
  profiles: defineEnt({
    bio: v.string(),
  }).edge('user', { to: 'users', field: 'userId', optional: true }),
});
```

In this example a profile can be created without a set `userId`.

#### 1:1 edges to system tables

You can connect ents to documents in system tables via 1:1 edges, see
[File Storage](/schema/files) and [Scheduled Functions](/schema/schedule) for
details.

### 1:many edges

1:many edges are very common, and map clearly to foreign keys. Take this
example, where the edge is defined via the `edge` (singular) and `edges`
(plural) method:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edges('messages', { ref: true }),
  messages: defineEnt({
    text: v.string(),
  }).edge('user'),
});
```

<Graphic src={edgesRef} dark={edgesRefDark} alt="1:many edge pictogram" />

This is a 1:many edge because the `edges` (plural) method is used on the "users"
ent and the `ref: true` option is specified. The `ref` option declares that the
edges are stored on a field in the other table. In this example each user can
have multiple associated messages.

The syntax shown is actually a shortcut for the following declaration:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edges('messages', { to: 'messages', ref: 'userId' }),
  messages: defineEnt({
    text: v.string(),
  }).edge('user', { to: 'users', field: 'userId' }),
});
```

The available options are:

- `to` is the table storing ents on the other end of the edge.
  - for the `edges` method, it defaults to the edge name (edges `messages` ->
    table `"messages"`)
  - for the `edge` method, it defaults to the edge name suffixed with `s` (edge
    `user` -> table `"users"`).
  - You'll need to specify `to` when the defaults don't match your table names.
- `ref` signifies that the edge is stored in a field on the other ent. It can
  either be the literal `true`, or the actual field's name. You must specify the
  name when you want to have another field edge between the same pair of tables
  (to identify the inverse edge).
- `field` is the name of the field that stores the foreign key. It defaults to
  the edge name suffixed with `Id` (edge `user` -> field `userId`).

#### Optional 1:many edges

You can make the field storing the edge optional with the `optional` option.

Shortcut syntax:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edges('messages', { ref: true }),
  messages: defineEnt({
    text: v.string(),
  }).edge('user', { field: 'userId', optional: true }),
});
```

You must specify the `field` name when using `optional`.

Fully specified:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edges('messages', { to: 'messages', ref: 'userId' }),
  messages: defineEnt({
    text: v.string(),
  }).edge('user', { to: 'users', field: 'userId', optional: true }),
});
```

In this case a message can be created without a set `userId`.

### many:many edges

Many:many edges are always stored in a separate table, and both ends of the edge
use the `edges` (plural) method:

```ts
defineEntSchema({
  messages: defineEnt({
    name: v.string(),
  }).edges('tags'),
  tags: defineEnt({
    text: v.string(),
  }).edges('messages'),
});
```

<Graphic src={edges} dark={edgesDark} alt="many:many edge pictogram" />

In this case the table storing the edge is called `messages_to_tags`, based on
the tables storing each end of the edge.

The syntax shown is actually a shortcut for the following declaration:

```ts
defineEntSchema({
  messages: defineEnt({
    name: v.string(),
  }).edges('tags', {
    to: 'tags',
    table: 'tags_to_messages',
    field: 'tagsId',
  }),
  tags: defineEnt({
    text: v.string(),
  }).edges('messages', {
    to: 'messages',
    table: 'tags_to_messages',
    field: 'messagesId',
  }),
});
```

The available options are:

- `to` is the table storing ents on the other end of the edge. It defaults to
  the edge name (edges `tags` -> table `"tags"`). You can specify it if you want
  to call the edge something more specific.
- `table` is the name of the table storing the edges. This table will have two
  ID fields, one for each end of the edge. You must specify the name when you
  want to have multiple different edges connecting the same pair of tables.
  These tables are only used by the framework under the hood, and won't appear
  in your code.
- `field` is the name of the field on `table` that stores the ID of the ent on
  this end of the edge. It defaults to the edge name with suffixed with `Id`
  (edge `tags` -> field `tagsId`). These fields will only be used by the
  framework under the hood, and won't appear in your code.

#### Asymmetrical self-directed many:many edges

Self-directed edges have the ents on both ends of the edge stored in the same
table.

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edges('followers', { to: 'users', inverse: 'followees' }),
});
```

<Graphic
  src={edgesSelfAsymmetric}
  dark={edgesSelfAsymmetricDark}
  alt="many:many edge pictogram"
/>

Self-directed edges point to the same table on which they are defined via the
`to` option. For the edge to be asymmetrical, it has to specify the `inverse`
name. In this example, if this edge is between user A and user B, B is a
"followee" of A (is being followed by A), and A is a "follower" of B.

The table storing the edges is named after the edges, and so are its fields. You
can also specify the `table`, `field` and `inverseField` options to control how
the edge is stored and to allow multiple self-directed edges.

#### Symmetrical self-directed many:many edges

Symmetrical edges also have the ents on both ends of the edge stored in the same
table, but additionally they "double-write" the edge for both directions:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edges('friends', { to: 'users' }),
});
```

<Graphic
  src={edgesSelfSymmetric}
  dark={edgesSelfSymmetricDark}
  alt="many:many edge pictogram"
/>

By not specifying the `inverse` name, you're declaring the edge as symmetrical.
You can also specify the `table`, `field` and `inverseField` options to control
how the edge is stored and to allow multiple symmetrical self-directed edges.

Other kinds of edges are possible, but less common.

## Rules

Rules allow collocating the logic for when an ent can be created, read, updated
or deleted.

See the [Rules page](/schema/rules).

# Helper Types

These types are helpful when you want to break down your backend code and pass
IDs and ents around.

## Ent ID type

Use the built-in `Id` type:

```ts filename=convex/myFunctions.ts
import { Id } from './_generated/dataModel';

// Note that a `MutationCtx` also satisfies the `QueryCtx` interface
export function myReadHelper(ctx: QueryCtx, id: Id<'tasks'>) {
  /* ... */
}
```

## Ent types & `ctx` types

```ts filename=convex/myFunctions.ts
import { Ent, EntWriter } from './shared/types';

export function myReadHelper(ctx: PublicQueryCtx, task: Ent<'tasks'>) {
  /* ... */
}

export function myWriteHelper(
  ctx: PublicMutationCtx,
  task: EntWriter<'tasks'>
) {
  /* ... */
}
```

# Cascading Deletes

Convex Ents are designed to simplify the creation of interconnected graphs of
documents in the database. Deleting ents connected through edges poses three
main challenges:

1. Propagating deletion across edges. When an ent is required on one end of an
   edge, and it is deleted, the edge and potentially the ent on the other end
   must be deleted as well.

   > Example: Consider an app with "teams" of "users". When a team is deleted,
   > its members, projects and other data belonging to the team should be
   > deleted as well.

2. Handling the volume of deleted documents. It is not possible to instantly
   erase a very large number of documents, from any database. Eventually there
   can be too many documents to delete, especially inside a single transaction.

   > Example: A team can have thousands of members and tens of thousands of
   > projects. These cannot all be deleted instantly.

3. Soft deleting and retaining data before final deletion. Often the data should
   not be immediately erased from the database.

   > Example: A team admin can delete a team, but you want to have the ability
   > to easily reinstate their data, in case the admin changes their mind, or
   > the request was fradulent.

   > Example: A user can leave a team and later rejoin it, reacquiring
   > attribution to data that was previously connected to them.

## Default deletion behavior

Without any additional configuration, ents and their edges are deleted
immediately. We'll also refer to this as "hard" deleted.

If the edge is required, as is the case for 1:many and 1:1 edges for the
[ents storing the edge as a field](/schema#understanding-how-edges-are-stored),
the ents on the other side of the edge are deleted as well.

The following scenarios are currently supported:

- 1:1 edge between ent A and ent B, ents A store the edge.
  - When ent A is deleted, only _it_ is deleted.
  - When ent B is deleted, the ent A connected to it is deleted as well (which
    might cause more edge and ent deletions).
- 1:many edge between ent A and ent B, ents A store the edge.
  - When ent A is deleted, only _it_ is deleted.
  - When ent B is deleted, ents A connected to it are deleted as well (which
    might cause more edge and ent deletions).
- many:many edge between ent A and ent B.
  - When ent A is deleted, the documents storing the edges to ents B are all
    deleted, but ents B are _not_ deleted.
  - When ent B is deleted, the documents storing the edges to ents A are all
    deleted, but ents A are _not_ deleted.

### Overriding deletion direction for 1:1 edges

For 1:1 edges, you can additionally configure deletion to propagate from the ent
that stores the edge to the other ent, by setting the `deletion` option on the
edge declaration:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).edge('profile', { optional: true }),
  profiles: defineEnt({
    bio: v.string(),
  }).edge('user', { deletion: 'hard' }),
});
```

In this example, when a user is deleted, their profile is deleted as well, which
is the default behavior, but also when a profile is deleted, the user is
deleted.

## Soft deletion behavior

You can configure an ent to use the `"soft"` deletion behavior with the
`deletion` method in your schema:

```ts
defineEntSchema({
  users: defineEnt({
    name: v.string(),
  }).deletion('soft'),
});
```

This behavior adds a `deletionTime` field to the ent. When the ent itself is
"deleted", via the `delete` method, the `deletionTime` field is set to the
current server time.

If the ent is being deleted as a result of cascading hard deletion, it is hard
deleted.

### Filtering soft deleted ents

You can include or exclude soft deleted ents from results by filtering:

```ts
const notDeletedUsers = await ctx
  .table('users')
  .filter((q) => q.eq(q.field('deletionTime'), undefined));
```

### Undeleting soft deleted ents

The ents can be "undeleted" by unsetting the `deletionTime` field, for example:

```ts
await ctx.table('users').getX(userId).patch({ deletionTime: undefined });
```

### Soft edge deletion

#### 1:1 and 1:many edges

By default soft deletion doesn't propagate. You can configure cascading deletes
for soft deletions for individual 1:1 and 1:many edges via the `deletion` option
on edge declarations:

```ts
defineEntSchema({
  users: defineEnt({
    email: v.string(),
  })
    .deletion('soft')
    .edges('profiles', { ref: true, deletion: 'soft' }),

  profiles: defineEnt({
    name: v.string(),
  })
    .deletion('soft')
    .edge('user'),
});
```

The ent on the other end of the edge has to have the `"soft"` deletion behavior.

In this example, when a user is deleted, it is soft deleted, and all its
profiles are also soft deleted immediately. When a profile itself is deleted, it
is also only soft deleted.

Soft deletion of edges happens immediately, within the same transaction.

<Aside title="What if an ent has too many 1:many edges to soft delete immediately?">

If an ent is connected to a large number of other ents, such that propagating
soft deletion to them all could fail the mutation, you should instead filter out
the soft deleted ents after traversing the edge.

</Aside>

#### many:many edges

Soft deletion doesn't affect many:many edges.

## Scheduled deletion behavior

The scheduled deletion behavior expands on the soft deletion behavior. The ent
is first immediately soft deleted. An actual hard deletion is then scheduled.

### Additional configuration

To enable scheduled ent deletion you need to add two lines of code to your
[`functions.ts` file](/setup/config):

```ts
// Add this import
import { scheduledDeleteFactory } from 'convex-ents';

// Add this export
export const scheduledDelete = scheduledDeleteFactory(entDefinitions);
```

This will expose an internal Convex mutation used by `ctx.table` when scheduling
deletions.

<Aside title="Alternatively, you can configure the mutation explicitly">

Expose the mutation somewhere in your `convex` folder, and its reference to the
factory function:

```ts filename="convex/someFileName.ts"
import { scheduledDeleteFactory } from 'convex-ents';
import { entDefinitions } from './schema';

export const myNameForScheduledDelete = scheduledDeleteFactory(entDefinitions, {
  scheduledDelete: internal.someFileName.myNameForScheduledDelete,
});
```

Also pass the function's reference in `functions.ts` file, wherever you set up
your custom `mutation` and `internalMutation` function constructors:

```ts filename="convex/functions.ts" {14, 40-42, 52-54}
export const mutation = customMutation(
  baseMutation,
  customCtx(async (ctx) => {
    return {
      table: entsTableFactory(ctx, entDefinitions, {
        scheduledDelete: internal.someFileName.myNameForScheduledDelete,
      }),
    };
  })
);

export const internalMutation = customMutation(
  baseInternalMutation,
  customCtx(async (ctx) => {
    return {
      table: entsTableFactory(ctx, entDefinitions, {
        scheduledDelete: internal.someFileName.myNameForScheduledDelete,
      }),
    };
  })
);
```

</Aside>

### Defining scheduled deletion behavior

You can configure an ent to use the `"scheduled"` deletion behavior with the
`deletion` method in your schema:

```ts
defineEntSchema({
  users: defineEnt({
    email: v.string(),
  })
    .deletion('scheduled')
    .edges('profiles', { ref: true }),

  profiles: defineEnt({
    name: v.string(),
  }).edge('user'),
});
```

When the ent is deleted, it is first soft deleted.
[Soft edge deletion](#soft-edge-deletion) can apply as well. This all happens
within the same mutation.

The hard deletion is scheduled to a separate mutation/mutations. Cascading
deletes are performed first, then the ent itself is hard deleted. There is no
guarantee on how long this can take, as it depends on the number of documents
that need to be deleted to finish the cascading deletion.

The hard deletion can be delayed into the future with the `delayMs` option:

```ts
defineEntSchema({
  users: defineEnt({
    email: v.string(),
  })
    .deletion('scheduled', { delayMs: 24 * 60 * 60 * 1000 })
    .edges('profiles', { ref: true }),

  profiles: defineEnt({
    name: v.string(),
  }).edge('user'),
});
```

In this example the user ent is soft deleted first, then after 24 hours its
profiles and the user itself are hard deleted.

The delay is only applied if the ent is itself being deleted, not when it is
being deleted as a result of a cascading delete.

### Canceling scheduled deletion

You can cancel a scheduled deletion by unsetting the `deletionTime` field, for
example:

```ts
await ctx.table('users').getX(userId).patch({ deletionTime: undefined });
```

### Correctness

You should make sure that while a scheduled hard deletion is running, there are
no new ents being inserted that would also be eligible for the same cascading
deletion.

You can do this by checking for the soft deletion state of the deleted ent in
your code.

# Scheduled Functions

You can connect
[scheduled functions](https://docs.convex.dev/scheduling/scheduled-functions) to
your ents.

The basic way to do this is the same as in vanilla Convex, adding a scheduled
function ID field:

```ts
defineEntSchema({
  answers: defineEnt({
    question: v.string(),
    actionId: v.id('_scheduled_functions'),
  }),
});
```

You can then retrieve the scheduled function status and cancel it. For example,
to retrieve all answers with the status of their action:

```ts
return ctx.table('answers').map(async ({ question, actionId }) => ({
  question,
  status:
    (await ctx.table('_scheduled_functions').get(actionId)?.state.kind) ??
    'stale',
}));
```

## Using edges for connecting scheduled functions

You can simplify the code above by declaring a 1:1 edge to scheduled functions:

```ts
defineEntSchema({
  answers: defineEnt({
    question: v.string(),
  }).edge('action', { to: '_scheduled_functions' }),
});
```

The field name is derived from the edge name (here `actionId`), you can also
specify it with the `field` option.

You can then retrieve scheduled function status via the edge:

```ts
return ctx.table('answers').map(async (answer) => ({
  question: answer.question,
  status: (await answer.edge('action')?.state.kind) ?? 'stale',
}));
```

### Automatic cancelation during cascading deletion

You can automatically cancel a connected scheduled function via the `deletion`
option:

```ts
defineEntSchema({
  answers: defineEnt({
    question: v.string(),
  }).edge('action', { to: '_scheduled_functions', deletion: 'hard' }),
});
```

Only the `"hard"` value is valid, since there is no way to mark soft deletion on
a scheduled function.

In this example when an answer is deleted, its action, if pending or
in-progress, is canceled.

Note that if you use `ctx.storage.delete` to delete a file that is referenced in
other ents, Ents will not cascade that deletion. Ideally do not use
`ctx.storage.delete`, or handle any other required deletions manually.

# Rules

The ents in your database are only accessible via server-side functions, and so
you can rely on their implementation to enforce authorization rules (also known
as "row level security").

But you might have multiple functions accessing the same data, and you might be
using the different methods provided by Convex Ents to access them:

- To read: `get`, `getX`, `edge`, `edgeX`, `unique`, `uniqueX`, `first`,
  `firstX`, `take`, etc.
- To write: `insert`, `insertMany`, `patch`, `replace`, `delete`

Enforcing rules about when an ent can be read, created, updated or deleted at
every callsite can be onerous and error-prone.

For this reason you can optionally define a set of "rules" implementations that
are automatically enforced by the `ctx.table` API. This is an advanced feature,
and so it requires a bit more setup.

## Setup

Before setting up rules, make sure you understand how Convex Ents are configured
via custom functions, see [Configuring Functions](/setup/config).

<Steps>

### Define your rules

Add a `rules.ts` file with the following contents:

```ts filename="convex/rules.ts" {8-16}
import { addEntRules } from 'convex-ents';
import { entDefinitions } from './schema';
import { QueryCtx } from './types';

export function getEntDefinitionsWithRules(
  ctx: QueryCtx
): typeof entDefinitions {
  return addEntRules(entDefinitions, {
    // "secrets" is one of our tables
    secrets: {
      read: async (secret) => {
        // Example: Only the viewer can see their secret
        return ctx.viewerId === secret.userId;
      },
    },
  });
}

// Example: Retrieve viewer ID using `ctx.auth`:
export async function getViewerId(
  ctx: Omit<QueryCtx, 'table' | 'viewerId' | 'viewer' | 'viewerX'>
): Promise<Id<'users'> | null> {
  const user = await ctx.auth.getUserIdentity();
  if (user === null) {
    return null;
  }
  const viewer = await ctx.skipRules
    .table('users')
    .get('tokenIdentifier', user.tokenIdentifier);
  return viewer?._id;
}
```

The rules are defined in the second argument to `addEntRules`, which takes
`entDefinitions` from our schema, adds any rules you specify and returns
augmented `entDefinitions`.

Authorization commonly has a concept of a viewer, although this is totally up to
your use case. The `rules.ts` file is a good place for defining how to retrieve
the viewer ID.

### Apply rules

Replace your `functions.ts` file with the following code, which uses your
implementations from `rules.ts`:

```ts filename="convex/functions.ts" {15}
export const query = customQuery(
  baseQuery,
  customCtx(async (baseCtx) => {
    return await queryCtx(baseCtx);
  })
);

export const internalQuery = customQuery(
  baseInternalQuery,
  customCtx(async (baseCtx) => {
    return await queryCtx(baseCtx);
  })
);

export const mutation = customMutation(
  baseMutation,
  customCtx(async (baseCtx) => {
    return await mutationCtx(baseCtx);
  })
);

export const internalMutation = customMutation(
  baseInternalMutation,
  customCtx(async (baseCtx) => {
    return await mutationCtx(baseCtx);
  })
);

async function queryCtx(baseCtx: QueryCtx) {
  const ctx = {
    db: baseCtx.db as unknown as undefined,
    skipRules: { table: entsTableFactory(baseCtx, entDefinitions) },
  };
  const entDefinitionsWithRules = getEntDefinitionsWithRules(ctx as any);
  const viewerId = await getViewerId({ ...baseCtx, ...ctx });
  (ctx as any).viewerId = viewerId;
  const table = entsTableFactory(baseCtx, entDefinitionsWithRules);
  (ctx as any).table = table;
  // Example: add `viewer` and `viewerX` helpers to `ctx`:
  const viewer = async () =>
    viewerId !== null ? await table('users').get(viewerId) : null;
  (ctx as any).viewer = viewer;
  const viewerX = async () => {
    const ent = await viewer();
    if (ent === null) {
      throw new Error('Expected authenticated viewer');
    }
    return ent;
  };
  (ctx as any).viewerX = viewerX;
  return { ...ctx, table, viewer, viewerX, viewerId };
}

async function mutationCtx(baseCtx: MutationCtx) {
  const ctx = {
    db: baseCtx.db as unknown as undefined,
    skipRules: { table: entsTableFactory(baseCtx, entDefinitions) },
  };
  const entDefinitionsWithRules = getEntDefinitionsWithRules(ctx as any);
  const viewerId = await getViewerId({ ...baseCtx, ...ctx });
  (ctx as any).viewerId = viewerId;
  const table = entsTableFactory(baseCtx, entDefinitionsWithRules);
  (ctx as any).table = table;
  // Example: add `viewer` and `viewerX` helpers to `ctx`:
  const viewer = async () =>
    viewerId !== null ? await table('users').get(viewerId) : null;
  (ctx as any).viewer = viewer;
  const viewerX = async () => {
    const ent = await viewer();
    if (ent === null) {
      throw new Error('Expected authenticated viewer');
    }
    return ent;
  };
  (ctx as any).viewerX = viewerX;
  return { ...ctx, table, viewer, viewerX, viewerId };
}
```

In this example we pulled out the logic for defining query and mutation `ctx`
into helper functions, so we don't have to duplicate the code between public and
internal constructors (but you can inline this code if you actually need
different setup for each).

The logic for setting up the query and mutation `ctx`s is the same, but we
define them separately to get the right types inferred by TypeScript.

<Aside title="Here's an annotated version of the code with explanation of each step:">

```ts
// The `ctx` object is mutated as we build it out.
// It starts off with `ctx.skipRules.table`, a version `ctx.table`
// that doesn't use the rules we defined in `rules.ts`:
const ctx = {
  db: undefined,
  skipRules: { table: entsTableFactory(baseCtx, entDefinitions) },
};
// We bind our rule implementations to this `ctx` object:
const entDefinitionsWithRules = getEntDefinitionsWithRules(ctx as any);
// We retrieve the viewer ID, without using rules (as our rules
// depend on having the viewer loaded), and add it to `ctx`:
const viewerId = await getViewerId({ ...baseCtx, ...ctx });
(ctx as any).viewerId = viewerId;
// We get a `ctx.table` using rules and add it to `ctx`:
const table = entsTableFactory(baseCtx, entDefinitionsWithRules);
(ctx as any).table = table;
// As an example we define helpers that allow retrieving
// the viewer as an ent. These have to be functions, to allow
// our rule implementations to use them as well.
// Anything that we want our rule implementations to have
// access to has to be added to the `ctx`.
const viewer = async () =>
  viewerId !== null ? await table('users').get(viewerId) : null;
(ctx as any).viewer = viewer;
const viewerX = async () => {
  const ent = await viewer();
  if (ent === null) {
    throw new Error('Expected authenticated viewer');
  }
  return ent;
};
(ctx as any).viewerX = viewerX;
// Finally we again list everything we want our
// functions to have access to. We have to do this
// for TypeScript to correctly infer the `ctx` type.
return { ...ctx, table, viewer, viewerX, viewerId };
```

</Aside>

</Steps>

## Read rules

For each table storing ents you can define a `read` rule implementation. The
implementation is given the ent that is being retrieved, and should return a
`boolean` of whether the ent is readable. This code runs before ents are
returned by `ctx.table`:

- If the retrieval method can return `null`, and the rule returns `false`, then
  `null` is returned. Examples: `get`, `first`, `unique` etc.
- If the retrieval method throws when the ent does not exist, it will also throw
  when the ent cannot be read. Examples: `getX`, `firstX`, `uniqueX`
- If the retrieval method returns a list of ents, then any ents that cannot be
  read will be filtered out.
  - except for `getManyX`, which will throw an `Error`

### Understanding read rules performance

A read rule is essentially a filter, performed in the Convex runtime running
your query or mutation. This means that adding a read rule to a table
fundamentally changes the way methods like `first`, `unique` and `take` are
implemented. These methods need to paginate through the underlying table (or
index range), on top of the scanning that is performed by the built-in `db` API.
You should be mindful of how many ents your read rules might filter out for a
given query.

<Aside title="How exactly do `first`, `unique` and `take` paginate?">
  The methods first try to load the requested number of ents (`1`, `2` or `n`
  respectively). If the ents loaded first get filtered out, the method loads 2
  times more documents, performs the filtering, and if again there aren't enough
  ents, it doubles the number again, and so on, for a maximum of 64 ents being
  evaluated at a time.
</Aside>

### Common read rule patterns

#### Delegating to another ent

Example: _When the user connected to the profile can be read, the profile can be
read_:

```ts
return addEntRules(entDefinitions, {
  profiles: {
    read: async (profile) => {
      return (await profile.edge('user')) !== null;
    },
  },
});
```

Watch out for infinite loops between read rules, and break them up using
`ctx.skipRules`.

#### Testing for an edge

Example: _A user ent can be read when it is the viewer or when there is a
`"friends"` edge between the viewer and the user_:

```ts
return addEntRules(entDefinitions, {
  users: {
    read: async (user) => {
      return (
        ctx.viewerId !== null &&
        (ctx.viewerId === user._id ||
          (await user.edge('friends').has(ctx.viewerId)))
      );
    },
  },
});
```

## Write rules

Write rules determine whether ents can be created, updated or deleted. They can
be specified using the `write` key:

```ts
return addEntRules(entDefinitions, {
  // "secrets" is one of our tables
  secrets: {
    // Note: The read rule is always checked for existing ents
    // for any updates or deletions
    read: async (secret) => {
      return ctx.viewerId === secret.userId;
    },
    write: async ({ operation, ent: secret, value }) => {
      if (operation === 'delete') {
        // Example: No one is allowed to delete secrets
        return false;
      }
      if (operation === 'create') {
        // Example: Only the viewer can create secrets
        return ctx.viewerId === value.ownerId;
      }
      // Example: secret's user edge is immutable
      return value.ownerId === undefined || value.ownerId === secret.ownerId;
    },
  },
});
```

If defined, the `read` rule is always checked first before ents are updated or
deleted.

The `write` rule is given an object with

- `operation`, one of `"create"`, `"update"` or `"delete"`
- `ent`, the existing ent if this is an update or delete
- `value`, the value provided to `.insert()`, `.replace()` or `.patch()`.

The methods `insert`, `insertMany`, `patch`, `replace` and `delete` throw an
`Error` if the `write` rule returns `false`.

## Ignoring rules

Sometimes you might want to read from or write to the database without abiding
by the rules you defined. Perhaps you are running with `ctx` that isn't
authenticated, or your code needs to perform some operation on behalf of a user
who isn't the current viewer.

For this purpose the [Setup](#setup) section above defines
`ctx.skipRules.table`, which is a version of `ctx.table` that can read and write
to the database without checking the rules.

**Remember that methods called on ents retrieved using `ctx.skipRules.table`
also ignore rules!** For this reason it's best to return plain documents or IDs
when using `ctx.skipRules.table`:

```ts
// Avoid!!!
return await ctx.skipRules.table('foos').get(someId);
// Return an ID instead:
return (await ctx.skipRules.table('foos').get(someId))._id;
```

It is preferable to still use Convex Ents over using the built-in `ctx.db` API
for this purpose, to maintain invariants around edges and unique fields. See
[Exposing built-in `db`](/setup/config#exposing-built-in-db).
import { Aside } from "../components/Aside.tsx";

# Reading Ents from the Database

Convex Ents provide a `ctx.table` method which replaces the built-in `ctx.db`
object in Convex [queries](https://docs.convex.dev/functions/query-functions).
The result of calling the method is a custom object which acts as a lazy
`Promise`. If you `await` it, you will get a list of results. But you can
instead call another method on it which will return a different lazy `Promise`,
and so on. This enables a powerful and efficient fluent API.

## Security

The Convex Ents API was designed to add an additional level of security to your
backend code. The built-in `ctx.db` object allows reading data from and writing
data to any table. Therefore in vanilla Convex you must correctly use both
[argument validation](https://docs.convex.dev/functions/args-validation) and
[strict schema validation](https://docs.convex.dev/database/schemas) to avoid
exposing a function which could be misused by an attacker to act on a table it
was not designed to act on.

All Convex Ents APIs require specifying a table up-front. If you need to read
data from an arbitrary table, you must the
[use the built-in `ctx.db`](/setup/config) object.

## Reading a single ent by ID

```ts
const task = await ctx.table('tasks').get(taskId);
```

<Aside title="This is equivalent to the built-in:">

```ts
const id = ctx.db.normalize('tasks', taskId);
if (id === null) {
  return null;
}
const task = await ctx.db.get(id);
```

with the addition of checking that `taskId` belongs to `"tasks"`.

</Aside>

## Reading a single ent by indexed field

```ts
const task = await ctx.table('users').get('email', '[email protected]');
```

<Aside title="This is equivalent to the built-in:">

```ts
const task = await ctx.db
  .query('users')
  .withIndex('email', (q) => q.eq('email', '[email protected]'))
  .unique();
```

</Aside>

## Reading a single ent via a compound index

```ts
const task = await ctx.table('users').get('nameAndRank', 'Steve', 10);
```

<Aside title="This is equivalent to the built-in:">

```ts
const task = await ctx.db
  .query('users')
  .withIndex('nameAndRank', (q) => q.eq('name', 'Steve').eq('rank', 10))
  .unique();
```

</Aside>

## Reading a single ent or throwing

The `getX` method (pronounced "get or throw") throws an `Error` if the read
produced no ents:

```ts
const task = await ctx.table('tasks').getX(taskId);
```

```ts
const task = await ctx.table('users').getX('email', '[email protected]');
```

## Reading multiple ents by IDs

Retrieve a list of ents or nulls:

```ts
const tasks = await ctx.table('tasks').getMany([taskId1, taskId2]);
```

<Aside title="This is equivalent to the built-in:">

```ts
const task = await Promise.all([taskId1, taskId2].map(ctx.db.get));
```

</Aside>

## Reading multiple ents by IDs or throwing

Retrieve a list of ents or throw an `Error` if any of the IDs didn't map to an
existing ent:

```ts
const tasks = await ctx.table('tasks').getManyX([taskId1, taskId2]);
```

Also throws if any of the ents fail a [read rule](/schema/rules).

## Listing all ents

To list all ents backed by a single table, simply `await` the `ctx.table` call:

```ts
const tasks = await ctx.table('users');
```

<Aside title="This is equivalent to the built-in:">

```ts
const tasks = await ctx.db.query('users').collect();
```

</Aside>

## Listing ents filtered by index

To list ents from a table and efficiently filter them by an index, pass the
index name and filter callback to `ctx.table`:

```ts
const posts = await ctx.table('posts', 'numLikes', (q) =>
  q.gt('numLikes', 100)
);
```

<Aside title="This is equivalent to the built-in:">

```ts
const posts = await ctx.db
  .query('posts')
  .withIndex('numLikes', (q) => q.gt('numLikes', 100))
  .collect();
```

</Aside>

## Searching ents via text search

To use [text search](https://docs.convex.dev/search) call the `search` method:

```ts
const awesomeVideoPosts = await ctx
  .table('posts')
  .search('text', (q) => q.search('text', 'awesome').eq('type', 'video'));
```

<Aside title="This is equivalent to the built-in:">

```ts
const awesomeVideoPosts = await ctx.db
  .query('posts')
  .withSearchIndex('text', (q) =>
    q.search('text', 'awesome').eq('type', 'video')
  )
  .collect();
```

</Aside>

## Filtering

Use the `filter` method, which works the same as the
[built-in `filter` method](https://docs.convex.dev/database/reading-data#filtering):

```ts
const posts = await ctx
  .table('posts')
  .filter((q) => q.gt(q.field('numLikes'), 100));
```

<Aside title="This is equivalent to the built-in:">

```ts
const posts = await ctx.db
  .query('posts')
  .filter((q) => q.gt(q.field('numLikes'), 100))
  .collect();
```

</Aside>

## Ordering

Use the `order` method. The default sort is by `_creationTime` in ascending
order (from oldest created to newest created):

```ts
const posts = await ctx.table('posts').order('desc');
```

<Aside title="This is equivalent to the built-in:">

```ts
const posts = await ctx.db.query('posts').order('desc').collect();
```

</Aside>

## Ordering by index

The `order` method takes an index name as a shortcut to sorting by a given
index:

```ts
const posts = await ctx.table('posts').order('desc', 'numLikes');
```

<Aside title="This is equivalent to the built-in:">

```ts
const posts = await ctx
  .table('posts')
  .withIndex('numLikes')
  .order('desc')
  .collect();
```

</Aside>

## Limiting results

### Taking first `n` ents

```ts
const users = await ctx.table('users').take(5);
```

### Loading a page of ents

The `paginate` method returns a page of ents. It takes the same argument object
as the [the built-in pagination](https://docs.convex.dev/database/pagination)
method.

```ts
const result = await ctx.table('posts').paginate(paginationOpts);
```

### Finding the first ent

Useful chained, either to a `ctx.table` call using an index, or after `filter`,
`order`, `edge`.

```ts
const latestUser = await ctx.table('users').order('desc').first();
```

### Finding the first ent or throwing

The `firstX` method (pronounced "first or throw") throws an `Error` if the read
produced no ents:

```ts
const latestUser = await ctx.table('users').order('desc').firstX();
```

### Finding a unique ent

Throws if the listing produced more than 1 result, returns `null` if there were
no results:

```ts
const counter = await ctx.table('counters').unique();
```

### Finding a unique ent or throwing

The `uniqueX` method (pronounced "unique or throw") throws if the listing
produced more or less than 1 result:

```ts
const counter = await ctx.table('counters').uniqueX();
```

## Accessing system tables

[System tables](https://docs.convex.dev/database/advanced/system-tables) can be
read with the `ctx.table.system` method:

```ts
const filesMetadata = await ctx.table.system('_storage');
const scheduledFunctions = await ctx.table.system('_scheduled_functions');
```

All the previously listed methods are also supported by `ctx.table.system`.

You might also access documents in system tables through edges, see
[File Storage](/schema/files) and [Scheduled Functions](/schema/schedule) for
details.

## Traversing edges

Convex Ents allow easily traversing edges declared in the
[schema](/schema#edges) with the `edge` method.

### Traversing 1:1 edge

For 1:1 edges, the `edge` method returns:

- The ent required on the other end of the edge (ex: given a `profile`, load the
  `user`)
- The ent or `null` on the optional end of the edge (ex: given a `user`, load
  their `profile`)

```ts
const user = await ctx.table('profiles').getX(profileId).edge('user');
const profileOrNull = await ctx.table('users').getX(userId).edge('profile');
```

<Aside title="This is equivalent to the built-in:">

```ts
const user = await ctx.db
  .query('users')
  .withIndex('profileId', (q) => q.eq('profileId', profileId))
  .unique();
```

</Aside>

The `edgeX` method (pronounced "edge or throw") throws if the edge does not
exist:

```ts
const profile = await ctx.table('users').getX(userId).edgeX('profile');
```

### Traversing 1:many edges

For 1:many edges, the `edge` method returns:

- The ent required on the other end of the edge (ex: given a `message`, load the
  `user`)
- A list of ents on the end of the multiple edges (ex: given a `user`, load all
  the `messages`)

```ts
const user = await ctx.table('message').getX(messageId).edge('user');
const messages = await ctx.table('users').getX(userId).edge('messages');
```

<Aside title="This is equivalent to the built-in:">

```ts
const messages = await ctx.db
  .query('messages')
  .withIndex('userId', (q) => q.eq('userId', userId))
  .collect();
```

</Aside>

In the case where a list is returned, [filtering](#filtering),
[ordering](#ordering) and [limiting results](#limiting-results) applies and can
be used.

### Traversing many:many edges

For many:many edges, the `edge` method returns a list of ents on the other end
of the multiple edges:

```ts
const tags = await ctx.table('messages').getX(messageId).edge('tags');
const messages = await ctx.table('tags').getX(tagId).edge('messages');
```

<Aside title="This is equivalent to the built-in:">

```ts
const tags = await Promise.all(
  (
    await ctx.db
      .query('messages_to_tags')
      .withIndex('messagesId', (q) => q.eq('messagesId', messageId))
      .collect()
  ).map((edge) => ctx.db.get(edge.tagsId))
);
```

</Aside>

The results are ordered by `_creationTime` of the edge, from oldest created to
newest created. The order can be changed with the [`order` method](#ordering)
and [the results can be limited](#limiting-results).

### Testing many:many edge presence

You can efficiently test whether two ents are connected by a many:many edge
using the `has` method:

```ts
const hasTag = ctx.table('messages').getX(messageId).edge('tags').has(tagId);
```

<Aside title="This is equivalent to the built-in:">

```ts
const hasTag =
  (await ctx.db
    .query('messages_to_tags')
    .withIndex('messagesId_tagsId', (q) =>
      q.eq('messagesId', message._id).eq('tagsId', tagId)
    )
    .first()) !== null;
```

</Aside>

## Retrieving related ents

Sometimes you want to retrieve both the ents from one table and the related ents
from another table. Other database systems refer to these as a _joins_ or
_nested reads_.

### Traversing edges from individual retrieved ents

All the methods for reading ents return the underlying document(s), enriched
with the `edge` and `edgeX` methods. This allows traversing edges after loading
the ent from the database:

```ts
const user = await ctx.table('users').firstX();
const profile = user.edgeX('profile');
```

### Mapping ents to include edges

The `map` method can be used to perform arbitrary transformation for each
returned ent. It can be used to add related ents via the `edge` and `edgeX`
methods:

```ts
const usersWithMessages = await ctx.table('users').map(async (user) => {
  return {
    name: user.name,
    messages: await user.edge('messages').take(5),
  };
});
```

```ts
const usersWithProfileAndMessages = await ctx.table("users").map(async (user) => {
  const [messages, profile] = await Promise.all([
    profile: user.edgeX("profile"),
    user.edge("messages").take(5),
  ])
  return {
    name: user.name,
    profile,
    messages,
  };
});
```

<Aside title="This code is equivalent to this code using only built-in Convex and JavaScript
functionality:">

```ts
const usersWithMessagesAndProfile = await Promise.all(
  (await ctx.db.query('users').collect()).map(async (user) => {
    const [posts, profile] = Promise.all([
      ctx.db
        .query('messages')
        .withIndex('userId', (q) => q.eq('userId', user._id))
        .collect(),
      (async () => {
        const profile = await ctx.db
          .query('profiles')
          .withIndex('userId', (q) => q.eq('userId', user._id))
          .unique();
        if (profile === null) {
          throw new Error(
            `Edge "profile" does not exist for document with ID "${user._id}"`
          );
        }
        return profile;
      })(),
    ]);
    return { name: user.name, posts, profile };
  })
);
```

</Aside>

As shown in this example, `map` can be used to transform the results by
selecting or excluding the returned fields, limiting the related ents, or
requiring that related ents exist.

The `map` method can beused to load ents by traversing more than one edge:

```ts
const usersWithMessagesAndTags = await ctx.table('users').map(async (user) => ({
  ...user,
  messages: await user.edge('messages').map(async (message) => ({
    text: message.text,
    tags: await message.edge('tags'),
  })),
}));
```

## Returning documents from functions

Consider the following query:

```ts filename="convex/messages.ts"
import { query } from './functions';

export const list = query({
  args: {},
  handler: async (ctx) => {
    return await ctx.table('messages');
  },
});
```

It returns all ents stored in the `messages` table. Ents include methods which
are not preserved when they are returned to the client. While this works fine at
runtime, TypeScript will think that the methods are still available even on the
client.

Note that it is generally not a good idea to return full documents directly to
clients. If you add a new field to an ent, you might not want that field to be
sent to clients. For this reason, and for backwards compatibility in general,
it's a good idea to pick specifically which fields to send:

```ts filename="convex/messages.ts"
import { query } from './functions';

export const list = query({
  args: {},
  handler: async (ctx) => {
    return await ctx.table('messages').map((message) => ({
      _id: message._id,
      _creationTime: message._creationTime,
      text: message.text,
      userId: message.userId,
    }));
  },
});
```

That said, you can easily retrieve raw documents, instead of ents with methods,
from the database using the `docs` and `doc` methods:

```ts filename="convex/messages.ts"
import { query } from './functions';

export const list = query({
  args: {},
  handler: async (ctx) => {
    return await ctx.table('messages').docs();
  },
});
```

```ts filename="convex/users.ts"
import { query } from './functions';

export const list = query({
  args: {},
  handler: async (ctx) => {
    const usersWithMessagesAndProfile = await ctx
      .table('users')
      .map(async (user) => ({
        ...user,
        profile: await user.edgeX('profile').doc(),
      }));
  },
});
```

You can also call the `doc` method on a retrieved ent, which returns the same
object, but is typed as a plain Convex document.
import { Aside } from "../components/Aside.tsx";

# Writing Ents to the Database

Just like for [reading](/read) ents from the database, for writing Convex Ents
provide a `ctx.table` method which replaces the built-in `ctx.db` object in
[Convex mutations](https://docs.convex.dev/functions/mutation-functions).

## Security

The same [added level of security](/read#security) applies to the writing ents
as it does to reading them.

## Inserting a new ent

You can insert new ents into the database with the `insert` method chained to
the result of calling `ctx.table`:

```ts
const taskId = await ctx.table('tasks').insert({ text: 'Win at life' });
```

You can retrieve the just created ent with the `get` method:

```ts
const task = await ctx.table('tasks').insert({ text: 'Win at life' }).get();
```

<Aside title="This is equivalent to the built-in:">

```ts
const taskId = await ctx.db.insert('tasks', { text: 'Win at life' });
const task = (await ctx.db.get(taskId))!;
```

</Aside>

## Inserting many new ents

```ts
const taskIds = await ctx
  .table('tasks')
  .insertMany({ text: 'Buy socks' }, { text: 'Buy socks' });
```

## Updating existing ents

To update an existing ent, call the `patch` or `replace` method on a
[lazy `Promise`](/read) of an ent, or on an already retrieved ent:

```ts
await ctx.table('tasks').getX(taskId).patch({ text: 'Changed text' });
await ctx.table('tasks').getX(taskId).replace({ text: 'Changed text' });
```

```ts
const task = await ctx.table('tasks').getX(taskId);
await task.patch({ text: 'Changed text' });
await task.replace({ text: 'Changed text' });
```

See the
[docs for the built-in `patch` and `replace` methods](https://docs.convex.dev/database/writing-data#updating-existing-documents)
for the difference between them.

## Deleting ents

To delete an ent, call the `delete` method on a [lazy `Promise`](/read) of an
ent, or on an already retrieved ent:

```ts
await ctx.table('tasks').getX(taskId).delete();
```

```ts
const task = await ctx.table('tasks').getX(taskId);
await task.delete();
```

### Cascading deletes

See the [Cascading Deletes](/schema/deletes) page for how to configure how
deleting an ent affects its edges and other ents connected to it.

## Writing edges

Edges can be created together with ents using the `insert` and `insertMany`
methods, or they can be created and deleted for two existing ents using the
`replace` and `patch` methods.

### Writing 1:1 and 1:many edges

A 1:1 or 1:many edge can be created by specifying the ID of the other ent on
[the ent which stores the edge](/schema#understanding-how-edges-are-stored),
either when inserting:

```ts
// First we need a user, which can have an optional profile edge
const userId = await ctx.table('users').insert({ name: 'Alice' });
// Now we can create a profile with the 1:1 edge to the user
const profileId = await ctx
  .table('profiles')
  .insert({ bio: 'In Wonderland', userId });
```

or when updating:

```ts
const profileId = await ctx.table('profiles').getX(profileId).patch({ userId });
```

<Aside title="This is equivalent to the built-in:">

```ts
const posts = await ctx.db.patch(profileId, { userId });
```

with the addition of checking that `profileId` belongs to `"profiles"`.

</Aside>

### Writing many:many edges

Many:many edges can be created by listing the IDs of the other ents when
inserting ents on either side of the edge:

```ts
// First we need a tag, which can have many:many edge to messages
const tagId = await ctx.table('tags').insert({ name: 'Blue' });
// Now we can create a message with a many:many edge to the tag
const messageId = await ctx
  .table('messages')
  .insert({ text: 'Hello world', tags: [tagId] });
```

But we could have equally created a message first, and then created a tag with a
list of message IDs.

The `replace` method can be used to create and delete many:many edges:

```ts
await ctx
  .table('messages')
  .getX(messageId)
  .replace({ text: 'Changed message', tags: [tagID, otherTagID] });
```

If a list is specified, the edges that need to be created are created, and all
other existing edges are deleted. If the edge name is ommited entirely, the
edges are left unchanged:

```ts
await ctx
  .table('messages')
  .getX(messageId)
  .replace({ text: 'Changed message' /* no `tags:`, so tags don't change */ });
```

The `patch` method on the other hand expects a description of the changes that
should be made, a list of IDs to `add` and `remove` edges for:

```ts
const message = await ctx.table('messages').getX(messageId);
await message.patch({ tags: { add: [tagID] } });
await message.patch({
  tags: { add: [tagID, otherTagID], remove: [tagToDeleteID] },
});
```

Any edges in the `add` list that didn't exist are created, and any edges in the
`remove` list that did exist are deleted. Edges to ents with ID not listed in
either list are not affected by `patch`.

## Updating ents connected by edges

The `patch`, `replace` and `delete` methods can be chained after `edge` calls to
update the ent on the other end of an edge:

```ts
await ctx
  .table('users')
  .getX(userId)
  .edgeX('profile')
  .patch({ bio: "I'm the first user" });
```

<Aside title="This is equivalent to the built-in:">

```ts
const profile = await ctx.db
  .query('profiles')
  .withIndex('userId', (q) => q.eq('userId', userId))
  .unique();
if (profile === null) {
  throw new Error(
    `Edge "profile" does not exist for document wit ID "${userId}"`
  );
}
await ctx.db.patch(profile._id, { bio: "I'm the first user" });
```

</Aside>

<Aside
  title={
    <>
      <b>Limitation:</b>&nbsp;<code>edge</code>&nbsp;called on a loaded ent
    </>
  }
>
The following code does not typecheck [currently](https://github.com/xixixao/convex-ents/issues/8):

```ts
const user = await ctx.table('users').getX(userId);
await user.edgeX('profile').patch({ bio: "I'm the first user" });
```

You can either disable typechecking via `// @ts-expect-error` or preferably
start with `ctx.table`:

```ts
const user = ... // loaded or passed via `map`
await ctx
  .table("users")
  .getX(user._id).edgeX("profile").patch({ bio: "I'm the first user" });
```

</Aside>

convex-optimize.mdc

---
description: Query optimization patterns for scaling Convex applications
globs: **/*.ts,**/*.tsx
alwaysApply: false
---

# Convex Query Optimization

This document covers optimization patterns for Convex queries at scale. Don't prematurely optimize - these patterns are for when you have thousands of documents.

## 1. Use Indexes Instead of Scanning

**Edit:** `convex/schema.ts` for indexes, `convex/*.ts` for queries

### Problem: Full table scans

```typescript
// ❌ WRONG: Scans every document looking for a match
const members = await ctx
  .table('members')
  .filter((q) => q.eq(q.field('teamId'), args.teamId))
  .collect();

// ✅ CORRECT: Uses index to jump to matching documents
const members = await ctx
  .table('members')
  .query()
  .withIndex('teamId', (q) => q.eq('teamId', args.teamId))
  .collect();
```

### Schema with indexes

````typescript
// convex/schema.ts
import { defineEnt, defineEntSchema } from 'convex-ents';
import { v } from 'convex/values';

const schema = defineEntSchema({
  members: defineEnt({
    teamId: v.id('teams'),
    status: v.union(v.literal('invited'), v.literal('active')),
    // ...
  })
    .index('teamId', ['teamId'])
    .index('teamId_status', ['teamId', 'status']), // Multi-field index
});

export default schema;

### Multi-field indexes for complex queries

```typescript
// Use multi-field index when filtering by multiple fields
const activeMembers = await ctx
  .table('members')
  .query()
  .withIndex('teamId_status', (q) =>
    q.eq('teamId', args.teamId).eq('status', 'active')
  )
  .collect();
````

### Codebase Audit: Search for `.filter(q =>`

Look at every `.filter()` call:

- Is the table always small? (< 1000 docs)
- Are you using an index first?
- Search for `q.eq(q.field(` - these should use indexes

**Rule:** If you see `.filter(q => q.eq(q.field("fieldName"), value))`, create an index on `fieldName`

## 2. Limit Documents with Pagination

**CRITICAL:** NEVER use `.collect()` without knowing the table size. Always use `ctx.table()` instead of `ctx.db`.

### Problem: Fetching unlimited documents

```typescript
// ❌ WRONG: Will break with thousands of posts
return await ctx.table('posts').query().order('desc').collect();

// ✅ OPTIMAL: Aggregates for counting with millions of documents (O(log n))
const count = await aggregatePosts.count(ctx, {
  namespace: categoryId,
  bounds: {} as any,
});

// ✅ GOOD: Pagination for user-facing lists
return await ctx.table('posts').query().order('desc').paginate(args.paginationOpts);

// ✅ ACCEPTABLE: Limits to 50 documents (ONLY for small bounded datasets)
return await ctx.table('posts').query().order('desc').take(50);
```

### When to use each pattern

**🚨 CRITICAL FOR SCALE: Aggregates FIRST for counting operations**

**Aggregates** - PRIMARY solution for counting with millions of documents (O(log n))

```typescript
// ✅ OPTIMAL: O(log n) count scales to millions of documents
const count = await aggregateMessages.count(ctx, {
  namespace: channelId,
  bounds: {} as any,
});
```

### Aggregates vs Denormalization for Small Lookups

**IMPORTANT**: Use aggregates even for ≤100 page size lookups. Here's why:

**Aggregates are preferred because:**

- **Flexibility with bounds**: Can query ranges, prefixes, and complex conditions
- **No schema changes**: Add new aggregates without modifying tables
- **Easier maintenance**: Triggers handle everything automatically
- **Still fast**: 100 O(log n) operations are very efficient
- **Dynamic filtering**: Supports complex queries without pre-computing every combination
- Only denormalize when aggregate can't be used

### When to Keep Separate Tables vs Using Aggregates

**Keep a separate table (like `characterActivity`) when:**

- **Domain relationship exists**: The table represents a meaningful business relationship
- **Additional metadata needed**: May add fields like `interaction_count`, `last_action_type`
- **Query patterns require it**: Need to query by multiple dimensions (userId + characterId)
- **Normalized design**: Avoids duplicating data across multiple aggregates

**Use aggregates when:**

- **Pure statistics**: Just counting or summing values
- **No domain meaning**: It's purely a performance optimization
- **Single-purpose**: Only used for one type of query

**Example: Why `characterActivity` is a good separate table:**

```typescript
// ✅ GOOD: Separate table for domain relationship
characterActivity: defineTable({
  userId: v.id('users'),
  characterId: v.id('characters'),
  lastMessageAt: v.number(),
  // Could extend with:
  // interactionCount: v.number(),
  // lastActionType: v.string(), // 'chat', 'star', 'view'
  // favoriteQuotes: v.array(v.string()),
})
  .index('userId', ['userId'])
  .index('userId_characterId', ['userId', 'characterId']);

// This represents "user's history with character" - a real domain concept
// Not just a cache or performance optimization
```

**`.paginate()`** - User-facing lists with "load more" functionality

```typescript
// Use with createPublicPaginatedQuery() or createAuthPaginatedQuery()
export const listPosts = createPublicPaginatedQuery()({
  handler: async (ctx, args) => {
    return await ctx.table('posts').query().order('desc').paginate(args.paginationOpts);
  },
});
```

**`.take(n)`** - ONLY for small bounded datasets where you don't need total count

```typescript
// Show "25+ messages" in UI (ONLY for bounded channels with enforced limits)
const recentMessages = await ctx
  .table('messages')
  .query()
  .withIndex('channelId', (q) => q.eq('channelId', channelId))
  .order('desc')
  .take(25);
```

**`.first()` / `.unique()`** - Single document

```typescript
const latestPost = await ctx.table('posts').query().order('desc').first();
const user = await ctx.table('users').get('email', email); // Unique lookup by indexed field
const singleDoc = await ctx.table('counters').query().unique(); // Throws if multiple matches
```

### Enforce limits at insert time

Instead of paginating everything, consider:

- Max 100 teams per user
- Max 10 email addresses per account
- Max 100 items per cart

```typescript
// ✅ OPTIMAL: Use aggregates for counting at scale (O(log n))
const teamCount = await aggregateTeamMembers.count(ctx, {
  namespace: userId,
  bounds: {} as any,
});
if (teamCount >= 100) {
  throw new ConvexError('Maximum 100 teams per user');
}

// ❌ AVOID: Fetching all to count (O(n) operation - breaks with millions)
const teamMembers = await ctx
  .table('teamMembers')
  .query()
  .withIndex('userId', (q) => q.eq('userId', userId))
  .collect();
if (teamMembers.length >= 100) {
  throw new ConvexError('Maximum 100 teams per user');
}
```

### Codebase Audit: Search for `.collect()`

Every `.collect()` call must either:

1. Use an index to limit documents
2. Be on a table with enforced size limits
3. Have a comment explaining why unlimited is safe
4. **PREFERRED**: Use aggregates for counting operations (O(log n) vs O(n))

**🚨 CRITICAL Rules for Scale:**

- **Counting (PRIMARY)**: Use aggregates from [convex-aggregate.mdc](mdc:.cursor/rules/convex-aggregate.mdc) for O(log n) performance with millions of documents
- **User-facing lists**: Use `.paginate()` for "load more" functionality
- **Small bounded datasets**: Use `.take(n)` ONLY when you don't need total count and table has enforced size limits
- **Never**: Use `.collect()` or `.collect().length` on unbounded tables - will break with millions of documents

## 3. Optimize for Query Caching

Convex automatically caches queries and invalidates when referenced documents change. Optimize by isolating frequently-changing data.

### Problem: Frequent cache invalidation

```typescript
// ❌ WRONG: User document changes every 10 seconds
// convex/schema.ts
users: defineEnt({
  name: v.string(),
  profilePicUrl: v.string(),
  lastSeen: v.number(), // Updates every heartbeat
});

// Every query reading users gets invalidated on heartbeat
const profile = await ctx.table('users').get(userId); // Invalidated every 10s!
```

### Solution: Isolate frequently-changing data

```typescript
// ✅ CORRECT: Separate frequently-changing data
// convex/schema.ts
users: defineEnt({
  name: v.string(),
  profilePicUrl: v.string(),
}).edge('heartbeat', { ref: true }), // 1:1 edge to heartbeat

heartbeats: defineEnt({
  lastSeen: v.number(),
}).edge('user'),

// Profile query not invalidated by heartbeats
const profile = await ctx.table('users').get(userId);

// Only queries that need online status read heartbeat
const heartbeat = profile && await profile.edge('heartbeat');
const isOnline = heartbeat && (Date.now() - heartbeat.lastSeen < 30000);
```

### Codebase Audit: Search for `.patch()` and `.replace()`

Look for frequently updated fields:

- Timestamps (lastSeen, lastActive)
- Counters (viewCount, messageCount)
- Status fields that change often

If these are in widely-referenced documents, move to separate tables.

## Performance Best Practices

### Index Guidelines

- **Use indexes for equality filters** - Replace `.filter()` with `.withIndex()`
- **Multi-field indexes don't add overhead** - Prefer one `["teamId", "status"]` over two separate indexes
- **Index limit** - Too many indexes slow inserts (16 indexes ≈ 17x insert cost)

### Query Limits

- **Read limit** - Convex limits documents read per transaction
- **Default take** - `.take()` defaults to 100 if not specified
- **Pagination** - Use for unbounded data sets

### Document Size

- **Arrays** - Keep to 10 elements max
- **Nested objects** - Avoid deep nesting
- **Large data** - Use separate tables with references

## Common Patterns

```typescript
// ✅ OPTIMAL: Use aggregates for counting at scale
const memberCount = await aggregateMembers.count(ctx, {
  namespace: teamId,
  bounds: {} as any,
});

// ✅ GOOD: Paginated list with filters
export const listPosts = createAuthPaginatedQuery()({
  args: { status: z.optional(z.string()) },
  handler: async (ctx, args) => {
    if (args.status) {
      return await ctx
        .table('posts')
        .query()
        .withIndex('status', (q) => q.eq('status', args.status))
        .order('desc')
        .paginate(args.paginationOpts);
    }
    return await ctx.table('posts').query().order('desc').paginate(args.paginationOpts);
  },
});

// ✅ ACCEPTABLE: Small bounded collection with enforced limits
const items = await ctx
  .table('cartItems')
  .query()
  .withIndex('cartId', (q) => q.eq('cartId', cartId))
  .collect();
// Safe: enforced max 100 at insert

// ❌ AVOID: Fetching all members to count or process
const members = await ctx
  .table('members')
  .query()
  .withIndex('teamId', (q) => q.eq('teamId', teamId))
  .collect();
// Dangerous: could be millions - use pagination instead
```

## 4. Full Text Search Optimization

**Edit:** `convex/schema.ts` for search indexes, `convex/*.ts` for queries

### When to Use Search Indexes vs Regular Indexes

**Use Regular Indexes** when:

- Exact matching on fields (`status === 'active'`)
- Range queries on numbers/dates
- Sorting by specific fields
- You need custom ordering

**Use Search Indexes** when:

- Text search within string fields
- Typeahead/autocomplete features
- Finding keywords in content
- Relevance-based results

### Search Index Best Practices

```typescript
// ✅ GOOD: Filter fields for common filters
characters: defineEnt({
  name: v.string(),
  label: v.string().optional(),
  userId: v.id('users'),
  private: v.boolean(),
}).searchIndex('search_name', {
  searchField: 'name',
  filterFields: ['userId', 'private'], // Indexed equality filters
});

// ❌ BAD: No filter fields = post-filtering required
// .searchIndex('search_name', {
//   searchField: 'name',
//   // Missing filterFields - all filtering happens in memory!
// });
```

### Efficient Search Queries

```typescript
// ✅ EFFICIENT: Use filter fields in search expression
const results = await ctx
  .table('characters')
  .search(
    'search_name',
    (q) =>
      q
        .search('name', args.query)
        .eq('private', false) // Uses index
        .eq('userId', excludeUserId) // Uses index
  )
  .paginate(args.paginationOpts);

// ❌ INEFFICIENT: Post-filtering after search
const results = await ctx
  .table('characters')
  .search('search_name', (q) => q.search('name', args.query))
  .filter((q) => q.eq(q.field('private'), false)) // Scans all results!
  .paginate(args.paginationOpts);
```

### Search vs Filter Performance

```typescript
// Problem: Can't search on multiple fields
// ❌ This doesn't work - only one search field allowed
// .searchIndex('search_name_label', {
//   searchField: ['name', 'label'], // NOT SUPPORTED
// })

// Solution 1: Search primary field, filter secondary
const results = await ctx
  .table('characters')
  .search('search_name', (q) => q.search('name', query))
  .filter((q) =>
    q.or(q.eq(q.field('name'), query), q.eq(q.field('label'), query))
  )
  .take(50);

// Solution 2: Concatenated search field
characters: defineEnt({
  name: v.string(),
  label: v.string().optional(),
  searchText: v.string(), // name + ' ' + label
}).searchIndex('search_all', {
  searchField: 'searchText',
  filterFields: ['private', 'userId'],
});
```

### Key Limitations & Workarounds

1. **1024 Document Scan Limit**

   ```typescript
   // Search can only scan 1024 documents
   // Use filter fields to narrow results BEFORE scanning
   ctx.table('users').search(
     'search_name',
     (q) => q.search('name', 'john').eq('teamId', teamId) // Narrow to team first
   );
   ```

2. **No Custom Ordering**

   ```typescript
   // ❌ Can't order search results by date
   ctx.table('posts').search(...).order('desc') // NOT SUPPORTED

   // ✅ Post-sort in memory for small result sets
   const results = await ctx.table('posts').search(...).take(50);
   results.sort((a, b) => b._creationTime - a._creationTime);
   ```

3. **Single Search Field**
   ```typescript
   // For multi-field search, consider:
   // 1. Denormalized search field (concatenated)
   // 2. Multiple search indexes (query separately)
   // 3. Post-filtering for secondary matches
   ```

### Search Index Patterns

```typescript
// Pattern 1: Optional search with fallback to regular index
export const listCharacters = createPublicPaginatedQuery()({
  args: { query: z.string().optional() },
  handler: async (ctx, args) => {
    if (args.query) {
      // Use search index when query provided
      return await ctx
        .table('characters')
        .search('search_name', (q) =>
          q.search('name', args.query!).eq('private', false)
        )
        .paginate(args.paginationOpts);
    } else {
      // Use regular index when no search
      return await ctx
        .table('characters')
        .query()
        .withIndex('private', (q) => q.eq('private', false))
        .order('desc')
        .paginate(args.paginationOpts);
    }
  },
});

// Pattern 2: For complex filters with search, use streams
// See [convex-streams.mdc](mdc:.cursor/rules/convex-streams.mdc)
```

## 5. Pagination with Filtering

**Edit:** `convex/*.ts` files using `.filter()` + `.paginate()`

### Important: Choose the Right Filtering Approach

1. **Simple filters:** Use built-in `.filter()` - maintains full page sizes with pagination!
2. **Complex filters WITHOUT pagination:** Use `filter` helper from `convex-helpers/server/filter`
3. **Complex filters WITH pagination:** Use streams from `convex-helpers/server/stream`

### Built-in `.filter()` Maintains Full Page Sizes

**Good news:** Built-in `.filter()` with `.paginate()` returns the requested number of items!

```typescript
// ✅ GOOD: Always returns 10 items (if available)
export const exploreCharacters = createPublicPaginatedQuery()({
  args: { status: z.string().optional() },
  handler: async (ctx, args) => {
    return await ctx
      .table('characters')
      .filter((q) => q.eq(q.field('status'), args.status))
      .paginate(args.paginationOpts);
    // Convex keeps fetching until it finds 10 matching items!
  },
});
```

### Complex Filtering Options

**Limitation:** Built-in `.filter()` only supports simple field comparisons (eq, neq, gt, lt).

```typescript
// ❌ CAN'T DO with built-in .filter():
// - Array operations: categories.includes('strategy')
// - String operations: name.toLowerCase().includes('john')
// - Async lookups: Check if author is premium
// - Complex logic: Multiple conditions with custom logic
```

#### Option 1: Filter Helper (No Pagination)

```typescript
import { filter } from 'convex-helpers/server/filter';

// ✅ GOOD: Complex filtering without pagination
const topPosts = await filter(
  ctx
    .table('posts')
    .query()
    .withIndex('status', (q) => q.eq('status', 'published')),
  (post) => {
    // Full TypeScript power
    return (
      post.tags.includes('featured') &&
      post.title.toLowerCase().includes(searchTerm) &&
      post.views > 1000
    );
  }
).take(10); // Works with .take(), .first(), unique()
```

#### Option 2: Streams (With Pagination)

```typescript
import { stream } from 'convex-helpers/server/stream';
import schema from './schema';

// ✅ BEST: Complex filtering with consistent pagination
export const searchCharacters = createPublicPaginatedQuery()({
  args: { category: z.string().optional() },
  handler: async (ctx, args) => {
    return await stream(ctx.db, schema) // ⚠️ Stream still uses ctx.db
      .query('characters')
      .withIndex('private', (q) => q.eq('private', false))
      .filterWith(async (char) => {
        // Full TypeScript power + consistent page sizes!
        return char.categories?.includes(args.category);
      })
      .paginate(args.paginationOpts);
  },
});
```

**Key Difference:** Filter helper with `.paginate()` returns variable page sizes. Streams always return the requested number of items.

### How It Works

1. Convex fetches documents up to the requested page size
2. The filter predicate is applied to each document
3. Non-matching documents are discarded
4. Result: Variable number of items per page

```typescript
// Example flow:
// 1. User requests: { numItems: 10 }
// 2. Convex fetches: 10 documents from the table
// 3. Filter applies: 3 match, 7 discarded
// 4. User receives: { page: [3 items], isDone: false }
```

### Solutions

#### Solution 1: Use Indexes for Filtering (Preferred)

```typescript
// ✅ BEST: Move filterable fields to indexes
characters: defineEnt({
  name: v.string(),
  category: v.string(), // Denormalized for indexing
  private: v.boolean(),
})
  .index('category', ['category'])
  .index('private_category', ['private', 'category']);

// Now filter at index level - consistent page sizes
export const exploreCharacters = createPublicPaginatedQuery()({
  args: { category: z.string().optional() },
  handler: async (ctx, args) => {
    if (args.category) {
      return await ctx
        .table('characters')
        .query()
        .withIndex('category', (q) => q.eq('category', args.category))
        .paginate(args.paginationOpts);
    }

    return await ctx.table('characters').query().paginate(args.paginationOpts);
    // Always returns requested number of items (if available)
  },
});
```

#### Solution 2: Denormalize Data (Use Sparingly)

**⚠️ CAUTION**: Only denormalize when aggregate can't be used.

```typescript
// ⚠️ ONLY IF: Shown in lists AND no complex queries needed
characters: defineEnt({
  name: v.string(),
  // Only denormalize simple booleans for index filtering
  hasStrategyCategory: v.boolean(), // ONLY if no bounds queries needed
  totalStarCount: v.number(), // ONLY if always shown in character lists
});

// ✅ BETTER: Use aggregates for counts and complex queries
const starCount = await aggregateCharacterStars.count(ctx, {
  namespace: characterId,
  bounds: {} as any, // Can add bounds for date ranges, user filtering, etc.
});
```

#### Solution 3: Streams for Consistent Page Sizes (Advanced)

```typescript
// ✅ BEST FOR COMPLEX CASES: Use streams from convex-helpers
import { stream } from 'convex-helpers/server/stream';
import schema from './schema';

export const searchCharacters = createPublicPaginatedQuery()({
  args: {
    category: z.string().optional(),
    minLevel: z.number().optional(),
    tags: z.array(z.string()).optional(),
  },
  handler: async (ctx, args) => {
    // Stream with filterWith - filters BEFORE pagination
    // ⚠️ IMPORTANT: Streams still use ctx.db, not yet migrated to Ents
    return await stream(ctx.db, schema) // Must use ctx.db here
      .query('characters')
      .withIndex('private', (q) => q.eq('private', false))
      // Complex filtering happens BEFORE pagination
      .filterWith(async (char) => {
        if (args.category && !char.categories?.includes(args.category)) {
          return false;
        }
        if (args.minLevel && char.level < args.minLevel) {
          return false;
        }
        if (args.tags && !args.tags.some((tag) => char.tags?.includes(tag))) {
          return false;
        }
        // Can even do async lookups
        const author = await ctx.table('users').get(char.userId);
        return author && !author.isBanned;
      })
      // Pagination happens AFTER filtering - consistent page sizes!
      .paginate(args.paginationOpts);
  },
});
```

**Key Differences: `filter` helper vs Streams**

| Feature               | `filter` helper             | Streams `filterWith`            |
| --------------------- | --------------------------- | ------------------------------- |
| Filtering timing      | During pagination           | Before pagination               |
| Page size consistency | Variable (0-N items)        | Consistent (N items)            |
| Performance           | Scans requested page size   | Scans until page is full        |
| Complexity            | Simple, drop-in replacement | Requires schema import          |
| Async predicates      | ✅ Supported                | ✅ Supported                    |
| Database lookups      | ✅ Supported                | ✅ Supported                    |
| Best for              | Simple cases, prototyping   | Production with complex filters |

**Stream Additional Features:**

- **Merge streams**: Combine multiple queries (UNION)
- **Joins**: Use `flatMap` for complex joins
- **Distinct**: Get unique values efficiently
- **Map**: Transform documents in the stream

```typescript
// Example: Merge multiple filtered streams
import { mergedStream } from 'convex-helpers/server/stream';

// ⚠️ Streams still use ctx.db
const activeUsers = stream(ctx.db, schema)
  .query('users')
  .withIndex('status', (q) => q.eq('status', 'active'));

const premiumUsers = stream(ctx.db, schema)
  .query('users')
  .withIndex('plan', (q) => q.eq('plan', 'premium'));

// Merge and get distinct users
const merged = mergedStream([activeUsers, premiumUsers], ['_creationTime'])
  .distinct(['_id'])
  .paginate(args.paginationOpts);
```

**When to Use Streams:**

1. Need consistent page sizes with complex filters
2. Combining data from multiple queries (UNION)
3. Complex joins that need pagination
4. Need distinct values with pagination
5. Building SQL-like query patterns

**Trade-offs:**

- More complex API (requires schema)
- Potentially more documents scanned
- Pagination cursors are more fragile
- Better for read-heavy operations

### Best Practices

1. **Prefer indexes** over post-query filtering
2. **Use aggregates** for counts and lookups (even ≤100 items)
3. **Document** variable page size behavior
4. **Design UI** to handle variable results gracefully
5. **Monitor** filter effectiveness (track avg results per page)
6. **Consider streams** for production apps with complex filtering needs
7. **Denormalize sparingly** - only when aggregate can't be used

### Common Patterns to Audit

```typescript
// Search for these patterns in your codebase:
// 1. filter(...).paginate()
// 2. Complex predicates in filter()
// 3. Array operations in filter predicates
// 4. Multiple conditions in filter predicates
```

#### Solution 3: Streams for Consistent Page Sizes (Advanced)

```typescript
// ✅ BEST FOR COMPLEX CASES: Use streams from convex-helpers
import { stream } from 'convex-helpers/server/stream';
import schema from './schema';

export const searchCharacters = createPublicPaginatedQuery()({
  args: {
    category: z.string().optional(),
    minLevel: z.number().optional(),
    tags: z.array(z.string()).optional(),
  },
  handler: async (ctx, args) => {
    // Stream with filterWith - filters BEFORE pagination
    // ⚠️ IMPORTANT: Streams still use ctx.db, not yet migrated to Ents
    const characters = stream(ctx.db, schema) // Must use ctx.db here
      .query('characters')
      .withIndex('private', (q) => q.eq('private', false));

    // Complex filtering happens BEFORE pagination
    const filtered = characters.filterWith(async (char) => {
      if (args.category && !char.categories?.includes(args.category)) {
        return false;
      }
      if (args.minLevel && char.level < args.minLevel) {
        return false;
      }
      if (args.tags && !args.tags.some((tag) => char.tags?.includes(tag))) {
        return false;
      }
      // Can even do async lookups
      const author = await ctx.table('users').get(char.userId);
      return author && !author.isBanned;
    });

    // Pagination happens AFTER filtering - consistent page sizes!
    return await filtered.paginate(args.paginationOpts);
  },
});
```

**Key Differences: `filter` helper vs Streams**

| Feature               | `filter` helper             | Streams `filterWith`            |
| --------------------- | --------------------------- | ------------------------------- |
| Filtering timing      | During pagination           | Before pagination               |
| Page size consistency | Variable (0-N items)        | Consistent (N items)            |
| Performance           | Scans requested page size   | Scans until page is full        |
| Complexity            | Simple, drop-in replacement | Requires schema import          |
| Async predicates      | ✅ Supported                | ✅ Supported                    |
| Database lookups      | ✅ Supported                | ✅ Supported                    |
| Best for              | Simple cases, prototyping   | Production with complex filters |

**Stream Additional Features:**

- **Merge streams**: Combine multiple queries (UNION)
- **Joins**: Use `flatMap` for complex joins
- **Distinct**: Get unique values efficiently
- **Map**: Transform documents in the stream

```typescript
// Example: Merge multiple filtered streams
import { mergedStream } from 'convex-helpers/server/stream';

// ⚠️ Streams still use ctx.db
const activeUsers = stream(ctx.db, schema)
  .query('users')
  .withIndex('status', (q) => q.eq('status', 'active'));

const premiumUsers = stream(ctx.db, schema)
  .query('users')
  .withIndex('plan', (q) => q.eq('plan', 'premium'));

// Merge and get distinct users
const merged = mergedStream([activeUsers, premiumUsers], ['_creationTime'])
  .distinct(['_id'])
  .paginate(args.paginationOpts);
```

**When to Use Streams:**

1. Need consistent page sizes with complex filters
2. Combining data from multiple queries (UNION)
3. Complex joins that need pagination
4. Need distinct values with pagination
5. Building SQL-like query patterns

**Trade-offs:**

- More complex API (requires schema)
- Potentially more documents scanned
- Pagination cursors are more fragile
- Better for read-heavy operations

### Best Practices

1. **Prefer indexes** over post-query filtering
2. **Use aggregates** for counts and lookups (even ≤100 items)
3. **Document** variable page size behavior
4. **Design UI** to handle variable results gracefully
5. **Monitor** filter effectiveness (track avg results per page)
6. **Consider streams** for production apps with complex filtering needs
7. **Denormalize sparingly** - only when aggregate can't be used

## 6. Batch Operations & Relationship Helpers

**Edit:** `convex/*.ts` files using batch operations

### Use convex-helpers and Ents for Readable Batch Operations

With Convex Ents, batch operations are simplified, but you can still use helpers from `convex-helpers`:

```typescript
import { asyncMap } from 'convex-helpers';
```

### asyncMap vs Promise.all

**Always prefer `asyncMap` over `Promise.all` for async operations:**

```typescript
// ❌ HARDER TO READ: Raw Promise.all
const results = await Promise.all(ids.map((id) => someAsyncOperation(id)));

// ✅ CLEANER: Use asyncMap from convex-helpers
const results = await asyncMap(ids, someAsyncOperation);

// Real example with aggregates
// ❌ Raw Promise.all
const starCounts = await Promise.all(
  characterIds.map((characterId) =>
    aggregateCharacterStarsByUser.count(ctx, {
      bounds: { key: characterId, inclusive: true },
      namespace: userId,
    })
  )
);

// ✅ Better with asyncMap
const starCounts = await asyncMap(characterIds, (characterId) =>
  aggregateCharacterStarsByUser.count(ctx, {
    bounds: { key: characterId, inclusive: true },
    namespace: userId,
  })
);
```

### getMany for Batch ID Lookups with Ents

**Use `getMany` or `getManyX` with Ents for fetching multiple documents by ID:**

```typescript
// ❌ Manual Promise.all pattern
const userPromises = [];
for (const userId of userIds) {
  userPromises.push(ctx.table('users').get(userId));
}
const users = await Promise.all(userPromises);

// ✅ Clean with Ents getMany
const users = await ctx.table('users').getMany(userIds);
// Returns (Doc | null)[] for each ID

// ✅ Or use getManyX to throw if any ID doesn't exist
const users = await ctx.table('users').getManyX(userIds);
// Returns Doc[] or throws if any missing
```

### WARNING: Fetching All Related Documents

**⚠️ CRITICAL**: With Ents, fetching all related documents via edges is still like `.collect()`!

```typescript
// ❌ DANGEROUS: Could return millions of documents!
const user = await ctx.table('users').get(userId);
const allUserPosts = await user.edge('posts'); // Fetches ALL posts!

// ✅ SAFE: Use pagination for unbounded data
const recentPosts = await ctx
  .table('posts')
  .query()
  .withIndex('userId', (q) => q.eq('userId', userId))
  .order('desc')
  .paginate(args.paginationOpts);

// ✅ ACCEPTABLE: Limit results when traversing edges
const user = await ctx.table('users').get(userId);
const recentPosts = await user.edge('posts').order('desc').take(5);
// Safe: limited to 5 posts
```

### When to Use Each Helper

| Helper                          | Use When                       | Warning                             |
| ------------------------------- | ------------------------------ | ----------------------------------- |
| `ctx.table().getMany(ids)`      | Batch fetching by IDs          | Returns (Doc \| null)[] for each ID |
| `ctx.table().getManyX(ids)`     | Batch fetching, must all exist | Throws if any ID missing            |
| `asyncMap(items, fn)`           | Any async operation over array | Better than Promise.all             |
| `ent.edge('relation')`          | Traverse 1:1 or 1:many edge    | ⚠️ Fetches ALL for 1:many           |
| `ent.edge('relation').take(n)`  | Limited edge traversal         | Safe with limit                     |
| `ent.edge('relation').paginate` | Paginated edge traversal       | Best for unbounded relations        |

### Safe Patterns for Unbounded Relationships

```typescript
// ✅ PATTERN 1: Use aggregates for counts
const postCount = await aggregatePosts.count(ctx, {
  namespace: userId,
  bounds: {} as any,
});

// ✅ PATTERN 2: Use pagination for lists
const posts = await ctx
  .table('posts')
  .query()
  .withIndex('userId', (q) => q.eq('userId', userId))
  .order('desc')
  .paginate(args.paginationOpts);

// ✅ PATTERN 3: Use .take() for small previews
const user = await ctx.table('users').getX(userId);
const recentPosts = await user.edge('posts').order('desc').take(5);

// ✅ PATTERN 4: Batch operations on current page only
const posts = await ctx.table('posts').query().paginate(args.paginationOpts);
const authorIds = posts.page.map((p) => p.authorId);
const authors = await ctx.table('users').getMany(authorIds); // Only current page!
```

### Best Practices

1. **Always use `asyncMap`** instead of raw Promise.all for readability
2. **Use `getMany/getManyX`** for batch ID lookups with Ents
3. **NEVER fetch all** via edges on tables that could grow unbounded
4. **Always limit edge traversal** with `.take()` or `.paginate()`
5. **For large relationships**, use pagination or aggregates instead
6. **Document edge cardinality** in your schema (1:1, 1:many, many:many)

### Common Anti-Patterns

```typescript
// ❌ Fetching all to filter/map/count
const user = await ctx.table('users').get(userId);
const allPosts = await user.edge('posts');
const publishedPosts = allPosts.filter((p) => p.status === 'published');
const count = allPosts.length;

// ✅ Use index + aggregate instead
const publishedPosts = await ctx
  .table('posts')
  .query()
  .withIndex('userId_status', (q) =>
    q.eq('userId', userId).eq('status', 'published')
  )
  .paginate(args.paginationOpts);

const count = await aggregatePosts.count(ctx, {
  namespace: userId,
  bounds: {} as any,
});
```

## 7. Convex Platform Limits

**Edit:** Understand these limits when designing your schema and queries

### Index Limits

- **Indexes per table:** 32 max
- **Fields per index:** 16 max (no duplicate fields)
- **Index name length:** 64 characters max
- **Reserved fields:** Cannot index `_id`, `_creationTime` (automatically indexed)
- **Search indexes per table:** 4 max
- **Vector indexes per table:** 4 max
- **Filters per search/vector index:** 16 max

### Query & Mutation Limits

- **Documents scanned per query/mutation:** 16,384 max
- **Data scanned per query/mutation:** 8 MiB max
- **db.get/db.query calls per function:** 4,096 max
- **JS execution time per function:** 1 second max
- **Documents written per mutation:** 8,192 max
- **Data written per mutation:** 8 MiB max

### Document Limits

- **Document size:** 1 MiB max
- **Fields per document:** 1,024 max
- **Field name length:** 64 characters max (nested keys up to 1,024)
- **Field nesting depth:** 16 max
- **Array elements:** 8,192 max per array

### Search Limits

- **Search terms per query:** 16 max
- **Filters per search query:** 8 max (full text), 64 max (vector)
- **Term length:** 32 bytes max
- **Result set:** 1,024 max (full text), 256 max (vector, defaults to 10)
- **Vector dimensions:** 2-4096 range

### Action Limits

- **Action timeout:** 10 minutes max
- **Memory limit:** 64 MB (Convex runtime), 512 MB (Node.js runtime)
- **Concurrent operations per action:** 1,000 max
- **Function argument/return size:** 16 MiB max
- **HTTP action response size:** 20 MiB max

### Design Implications

```typescript
// ❌ BAD: Too many indexes (approaching 32 limit)
defineEnt({
  // 20+ fields...
})
  .index('field1', ['field1'])
  .index('field2', ['field2']);
// ... 30+ indexes - SLOW INSERTS!

// ✅ GOOD: Consolidate with compound indexes
defineEnt({
  // fields...
})
  .index('user_status', ['userId', 'status'])
  .index('user_date', ['userId', 'createdAt']);
// Reuse compound indexes for single-field queries too
```

```typescript
// ❌ BAD: Large arrays in documents
const character = {
  followers: ['userId1', 'userId2', ...], // Could hit 8,192 limit!
}

// ✅ GOOD: Use many:many edge for unbounded relationships
characters: defineEnt({
  name: v.string(),
  // ...
}).edges('followers', { to: 'users' }), // many:many edge

users: defineEnt({
  name: v.string(),
  // ...
}).edges('followingCharacters', { to: 'characters' })
```

```typescript
// ❌ BAD: Deep nesting (approaching 16 levels)
const data = {
  level1: {
    level2: {
      level3: {
        // ... up to level 16
      }
    }
  }
}

// ✅ GOOD: Flatten structure or use edges
const mainEntity = defineEnt({
  mainData: v.object({ ... }),
}).edge('details'), // 1:1 edge to details

const details = defineEnt({
  // Details stored in separate table
}).edge('mainEntity', { ref: true })
```

## Summary

1. **Always use indexes** for equality filters with `ctx.table()`
2. **Use aggregates for counting** - O(log n) vs O(n) performance with millions of documents
3. **Never fetch all documents** - use `.paginate()` for user-facing lists
4. **Isolate frequently-changing data** with separate tables and edges
5. **Enforce limits at insert time** when possible, using aggregates for counting
6. **Use search indexes** for text search with proper filter fields
7. **Understand filter + paginate behavior** - filtering happens during, not after
8. **Know platform limits** - design within Convex's boundaries for optimal performance
9. **Limit edge traversals** - use `.take()` or `.paginate()` on 1:many edges
10. **Streams still use ctx.db** - not yet migrated to Ents API

**Critical for Scale**: Replace fetching all documents with aggregates from [convex-aggregate.mdc](mdc:.cursor/rules/convex-aggregate.mdc). These patterns are essential for millions of documents.

## Decision Tree: Which Pattern to Use?

```
Need to COUNT documents?
  ↓
✅ Use AGGREGATES (O(log n) - scales to millions)
  - await aggregateItems.count(ctx, { bounds: {} as any })
  - Set up once with triggers, works automatically

Need a USER-FACING LIST with "load more"?
  ↓
✅ Use PAGINATION (.paginate())
  - createPublicPaginatedQuery() or createAuthPaginatedQuery()
  - With Ents: ctx.table('items').query().paginate(args.paginationOpts)
  - Consistent page sizes, good UX

Need a SMALL PREVIEW of items (< 100) from BOUNDED table?
  ↓
✅ Use TAKE (.take(n))
  - With Ents: ctx.table('items').query().take(5) or ent.edge('items').take(5)
  - ONLY when table has enforced size limits
  - ONLY when you don't need total count
  - Example: "Show 5 most recent notifications"

❌ NEVER fetch all documents without limits on production tables
❌ NEVER traverse 1:many edges without .take() or .paginate()
❌ NEVER count by fetching all documents - use aggregates
```

convex-review.mdc

---
description: Compact checklist for reviewing Convex queries/mutations for performance and best practices at scale (millions of docs per table)
globs: convex/**/*.ts
alwaysApply: false
---

**User Request:** $ARGUMENTS

# Convex Performance Review Checklist

Review each query/mutation in `convex/*.ts` files assuming millions of documents per table.

**Related Documentation:**

- [convex.mdc](mdc:.cursor/rules/convex.mdc) - Core Convex usage patterns
- [convex-optimize.mdc](mdc:.cursor/rules/convex-optimize.mdc) - Detailed optimization strategies
- [convex-aggregate.mdc](mdc:.cursor/rules/convex-aggregate.mdc) - Aggregation best practices
- [convex-streams.mdc](mdc:.cursor/rules/convex-streams.mdc) - Complex query patterns
- [convex-trigger.mdc](mdc:.cursor/rules/convex-trigger.mdc) - Database triggers

## Critical Rule: No ctx.db Usage

**🚨 FORBIDDEN:** Never use `ctx.db` in any Convex functions. Always use `ctx.table()` instead.

**⚠️ ONLY EXCEPTION:** Streams from `convex-helpers/server/stream` still require `ctx.db`.

## Query Review Checklist

### 1. Index Usage ⚡

- [ ] **Using index for equality filters?** Replace `.filter(q => q.eq(q.field('x'), y))` with `.withIndex('x', q => q.eq('x', y))`
- [ ] **Index exists in schema?** Check `convex/schema.ts` for `.index('field', ['field'])`
- [ ] **Compound index for multiple filters?** Use `.index('x_y', ['x', 'y'])` for `.eq('x', a).eq('y', b)`
- [ ] **Search index has filterFields?** Add common filters to `filterFields: ['userId', 'private']`

### 2. Document Limits 🚫

- [ ] **Never using `.collect()` without limit?** Use aggregates for counting, `.paginate()` for lists
- [ ] **Using aggregates for counting?** O(log n) vs O(n) performance with millions of docs
- [ ] **Using `.paginate()` for user-facing lists?** Required for unbounded data
- [ ] **Using `.collect()` on indexed queries?** Consider aggregates or pagination for 1000+ docs

### 3. Counting & Aggregation 📊

- [ ] **Never using `.collect().length`?** Use aggregates for O(log n) counts
- [ ] **Aggregate registered in triggers?** Check `convex/triggers.ts` for `triggers.register('table', aggregate.trigger())`
- [ ] **Using `aggregate.count()` not array.length?** For all count operations

### 4. Aggregate Lookup Guidelines 🔄

- [ ] **N+1 aggregate lookups are FINE for bounded data** (≤100 items per page)
- [ ] **Aggregate lookups in paginated queries?** ✅ GOOD - provides flexibility
- [ ] **Aggregate lookups in stream filters?** ✅ GOOD - each page is bounded
- [ ] **Only flag if looping over 1000+ of items** - That's when it becomes a problem

### 5. Filtering Best Practices 🔍

- [ ] **Simple filters: Using built-in `.filter()`?** Maintains full page sizes
- [ ] **Complex filters + pagination: Using streams?** For arrays/strings/async filters
- [ ] **Complex filters without pagination: Using filter helper?** For `.take()/.first()`

### 6. Query Optimization 🎯

- [ ] **Frequently-changing fields isolated?** Move timestamps/counters to separate tables
- [ ] **Using `.first()` not `.collect()[0]`?** More efficient for single doc
- [ ] **Batch operations use `getAll()`?** For fetching by multiple IDs
- [ ] **Using `asyncMap()` for async operations?** Not regular `map()` with await inside
- [ ] **Avoiding `.collect()` on large tables?** Even with indexes, it fetches ALL matching docs

### 7. Computed Data Patterns 🔄

- [ ] **Computing derived data on read?** Look for functions that scan many docs to compute per-entity values
- [ ] **Finding last/first/max/min across docs?** Common pattern: `getLastMessageMap`, `getLatestActivity`, etc.
- [ ] **O(n) aggregations in queries?** Use aggregates with proper bounds (O(log n))
- [ ] **Repeatedly calculating same values?** Pre-compute with triggers instead

## Mutation Review Checklist

### 1. Function Wrappers ✅

- [ ] **Using `zid()` for all IDs?** Never `z.string()` for document IDs
- [ ] **Rate limiting configured?** Add `{ rateLimit: 'feature/action' }` for user-facing mutations
- [ ] **Rate limit defined in `rateLimiter.ts`?** Check if the rate limit key exists with `:free` and `:premium` variants

### 2. Write Patterns 📝

- [ ] **Using `.patch()` for partial updates?** Not `.replace()` unless needed
- [ ] **Bulk inserts use loop not Promise.all?** Convex batches automatically mutations (not queries)
- [ ] **Large batches processed in chunks?** 500-1000 items per batch
- [ ] **Edge definitions handle cascades?** Check schema for proper edge relationships (hard deletion is default)
- [ ] **No manual cascade deletes needed?** Convex Ents handles cascade deletes automatically via edges

### 3. Error Handling 🛡️

- [ ] **Throwing `ConvexError` not `Error`?** With proper error codes
- [ ] **Checking document exists before update?** Handle null from `.get()`
- [ ] **Transaction safety considered?** All operations atomic within mutation

## Common Anti-Patterns to Flag 🚨

```typescript
// ❌ BAD: Full table scan
.filter(q => q.eq(q.field('userId'), userId))
// ✅ GOOD: Use index
.withIndex('userId', q => q.eq('userId', userId))

// ❌ BAD: Unbounded collection
const all = await ctx.table('items').query().collect();
// ✅ GOOD: Use aggregates for counting
const count = await aggregateItems.count(ctx, { bounds: {} as any });
// ✅ GOOD: Paginated for lists
const items = await ctx.table('items').query().paginate(paginationOpts);

// ❌ BAD: Count with collect
const count = (await ctx.table('items').query().withIndex('x', q => q.eq('x', x)).collect()).length;
// ✅ GOOD: Use aggregate
const count = await aggregateItems.count(ctx, { namespace: x });

// ❌ BAD: Manual aggregate update
await aggregate.insert(ctx, doc);
// ✅ GOOD: Let trigger handle it
await ctx.table('items').insert(doc); // Trigger fires automatically

// ❌ BAD: Raw function
export const myQuery = query({...});
// ✅ GOOD: Wrapped function
export const myQuery = createPublicQuery()({...});

// ⚠️ IMPORTANT: N+1 aggregate lookups are FINE for bounded data!
// ✅ GOOD: Aggregate lookups for paginated/bounded data (≤100 items)
for (const message of messages) { // 10-100 messages from paginated query
  const voteCount = await aggregateVotes.count(ctx, {
    namespace: chatId,
    bounds: { /* can filter by date, user, etc. */ }
  });
}
// ✅ 100 O(log n) operations are still very efficient

// ✅ GOOD: Aggregate lookups in stream filters (bounded by pagination)
// Note: streams still use ctx.db, not yet migrated to Ents
await stream(ctx.db, schema)
  .query('characters')
  .filterWith(async (char) => {
    // This is FINE - each page is limited (e.g., 10-50 items)
    const isStarred = await aggregateStars.count(ctx, {
      namespace: userId,
      bounds: { lower: { key: char._id }, upper: { key: char._id } }
    });
    return isStarred > 0;
  })
  .paginate(args.paginationOpts);

// ❌ BAD: Only when dealing with unbounded/thousands of items
const allUsers = await ctx.table('users').query().collect(); // Could be millions!
for (const user of allUsers) { // ❌ Looping over unbounded data
  const stats = await aggregateUserStats.count(ctx, { namespace: user._id });
}

// ❌ BAD: Computing derived data on every query
async function getLastMessageMap(ctx, userId, characterIds) {
  const chats = await ctx.table('chats')
    .query()
    .withIndex('userId', q => q.eq('userId', userId))
    .take(1000); // Scans up to 1000 docs every time!

  // O(n) computation on every call
  const map = new Map();
  for (const chat of chats) {
    if (characterIds.includes(chat.characterId)) {
      const existing = map.get(chat.characterId);
      if (!existing || chat.lastMessageAt > existing) {
        map.set(chat.characterId, chat.lastMessageAt);
      }
    }
  }
  return map;
}

// ✅ GOOD: Use aggregates for flexibility
// Even for 100 lookups, aggregates provide bounds queries
for (const characterId of characterIds) {
  const lastActivity = await aggregateCharacterActivity.max(ctx, {
    namespace: userId,
    bounds: {
      lower: { key: characterId, inclusive: true },
      upper: { key: characterId, inclusive: true }
    } as any
  });
  lastMessageMap.set(characterId, lastActivity);
}
// Still efficient and supports date ranges, filtering, etc.

// ❌ BAD: Raw Promise.all for batch operations
const results = await Promise.all(
  ids.map(id => someAsyncOperation(id))
);
// ✅ GOOD: Use asyncMap from convex-helpers
import { asyncMap } from 'convex-helpers';
const results = await asyncMap(ids, someAsyncOperation);

// ❌ DANGEROUS: getManyFrom on unbounded table
const allUserPosts = await ctx.table('posts').query().withIndex('userId', q => q.eq('userId', userId)).collect();
// Could return millions of posts!
// ✅ SAFE: Use pagination or take a limited number
const recentPosts = await ctx
  .table('posts')
  .query()
  .withIndex('userId', q => q.eq('userId', userId))
  .order('desc')
  .take(10);
```

## Performance Red Flags 🚩

1. **Missing indexes**: Any `.filter()` without preceding `.withIndex()`
2. **Unbounded queries**: `.collect()` without enforced size limits
3. **Manual counting**: Any `.length` on collections (use aggregates)
4. **Missing aggregates**: Count operations without O(log n) aggregates
5. **Cache thrashing**: Frequently-updated fields in widely-referenced docs
6. **Complex filters + paginate**: Not using streams for consistent page sizes
7. **Missing triggers**: Aggregates updated manually in mutations
8. **Manual cascade deletes**: Redundant deletes in mutations when Convex Ents handles them automatically
9. **Computed data patterns**: Functions that scan many docs to compute per-entity values
10. **O(n) aggregations**: Computing last/first/max/min across unbounded collections
11. **Raw Promise.all**: Not using `asyncMap` from convex-helpers for async operations
12. **Unbounded indexed queries**: Using `.collect()` that could match millions of documents

## ✅ Performance Green Flags (Don't Flag These!)

1. **Aggregate lookups in loops**: Fine for ≤100 items (paginated data)
2. **Aggregate lookups in stream filters**: Each page is bounded, so it's efficient
3. **Multiple aggregate queries**: O(log n) × 100 is still very fast
4. **asyncMap with aggregates**: Proper pattern for parallel aggregate lookups
5. **Starred/favorited checks**: Using aggregates for user preferences is correct

## Domain Modeling vs Performance Optimization

### When to Keep Separate Tables (like `characterActivity`)

```typescript
// ✅ GOOD: Separate table for domain relationships
// characterActivity represents user's interaction history with characters
characterActivity: defineEnt({
  userId: v.id('users'),
  characterId: v.id('characters'),
  lastMessageAt: v.number(),
  // Could extend with:
  // interactionCount: v.number(),
  // lastActionType: v.string(), // 'chat', 'star', 'view'
  // favoriteLevel: v.number(),
})
  .index('userId', ['userId'])
  .index('characterId', ['characterId'])
  .index('user_character', ['userId', 'characterId']);

// ✅ Appropriate because:
// - Represents meaningful domain relationship
// - May need additional metadata
// - Enables queries by userId AND characterId
// - Not just a performance cache
```

### When to Use Aggregates Instead

```typescript
// ✅ GOOD: Aggregates for pure statistics
const aggregateMessages = new TableAggregate<{
  Namespace: Id<'chats'>;
  Key: null;
  DataModel: DataModel;
  TableName: 'messages';
}>(components.aggregateMessages, {
  namespace: (doc) => doc.chatId,
  sortKey: () => null,
});

// ✅ Appropriate because:
// - Just counting messages
// - No domain meaning
// - Single-purpose optimization
// - No additional metadata needed
```

## Convex Platform Limits 📏

### Quick Reference

- **Indexes per table:** 32 (too many = slow inserts)
- **Documents per query:** 16,384 max scanned
- **Query time limit:** 1 second JS execution
- **Document size:** 1 MiB max
- **Array elements:** 8,192 max
- **Search results:** 1,024 max

**Full limits documentation:** [convex-optimize.mdc#7](mdc:.cursor/rules/convex-optimize.mdc)

## Quick Wins 🏆

1. **Add index**: For every equality filter
2. **Setup aggregates**: For all count operations (O(log n) vs O(n))
3. **Use pagination**: Replace `.collect()` with `.paginate()` for lists
4. **Use streams**: For complex filtered pagination
5. **Register triggers**: For automatic aggregate maintenance
6. **Isolate counters**: Move to separate tables
7. **Use aggregates for computed data**: Replace O(n) scans with O(log n) aggregates

## Review Example

```typescript
// BEFORE: Multiple issues
export const getTeamMembers = query({
  args: { teamId: v.id('teams') },
  handler: async (ctx, args) => {
    const members = await ctx
      .table('members')
      .query()
      .filter((q) => q.eq(q.field('teamId'), args.teamId)) // 🚩 No index
      .collect(); // 🚩 Unbounded

    const count = members.length; // 🚩 O(n) count
    return { members, count };
  },
});

// AFTER: Optimized for millions of documents
export const getTeamMembers = createPublicPaginatedQuery()({
  args: { teamId: zid('teams') }, // ✅ zid for IDs
  handler: async (ctx, args) => {
    // ✅ Using index + pagination
    const results = await ctx
      .table('members')
      .query()
      .withIndex('teamId', (q) => q.eq('teamId', args.teamId))
      .paginate(args.paginationOpts);

    // ✅ O(log n) count with aggregate - scales to millions
    const count = await aggregateMembers.count(ctx, {
      namespace: args.teamId,
      bounds: {} as any,
    });

    return { ...results, totalCount: count };
  },
});

// In triggers.ts:
triggers.register('members', aggregateMembers.trigger()); // ✅ Automatic maintenance
```