Shopify Webhook Filtering Beta Docs.md

Filtering Shopify webhooks

Shopify allows you to filter the events that trigger a webhook for your application. There's two ways to filter webhooks in Shopify:

limit which records are delivered to your app, which you can do with a webhook filter
limit which fields of each record are delivered to your app, which you can do with an includeFields list

Both of these strategies can be used individually or together to limit the number of webhook Shopify sends to your app, and lower your total Gadget compute time.

Webhook filters

In Gadget, you can use the same filter syntax provided by Shopify to filter your action's Shopify data triggers. When a filter is added, Gadget will register the webhook with the filter using Shopify's API, and Shopify will start only sending data to your app that passes the filter. Your filter must be written in Shopify's filter syntax, and can be applied to any Shopify model actions or global actions with the Shopify Data trigger added.

Note that there are some notable differences between Shopify's filtering and search syntax that are important to be aware of when building webhook filters.

Applying a webhook filter

You can apply webhook filters or include fields lists to root model actions triggered by Shopify webhooks, or to webhook-triggered global actions.

To apply a filter to a Shopify data trigger:

Navigate to an action triggered by a Shopify webhook.
Open the menu on the filter card and select Filter.
Enter your filter in the trigger card on the action. This will add the trigger to your action options in your source code.
Gadget will automatically re-register your webhooks in your development environment, and re-register them in your production environment when you deploy.

Do not define your filters in your `shopify.app.toml` file in Gadget. They will not be applied to your webhook subscriptions.

For example, you could use the following filter on product webhook triggers if you only wanted webhooks to be fired when the product vendor is Nike:

// in api/actions/shopifyProduct/update.ts

// a sample filter for a product webhook
export const options: ActionOptions = {
  triggers: {
    shopify: {
      shopifyFilter: "vendor:Nike",
    },
  },
};

You can also use filters on fields of child models, such as the price of product variants. Webhooks will be fired if ANY of the children pass the filter condition.

For example, you can filter to only receive webhooks for products if any of that product's variants have a price greater than $10.00:

// in api/actions/shopifyProduct/update.ts

// a sample filter for a product webhook that filters on variant fields
export const options: ActionOptions = {
  triggers: {
    shopify: {
      shopifyFilter: "variants.price:>=10.00",
    },
  },
};

Filters on child models must be set on the parent model's action where the webhook is actually received, *not* the child model's action. Follow the links in the **Shopify Data** trigger card to find the parent model's action.

Filters are also applied to the nightly reconciliation job that Gadget runs for your Shopify apps. Only missed events that would have triggered a webhook will be re-run.

Filters are not applied to any Shopify data syncs.

Syntax errors in webhook filters will be reported by Shopify and will produce an error in your Gadget logs. Gadget will also alert you to a failed webhook registration. See Shopify's guide on debugging filters for more details.

Modify webhook payloads with `includeFields`

Gadget also supports Shopify's includeFields option that allows you to modify your webhook payload. This allows you to limit the data Shopify adds to a webhook payload so only a subset of the typical payload data is returned. If a record changes, but no fields in your includeFields list change, Shopify will not send a webhook, and you won't incur any Gadget compute time. If fields in your includeFields list do change, the webhook will be sent with only those fields, so you can still subscribe to the data you care about, and ignore the data you don't.

To modify webhook payloads in Gadget:

Navigate to an action triggered by a Shopify webhook.
Open the menu on the filter card and select Include fields.
Enter a comma-separated list of fields that will be included in the webhook payload.
Gadget will automatically re-register your webhooks in your development environment, and re-register them in your production environment when you deploy.

Shopify recommends you always include a resource's id and updated_at fields in a webhook payload.

For example, here is how you could limit a products/create webhook payloads to only include the product title:

// in api/actions/shopifyProduct/update.ts

// a sample includeFields for a products/update webhook
// note: we must the id field, and Shopify recommends including the updated_at field
export const options: ActionOptions = {
  triggers: {
    shopify: {
      includeFields: ["id", "updated_at", "title"],
    },
  },
};

You can also include fields across child models, such as the price on all a product's variants:

// in api/actions/shopifyProduct/update.ts

// a sample includeFields for a products/update webhook
// note: we must the id field, and Shopify recommends including the updated_at field
export const options: ActionOptions = {
  triggers: {
    shopify: {
      includeFields: ["id", "updated_at", "variants.id", "variants.updated_at", "variants.price"],
    },
  },
};

Field names in the `includeFields` list must match Shopify's field names, not Gadget's field names, so use camel case! For example: `email_marketing_consent`, not `emailMarketingConsent`.

Combining webhook filters and payload modification

You can combine webhook filters and includeFields to limit both which records and which fields are delivered to your app. If you are using webhook filtering at the same time as includeFields for the same webhook, you must ensure that all fields used in the filter are part of the webhook payload. Otherwise Shopify may not be able to register the webhook, or the filter condition could not work as expected.

Using different filters on different model actions

You can use different webhook filters or includeFields trigger settings for topics on the same Shopify resource. To do this, set the filter for the Gadget model action corresponding to the topic you want to adjust.

For the shopifyProduct model for example, here's how the model actions correspond to the Shopify webhook topics Gadget will register:

Gadget model action	Shopify topic
`shopifyProduct.create`	`products/create`
`shopifyProduct.update`	`products/update`
`shopifyProduct.delete`	`products/delete`

To filter on nested models, like shopifyProductVariant, you generally need to set filters or includeFields on both the parent model's create and update actions. This is because the update topic for a resource can create, update, or delete children records, so you need to set the filter or includeFields on both actions to ensure you get all the data you need.

For example, if you want to know everything about a product when it is created, but then only find out about changes to the title in the future as products are updated, you can set no filter on the shopifyProduct.create action, and set includeFields on the shopifyProduct.update action to only include id and title.

Using multiple different filters on the same topics

Gadget supports only one filter and includeFields per Shopify Data trigger, so Shopify model actions can only have one filter or includeFields set. You can combine different filters with Shopify's filtering syntax using the and and or operators.

If you need two independent filters, you can also use Gadget's global actions with the Shopify Data trigger. To register two different, independent filters, create two different global actions, and add the Shopify Data trigger to each. You can then pick which topics each global action should register webhooks for, and set independent filters or includeFields on each. Under the hood, Gadget will register one webhook for each global action if their filters are different, and Shopify will send webhooks to the global actions accordingly.

Filling in ignored records or fields later

If you are using a webhook filter to ignore certain records, or includeFields to ignore changes to certain fields, Gadget won't automatically populate that data later. If you do need the data eventually, you can fill in those records or fields later in your app by running a sync. It's up to you to decide how often you want to run a sync to fill in the data.

A common pattern is to schedule a daily sync to run against models you want to fill in all the data for. You can run syncs for all your installed shops on a schedule with a global action like this:

// in api/actions/syncWebhookFilteredModels.ts
import { ActionOptions } from "gadget-server";
import { globalShopifySync } from "gadget-server/shopify";

const HourInMs = 60 * 60 * 1000;

export const run: ActionRun = async ({ params, logger, api, connections }) => {
  const syncSince = new Date(Date.now() - 25 * HourInMs);

  await globalShopifySync({
    apiKeys: connections.shopify.apiKeys,
    syncSince,
    models: ["shopifyProduct", "shopifyProductVariant"],
  });
};

export const options: ActionOptions = {
  triggers: {
    scheduler: [
      {
        every: "day",
        at: "08:00 UTC",
      },
    ],
  },
};

This strategy achieves a good balance of cost vs data freshness. Webhooks will deliver up to date data in real-time for your app, so for the things you really care about freshness for you can use them, but filter out data you don't care about. You can then ensure it doesn't drift too far out of sync by running a sync to refresh the other data at whatever schedule meets your requirements.

airhorns/Shopify Webhook Filtering Beta Docs.md