Getting triggered by Gadget - Trigger.dev integration with Gadget.dev 🤝

Trigger.dev is "The open source Background Jobs framework for TypeScript".

It allows you to "Create long-running Jobs directly in your codebase with features like API integrations, webhooks, scheduling and delays."

It has support for frameworks such as Next.js, Express, and Remix. But can it work with Gadget? 🤔

Turns out the answer is "Yes", and it works great!

This tutorial is based on the code in a PR for a Fastify adaptor for Trigger. Seeing as Gadget uses Fastify under the hood, I figured it'd be easy enough to port that code over.

The trick is with Gadget we don't have a local environment running, but we do get our code changes pushed live to dev, so that's good enough to work with.

I loosely followed the Express tutorial to get started: https://trigger.dev/docs/documentation/guides/manual/express, and then tweaked it based on the Fastify PR mentioned above.

So, to start:

1. Install packages:

yarn add "@trigger.dev/sdk" "@remix-run/web-fetch"

Note: We don't need the @trigger.dev/express adaptor mentioned in the Express tutorial. We also want to add the "@remix-run/web-fetch" dependency, which is used in the Fastify adaptor to convert a Fastify request to a standard web one...

2. Get your API key

Follow the instructions at: https://trigger.dev/docs/documentation/guides/manual/express#obtaining-the-development-server-api-key to get your dev server API key.

Add it to an environment variable in Gadget: {your-gadget-app}/edit/settings/env, i.e.

TRIGGER_API_KEY: {your-key}

You may also want to add your Trigger project name as an env var, i.e.

TRIGGER_PROJECT_NAME: {your-project-name}

3. Configure the Trigger Client

Similar to this step: https://trigger.dev/docs/documentation/guides/manual/express#configuring-the-trigger-client, create a file called trigger.js in your Gadget app (we'll use the root folder for simplicity), and add the following:

(Note: Omit the Sentry parts if you've not got that setup in Gadget)

import { TriggerClient } from "@trigger.dev/sdk";

// Import the logger from gadget-server so we can log to the Gadget logs
import { logger } from "gadget-server";

// Require Sentry so we can log errors to Sentry and get notified
const Sentry = require("@sentry/node");

export const client = new TriggerClient({
  id: process.env.TRIGGER_PROJECT_NAME, // Or hard code it as a string if you prefer...
  apiKey: process.env.TRIGGER_API_KEY,
  apiUrl: process.env.TRIGGER_API_URL, // Not really required as it's only used for self-hosting, which we're not...
});

// Send failures to Sentry so we can be notified
client.on("runFailed", async (notification) => {
  logger.error({ notification }, `Run on job ${notification.job?.id} failed`);

  Sentry.setContext("notification", notification);

  const scope = new Sentry.Scope();
  scope.setTag("jobId", notification.job?.id);
  scope.setTag("taskId", notification.task?.id);

  Sentry.captureException(notification.error, scope);
});

4. Add a Trigger job

We skip ahead a bit here and create our job first, so we can register it in the next step.

Based on this step: https://trigger.dev/docs/documentation/guides/manual/express#creating-the-example-job, but we add some more Gadget bits and bobs for good measure. Tweak as you see fit.

Create a folder called jobs in your Gadget app, and add a file called example.js (or whatever you want to call it), i.e. {your-gadget-app}/edit/files/jobs/example.js

import { eventTrigger } from "@trigger.dev/sdk";
import { client } from "../trigger";

// Import the logger from gadget-server so we can log to the Gadget logs from our job
import { logger } from "gadget-server";

// Import the api from gadget-server so we can make requests to the Gadget API from our job
// Note: This api client has the `system-admin` role and can perform any action in your Gadget app!
import { api } from "gadget-server";

// your first job
client.defineJob({
  id: "example-job",
  name: "Example Job",
  version: "0.0.1",
  trigger: eventTrigger({
    name: "example.event",
  }),
  run: async (payload, io, ctx) => {
    // Logs to the Gadget logs, view in Gadget UI
    logger.info(
      { payload, env: process.env.GADGET_ENV },
      "Gadget logs => Hello world!"
    );

    // Logs to the Trigger logs, view in Trigger UI
    await io.logger.info("Trigger logs => Hello world!", {
      payload,
      env: process.env.GADGET_ENV,
    });

    // Find a model in our Gadget app, using the generated API
    const shop = await api.shopifyShop.maybeFindFirst({
      filter: {
        name: { equals: "trigger-gadget-dev" }, // Replace with a shop name in your Gadget app
      },
      select: { id: true, name: true, domain: true },
    });

    logger.info({ shop }, "Gadget logs => shop");
    await io.logger.info("Trigger logs => shop", { shop });

    return {
      message: "Hello world!",
    };
  },
});

5. Add a Gadget route

In your routes folder in the Gadget app, add a new folder called: api

Then, a route file to your Gadget app called: POST-trigger.js, i.e. {your-gadget-app}/edit/files/routes/api/POST-trigger.js

It is prefixed with POST- because we want to trigger it via POST requests only.

This route file acts as an equivalent of the Fastify adaptor mentioned above, which only really registers some code against an /api/trigger route anyways.

Note the use of the "@remix-run/web-fetch" package to convert the Fastify request to a standard web one.

Also note the important step of importing all our jobs so they can be registered with Trigger.

// Custom trigger route, based on: https://github.com/triggerdotdev/trigger.dev/blob/d7b1b234edd27b722a5f80c10e258d2bbe290b00/packages/fastify/src/plugin.ts

import { RouteContext } from "gadget-server";

// Used to standardize the request and headers, doesn't actually do anything Remix specific:
import {
  Request as StandardRequest,
  Headers as StandardHeaders,
} from "@remix-run/web-fetch";

// Import our instantiated Trigger client instead of the one from the Trigger SDK
import { client as triggerClient } from "../../trigger";

// !!! Important !!!  Import all your jobs here so they can be registered.
// Otherwise, Trigger will not know about them and they won't show up in the "Jobs" tab (in the Trigger UI).
//
import "../../jobs/example"; // Our example job from step 4.

const convertToStandardRequest = (req) => {
  const { headers: nextHeaders, method } = req;

  const headers = new StandardHeaders();

  Object.entries(nextHeaders).forEach(([key, value]) => {
    headers.set(key, value);
  });

  // Create a new Request object (hardcode the url because it doesn't really matter what it is)
  return new StandardRequest("https://fastify.js/api/trigger", {
    headers,
    method,
    // @ts-ignore
    body: req.body ? JSON.stringify(req.body) : req,
  });
};

/**
 * Route handler for POST api/trigger
 *
 * @param { RouteContext } route context - see: https://docs.gadget.dev/guides/http-routes/route-configuration#route-context
 *
 */
export default async function route({
  request,
  reply,
  api,
  logger,
  connections,
}) {
  try {
    logger.debug({ request }, "Trigger Dev SDK request received");

    const standandRequest = convertToStandardRequest(request);
    const response = await triggerClient.handleRequest(standandRequest);

    if (!response) {
      logger.error({ error }, "Trigger - Not found");
      reply.status(404).send({ error: "Not found" });
      return reply;
    }

    logger.debug({ response }, "Trigger Dev SDK sending response");
    // Updated to include response headers based on advice from Trigger support:
    reply.status(response.status).headers(response.headers).send(response.body);
    return reply;
  } catch (error) {
    logger.error({ error }, "Trigger Dev SDK error occurred");

    reply
      .status(500)
      .send({ message: `Trigger Dev SDK error occurred: ${error}` });
    return reply;
  }
}

6. Test it out

Assuming all the above files are live on Gadget in the dev environment, you should now be able to register your Gadget app endpoint as the Endpoint URL for your Trigger Dev environment.

In the Trigger UI, go to the "Environments & API Keys" page, i.e. https://cloud.trigger.dev/orgs/{your-org}/projects/{your-project}/environments

In the "Endpoints" section, click on the DEV environment option, then in the Endpoint URL field, enter the URL to your Gadget app route, i.e. https://{your-gadget-app}.gadget.dev/api/trigger

Then, click the "Save" button.

When you save, Trigger will send a request to your Gadget app to verify it's working. If it is, you should see a success message in the Trigger UI: "Status => Endpoint configured"

Important! You'll need to "Refresh" this endpoint each time you create a new job (or edit the metadata of an existing one), so that Trigger can get the latest metadata about your jobs.

There's a Trigger CLI which does this for you automatically when developing locally, but not sure if that will work with Gadget or not, need to try it and see...

If not, check your Gadget logs to see what the problem is.

When you save the Endpoint URL, Trigger requests metadata about the jobs you have registered in code, so it can display them in the UI.

Check the "Jobs" tab in the Trigger UI to see your jobs.

There should be a, Example Job job listed there. Click on it to see the details, and then click the Test button to run it.

If all goes well, you should see the job run successfully, and you should see the logs in both the Gadget and Trigger UIs. 🎉

Click on the Run of the job to see the details of the run, including the logs.

7. ??? - Profit? 💸

You should now have a nicely integrated way of running background jobs in your Gadget app, using Trigger.dev. Explore the rest of the Trigger docs to see what's possible!

edlaver/readme.md