Skip to content

Instantly share code, notes, and snippets.

@bobsilverberg
Created April 4, 2025 19:33
Show Gist options
  • Save bobsilverberg/59cf1882d0185f1c68132a8fa9a2cb7c to your computer and use it in GitHub Desktop.
Save bobsilverberg/59cf1882d0185f1c68132a8fa9a2cb7c to your computer and use it in GitHub Desktop.
i functions: Beginning execution of "northamerica-northeast1-ext-firestore-bigquery-export-fsexportbigquery"
> {"bqProjectId":"dev-shelter-movers","databaseId":"(default)","collectionPath":"sync-bigquerydocs","datasetId":"dev_bob","tableId":"main","initialized":false,"datasetLocation":"northamerica-northeast1","timePartitioning":null,"clustering":null,"wildcardIds":false,"useNewSnapshotQuerySyntax":true,"excludeOldData":false,"viewType":"materialized_incremental","instanceId":"firestore-bigquery-export","maxDispatchesPerSecond":100,"maxEnqueueAttempts":3,"backupToGCS":false,"backupBucketName":"dev-shelter-movers.appspot.com","backupDir":"_firestore-bigquery-export","logLevel":"info","severity":"INFO","message":"Started execution of extension with configuration"}
> {"document_name":"projects/dev-shelter-movers/databases/(default)/documents/sync-bigquerydocs/WSSGFCz5dEqvhGGYmIED","event_id":"a86bb294-e3eb-4333-b3d2-c97e8e746840","operation":0,"severity":"INFO","message":"Firestore event received by onWrite trigger"}
⚠ Google API requested!
- URL: "https://oauth2.googleapis.com/token"
- Be careful, this may be a production service.
⚠ Google API requested!
- URL: "https://bigquery.googleapis.com/bigquery/v2/projects/dev-shelter-movers/datasets/dev_bob/tables/main_raw_changelog/insertAll?prettyPrint=false"
- Be careful, this may be a production service.
> {"severity":"WARNING","message":"Error when inserting data to table."}
> {"severity":"WARNING","message":"ROW DATA JSON:"}
> {"severity":"WARNING","message":"undefined"}
> {"severity":"WARNING","message":"Failed to write event to BigQuery Immediately. Will attempt to Enqueue to Cloud Tasks. ApiError: Caller does not have required permission to use project dev-shelter-movers. Grant the caller the roles/serviceusage.serviceUsageConsumer role, or a custom role with the serviceusage.services.use permission, by visiting https://console.developers.google.com/iam-admin/iam/project?project=dev-shelter-movers and then retry. Propagation of the new permission may take a few minutes.\n at new ApiError (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/@google-cloud/common/build/src/util.js:75:15)\n at Util.parseHttpRespBody (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/@google-cloud/common/build/src/util.js:210:38)\n at Util.handleResp (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/@google-cloud/common/build/src/util.js:151:117)\n at /Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/@google-cloud/common/build/src/util.js:534:22\n at onResponse (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/@google-cloud/common/node_modules/retry-request/index.js:259:7)\n at /Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/teeny-request/build/src/index.js:226:13\n at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {\n code: 403,\n errors: [\n {\n message: 'Caller does not have required permission to use project dev-shelter-movers. Grant the caller the roles/serviceusage.serviceUsageConsumer role, or a custom role with the serviceusage.services.use permission, by visiting https://console.developers.google.com/iam-admin/iam/project?project=dev-shelter-movers and then retry. Propagation of the new permission may take a few minutes.',\n domain: 'global',\n reason: 'forbidden'\n }\n ],\n response: Gunzip {\n _writeState: Uint32Array(2) [ 14244, 0 ],\n _events: {\n close: undefined,\n error: [Array],\n prefinish: [Function: prefinish],\n finish: undefined,\n drain: undefined,\n data: [Function (anonymous)],\n end: [Function (anonymous)],\n readable: undefined,\n unpipe: undefined\n },\n _readableState: ReadableState {\n highWaterMark: 65536,\n buffer: [],\n bufferIndex: 0,\n length: 0,\n pipes: [],\n awaitDrainWriters: null,\n [Symbol(kState)]: 194512764\n },\n _writableState: WritableState {\n highWaterMark: 65536,\n length: 0,\n corked: 0,\n onwrite: [Function: bound onwrite],\n writelen: 0,\n bufferedIndex: 0,\n pendingcb: 0,\n [Symbol(kState)]: 1091466620,\n [Symbol(kBufferedValue)]: null,\n [Symbol(kWriteCbValue)]: null\n },\n allowHalfOpen: true,\n _maxListeners: undefined,\n _eventsCount: 4,\n bytesWritten: 566,\n _handle: null,\n _outBuffer: <Buffer 7b 0a 20 20 22 65 72 72 6f 72 22 3a 20 7b 0a 20 20 20 20 22 63 6f 64 65 22 3a 20 34 30 33 2c 0a 20 20 20 20 22 6d 65 73 73 61 67 65 22 3a 20 22 43 61 ... 16334 more bytes>,\n _outOffset: 2140,\n _chunkSize: 16384,\n _defaultFlushFlag: 2,\n _finishFlushFlag: 2,\n _defaultFullFlushFlag: 3,\n _info: undefined,\n _maxOutputLength: 9007199254740991,\n _level: -1,\n _strategy: 0,\n statusCode: 403,\n statusMessage: 'Forbidden',\n request: {\n agent: [Agent],\n headers: [Object],\n href: 'https://bigquery.googleapis.com/bigquery/v2/projects/dev-shelter-movers/datasets/dev_bob/tables/main_raw_changelog/insertAll?prettyPrint=false'\n },\n body: '{\\n' +\n ' \"error\": {\\n' +\n ' \"code\": 403,\\n' +\n ' \"message\": \"Caller does not have required permission to use project dev-shelter-movers. Grant the caller the roles/serviceusage.serviceUsageConsumer role, or a custom role with the serviceusage.services.use permission, by visiting https://console.developers.google.com/iam-admin/iam/project?project=dev-shelter-movers and then retry. Propagation of the new permission may take a few minutes.\",\\n' +\n ' \"errors\": [\\n' +\n ' {\\n' +\n ' \"message\": \"Caller does not have required permission to use project dev-shelter-movers. Grant the caller the roles/serviceusage.serviceUsageConsumer role, or a custom role with the serviceusage.services.use permission, by visiting https://console.developers.google.com/iam-admin/iam/project?project=dev-shelter-movers and then retry. Propagation of the new permission may take a few minutes.\",\\n' +\n ' \"domain\": \"global\",\\n' +\n ' \"reason\": \"forbidden\"\\n' +\n ' }\\n' +\n ' ],\\n' +\n ' \"status\": \"PERMISSION_DENIED\",\\n' +\n ' \"details\": [\\n' +\n ' {\\n' +\n ' \"@type\": \"type.googleapis.com/google.rpc.ErrorInfo\",\\n' +\n ' \"reason\": \"USER_PROJECT_DENIED\",\\n' +\n ' \"domain\": \"googleapis.com\",\\n' +\n ' \"metadata\": {\\n' +\n ' \"service\": \"bigquery.googleapis.com\",\\n' +\n ' \"consumer\": \"projects/dev-shelter-movers\",\\n' +\n ' \"containerInfo\": \"dev-shelter-movers\"\\n' +\n ' }\\n' +\n ' },\\n' +\n ' {\\n' +\n ' \"@type\": \"type.googleapis.com/google.rpc.LocalizedMessage\",\\n' +\n ' \"locale\": \"en-US\",\\n' +\n ' \"message\": \"Caller does not have required permission to use project dev-shelter-movers. Grant the caller the roles/serviceusage.serviceUsageConsumer role, or a custom role with the serviceusage.services.use permission, by visiting https://console.developers.google.com/iam-admin/iam/project?project=dev-shelter-movers and then retry. Propagation of the new permission may take a few minutes.\"\\n' +\n ' },\\n' +\n ' {\\n' +\n ' \"@type\": \"type.googleapis.com/google.rpc.Help\",\\n' +\n ' \"links\": [\\n' +\n ' {\\n' +\n ' \"description\": \"Google developer console IAM admin\",\\n' +\n ' \"url\": \"https://console.developers.google.com/iam-admin/iam/project?project=dev-shelter-movers\"\\n' +\n ' }\\n' +\n ' ]\\n' +\n ' }\\n' +\n ' ]\\n' +\n ' }\\n' +\n '}\\n',\n headers: {\n 'alt-svc': 'h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000',\n 'content-encoding': 'gzip',\n 'content-type': 'application/json; charset=UTF-8',\n date: 'Fri, 04 Apr 2025 19:30:58 GMT',\n server: 'ESF',\n 'transfer-encoding': 'chunked',\n vary: 'Origin, X-Origin, Referer',\n 'x-frame-options': 'SAMEORIGIN',\n 'x-xss-protection': '0'\n },\n toJSON: [Function: toJSON],\n [Symbol(shapeMode)]: true,\n [Symbol(kCapture)]: false,\n [Symbol(kCallback)]: null,\n [Symbol(kError)]: null\n }\n}"}
> {"document_name":"projects/dev-shelter-movers/databases/(default)/documents/sync-bigquerydocs/WSSGFCz5dEqvhGGYmIED","event_id":"a86bb294-e3eb-4333-b3d2-c97e8e746840","operation":"CREATE","error":{"code":"functions/unknown-error","message":"Failed to determine service account. Initialize the SDK with service account credentials or set service account ID as an app option."},"severity":"ERROR","message":"Error: Failed to enqueue event to Cloud Tasks from onWrite handler\n at entryFromArgs (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/firebase-functions/lib/logger/index.js:130:19)\n at error (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/firebase-functions/lib/logger/index.js:116:11)\n at Logger.runIfLogLevel (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/lib/logger.js:64:13)\n at Logger.error (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/node_modules/@firebaseextensions/firestore-bigquery-change-tracker/lib/logger.js:57:14)\n at Object.logFailedEventAction (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/lib/logs.js:175:20)\n at attemptToEnqueue (/Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/lib/index.js:236:14)\n at async /Users/robertsilverberg/.cache/firebase/extensions/firebase/[email protected]/functions/lib/index.js:159:9\n at async runFunction (/Users/robertsilverberg/Documents/gitRepos/sm-operations-app/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:9)\n at async runBackground (/Users/robertsilverberg/Documents/gitRepos/sm-operations-app/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:517:5)\n at async processBackground (/Users/robertsilverberg/Documents/gitRepos/sm-operations-app/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:501:5)"}
> {"severity":"INFO","message":"Completed execution of extension"}
i functions: Finished "northamerica-northeast1-ext-firestore-bigquery-export-fsexportbigquery" in 1823.138864ms
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment