Asynchronous execution for Google App Scripts (gas)

Async.gs

/*
* Async.gs
*
* Manages asyncronous execution via time-based triggers.
*
* Note that execution normally takes 30-60s due to scheduling of the trigger.
*
* @see https://developers.google.com/apps-script/reference/script/clock-trigger-builder.html
*/

var Async = Async || {};
var GLOBAL = this;

// Triggers asynchronous execution of a function (with arguments as extra parameters)
Async.call = function(handlerName) {
  return Async.apply(handlerName, Array.prototype.slice.call(arguments, 1));
};

// Triggers asynchronous execution of a function (with arguments as an array)
Async.apply = function(handlerName, args) {
  var trigger = ScriptApp
  .newTrigger('Async_handler')
  .timeBased()
  .after(1)
  .create();
  CacheService.getScriptCache().put(String(trigger.getUniqueId()), JSON.stringify({ handlerName: handlerName, args: args }));
  return { 
    triggerUid: trigger.getUniqueId(),
    source: String(trigger.getTriggerSource()), 
    eventType: String(trigger.getEventType()), 
    handlerName: handlerName,
    args: args
  };
};

// GENERIC HANDLING BELOW
// 
function Async_handler(e) {
  var triggerUid = e && e.triggerUid;
  var cache = CacheService.getScriptCache().get(triggerUid);
  if (cache) {
    try {
      var event = JSON.parse(cache);
      var handlerName = event && event.handlerName;
      var args = event && event.args;
      if (handlerName) {
        var context, fn = handlerName.split('.').reduce(function(parent, prop) {
          context = parent;
          return parent && parent[prop];
        }, GLOBAL);
        if (!fn || !fn.apply) throw "Handler `" + handlerName + "` does not exist! Exiting..";
        // Execute with arguments
        fn.apply(context, args || []);
      }
    } catch (e) {
      console.error(e);
    }
  }
  // Delete the trigger, it only needs to be executed once
  ScriptApp.getProjectTriggers().forEach(function(t) {
    if (t.getUniqueId() === triggerUid) {
      ScriptApp.deleteTrigger(t);
    }
  });
};

Hello @sdesalas, thanks for your code it's nice workaround to async functions with gas! (even with V8)
Question: CacheService parameter size limit seems to be 100kbytes, which is not enough for PDF I would like to build async way. Any clue of different way to do it? Thanks

@Zained I see four possible options here with (of course) different advantages and disadvantages:

1. Use the PropertiesService instead of the CacheService. However, this is still subject to certain quota limits and most likely will still not meet your requirements.
2. Connect to a database using the Jdbc Service. That one would require you to have that database set up and available. Working with the JDBC interface directly take a bit of effort. A ready-to-use abstraction I know of is for Cloud Firestore, which is, however, a NoSQL database. Maybe there are also others which are maintained more frequently.
3. Make use of Cloud Storage. There's also a free tier. However, I do not have any clue how to make this available in Apps Script.
4. Store files in Google Drive. By far the most straightforward option. You put your PDF somewhere in Drive and store its ID either in the CacheService or the PropertiesService under the trigger's unique identifier. I would go for the PropertiesService if applicable, since it is permanent. Apps Script triggers sometimes seem to have some hiccups and may not be executed properly. In such edge-case scenarios you would still be able to find your documents by investigating the stored properties. With the CacheService they would be gone after 6h latest and you cannot restore any relations.

Hope that helps

Hi @Zained @edmundmaruhn, a bit late to the party... my 2c:

I would go with 4 (Store Files in Google Drive). Then just pass the file ID to your async function.

var file = DriveApp.getRootFolder().createFile(blob);
Async.call('processPDF', file.getId());

function processPDF(fileId) {
   // some logic
}

Alternatively, if the creating the blob or saving the file is the time-consuming task you want to avoid, then pass what you need in order to generate the PDF into an async task: ie

var data = { user: 1842559, order: 6621399 };
Async.call('generateReport', data);

function generateReport(data) {
   // some logic
}

Juste want to say that this is exactly what I was trying to do all day. I wouldn't have been able to do something like this! Thank you sooo much

Very cool! Please make the trigger name from the name of the function being executed. This way you can find the logs faster. I tried to specify a different trigger name, an error occurs.

Async.apply = function(handlerName, args) {
var trigger = ScriptApp
.newTrigger('Async_handler') // i tried 'String(args[0])'+'_Async_handler'

Thank you so much for this.

One more issue, my Function after that async update have a ping about 1-2 min. How can i minimize this time?

Hi @steevab:

See the comments:

* Note that execution normally takes 30-60s due to scheduling of the trigger.

There is just no way around that.

Hi @sdesalas,
First of all thanks, what you provide is really great, it's been a while since I try to find this kind of trick for scripts that are very long to run that I want to split into smaller batches and have them run asynchronously.
I just tried to apply it to one of these scipts and unfortunatly I don't get the expected result.
I was wondering if there were known limitations to have it work?

My script is quite simple :

function createBatchScript(num = 50){
  var total = getTotalNumber(); // Get the number of rows in a specific sheet with the data to treat
  var treated = getNumberTreated(); // Get the number of rows in another sheet with the result of the function
  var iterations = ((total - treated)/num).toFixed(0) + 1;
  for (var i = 0; i < iterations; i++){
    var start = treated + i * num + 1;
    Async.call(identifyRedirections,[start, num]); // Program the Trigger of the function for each batch
  }
}

I see the triggers properly created, I see the Async_handler functions running in the execution pannel, but the result of my function is not added to the result sheet as expected.

Do you have an idea what could happen or what I could do to debug?

Thanks in advance for your help

Hi @jberdah,

Triggers are expensive things for GAS to keep track off so I would avoid putting Async.call(blah) inside a loop.

A better way to acomplish this would be to put the for..loop inside the asynchronus function you are calling from within your createBatchScript function, so you dont create a zillion triggers inside a loop but just a single one.

Regards,

Is there a way to pass complex objects as arguments to Async.call ? My script works fine as long as I pass strings as parameters, but if I pass an object of type Spreadsheet, for example, the handler function sees an empty object instead of the Spreadsheet...

Hi @Tamarindo94

Only primitives will work as callback arguments, however its relatively straightforward to pass the ID and then recreate the Spreadsheet object inside the handler method.

Exception: This script has too many triggers. Triggers must be deleted from the script before more can be added.

I just copied your code
Did the error message appear because the google script was updated? Or am I doing something wrong?
Thanks

Hi @designer-1407

The most likely explanation for your script having too many triggers is because you are calling the Async function inside a loop.

See comment above

Triggers are expensive things for GAS to keep track off so I would avoid putting Async.call(blah) inside a loop.

A better way to acomplish this would be to put the for..loop inside the asynchronus function you are calling from within your createBatchScript function, so you dont create a zillion triggers inside a loop but just a single one.

You should be able to go into the list of Triggers for your script (side menu) and delete them one by one.

u should be able to go into the list of Triggers for your script (side menu) and del

Thank you so much for your reply!
I was looking for a solution to speed up data processing in my table by running several algorithms at once.
The solution was 2 things:

1. Run Google apps script in 2 browser tabs and run code in them from one file (it works even with 3 or more browser tabs) Such multi-threading =)
2. I switched to python, though. Everything is much faster there, and there is no limit on the algorithm in 6 minutes.

I tried implementing this code. If I call Async.apply directly while testing, it works fine. If placed then call into MyOnOpen.gs which is itself invoked by a trigger. This does not seem to work and I get the error message: 'Script function not found: Async_handler'
Can anybody provide me direction on how to fix this?

@sdesalas Hi! I tried to apply your async solution, but on first iteration an error pops up when calling the object's method. What do you think is the problem?

Error: [TypeError: folderSecond.createFile is not a function]

Code:
function uploadFilesFromDrive(url,filename,folderSecond){
const fileID = getIdFrom(url);
const getFile = DriveApp.getFileById(fileID);
const blob = getFile.getBlob().setName(filename).getAs('application/pdf');
const file = folderSecond.createFile(blob);
};

@ShadrinSpock see comment above

Only primitives will work as callback arguments, however its relatively straightforward to pass the ID and then recreate the Spreadsheet object inside the handler method.

https://gist.github.com/sdesalas/2972f8647897d5481fd8e01f03122805?permalink_comment_id=4428924#gistcomment-4428924

In your case you probably want the ID of the folder. Instead of the folder object.

@sdesalas Thank you so much for writing this! I've only just begun my journey as a developer and learned how to write asynchronous functions in Javascript today, only to find that it doesn't work in Google Apps Script. Your library worked like a charm for my purposes though. I managed to get the expected result on the first try, which is honestly incredibly surprising knowing me.

It seems so weird that you can write async function blah(), or return promises and write .then(), but Apps Script still runs it synchronously. Do you know why that is? What is the point of having the syntax if it doesn't work the way it was intended?

@mojoro https://stackoverflow.com/questions/31241396/is-google-apps-script-synchronous