-
-
Save rudimusmaximus/133ef10736888e42f0c9ba89c07be546 to your computer and use it in GitHub Desktop.
/** | |
* quickly test our function | |
*/ | |
function test(){ | |
var result = queryASpreadsheet('1sPevvtTMSd9LUptX8qdsw4VJf07nOal_1qn9JLwO4fQ', | |
'Example Data', | |
'A1:C', | |
'SELECT A,B,C WHERE B < 7'); | |
var rows = result.length;//7 | |
var columns = result[0].length;//3 | |
// first clear anything in a sheet by that name in case running twice | |
var thisWorkbook = SpreadsheetApp.getActiveSpreadsheet(); | |
var outputSheet = thisWorkbook.getSheetByName('outputSheet'); | |
if (outputSheet) { | |
outputSheet.clear(); | |
} | |
else { | |
// insert a new sheet at the beginning | |
outputSheet = thisWorkbook.insertSheet('outputSheet' , 0 ); | |
} | |
// write to the outputSheet | |
var outputSheet = outputSheet.getRange(1,1,rows, columns).setValues(result); | |
return true; | |
}//end test | |
/** | |
* This function uses url fetch to get data from a spreadsheet using a query style | |
* | |
* @param {String} sheetId | |
* @param {String} sheetName | |
* @param {String} rangeSyntax a few example range syntax arguments: "A1:B10" - A range from cell A1 through B10", "5:7" - Rows 5-7, "D:F" - Columns D-F, "A:A70" - The first 70 cells in column A, "A70:A" - Column A from row 70 to the end, "B5:5" - B5 to the end of row 5, "D3:D" - D3 to the end of column D,"C:C10" - From the beginning of column C to C10 | |
* @param {String} queryString | |
* @return {Object[][]} dataTwoD is a JavaScript 2d array; rather this is an array of rows where the rows are arrays of values. | |
*/ | |
function queryASpreadsheet(sheetId, sheetName, rangeSyntax, queryString) { | |
//DriveApp.getRootFolder() | |
var url = 'https://docs.google.com/spreadsheets/d/'+sheetId+'/gviz/tq?'+ | |
'range=' + rangeSyntax + | |
'&tqx=out:csv' + | |
'&sheet='+sheetName+ | |
'&tq=' + encodeURIComponent(queryString); | |
var params = { | |
headers: { | |
'Authorization': 'Bearer ' + ScriptApp.getOAuthToken() | |
}, | |
muteHttpExceptions: true | |
}; | |
var csvData = UrlFetchApp.fetch(url, params); | |
var dataTwoD = Utilities.parseCsv(csvData);// array of the format [[a, b, c], [d, e, f]] | |
return dataTwoD; | |
} |
example call
/**
* quickly test our function
*/
function test(){
var result = queryASpreadsheet('1sPevvtTMSd9LUptX8qdsw4VJf07nOal_1qn9JLwO4fQ',
'Example Data',
'A1:C',
'SELECT A,B,C WHERE B < 7');
var rows = result.length;//7
var columns = result[0].length;//3
// first clear anything in a sheet by that name in case running twice
var thisWorkbook = SpreadsheetApp.getActiveSpreadsheet();
var outputSheet = thisWorkbook.getSheetByName('outputSheet');
if (outputSheet) {
outputSheet.clear();
}
else {
// insert a new sheet at the beginning
outputSheet = thisWorkbook.insertSheet('outputSheet' , 0 );
}
// write to the outputSheet
var outputSheet = outputSheet.getRange(1,1,rows, columns).setValues(result);
return true;
}//end test
I have made the spreadsheet available but this will work on any sheet you have access to.
The example data looks like this:
Create an empty sheet, open the editor and paste both the function and the test.
You see this:
If you like, star this gist :)
This is my first Gist.
Please comment with any improvements or suggestions or problems you learn along the way.
Cheers,
r
Query how to
One place to find out more about structuring queries is to look at the query function in sheets
This is a deep rabbit hole, BUT my thought is you can test your intention in a sheet where it's easier to tweek and test your query until you get your expected results. THEN, use the data range from your query formula and pass in the query you created in our queryASheet function.
This is possible because i think these approaches are all using the Google Visualization API Query Language. Correct me if i'm mistaken.
Guidance on Google Visualization API Query Language
TESTING UPDATE
I simplified above a little. no functional change, just removed redundant getContentText since we already ask for csv output in the
'&tqx=out:csv' +
part of the url build.
In testing, I discovered that the Utilities.parseCsv(csvString) assumes new row on line breaks within double quotes. This means if you have a cell in your source sheet with a line break (alt+return when entering text in a cell), then the parse will break the line. This means your dataTwoD above will return extra lines.
I've seen talk about Regex workarounds on stack overflow which mention an old issue on parseCsv from 2012, but I'm not comfortable with anything I'm seeing.
stackoverflow
app script issue 1871
Does anyone know of any better work around or is there a preferred stack exchange solution for our purposes in google app script?
Also, is there a way to get the number of rows returned from the query in the fetch? At least this provides a 'control total' to know if the parse was clean based on the length of the array[0] created.
Any suggestions appreciated.
Lesson Learned
Recently discovered that when the source data is filtered, a query formula in a sheet will ignore the filter but a query via this url fetch approach will honor the filter in it's result - good and bad, but important to note.
Anyone, know of a way to specify the query to manage this?
Here's a link about building queries for url fetch, but I'm not sure where else to look now.
https://developers.google.com/apps-script/guides/services/external
Thanks to the community and to Spencer Easton.
Helpful links in researching:
implementing data source
Utilities.parseCsv
Example range syntax arguments, see query source ranges