Skip to content

Instantly share code, notes, and snippets.

@DIVISHAD
Created August 19, 2024 07:07
Show Gist options
  • Save DIVISHAD/178c306807aa75ee587e3af5c41f626d to your computer and use it in GitHub Desktop.
Save DIVISHAD/178c306807aa75ee587e3af5c41f626d to your computer and use it in GitHub Desktop.
To export tickertape screener results
const scrapeData = () => {
const sections = document.querySelectorAll("#screener-table > table");
const headingElements = sections[0].querySelectorAll("th");
const scrapedData = Array.from(headingElements).map(elm => {
const selector = elm.getAttribute("id") === "name" ? "data-col" : elm.getAttribute("id") + "-col";
const rowElements = sections[0].querySelectorAll('tbody')[0].querySelectorAll("." + selector + " .ellipsis .desktop--only")
return {
column: elm.querySelector(".data-cell .desktop--only") ? elm.querySelector(".data-cell .desktop--only").textContent : "#",
rows: Array.from(rowElements).map(el => el.textContent.replaceAll(",",""))
}
});
return scrapedData;
}
const generateCSVData = scrapedData => {
scrapedData = scrapedData.slice(1, scrapedData.length - 1);
const count = scrapedData[0].rows.length;
// generate structured data
const csvData = Array.from({
length: count
}, (_, rowIndex) => {
var obj = {};
scrapedData.forEach(data => {
obj[data.column] = `"${data.rows[rowIndex]}"`
})
return obj;
});
const headers = Object.keys(csvData[0]).toString();
// Get and stringify the keys of the first object in the array
const main = csvData.map(item => Object.values(item).toString());
// Map finally returns array of arrays of values in each object
const csv = [headers, ...main].join('\n');
// Creates new array, where first row is keys and further rows the values in each object
return csv;
}
const downloadFile = csvData => {
const anchor = document.createElement('a');
anchor.href = 'data:text/csv;charset=utf-8,' + encodeURI(csvData);
anchor.target = '_blank';
anchor.download = `${document.title}`;
anchor.click();
}
downloadFile(generateCSVData(scrapeData()));
@rajvermacas
Copy link

rajvermacas commented Oct 20, 2024

Error processing header element 5: SyntaxError: Failed to execute 'querySelectorAll' on 'Element': 'tbody .26wpct-col .ellipsis .desktop--only' is not a valid selector.
at Array.map ()
at scrapeData (:20:53)
at :89:18

RCA:
A CSS class selector cannot start with a number
In this case, .26wpct-col starts with number "26"
This violates CSS naming conventions and causes a syntax error

Fix:
Please use:
tbody [class~="${selector}"] .ellipsis .desktop--only
instead of
"." + selector + " .ellipsis .desktop--only"

at line 8 i.e.
https://gist.github.com/DIVISHAD/178c306807aa75ee587e3af5c41f626d#file-tickertape-report-exporter-latest-js-L8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment