Skip to content

Instantly share code, notes, and snippets.

@jimbol
Created June 7, 2018 02:45
Show Gist options
  • Save jimbol/1d912d809ce874d8a5265f069d09cd1f to your computer and use it in GitHub Desktop.
Save jimbol/1d912d809ce874d8a5265f069d09cd1f to your computer and use it in GitHub Desktop.
Scraper concept
Searcher({
baseUrl: String,
queries: Array<String>,
executeQuery: Function(Driver): Driver,
pages: 3,
goToNextPage: Function(Driver): NextPageDriver
scraper: Scraper,
})
Scraper({
getElements: Function(driver: Driver, createRow: Function): Array<Element>,
schema: {
url: Function(Element): string
title: Function(Element): string
}
})
getElements('//div[@class="webiste"]') = []
const BringMapsScraper = Scraper.extend(
getElements: function(driver, createRow) {
// rows = getRows
const output = rows.forEach((row) => {
row.click();
const el = driver.get('overlay-container');
return this.createRow(element;)
});
}
)
const scraper = new BringMapsScraper(Driver);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment