Skip to content

Instantly share code, notes, and snippets.

@g-rohit
Created December 13, 2019 07:46
Show Gist options
  • Save g-rohit/030ce596d9d50205f47ee99157bce0ff to your computer and use it in GitHub Desktop.
Save g-rohit/030ce596d9d50205f47ee99157bce0ff to your computer and use it in GitHub Desktop.
Fetch and export tweets into a html file
const fs = require("fs");
const $ = require("cheerio");
const rp = require("request-promise");
var url = "https://twitter.com/india";
const fileName = "tweets-file-" + url.substr(20) + ".html";
const tweets = fs.createWriteStream(fileName);
// write headers
tweets.write(`Tweets:
<style>
@import url(https://cdn.jsdelivr.net/npm/[email protected]/build/web/hack.css);
body{white-space: break-spaces;font-family: hack, monospace}
</style>\n`);
rp(url)
.then(response => {
// console.log(
// $(".js-tweet-text-container", response)
// .text()
// .replace(/\s\s+/g, "\n\n")
// );
const tweetsScrapped = $(".js-tweet-text-container", response)
.text()
.replace(/\s\s+/g, "\n\n");
// write to file
tweets.write(tweetsScrapped);
console.log("tweets scrapped... Succesfully!!");
})
.catch(err => console.log(err));
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment