Skip to content

Instantly share code, notes, and snippets.

@chilts

chilts/alexa.js

Created October 30, 2013 09:27
Show Gist options
  • Save chilts/7229605 to your computer and use it in GitHub Desktop.
Save chilts/7229605 to your computer and use it in GitHub Desktop.
Download ZIP
Getting the Alexa top 1 million sites directly from the server, unzipping it, parsing the csv and getting each line as an array.
Raw
alexa.js
var request = require('request');
var unzip = require('unzip');
var csv2 = require('csv2');
request.get('http://s3.amazonaws.com/alexa-static/top-1m.csv.zip')
.pipe(unzip.Parse())
.on('entry', function (entry) {
entry.pipe(csv2()).on('data', console.log);
})
;
@kostasmaneadis
Copy link

kostasmaneadis commented May 17, 2023 •
edited
Loading

Hey everyone, when I download http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip , the csv has ".deprecated" as file extension. This is it ? Its done ?

@skacurt
Copy link

skacurt commented May 17, 2023

@kostasmaneadis Yes, it's no more.

-----------------------------------------------------------------
Notice: This file is deprecated and is not being updated anymore.
        This file was last updated on February 1, 2023.
        This file will not be available from
        http://s3.amazonaws.com/alexa-static/top-1m.csv.zip after
        July 31, 2023.
-----------------------------------------------------------------

@ggmartins
Copy link

https://radar.cloudflare.com/domains
top 1000000 unordered 🤢

@securitybd
Copy link

I am in trouble with my new domain securelines.net to install WordPress,

@d668
Copy link

d668 commented May 31, 2024

I get access denied when accessing http://s3.amazonaws.com/alexa-static/top-1m.csv.zip

@kostasmaneadis
Copy link

kostasmaneadis commented May 31, 2024 •
edited
Loading

I get access denied when accessing http://s3.amazonaws.com/alexa-static/top-1m.csv.zip

try this one http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip

@evilpie
Copy link

evilpie commented Jul 28, 2024 •
edited
Loading

This is a link to the Cisco Umbrella popularity list. Archive.org has luckily archived the zip: https://web.archive.org/web/20230401000000*/https://s3.amazonaws.com/alexa-static/top-1m.csv.zip

@muserk1977
Copy link

You can check our website for this notebookdepo

@grayguest
Copy link

You can download from here https://www.kaggle.com/datasets/cheedcheed/top1m

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment