Skip to content

Instantly share code, notes, and snippets.

View rudSarkar's full-sized avatar
:electron:
Negatively Charged

Rudra Sarkar rudSarkar

:electron:
Negatively Charged
156 followers · 98 following
View GitHub Profile
@calimaborges
calimaborges / readme.md
Last active July 3, 2019 12:42
Laravel + Heroku
@1N3
1N3 / gist:de48ab54edd831cb12fb
Created July 2, 2015 09:31
Open Redirect Fuzz List
http://google.com
//google.com
\\google.com
\/google.com
\/\/google.com
/\google.com
/\/\google.com
|/google.com
/%09/google.com
/google.com
@dnozay
dnozay / README.md
Created November 23, 2014 06:01
simple distributed web crawler using flask + scrapy + redis

design

Requests are handled by flask, a bunch of urls are inserted in the object store (redis) and arguments are put on the queue (redis again) for workers to consume. More workers would mean more items processed in parallel.

Other possible implementations:

  • multiprocessing module for consuming all cpus.
  • multiprocessing.managers.SyncManager for distributing task to other machines.