You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
simple distributed web crawler using flask + scrapy + redis
design
Requests are handled by flask, a bunch of urls are inserted in the object store (redis)
and arguments are put on the queue (redis again) for workers to consume. More workers
would mean more items processed in parallel.
Other possible implementations:
multiprocessing module for consuming all cpus.
multiprocessing.managers.SyncManager for distributing task to other machines.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
And indeed, after a while, Authy changed something in their backend which now prevents the old desktop app from logging in. If you are already logged in, then you are in luck, and you can follow the instructions below to export your tokens.
If you are not logged in anymore, but can find a backup of the necessary files, then restore those files, and re-install Authy 2.2.3 following the instructions below, and it should work as expected.
Using error messages to decloak an S3 bucket. Uses soap, unicode, post, multipart, streaming and index listing as ways of figure it out. You do need a valid aws-key (never the secret) to properly get the error messages
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Example usage of MongoDB's official Go Driver with nested struct types
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters