This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import grequests | |
| from collections import deque | |
| class RequestQueue(object): | |
| """ | |
| This is a lame imitation of a Typhoeus Hydra using GRequests. | |
| The main thing this allows is building up a queue of requests and then | |
| executing them, and potentially adding requests to the queue in a callback | |
| so that you can build requests that depend on other requests more naturally. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <?xml version="1.0" encoding="UTF-8" ?> | |
| <Data> | |
| <Series> | |
| <id>83462</id> | |
| <Actors>|Nathan Fillion|Stana Katic|Molly C. Quinn|Jon Huertas|Seamus Dever|Tamala Jones|Susan Sullivan|Ruben Santiago-Hudson|Monet Mazur|</Actors> | |
| <Airs_DayOfWeek>Monday</Airs_DayOfWeek> | |
| <Airs_Time>10:00 PM</Airs_Time> | |
| <ContentRating>TV-PG</ContentRating> | |
| <FirstAired>2009-03-09</FirstAired> | |
| <Genre>|Drama|</Genre> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from scrapy import log | |
| from scrapy.item import Item | |
| from scrapy.http import Request | |
| from scrapy.contrib.spiders import XMLFeedSpider | |
| def NextURL(): | |
| """ | |
| Generate a list of URLs to crawl. You can query a database or come up with some other means | |
| Note that if you generate URLs to crawl from a scraped URL then you're better of using a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /* ========================================================================= | |
| CurveZMQ - authentication and confidentiality for 0MQ | |
| ------------------------------------------------------------------------- | |
| Copyright (c) 1991-2013 iMatix Corporation <www.imatix.com> | |
| Copyright other contributors as noted in the AUTHORS file. | |
| This is free software; you can redistribute it and/or modify it under | |
| the terms of the GNU Lesser General Public License as published by | |
| the Free Software Foundation; either version 3 of the License, or (at |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Rewritten code from /r2/r2/lib/db/_sorts.pyx | |
| # http://amix.dk/blog/post/19588 # How Reddit ranking algorithms work | |
| # http://amix.dk/blog/post/19574 # How Hacker News ranking algorithm works | |
| # http://www.evanmiller.org/how-not-to-sort-by-average-rating.html | |
| from math import sqrt | |
| def _confidence(ups, downs): | |
| n = ups + downs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #! /usr/bin/env python | |
| # See http://preshing.com/20130115/view-your-filesystem-history-using-python | |
| import optparse | |
| import os | |
| import fnmatch | |
| import time | |
| # Parse options | |
| parser = optparse.OptionParser(usage='Usage: %prog [options] path [path2 ...]') | |
| parser.add_option('-g', action='store', type='long', dest='secs', default=10, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # coding=UTF-8 | |
| import nltk | |
| from nltk.corpus import brown | |
| # This is a fast and simple noun phrase extractor (based on NLTK) | |
| # Feel free to use it, just keep a link back to this post | |
| # http://thetokenizer.com/2013/05/09/efficient-way-to-extract-the-main-topics-of-a-sentence/ | |
| # Create by Shlomi Babluki | |
| # May, 2013 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # coding=UTF-8 | |
| from __future__ import division | |
| import re | |
| # This is a naive text summarization algorithm | |
| # Created by Shlomi Babluki | |
| # April, 2013 | |
| class SummaryTool(object): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /*jshint globalstrict:true */ | |
| /*global angular:true */ | |
| 'use strict'; | |
| angular.module('demo', [ | |
| 'demo.controllers', | |
| 'demo.directives', | |
| 'elasticjs.service' | |
| ]); |