The following gist is an extract of the article Building a simple crawler. It allows crawling from a URL and for a given number of bounce.
from crawler import Crawler
crawler = Crawler()
crawler.crawl('http://techcrunch.com/')
| To remove a submodule you need to: | |
| Delete the relevant line from the .gitmodules file. | |
| Delete the relevant section from .git/config. | |
| Run git rm --cached path_to_submodule (no trailing slash). | |
| Commit and delete the now untracked submodule files. |
The following gist is an extract of the article Building a simple crawler. It allows crawling from a URL and for a given number of bounce.
from crawler import Crawler
crawler = Crawler()
crawler.crawl('http://techcrunch.com/')
| REGEDIT4 | |
| [HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\FontLink\SystemLink] | |
| "Arial"="wqy-microhei.ttc" | |
| "Arial Black"="wqy-microhei.ttc" | |
| "Arial CE,238"="wqy-microhei.ttc" | |
| "Arial CYR,204"="wqy-microhei.ttc" | |
| "Arial Greek,161"="wqy-microhei.ttc" | |
| "Arial TUR,162"="wqy-microhei.ttc" | |
| "Courier New"="wqy-microhei.ttc" |
| /** | |
| * Compares two software version numbers (e.g. "1.7.1" or "1.2b"). | |
| * | |
| * This function was born in http://stackoverflow.com/a/6832721. | |
| * | |
| * @param {string} v1 The first version to be compared. | |
| * @param {string} v2 The second version to be compared. | |
| * @param {object} [options] Optional flags that affect comparison behavior: | |
| * <ul> | |
| * <li> |
| # The initial version | |
| if [ ! -f .env ] | |
| then | |
| export $(cat .env | xargs) | |
| fi | |
| # My favorite from the comments. Thanks @richarddewit & others! | |
| set -a && source .env && set +a |
| /**************************\ | |
| Basic Modal Styles | |
| \**************************/ | |
| .modal { | |
| font-family: -apple-system,BlinkMacSystemFont,avenir next,avenir,helvetica neue,helvetica,ubuntu,roboto,noto,segoe ui,arial,sans-serif; | |
| } | |
| .modal__overlay { | |
| position: fixed; |
| wget -O /tmp/YaHei.Consolas.1.12.zip https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/uigroupcode/YaHei.Consolas.1.12.zip | |
| unzip /tmp/YaHei.Consolas.1.12.zip | |
| sudo mkdir -p /usr/share/fonts/consolas | |
| sudo mv YaHei.Consolas.1.12.ttf /usr/share/fonts/consolas/ | |
| sudo chmod 644 /usr/share/fonts/consolas/YaHei.Consolas.1.12.ttf | |
| cd /usr/share/fonts/consolas | |
| sudo mkfontscale && sudo mkfontdir && sudo fc-cache -fv |
| #!/bin/sh | |
| until docker-compose exec mysql mysql -h 127.0.0.1 -u $DB_USERNAME -p$DB_PASSWORD -D $DB_DATABASE --silent -e "show databases;" | |
| do | |
| echo "Waiting for database connection..." | |
| sleep 5 | |
| done |
| # | |
| # ZSH functions to start/stop OpenConnect VPN client | |
| # | |
| # In my setup the VPN username is the same as $USER | |
| # | |
| export VPN_HOST=<your VPN host> | |
| function vpn-up() { | |
| if [[ -z $VPN_HOST ]] | |
| then |