Konstantin Bryzgalin constb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var proxyDomainsRu = [ | |
| // ".whatismyip.com", | |
| ".ident.me", | |
| ".nalog.ru", | |
| ".gosuslugi.ru", | |
| ".gu-st.ru", | |
| ".gov.ru", | |
| ".tomskenergosbyt.ru", | |
| ".tomrc.ru", | |
| ".tom.ru", |
constb
/ tour_of_go_exercise_web_crawler.go
Created
November 13, 2021 11:51
A Tour of Go: Exercise: Web Crawler
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // In this exercise you'll use Go's concurrency features to parallelize a web crawler. | |
| // | |
| // Modify the Crawl function to fetch URLs in parallel without fetching the same URL twice. | |
| // | |
| // Hint: you can keep a cache of the URLs that have been fetched on a map, but maps alone are not safe for concurrent use! | |
| package main | |
| import ( | |
| "fmt" |