Skip to content

Instantly share code, notes, and snippets.

@sasezaki
Created July 13, 2013 17:16
Show Gist options
  • Select an option

  • Save sasezaki/5991396 to your computer and use it in GitHub Desktop.

Select an option

Save sasezaki/5991396 to your computer and use it in GitHub Desktop.
temoto/robotstxt-go がURLエンコードを考慮してないように見える
package main
import (
"github.com/temoto/robotstxt-go"
"fmt"
)
func main() {
robots, err := robotstxt.FromBytes([]byte(`
User-agent: Googlebot
Disallow: /
Allow: /~joe/index.html
`))
if err != nil {
fmt.Println("Error")
}
allow := robots.TestAgent("/%7Ejoe/index.html", "Googlebot")
fmt.Println(allow)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment