cr0hn/agents.md

Forked from Paratron/agents.md

Last active October 28, 2020 03:17

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/cr0hn/ff254d47376f752795d3.js"></script>
Save cr0hn/ff254d47376f752795d3 to your computer and use it in GitHub Desktop.

Raw

#Social Network Crawler User Agents Users can post URLs on a lot of different platforms nowadays. Most of those platforms will send a request to that URL to generate some preview data from it.

These are a couple of user agents I quickly tested out.

##Facebook

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

##Google+

Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)

##Xing

XING-contenttabreceiver/2.0

##LinkedIn

LinkedInBot/1.0 (compatible; Mozilla/5.0; Jakarta Commons-HttpClient/3.1 +http://www.linkedin.com)

##Delicious

Java/1.7.0_45

##Twitter

Twitterbot

Pinterest

Pinterest/0.1 +http://pinterest.com/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment