Skip to content

Instantly share code, notes, and snippets.

@Exlord
Created June 25, 2015 04:31
Show Gist options
  • Save Exlord/b806aa26941358aeebb8 to your computer and use it in GitHub Desktop.
Save Exlord/b806aa26941358aeebb8 to your computer and use it in GitHub Desktop.
C# WebClient or WebRequest retrives unreadable data where browser recives correct html
//if you are trying to read a webpage using webclient or webrequest and getting very diffrent data than the browser
//the chances are that the content is compressed
//but if you are reciving correct html but diffrent than the browser then you should set a correct UserAgent
System.Net.HttpWebRequest r = (System.Net.HttpWebRequest)System.Net.WebRequest.Create("http://kat.cr/");
r.AutomaticDecompression = System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate;
r.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0";
System.Net.HttpWebResponse resp = (System.Net.HttpWebResponse)r.GetResponse();
System.IO.Stream respS = resp.GetResponseStream();
System.IO.StreamReader sr = new System.IO.StreamReader(respS, System.Text.Encoding.GetEncoding(1252));
string htmlcode = sr.ReadToEnd();
sr.Close();
resp.Close();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment