cpoDesign · July 6, 2016 12:26
diff --git a/blogpostToRead.txt b/blogpostToRead.txt
 Statistical analysis of page load speed data collected using the Navigation Timing API shows that an HTTP request can be reasonably approximated to 0.5 seconds.

 Permalink to this heading.
 Introduction
 Whilst newer browsers implement a timing API that can give the time for a request including network latency, this is not available in all browsers. In browsers that do not support the timing API it is only possible, using on page JavaScript, to measure the time taken from the start of the DOM loading to a subsequent event - such as DOM ready. Using information from browsers that support the timing API it is possible to measure both the time from request to DOM ready and the time from DOM start to DOM ready and establish a relationship such that request_time = multiplier x dom_time + constant. Once this relationship is known it can be applied when only the DOM loading to ready time is available to estimate the full request time.

 Permalink to this heading.
 Method
 From visitors that were using a browser that had support for the Navigation Timing API a random 1% sample was taken over the course of a week on a site with a very high level of traffic. For the sampled visitors two measurements were sent as query string parameters on a tracking pixel located on a dedicated subdomain. The code to insert the tracking pixels was similar to the following:

 window.onload = function() {
  var body,
      img,
      t_dom,
      t_request,
      tracking_pixel = 'http://subdomain.example.com/tracking.gif';
  if (typeof window.performance !== "undefined" && typeof window.performance.timing !== "undefined") {
    if (!Math.floor(Math.random()*100)) {
      t_dom = window.performance.timing.domInteractive - window.performance.timing.domLoading;
      t_request = window.performance.timing.domInteractive - window.performance.timing.requestStart;
      body = document.getElementsByTagName('body')[0];
      img = document.createElement("img");
      img.src = tracking_pixel + '?dom=' + t_dom + '&request=' + t_request;
      img.alt = '';
      img.height = 1;
      img.width = 1;
      body.insertBefore(img, body.lastChild);
    }
  }
 };
 The tracking pixel returned a 204 (No content) status code.

 The server logs for the test period were filtered for references to the tracking pixel, concatenated and passed through a Python data processing script. This performs a liner regression using least squares to find the best fit to the equation: request_time = multiplier x dom_time + constant.

 In order to prevent outlier results adversely affecting the result a cut off was placed on both the request time and the DOM time: if either is greater than six seconds then both are discarded.

 In order to validate that the performance timing API provides equivalent data to manually setting in-page timers a page similar to the following was set up:

 <!DOCTYPE html>
 <html>
  <head>
    <script>var t_head = new Date();</script> <!-- Must be the very first item in the head. -->
  </head>
  <body>
    ...
    <script>
      if (typeof window.performance !== "undefined" && typeof window.performance.timing !== "undefined" && typeof console.log !== "undefined" && typeof document.addEventListener !== "undefined") {
        document.addEventListener("DOMContentLoaded", function () {
          var jitter,
              t_dom = new Date();
          jitter = (window.performance.timing.domInteractive - window.performance.timing.domLoading) - (t_dom - t_head);
          console.log(jitter);
        });
      }
    </script>
  </body>
 </html>
 Permalink to this heading.
 Results
 The following graph is the output from the data processing script:

 Request time vs DOM time

 Based on 1641 data points, the average request time for a page was 1.475 seconds.

 The discrepancy between the in-page timers and the performance timing API was found to be consistently less than 50ms which is an error of ~0.05% on the average measured DOM load time and so could be effectively ignored. It also shows that using in-page timers is a valid substitute for the the timing API.

 The measured values for the formula request_time = multiplier x dom_time + constant give a multiplier of 0.997 and a constant of 0.442 seconds.

 Permalink to this heading.
 Conclusion
 It is important to note that, at the time of conducting the experiment, no common mobile browsers supported the timing API so it is reasonable to suppose that the majority of this data was gathered on fast connections where we would expect the overhead of a request to be minimal.

 The browser breakdown is as follows:

 Internet Exporer 9
 46%
 Chrome
 29%
 Firefox
 21%
 Based on this research is it reasonable to use the approximation of adding 0.5 seconds to the time measured to load the DOM using in-page timers to estimate the time taken for the full request.
	Statistical analysis of page load speed data collected using the Navigation Timing API shows that an HTTP request can be reasonably approximated to 0.5 seconds.

	Permalink to this heading.
	Introduction
	Whilst newer browsers implement a timing API that can give the time for a request including network latency, this is not available in all browsers. In browsers that do not support the timing API it is only possible, using on page JavaScript, to measure the time taken from the start of the DOM loading to a subsequent event - such as DOM ready. Using information from browsers that support the timing API it is possible to measure both the time from request to DOM ready and the time from DOM start to DOM ready and establish a relationship such that request_time = multiplier x dom_time + constant. Once this relationship is known it can be applied when only the DOM loading to ready time is available to estimate the full request time.

	Permalink to this heading.
	Method
	From visitors that were using a browser that had support for the Navigation Timing API a random 1% sample was taken over the course of a week on a site with a very high level of traffic. For the sampled visitors two measurements were sent as query string parameters on a tracking pixel located on a dedicated subdomain. The code to insert the tracking pixels was similar to the following:

	window.onload = function() {
	var body,
	img,
	t_dom,
	t_request,
	tracking_pixel = 'http://subdomain.example.com/tracking.gif';
	if (typeof window.performance !== "undefined" && typeof window.performance.timing !== "undefined") {
	if (!Math.floor(Math.random()*100)) {
	t_dom = window.performance.timing.domInteractive - window.performance.timing.domLoading;
	t_request = window.performance.timing.domInteractive - window.performance.timing.requestStart;
	body = document.getElementsByTagName('body')[0];
	img = document.createElement("img");
	img.src = tracking_pixel + '?dom=' + t_dom + '&request=' + t_request;
	img.alt = '';
	img.height = 1;
	img.width = 1;
	body.insertBefore(img, body.lastChild);
	}
	}
	};
	The tracking pixel returned a 204 (No content) status code.

	The server logs for the test period were filtered for references to the tracking pixel, concatenated and passed through a Python data processing script. This performs a liner regression using least squares to find the best fit to the equation: request_time = multiplier x dom_time + constant.

	In order to prevent outlier results adversely affecting the result a cut off was placed on both the request time and the DOM time: if either is greater than six seconds then both are discarded.

	In order to validate that the performance timing API provides equivalent data to manually setting in-page timers a page similar to the following was set up:

	<!DOCTYPE html>
	<html>
	<head>
	<script>var t_head = new Date();</script> <!-- Must be the very first item in the head. -->
	</head>
	<body>
	...
	<script>
	if (typeof window.performance !== "undefined" && typeof window.performance.timing !== "undefined" && typeof console.log !== "undefined" && typeof document.addEventListener !== "undefined") {
	document.addEventListener("DOMContentLoaded", function () {
	var jitter,
	t_dom = new Date();
	jitter = (window.performance.timing.domInteractive - window.performance.timing.domLoading) - (t_dom - t_head);
	console.log(jitter);
	});
	}
	</script>
	</body>
	</html>
	Permalink to this heading.
	Results
	The following graph is the output from the data processing script:

	Request time vs DOM time

	Based on 1641 data points, the average request time for a page was 1.475 seconds.

	The discrepancy between the in-page timers and the performance timing API was found to be consistently less than 50ms which is an error of ~0.05% on the average measured DOM load time and so could be effectively ignored. It also shows that using in-page timers is a valid substitute for the the timing API.

	The measured values for the formula request_time = multiplier x dom_time + constant give a multiplier of 0.997 and a constant of 0.442 seconds.

	Permalink to this heading.
	Conclusion
	It is important to note that, at the time of conducting the experiment, no common mobile browsers supported the timing API so it is reasonable to suppose that the majority of this data was gathered on fast connections where we would expect the overhead of a request to be minimal.

	The browser breakdown is as follows:

	Internet Exporer 9
	46%
	Chrome
	29%
	Firefox
	21%
	Based on this research is it reasonable to use the approximation of adding 0.5 seconds to the time measured to load the DOM using in-page timers to estimate the time taken for the full request.