Filter Performance

6-months-with-attribute-filter.md

6 Months of historical Well License's with an Attribute filter

tl;dr - Improvement from 16s down to < 1s

With 6 months of well license data: start_date: 2014-10-18, end_date: 2015-03-05 and a single filter by operator name: Standard Exploration Ltd.

Request:

bcurl 'http://localhost:3000/api/v1/maps/asset_list' \
  -H 'Accept-Encoding: gzip, deflate' \
  -H 'X-CSRF-Token: ZvnjmboKTk/1Q00175Bh3aUaU6Gn04GBrWCZXSWT3cM=' \
  -H 'Accept-Language: en-US,en;q=0.8,pt;q=0.6' \
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/javascript, */*; q=0.01' \
  -H 'X-Requested-With: XMLHttpRequest' \
  -H 'Connection: keep-alive' \
  -H 'DNT: 1' \
  --compressed --concurrency 2 --requests 30 \
  --data '{"start_date":"2014-10-18","end_date":"2015-03-05","filters":{"story_types":["well_licenses"],"filter_set_attributes":{"filters_attributes":[{"entity":"wells", "filter_field":"operator_name", "filter_values":["Standard Exploration Ltd."]}]}}}'

# Without Fix: 

Summary:
  Total:	265.3317 secs.
  Slowest:	18.8894 secs.
  Fastest:	16.5859 secs.
  Average:	17.6509 secs.
  Requests/sec:	0.1131

Status code distribution:
  [200]	30 responses

Response time histogram:
  16.586 [1]	|∎∎∎∎∎∎
  16.816 [2]	|∎∎∎∎∎∎∎∎∎∎∎∎∎
  17.047 [5]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  17.277 [2]	|∎∎∎∎∎∎∎∎∎∎∎∎∎
  17.507 [3]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  17.738 [5]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  17.968 [2]	|∎∎∎∎∎∎∎∎∎∎∎∎∎
  18.198 [3]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  18.429 [1]	|∎∎∎∎∎∎
  18.659 [0]	|
  18.889 [6]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎

Latency distribution:
  10% in 16.8319 secs.
  25% in 17.1345 secs.
  50% in 17.5281 secs.
  75% in 18.2657 secs.
  90% in 18.7712 secs.
  95% in 18.8894 secs.

# With Fix: 

Summary:
  Total:  5.0707 secs.
  Slowest:  0.5725 secs.
  Fastest:  0.2393 secs.
  Average:  0.3341 secs.
  Requests/sec: 5.9163

Status code distribution:
  [200] 30 responses

Response time histogram:
  0.239 [1] |∎∎∎
  0.273 [11]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.306 [2] |∎∎∎∎∎∎∎
  0.339 [5] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.373 [2] |∎∎∎∎∎∎∎
  0.406 [2] |∎∎∎∎∎∎∎
  0.439 [0] |
  0.473 [5] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.506 [0] |
  0.539 [1] |∎∎∎
  0.572 [1] |∎∎∎

Latency distribution:
  10% in 0.2470 secs.
  25% in 0.2531 secs.
  50% in 0.3099 secs.
  75% in 0.4465 secs.
  90% in 0.4580 secs.
  95% in 0.5725 secs.

6-months-with-geo-filter.md

6 Months of historical Well License's with a geo filter

tl;dr - No noticable change in performance

Request:

  bcurl 'http://localhost:3000/api/v1/maps/asset_list' \
  -H 'Accept-Encoding: gzip, deflate' \
  -H 'X-CSRF-Token: ZvnjmboKTk/1Q00175Bh3aUaU6Gn04GBrWCZXSWT3cM=' \
  -H 'Accept-Language: en-US,en;q=0.8,pt;q=0.6' \
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/javascript, */*; q=0.01' \
  -H 'X-Requested-With: XMLHttpRequest' \
  -H 'Connection: keep-alive' \
  -H 'DNT: 1' \
  --compressed --concurrency 2 --requests 30 \
  --data '{"start_date":"2014-10-18","end_date":"2015-03-05","filters":{"story_types":["well_licenses"],"filter_set_attributes":{"filters_attributes":[{"entity":"wells","filter_field":"geospatial","filter_values":[[["51.67255514839676","-102.3486328125"],["48.9513664709477","-113.90625"]]]}]}}}'

# Without Fix:

Summary:
  Total:	96.1675 secs.
  Slowest:	7.4564 secs.
  Fastest:	6.0458 secs.
  Average:	6.4063 secs.
  Requests/sec:	0.3120

Status code distribution:
  [200]	30 responses

Response time histogram:
  6.046 [1]	|∎∎
  6.187 [1]	|∎∎
  6.328 [17]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  6.469 [5]	|∎∎∎∎∎∎∎∎∎∎∎
  6.610 [0]	|
  6.751 [0]	|
  6.892 [3]	|∎∎∎∎∎∎∎
  7.033 [1]	|∎∎
  7.174 [0]	|
  7.315 [0]	|
  7.456 [2]	|∎∎∎∎

Latency distribution:
  10% in 6.2075 secs.
  25% in 6.2360 secs.
  50% in 6.2598 secs.
  75% in 6.3824 secs.
  90% in 6.9804 secs.
  95% in 7.4564 secs.

# With Fix:

Summary:
  Total:	100.4661 secs.
  Slowest:	8.7832 secs.
  Fastest:	6.1321 secs.
  Average:	6.6850 secs.
  Requests/sec:	0.2986

Status code distribution:
  [200]	30 responses

Response time histogram:
  6.132 [1]	|∎∎
  6.397 [14]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  6.662 [3]	|∎∎∎∎∎∎∎∎
  6.927 [6]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  7.193 [0]	|
  7.458 [4]	|∎∎∎∎∎∎∎∎∎∎∎
  7.723 [0]	|
  7.988 [0]	|
  8.253 [0]	|
  8.518 [0]	|
  8.783 [2]	|∎∎∎∎∎

Latency distribution:
  10% in 6.1602 secs.
  25% in 6.2935 secs.
  50% in 6.4194 secs.
  75% in 6.8624 secs.
  90% in 7.3240 secs.
  95% in 8.7832 secs.

6-months-without-filter.md

6 Months of historical Well License's without a filter

tl;dr - Loss of performance from 31s to 33s

With 6 months of well license data: start_date: 2014-10-18, end_date: 2015-03-05 and a single filter by operator name: Standard Exploration Ltd.

Request:

bcurl 'http://localhost:3000/api/v1/maps/asset_list' \
  -H 'Accept-Encoding: gzip, deflate' \
  -H 'X-CSRF-Token: ZvnjmboKTk/1Q00175Bh3aUaU6Gn04GBrWCZXSWT3cM=' \
  -H 'Accept-Language: en-US,en;q=0.8,pt;q=0.6' \
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/javascript, */*; q=0.01' \
  -H 'X-Requested-With: XMLHttpRequest' \
  -H 'Connection: keep-alive' \
  -H 'DNT: 1' \
  --compressed --concurrency 2 --requests 30 \
  --data '{"start_date":"2014-10-18","end_date":"2015-03-05","filters":{"story_types":["well_licenses"],"filter_set_attributes":{"filters_attributes":[]}}}'

# Without Fix: 

Summary:
  Total:	479.6014 secs.
  Slowest:	43.6604 secs.
  Fastest:	30.3769 secs.
  Average:	31.9655 secs.
  Requests/sec:	0.0626

Status code distribution:
  [200]	30 responses

Response time histogram:
  30.377 [1]	|∎
  31.705 [21]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  33.034 [4]	|∎∎∎∎∎∎∎
  34.362 [2]	|∎∎∎
  35.690 [0]	|
  37.019 [0]	|
  38.347 [0]	|
  39.675 [0]	|
  41.004 [0]	|
  42.332 [0]	|
  43.660 [2]	|∎∎∎

Latency distribution:
  10% in 30.4849 secs.
  25% in 30.5922 secs.
  50% in 30.8104 secs.
  75% in 32.4529 secs.
  90% in 33.6177 secs.
  95% in 43.6604 secs.

# With Fix: 

Summary:
  Total:	499.0733 secs.
  Slowest:	34.4540 secs.
  Fastest:	32.4062 secs.
  Average:	33.2305 secs.
  Requests/sec:	0.0601

Status code distribution:
  [200]	30 responses

Response time histogram:
  32.406 [1]	|∎∎∎∎∎∎
  32.611 [4]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  32.816 [4]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  33.021 [6]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  33.225 [1]	|∎∎∎∎∎∎
  33.430 [3]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  33.635 [3]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  33.840 [4]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  34.044 [0]	|
  34.249 [2]	|∎∎∎∎∎∎∎∎∎∎∎∎∎
  34.454 [2]	|∎∎∎∎∎∎∎∎∎∎∎∎∎

Latency distribution:
  10% in 32.5599 secs.
  25% in 32.7530 secs.
  50% in 33.1989 secs.
  75% in 33.6942 secs.
  90% in 34.1343 secs.
  95% in 34.4540 secs.

7-days-with-attribute-filter.md

tl;dr - 50% improvement in performance with fix

Request:

bcurl 'http://localhost:3000/api/v1/maps/asset_list' \
  -H 'Accept-Encoding: gzip, deflate' \
  -H 'X-CSRF-Token: ZvnjmboKTk/1Q00175Bh3aUaU6Gn04GBrWCZXSWT3cM=' \
  -H 'Accept-Language: en-US,en;q=0.8,pt;q=0.6' \
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/javascript, */*; q=0.01' \
  -H 'X-Requested-With: XMLHttpRequest' \
  -H 'Connection: keep-alive' \
  -H 'DNT: 1' \
  --compressed --concurrency 2 --requests 30 \
  --data '{"start_date":"2015-03-11","end_date":"2015-03-18","filters":{"story_types":["well_licenses"],"filter_set_attributes":{"filters_attributes":[{"entity":"wells", "filter_field":"operator_name", "filter_values":["Penn West"]}]}}}'

# Without Fix:

Summary:
  Total:	6.3697 secs.
  Slowest:	0.7906 secs.
  Fastest:	0.3106 secs.
  Average:	0.4118 secs.
  Requests/sec:	4.7098

Status code distribution:
  [200]	30 responses

Response time histogram:
  0.311 [1]	|∎∎
  0.359 [5]	|∎∎∎∎∎∎∎∎∎∎
  0.407 [3]	|∎∎∎∎∎∎
  0.455 [20]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.503 [0]	|
  0.551 [0]	|
  0.599 [0]	|
  0.647 [0]	|
  0.695 [0]	|
  0.743 [0]	|
  0.791 [1]	|∎∎

Latency distribution:
  10% in 0.3286 secs.
  25% in 0.4061 secs.
  50% in 0.4139 secs.
  75% in 0.4223 secs.
  90% in 0.4412 secs.
  95% in 0.7906 secs.

# With Fix:

Summary:
  Total:	3.5320 secs.
  Slowest:	0.7556 secs.
  Fastest:	0.1354 secs.
  Average:	0.2355 secs.
  Requests/sec:	8.4938

Status code distribution:
  [200]	30 responses

Response time histogram:
  0.135 [1]	|∎∎
  0.197 [7]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.259 [20]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.321 [0]	|
  0.383 [0]	|
  0.445 [0]	|
  0.507 [0]	|
  0.570 [0]	|
  0.632 [0]	|
  0.694 [0]	|
  0.756 [2]	|∎∎∎∎

Latency distribution:
  10% in 0.1370 secs.
  25% in 0.2148 secs.
  50% in 0.2212 secs.
  75% in 0.2246 secs.
  90% in 0.2321 secs.
  95% in 0.7556 secs.

7-days-with-geo-filter.md

7 Days historical Well License's with Geo Filter

tl;dr - No noticable change in performance

Request:

bcurl 'http://localhost:3000/api/v1/maps/asset_list' \
  -H 'Accept-Encoding: gzip, deflate' \
  -H 'X-CSRF-Token: ZvnjmboKTk/1Q00175Bh3aUaU6Gn04GBrWCZXSWT3cM=' \
  -H 'Accept-Language: en-US,en;q=0.8,pt;q=0.6' \
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/javascript, */*; q=0.01' \
  -H 'X-Requested-With: XMLHttpRequest' \
  -H 'Connection: keep-alive' \
  -H 'DNT: 1' \
  --compressed --concurrency 2 --requests 30 \
  --data '{"start_date":"2015-03-11","end_date":"2015-03-18","filters":{"story_types":["well_licenses"],"filter_set_attributes":{"filters_attributes":[{"entity":"wells","filter_field":"geospatial","filter_values":[[["51.67255514839676","-102.3486328125"],["48.9513664709477","-113.90625"]]]}]}}}'

# Without Fix:

Summary:
  Total:	3.7613 secs.
  Slowest:	0.7433 secs.
  Fastest:	0.1444 secs.
  Average:	0.2507 secs.
  Requests/sec:	7.9759

Status code distribution:
  [200]	30 responses

Response time histogram:
  0.144 [1]	|∎
  0.204 [5]	|∎∎∎∎∎∎∎∎∎
  0.264 [22]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.324 [0]	|
  0.384 [0]	|
  0.444 [0]	|
  0.504 [0]	|
  0.564 [0]	|
  0.624 [0]	|
  0.683 [0]	|
  0.743 [2]	|∎∎∎

Latency distribution:
  10% in 0.1477 secs.
  25% in 0.2291 secs.
  50% in 0.2353 secs.
  75% in 0.2367 secs.
  90% in 0.2390 secs.
  95% in 0.7433 secs.

# With Fix:

Summary:
  Total:	3.2272 secs.
  Slowest:	0.2402 secs.
  Fastest:	0.1344 secs.
  Average:	0.2151 secs.
  Requests/sec:	9.2960

Status code distribution:
  [200]	30 responses

Response time histogram:
  0.134 [1]	|∎∎
  0.145 [5]	|∎∎∎∎∎∎∎∎∎∎∎
  0.156 [0]	|
  0.166 [0]	|
  0.177 [0]	|
  0.187 [0]	|
  0.198 [0]	|
  0.208 [0]	|
  0.219 [0]	|
  0.230 [6]	|∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.240 [18]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎

Latency distribution:
  10% in 0.1436 secs.
  25% in 0.2263 secs.
  50% in 0.2322 secs.
  75% in 0.2372 secs.
  90% in 0.2386 secs.
  95% in 0.2402 secs.

7-days-without-filter.md

7 Days historical Well License's without Filter

tl;dr - No noticable change in performance

Request:

bcurl 'http://localhost:3000/api/v1/maps/asset_list' \
  -H 'Accept-Encoding: gzip, deflate' \
  -H 'X-CSRF-Token: ZvnjmboKTk/1Q00175Bh3aUaU6Gn04GBrWCZXSWT3cM=' \
  -H 'Accept-Language: en-US,en;q=0.8,pt;q=0.6' \
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/javascript, */*; q=0.01' \
  -H 'X-Requested-With: XMLHttpRequest' \
  -H 'Connection: keep-alive' \
  -H 'DNT: 1' \
  --compressed --concurrency 2 --requests 30 \
  --data '{"start_date":"2015-03-11","end_date":"2015-03-18","filters":{"story_types":["well_licenses"],"filter_set_attributes":{"filters_attributes":[]}}}'

# Without Fix:

Summary:
  Total:	9.2692 secs.
  Slowest:	1.1322 secs.
  Fastest:	0.5193 secs.
  Average:	0.6179 secs.
  Requests/sec:	3.2365

Status code distribution:
  [200]	30 responses

Response time histogram:
  0.519 [1]	|∎∎
  0.581 [9]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.642 [18]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.703 [0]	|
  0.764 [0]	|
  0.826 [0]	|
  0.887 [0]	|
  0.948 [0]	|
  1.010 [0]	|
  1.071 [0]	|
  1.132 [2]	|∎∎∎∎

Latency distribution:
  10% in 0.5263 secs.
  25% in 0.5340 secs.
  50% in 0.6077 secs.
  75% in 0.6152 secs.
  90% in 0.6239 secs.
  95% in 1.1322 secs.

# With Fix:

Summary:
  Total:	8.4721 secs.
  Slowest:	0.6665 secs.
  Fastest:	0.5434 secs.
  Average:	0.5592 secs.
  Requests/sec:	3.5410

Status code distribution:
  [200]	30 responses

Response time histogram:
  0.543 [1]	|∎∎
  0.556 [16]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.568 [6]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.580 [6]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.593 [0]	|
  0.605 [0]	|
  0.617 [0]	|
  0.630 [0]	|
  0.642 [0]	|
  0.654 [0]	|
  0.667 [1]	|∎∎

Latency distribution:
  10% in 0.5453 secs.
  25% in 0.5467 secs.
  50% in 0.5531 secs.
  75% in 0.5683 secs.
  90% in 0.5783 secs.
  95% in 0.6665 secs.

template.md

**tl;dr - **

Request:

# Without Fix:

# With Fix: