Skip to content

Instantly share code, notes, and snippets.

@adamsilverstein

adamsilverstein/wpp-metrics-tracking.ipynb

Last active December 11, 2024 19:38
Show Gist options
  • Save adamsilverstein/20fbb28b6db2c280089b79b9fadb2ca1 to your computer and use it in GitHub Desktop.
Save adamsilverstein/20fbb28b6db2c280089b79b9fadb2ca1 to your computer and use it in GitHub Desktop.
Download ZIP
WPP Metrics Tracking colab
Display the source blob
Display the rendered blob
Raw
wpp-metrics-tracking.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/adamsilverstein/20fbb28b6db2c280089b79b9fadb2ca1/wpp-metrics-tracking.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KcAZ2RHCg_Ze"
},
"source": [
"# WPP Tracking\n",
"\n",
"Track CWV metric pass rate and feature adoption metrics across initiatives."
]
},
{
"cell_type": "markdown",
"source": [
"# Setup"
],
"metadata": {
"id": "4G2WkwMPzxbT"
}
},
{
"cell_type": "code",
"source": [
"project_id = 'wpp-research'"
],
"metadata": {
"id": "YhCtDqb8AEcY"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "qTmLBxDxBAZL"
},
"source": [
"### Provide your credentials to the runtime"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "SeTJb51SKs_W",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "1dbe1889-3919-4c00-f7eb-dfe6f9fe85cb"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Authenticated\n"
]
}
],
"source": [
"from google.colab import auth\n",
"auth.authenticate_user()\n",
"print('Authenticated')\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "goQQ96EDKs_7"
},
"source": [
"### Declare the Cloud project ID which will be used throughout this notebook\n",
"\n"
]
},
{
"cell_type": "code",
"source": [
"from google.cloud.bigquery import magics\n",
"# Update with your own Google Cloud Platform project name\n",
"magics.context.project = project_id"
],
"metadata": {
"id": "YdTgQYtSoOoE"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### Add a helper to get the latest dataset"
],
"metadata": {
"id": "yV85Ec6A9FED"
}
},
{
"cell_type": "code",
"source": [
"from datetime import datetime, timedelta\n",
"\n",
"def get_first_of_previous_month():\n",
" today = datetime.now()\n",
" first_day_previous_month = datetime(today.year, today.month - 1, 1) if today.month > 1 else datetime(today.year - 1, 12, 1)\n",
" return first_day_previous_month.strftime('%Y_%m_%d')\n",
"\n",
"latest_dataset = get_first_of_previous_month() # eg. \"2023_06_01\" - datasets are updated monthly, indicate the latest"
],
"metadata": {
"id": "stNLljYnR355"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "UMKGkkZEPVRu"
},
"source": [
"### Enable data table display"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "LMNA-vBHPyHz"
},
"outputs": [],
"source": [
"%load_ext google.colab.data_table"
]
},
{
"cell_type": "code",
"source": [
"from google.colab import data_table\n",
"data_table.enable_dataframe_formatter()"
],
"metadata": {
"id": "JlBfb2k3JpRS"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# Helper Functions\n",
"\n",
"Define helper functions that are used repeatedly throughout the colab.\n",
"\n",
"`query_cwv_for_technology_over_dates` - Calculate CWV metrics over time for technology\n"
],
"metadata": {
"id": "7Zntzp87Q0_y"
}
},
{
"cell_type": "markdown",
"source": [
"### Helper to show CWV metrics over time for technology"
],
"metadata": {
"id": "6X4dlDq8ukqL"
}
},
{
"cell_type": "code",
"source": [
"from google.cloud import bigquery\n",
"from datetime import datetime, timedelta\n",
"\n",
"client = bigquery.Client(project=project_id)\n",
"\n",
"def query_cwv_for_technology_over_dates(technology, from_date=None, to_date=None):\n",
" \"\"\"\n",
" Queries Core Web Vitals metrics from HTTP Archive for a specific technology\n",
" over the past 12 months.\n",
"\n",
" Args:\n",
" technology (str): The technology to filter by (e.g., 'React').\n",
" from_date (str, optional): The start date in YYYY_MM_DD format.\n",
" to_date (str, optional): The end date in YYYY_MM_DD format.\n",
" Returns:\n",
" A BigQuery query result object containing CWV data.\n",
" \"\"\"\n",
" # set to_date if not passed\n",
" if not to_date:\n",
" to_date = latest_dataset\n",
"\n",
" # set from date if not passed\n",
" if not from_date:\n",
" from_date = (datetime.strptime(to_date, '%Y_%m_%d').date() - timedelta(days=365)).strftime('%Y_%m_%d')\n",
"\n",
"\n",
" query = f\"\"\"\n",
"CREATE TEMP FUNCTION IS_GOOD(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good / (good + needs_improvement + poor) >= 0.75\n",
" );\n",
"\n",
" CREATE TEMP FUNCTION IS_NON_ZERO(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good + needs_improvement + poor > 0\n",
" );\n",
" WITH sites_using_technology AS (\n",
" SELECT\n",
" page AS origin,\n",
" client AS device,\n",
" date,\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = '{technology}'\n",
" AND date BETWEEN PARSE_DATE('%Y_%m_%d', '{from_date}') AND PARSE_DATE('%Y_%m_%d', '{to_date}' )\n",
" AND is_root_page = TRUE\n",
" GROUP BY\n",
" page, device, date\n",
" ),\n",
" cwv_metrics_for_origins AS (\n",
" SELECT\n",
" cwvs.date AS date,\n",
" sites_using_technology.origin AS origin,\n",
" # Device phone or tablet as mobile, otherwise desktop.\n",
" IF(cwvs.device = 'phone' OR cwvs.device = 'tablet', 'mobile', 'desktop') AS device,\n",
" IS_NON_ZERO(fast_inp, avg_inp, slow_inp) AS any_inp,\n",
" IS_GOOD(fast_inp, avg_inp, slow_inp) AS good_inp,\n",
" IS_NON_ZERO(small_cls, medium_cls, large_cls) AS any_cls,\n",
" IS_GOOD(small_cls, medium_cls, large_cls) AS good_cls,\n",
" IS_NON_ZERO(fast_lcp, avg_lcp, slow_lcp) AS any_lcp,\n",
" IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_lcp,\n",
" (IS_GOOD(fast_inp, avg_inp, slow_inp) OR fast_inp IS NULL) AND IS_GOOD(small_cls, medium_cls, large_cls) AND IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_cwv\n",
" FROM\n",
" `chrome-ux-report.materialized.device_summary` AS cwvs\n",
" JOIN\n",
" sites_using_technology\n",
" ON (\n",
" sites_using_technology.origin = CONCAT(cwvs.origin, '/') AND\n",
" sites_using_technology.date = cwvs.date AND\n",
" sites_using_technology.device = IF(cwvs.device = 'phone' OR cwvs.device = 'tablet', 'mobile', cwvs.device)\n",
" )\n",
" WHERE\n",
" cwvs.date BETWEEN PARSE_DATE('%Y_%m_%d', '{from_date}') AND PARSE_DATE('%Y_%m_%d', '{to_date}' )\n",
" AND cwvs.device IN ('desktop', 'tablet', 'phone')\n",
"\n",
" ),\n",
"\n",
"\n",
" cwvs_by_date AS (\n",
" SELECT\n",
" date,\n",
" device,\n",
" COUNT(DISTINCT origin) AS origins,\n",
" SAFE_DIVIDE(COUNTIF(good_cwv), COUNTIF(any_lcp AND any_cls)) AS pct_eligible_origins_with_good_cwv,\n",
" SAFE_DIVIDE(COUNTIF(good_lcp), COUNTIF(any_lcp)) AS pct_eligible_origins_with_good_lcp,\n",
" SAFE_DIVIDE(COUNTIF(good_cls), COUNTIF(any_cls)) AS pct_eligible_origins_with_good_cls,\n",
" SAFE_DIVIDE(COUNTIF(good_inp), COUNTIF(any_inp)) AS pct_eligible_origins_with_good_inp\n",
" FROM\n",
" cwv_metrics_for_origins\n",
" GROUP BY\n",
" date, device\n",
" ORDER BY\n",
" date, device\n",
" )\n",
" SELECT * FROM cwvs_by_date\n",
" \"\"\"\n",
"\n",
" query_job = client.query(query)\n",
" return query_job.result() # Return the results of the query\n",
"\n",
"# Restrict the query to on the from and to dates, and to the specified from and to versions\n",
"def query_cwv_for_technology_compare_dates_by_version(technology, from_date, to_date, from_version, to_version):\n",
" \"\"\"\n",
" Queries Core Web Vitals metrics from HTTP Archive for a specific technology\n",
" over the past 12 months.\n",
"\n",
" Args:\n",
" technology (str): The technology to filter by (e.g., 'React').\n",
" from_date (str): The start date in YYYY_MM_DD format.\n",
" to_date (str): The end date in YYYY_MM_DD format.\n",
" from_version (str): The start version.\n",
" to_version (str): The end version.\n",
" Returns:\n",
" A BigQuery query result object containing CWV data.\n",
" \"\"\"\n",
" # set to_date if not passed\n",
" if not to_date:\n",
" to_date = latest_dataset\n",
"\n",
" # set from date if not passed\n",
" if not from_date:\n",
" from_date = (datetime.strptime(to_date, '%Y_%m_%d').date() - timedelta(days=365)).strftime('%Y_%m_%d')\n",
"\n",
"\n",
" query = f\"\"\"\n",
"CREATE TEMP FUNCTION IS_GOOD(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good / (good + needs_improvement + poor) >= 0.75\n",
" );\n",
"\n",
" CREATE TEMP FUNCTION IS_NON_ZERO(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good + needs_improvement + poor > 0\n",
" );\n",
" WITH sites_using_technology AS (\n",
" SELECT\n",
" page AS origin,\n",
" client AS device,\n",
" date,\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = '{technology}'\n",
" AND (\n",
" ( date = PARSE_DATE('%Y_%m_%d', '{from_date}') AND REGEXP_EXTRACT(JSON_EXTRACT( JSON_EXTRACT(payload, '$._detected_apps'),'$.{technology}'),r'(\\\\d.\\\\d+.\\\\d+).*') = '{from_version}' ) OR\n",
" ( date = PARSE_DATE('%Y_%m_%d', '{to_date}' ) AND REGEXP_EXTRACT(JSON_EXTRACT( JSON_EXTRACT(payload, '$._detected_apps'),'$.{technology}'),r'(\\\\d.\\\\d+.\\\\d+).*') = '{to_version}' )\n",
" )\n",
" AND is_root_page = TRUE\n",
" GROUP BY\n",
" page, device, date\n",
" ),\n",
" cwv_metrics_for_origins AS (\n",
" SELECT\n",
" cwvs.date AS date,\n",
" sites_using_technology.origin AS origin,\n",
" # Device phone or tablet as mobile, otherwise desktop.\n",
" IF(cwvs.device = 'phone' OR cwvs.device = 'tablet', 'mobile', cwvs.device) AS device,\n",
" IS_NON_ZERO(fast_inp, avg_inp, slow_inp) AS any_inp,\n",
" IS_GOOD(fast_inp, avg_inp, slow_inp) AS good_inp,\n",
" IS_NON_ZERO(small_cls, medium_cls, large_cls) AS any_cls,\n",
" IS_GOOD(small_cls, medium_cls, large_cls) AS good_cls,\n",
" IS_NON_ZERO(fast_lcp, avg_lcp, slow_lcp) AS any_lcp,\n",
" IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_lcp,\n",
" (IS_GOOD(fast_inp, avg_inp, slow_inp) OR fast_inp IS NULL) AND IS_GOOD(small_cls, medium_cls, large_cls) AND IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_cwv\n",
" FROM\n",
" `chrome-ux-report.materialized.device_summary` AS cwvs\n",
" JOIN\n",
" sites_using_technology\n",
" ON (\n",
" sites_using_technology.origin = CONCAT(cwvs.origin, '/') AND\n",
" sites_using_technology.date = cwvs.date AND\n",
" sites_using_technology.device = IF(cwvs.device = 'phone' OR cwvs.device = 'tablet', 'mobile', cwvs.device)\n",
" )\n",
" WHERE\n",
" cwvs.date BETWEEN PARSE_DATE('%Y_%m_%d', '{from_date}') AND PARSE_DATE('%Y_%m_%d', '{to_date}' )\n",
" AND cwvs.device IN ('desktop', 'tablet', 'phone')\n",
" # Restrict to sites that are in both dates\n",
" AND CONCAT(cwvs.origin, '/') IN (\n",
" SELECT\n",
" origin\n",
" FROM\n",
" sites_using_technology\n",
" GROUP BY\n",
" origin\n",
" HAVING\n",
" COUNT(*) = 2\n",
" )\n",
" ),\n",
"\n",
"\n",
" cwvs_by_date AS (\n",
" SELECT\n",
" date,\n",
" device,\n",
" COUNT(DISTINCT origin) AS origins,\n",
" SAFE_DIVIDE(COUNTIF(good_cwv), COUNTIF(any_lcp AND any_cls)) AS pct_eligible_origins_with_good_cwv,\n",
" SAFE_DIVIDE(COUNTIF(good_lcp), COUNTIF(any_lcp)) AS pct_eligible_origins_with_good_lcp,\n",
" SAFE_DIVIDE(COUNTIF(good_cls), COUNTIF(any_cls)) AS pct_eligible_origins_with_good_cls,\n",
" SAFE_DIVIDE(COUNTIF(good_inp), COUNTIF(any_inp)) AS pct_eligible_origins_with_good_inp\n",
" FROM\n",
" cwv_metrics_for_origins\n",
" GROUP BY\n",
" date, device\n",
" ORDER BY\n",
" date, device\n",
" )\n",
" SELECT * FROM cwvs_by_date\n",
" \"\"\"\n",
"\n",
" query_job = client.query(query)\n",
" return query_job.result() # Return the results of the query"
],
"metadata": {
"id": "zxFgf3WUQ5IA"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Feature performance analysis helpers"
],
"metadata": {
"id": "_59x7rfyWRYO"
}
},
{
"cell_type": "code",
"source": [
"#@markdown ## Get sites at `after_date` with feature active.\n",
"from google.cloud import bigquery\n",
"import pandas as pd\n",
"\n",
"# This limit is used throughout, adjust to increase sample or speed.\n",
"limit = 50000\n",
"\n",
"\n",
"# Get sites with the feature active on a specific date\n",
"def get_sites_with_feature_active(generator_tag, after_date):\n",
" client = bigquery.Client(project=project_id)\n",
" # 1. Identify Active and Non-Active Sites\n",
" active_sites_query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING, generator_tag STRING)\n",
" RETURNS STRING\n",
" LANGUAGE js\n",
" AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith(generator_tag));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
" ''';\n",
"\n",
"\n",
" SELECT\n",
" page AS origin\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = 'WordPress'\n",
" AND date = PARSE_DATE('%Y-%m-%d', '{after_date}' )\n",
" AND is_root_page = TRUE\n",
" AND client = 'mobile'\n",
" AND getFeature(payload, '{generator_tag}') != \"\"\n",
"\n",
"\n",
" LIMIT {limit}\n",
" \"\"\"\n",
"\n",
" active_sites_df = client.query(active_sites_query).to_dataframe()\n",
" return active_sites_df"
],
"metadata": {
"id": "ntnEK4tkv7Y1"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### Get sites with features active"
],
"metadata": {
"id": "bC13eOJnR__7"
}
},
{
"cell_type": "code",
"source": [
"#@markdown ## Get sites at `after_date` with feature active.\n",
"from google.cloud import bigquery\n",
"import pandas as pd\n",
"\n",
"# This limit is used throughout, adjust to increase sample or speed.\n",
"limit = 50000\n",
"\n",
"\n",
"# Get sites with the feature active on a specific date\n",
"def get_sites_with_features_active(generator_tag1, generator_tag2, after_date):\n",
" client = bigquery.Client(project=project_id)\n",
" # 1. Identify Active and Non-Active Sites\n",
" active_sites_query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING, generator_tag STRING)\n",
" RETURNS STRING\n",
" LANGUAGE js\n",
" AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith(generator_tag));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
" ''';\n",
"\n",
"\n",
" SELECT\n",
" page AS origin\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = 'WordPress'\n",
" AND date = PARSE_DATE('%Y-%m-%d', '{after_date}' )\n",
" AND is_root_page = TRUE\n",
" AND client = 'mobile'\n",
" AND getFeature(payload, '{generator_tag1}') != \"\"\n",
" AND getFeature(payload, '{generator_tag2}') != \"\"\n",
"\n",
"\n",
" LIMIT {limit}\n",
" \"\"\"\n",
"\n",
" active_sites_df = client.query(active_sites_query).to_dataframe()\n",
" return active_sites_df"
],
"metadata": {
"id": "Gh97AHI8QqPY"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ## Get sites at `after_date` with feature *not* active.\n",
"from google.cloud import bigquery\n",
"import pandas as pd\n",
"\n",
"\n",
"# Get sites with the feature active on a specific date\n",
"def get_sites_with_feature_inactive(generator_tag, after_date):\n",
" client = bigquery.Client(project=project_id)\n",
" # 1. Identify Active and Non-Active Sites\n",
" inactive_sites_query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING, generator_tag STRING)\n",
" RETURNS STRING\n",
" LANGUAGE js\n",
" AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith(generator_tag));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
" ''';\n",
"\n",
"\n",
" SELECT\n",
" page AS origin\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = 'WordPress'\n",
" AND date = PARSE_DATE('%Y-%m-%d', '{after_date}' )\n",
" AND is_root_page = TRUE\n",
" AND client = 'mobile'\n",
" AND getFeature(payload, '{generator_tag}') = \"\"\n",
"\n",
"\n",
" LIMIT {limit}\n",
" \"\"\"\n",
"\n",
" inactive_sites_df = client.query(inactive_sites_query).to_dataframe()\n",
" return inactive_sites_df"
],
"metadata": {
"id": "Pqejonkx4rU2"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ## Get sites at `after_date` with feature *not* active.\n",
"from google.cloud import bigquery\n",
"import pandas as pd\n",
"\n",
"\n",
"# Get sites with the feature active on a specific date\n",
"def get_sites_with_features_inactive(generator_tag1, generator_tag2, after_date):\n",
" client = bigquery.Client(project=project_id)\n",
" # 1. Identify Active and Non-Active Sites\n",
" inactive_sites_query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING, generator_tag STRING)\n",
" RETURNS STRING\n",
" LANGUAGE js\n",
" AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith(generator_tag));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
" ''';\n",
"\n",
"\n",
" SELECT\n",
" page AS origin\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = 'WordPress'\n",
" AND date = PARSE_DATE('%Y-%m-%d', '{after_date}' )\n",
" AND is_root_page = TRUE\n",
" AND client = 'mobile'\n",
" AND getFeature(payload, '{generator_tag1}') = \"\"\n",
" AND getFeature(payload, '{generator_tag2}') = \"\"\n",
"\n",
"\n",
" LIMIT {limit}\n",
" \"\"\"\n",
"\n",
" inactive_sites_df = client.query(inactive_sites_query).to_dataframe()\n",
" return inactive_sites_df"
],
"metadata": {
"id": "I9SXaAtfn3ga"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Helpers to compare before and after, minus baseline"
],
"metadata": {
"id": "O6jI1ChWXOJs"
}
},
{
"cell_type": "code",
"source": [
"#@markdown ## Get sites at `before_date` with feature active.\n",
"#@markdown Ideally, this should be 0.\n",
"\n",
"# Calculate how many of the sites where the feature was active at the after_date\n",
"# did not have the feature active at the before_date\n",
"from google.cloud import bigquery\n",
"import pandas as pd\n",
"\n",
"\n",
"# Get sites from a list with the feature not active on a specific date\n",
"def get_active_sites_also_active_at_before_date(generator_tag, before_date, existing_sites):\n",
" client = bigquery.Client(project=project_id)\n",
"\n",
" # Extract the origins from the existing_sites dataframe as a comma separated list\n",
" existing_sites_list = ', '.join(f\"'{origin}'\" for origin in existing_sites['origin'])\n",
"\n",
"\n",
" # 1. Identify Active and Non-Active Sites\n",
" active_sites_query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING, generator_tag STRING)\n",
" RETURNS STRING\n",
" LANGUAGE js\n",
" AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith(generator_tag));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
" ''';\n",
"\n",
"\n",
" SELECT\n",
" page AS origin\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
"\n",
" WHERE\n",
" technologies.technology = 'WordPress'\n",
" AND date = PARSE_DATE('%Y-%m-%d', '{before_date}' )\n",
" AND is_root_page = TRUE\n",
" AND client = 'mobile'\n",
" AND page IN ({existing_sites_list})\n",
" AND getFeature(payload, '{generator_tag}') != \"\"\n",
" LIMIT {limit}\n",
" \"\"\"\n",
"\n",
" active_sites_df = client.query(active_sites_query).to_dataframe()\n",
" return active_sites_df"
],
"metadata": {
"id": "u-VvZMmfSBJ8"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ## Get sites at `before_date` with feature active.\n",
"#@markdown Ideally, this should be 0.\n",
"\n",
"# Calculate how many of the sites where the feature was active at the after_date\n",
"# did not have the feature active at the before_date\n",
"from google.cloud import bigquery\n",
"import pandas as pd\n",
"\n",
"\n",
"# Get sites from a list with the feature not active on a specific date\n",
"def get_active_sites_also_active_at_before_date_features(generator_tag1, generator_tag2, before_date, existing_sites):\n",
" client = bigquery.Client(project=project_id)\n",
"\n",
" # Extract the origins from the existing_sites dataframe as a comma separated list\n",
" existing_sites_list = ', '.join(f\"'{origin}'\" for origin in existing_sites['origin'])\n",
"\n",
"\n",
" # 1. Identify Active and Non-Active Sites\n",
" active_sites_query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING, generator_tag STRING)\n",
" RETURNS STRING\n",
" LANGUAGE js\n",
" AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith(generator_tag));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
" ''';\n",
"\n",
"\n",
" SELECT\n",
" page AS origin\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
"\n",
" WHERE\n",
" technologies.technology = 'WordPress'\n",
" AND date = PARSE_DATE('%Y-%m-%d', '{before_date}' )\n",
" AND is_root_page = TRUE\n",
" AND client = 'mobile'\n",
" AND page IN ({existing_sites_list})\n",
" AND getFeature(payload, '{generator_tag1}') != \"\"\n",
" AND getFeature(payload, '{generator_tag2}') != \"\"\n",
" LIMIT {limit}\n",
" \"\"\"\n",
"\n",
" active_sites_df = client.query(active_sites_query).to_dataframe()\n",
" return active_sites_df"
],
"metadata": {
"id": "i2DT6JXgPlDO"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"id": "iqE8_d5kKddc"
}
},
{
"cell_type": "code",
"source": [
"#@markdown ## Get CWV changes for a set of sites between before and after dates\n",
"#@markdown\n",
"#@markdown Note: sites_df is a dataframe with an origin column containing the URLs.\n",
"#@markdown\n",
"#@markdown Returns:\n",
"#@markdown * pct_eligible_origins_with_good_cwv_change\n",
"#@markdown * pct_eligible_origins_with_good_lcp_change\n",
"#@markdown * pct_eligible_origins_with_good_cls_change\n",
"#@markdown * pct_eligible_origins_with_good_inp_change\n",
"#@markdown\n",
"\n",
"\n",
"\n",
"import pdb\n",
"# Get the CWV change before/after for sites\n",
"def get_cwv_change_before_after(sites_df, before_date, after_date):\n",
" client = bigquery.Client(project=project_id)\n",
" # 0. Ensure the 'temp_dataset' dataset exists, or create it\n",
" dataset_id = f\"{project_id}.temp_dataset\"\n",
" try:\n",
" client.get_dataset(dataset_id) # Check if dataset exists\n",
" except NotFound:\n",
" dataset = bigquery.Dataset(dataset_id)\n",
" dataset.location = \"US\" # Set the location (adjust if needed)\n",
" client.create_dataset(dataset)\n",
"\n",
" # 1. Create a temporary table with the list of origins\n",
" schema = [bigquery.SchemaField(\"origin\", \"STRING\")]\n",
" job_config = bigquery.LoadJobConfig(schema=schema)\n",
" job = client.load_table_from_dataframe(sites_df, f\"{project_id}.temp_dataset.origins\", job_config=job_config)\n",
" job.result() # Wait for the job to complete\n",
"\n",
" client = bigquery.Client(project=project_id)\n",
" # 2. Collect CWV Data and Calculate Baselines\n",
" cwv_query = f\"\"\"\n",
" CREATE TEMP FUNCTION IS_GOOD(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good / (good + needs_improvement + poor) >= 0.75\n",
" );\n",
"\n",
" CREATE TEMP FUNCTION IS_NON_ZERO(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good + needs_improvement + poor > 0\n",
" );\n",
"\n",
"\n",
" SELECT\n",
" date,\n",
" COUNT(DISTINCT origin) AS origins,\n",
" SAFE_DIVIDE(COUNTIF(good_cwv), COUNTIF(any_lcp AND any_cls)) AS pct_eligible_origins_with_good_cwv,\n",
" SAFE_DIVIDE(COUNTIF(good_lcp), COUNTIF(any_lcp)) AS pct_eligible_origins_with_good_lcp,\n",
" SAFE_DIVIDE(COUNTIF(good_cls), COUNTIF(any_cls)) AS pct_eligible_origins_with_good_cls,\n",
" SAFE_DIVIDE(COUNTIF(good_inp), COUNTIF(any_inp)) AS pct_eligible_origins_with_good_inp\n",
" FROM\n",
" (\n",
" SELECT\n",
" date AS date,\n",
" origin,\n",
" # Device phone or tablet as mobile, otherwise desktop.\n",
" IF(device = 'phone' OR device = 'tablet', 'mobile', device) AS device,\n",
" IS_NON_ZERO(fast_inp, avg_inp, slow_inp) AS any_inp,\n",
" IS_GOOD(fast_inp, avg_inp, slow_inp) AS good_inp,\n",
" IS_NON_ZERO(small_cls, medium_cls, large_cls) AS any_cls,\n",
" IS_GOOD(small_cls, medium_cls, large_cls) AS good_cls,\n",
" IS_NON_ZERO(fast_lcp, avg_lcp, slow_lcp) AS any_lcp,\n",
" IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_lcp,\n",
" (IS_GOOD(fast_inp, avg_inp, slow_inp) OR fast_inp IS NULL) AND IS_GOOD(small_cls, medium_cls, large_cls) AND IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_cwv\n",
" FROM\n",
" `chrome-ux-report.materialized.device_summary` AS cwvs\n",
" WHERE\n",
" (\n",
" date = PARSE_DATE('%Y-%m-%d', '{before_date}') OR\n",
" date = PARSE_DATE('%Y-%m-%d', '{after_date}' )\n",
" )\n",
" AND cwvs.device IN ('desktop', 'tablet', 'phone')\n",
" # Restrict to sites that are in both dates\n",
" AND CONCAT(cwvs.origin, '/') IN (SELECT origin FROM `{project_id}.temp_dataset.origins`)\n",
" )\n",
" GROUP BY\n",
" date\n",
" ORDER BY\n",
" date\n",
" \"\"\"\n",
"\n",
" cwv_data = client.query(cwv_query).to_dataframe()\n",
"\n",
" # select the data from the after_date (by date column)\n",
" cwv_after = cwv_data[cwv_data['date'] == pd.to_datetime(after_date)]\n",
" cwv_before = cwv_data[cwv_data['date'] == pd.to_datetime(before_date)]\n",
"\n",
" # subtract the 'pct_eligible_origins_with_good_cwv' value at cwv_before from cwv_after\n",
" pct_eligible_origins_with_good_cwv_change = cwv_after['pct_eligible_origins_with_good_cwv'].values[0] - cwv_before['pct_eligible_origins_with_good_cwv'].values[0]\n",
" pct_eligible_origins_with_good_lcp_change = cwv_after['pct_eligible_origins_with_good_lcp'].values[0] - cwv_before['pct_eligible_origins_with_good_lcp'].values[0]\n",
" pct_eligible_origins_with_good_cls_change = cwv_after['pct_eligible_origins_with_good_cls'].values[0] - cwv_before['pct_eligible_origins_with_good_cls'].values[0]\n",
" pct_eligible_origins_with_good_inp_change = cwv_after['pct_eligible_origins_with_good_inp'].values[0] - cwv_before['pct_eligible_origins_with_good_inp'].values[0]\n",
"\n",
" # 3. Clean up: Delete the temporary table (optional but good practice)\n",
" client.delete_table(f\"{project_id}.temp_dataset.origins\")\n",
"\n",
" return [\n",
" pct_eligible_origins_with_good_cwv_change,\n",
" pct_eligible_origins_with_good_lcp_change,\n",
" pct_eligible_origins_with_good_cls_change,\n",
" pct_eligible_origins_with_good_inp_change\n",
" ]"
],
"metadata": {
"id": "WVcPDbM75Sz_"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### Breakout by device"
],
"metadata": {
"id": "nmMTyZvMKRgw"
}
},
{
"cell_type": "code",
"source": [
"#@markdown ## Get CWV changes for a set of sites between before and after dates\n",
"#@markdown ### Broken out by device\n",
"#@markdown `get_cwv_change_before_after_by_device`\n",
"#@markdown\n",
"#@markdown Note: sites_df is a dataframe with an origin column containing the URLs.\n",
"#@markdown\n",
"#@markdown Returns:\n",
"#@markdown * pct_eligible_origins_with_good_cwv_change_mobile\n",
"#@markdown * pct_eligible_origins_with_good_cwv_change_mobile,\n",
"#@markdown * pct_eligible_origins_with_good_lcp_change_mobile,\n",
"#@markdown * pct_eligible_origins_with_good_cls_change_mobile,\n",
"#@markdown * pct_eligible_origins_with_good_inp_change_mobile,\n",
"#@markdown * pct_eligible_origins_with_good_cwv_change_desktop,\n",
"#@markdown * pct_eligible_origins_with_good_lcp_change_desktop,\n",
"#@markdown * pct_eligible_origins_with_good_cls_change_desktop,\n",
"#@markdown * pct_eligible_origins_with_good_inp_change_desktop\n",
"\n"
],
"metadata": {
"id": "heT91UXwKUWi"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"\n",
"import pdb\n",
"# Get the CWV change before/after for sites\n",
"def get_cwv_change_before_after_by_device(sites_df, before_date, after_date):\n",
" client = bigquery.Client(project=project_id)\n",
" # 0. Ensure the 'temp_dataset' dataset exists, or create it\n",
" dataset_id = f\"{project_id}.temp_dataset\"\n",
" try:\n",
" client.get_dataset(dataset_id) # Check if dataset exists\n",
" except NotFound:\n",
" dataset = bigquery.Dataset(dataset_id)\n",
" dataset.location = \"US\" # Set the location (adjust if needed)\n",
" client.create_dataset(dataset)\n",
"\n",
" # 1. Create a temporary table with the list of origins\n",
" schema = [bigquery.SchemaField(\"origin\", \"STRING\")]\n",
" job_config = bigquery.LoadJobConfig(schema=schema)\n",
" job = client.load_table_from_dataframe(sites_df, f\"{project_id}.temp_dataset.origins\", job_config=job_config)\n",
" job.result() # Wait for the job to complete\n",
"\n",
" client = bigquery.Client(project=project_id)\n",
" # 2. Collect CWV Data and Calculate Baselines\n",
" cwv_query = f\"\"\"\n",
" CREATE TEMP FUNCTION IS_GOOD(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good / (good + needs_improvement + poor) >= 0.75\n",
" );\n",
"\n",
" CREATE TEMP FUNCTION IS_NON_ZERO(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good + needs_improvement + poor > 0\n",
" );\n",
"\n",
"\n",
" SELECT\n",
" date,\n",
" device,\n",
" COUNT(DISTINCT origin) AS origins,\n",
" SAFE_DIVIDE(COUNTIF(good_cwv), COUNTIF(any_lcp AND any_cls)) AS pct_eligible_origins_with_good_cwv,\n",
" SAFE_DIVIDE(COUNTIF(good_lcp), COUNTIF(any_lcp)) AS pct_eligible_origins_with_good_lcp,\n",
" SAFE_DIVIDE(COUNTIF(good_cls), COUNTIF(any_cls)) AS pct_eligible_origins_with_good_cls,\n",
" SAFE_DIVIDE(COUNTIF(good_inp), COUNTIF(any_inp)) AS pct_eligible_origins_with_good_inp\n",
" FROM\n",
" (\n",
" SELECT\n",
" date AS date,\n",
" origin,\n",
" # Device phone or tablet as mobile, otherwise desktop.\n",
" IF(device = 'phone' OR device = 'tablet', 'mobile', device) AS device,\n",
" IS_NON_ZERO(fast_inp, avg_inp, slow_inp) AS any_inp,\n",
" IS_GOOD(fast_inp, avg_inp, slow_inp) AS good_inp,\n",
" IS_NON_ZERO(small_cls, medium_cls, large_cls) AS any_cls,\n",
" IS_GOOD(small_cls, medium_cls, large_cls) AS good_cls,\n",
" IS_NON_ZERO(fast_lcp, avg_lcp, slow_lcp) AS any_lcp,\n",
" IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_lcp,\n",
" (IS_GOOD(fast_inp, avg_inp, slow_inp) OR fast_inp IS NULL) AND IS_GOOD(small_cls, medium_cls, large_cls) AND IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_cwv\n",
" FROM\n",
" `chrome-ux-report.materialized.device_summary` AS cwvs\n",
" WHERE\n",
" (\n",
" date = PARSE_DATE('%Y-%m-%d', '{before_date}') OR\n",
" date = PARSE_DATE('%Y-%m-%d', '{after_date}' )\n",
" )\n",
" AND cwvs.device IN ('desktop', 'tablet', 'phone')\n",
" # Restrict to sites that are in both dates\n",
" AND CONCAT(cwvs.origin, '/') IN (SELECT origin FROM `{project_id}.temp_dataset.origins`)\n",
" )\n",
" GROUP BY\n",
" date, device\n",
" ORDER BY\n",
" date\n",
" \"\"\"\n",
"\n",
" cwv_data = client.query(cwv_query).to_dataframe()\n",
"\n",
" # select the data from the after_date (by date and device columns)\n",
" # mobile data\n",
" cwv_after_mobile = cwv_data[(cwv_data['date'] == pd.to_datetime(after_date)) & (cwv_data['device'] == 'mobile')]\n",
" cwv_before_mobile = cwv_data[(cwv_data['date'] == pd.to_datetime(before_date)) & (cwv_data['device'] == 'mobile')]\n",
"\n",
" # desktop data\n",
" cwv_after_desktop = cwv_data[(cwv_data['date'] == pd.to_datetime(after_date)) & (cwv_data['device'] == 'desktop')]\n",
" cwv_before_desktop = cwv_data[(cwv_data['date'] == pd.to_datetime(before_date)) & (cwv_data['device'] == 'desktop')]\n",
"\n",
"\n",
" # subtract the 'pct_eligible_origins_with_good_cwv' value at cwv_before from cwv_after\n",
" pct_eligible_origins_with_good_cwv_change_mobile = cwv_after_mobile['pct_eligible_origins_with_good_cwv'].values[0] - cwv_before_mobile['pct_eligible_origins_with_good_cwv'].values[0]\n",
" pct_eligible_origins_with_good_lcp_change_mobile = cwv_after_mobile['pct_eligible_origins_with_good_lcp'].values[0] - cwv_before_mobile['pct_eligible_origins_with_good_lcp'].values[0]\n",
" pct_eligible_origins_with_good_cls_change_mobile = cwv_after_mobile['pct_eligible_origins_with_good_cls'].values[0] - cwv_before_mobile['pct_eligible_origins_with_good_cls'].values[0]\n",
" pct_eligible_origins_with_good_inp_change_mobile = cwv_after_mobile['pct_eligible_origins_with_good_inp'].values[0] - cwv_before_mobile['pct_eligible_origins_with_good_inp'].values[0]\n",
"\n",
" pct_eligible_origins_with_good_cwv_change_desktop = cwv_after_desktop['pct_eligible_origins_with_good_cwv'].values[0] - cwv_before_desktop['pct_eligible_origins_with_good_cwv'].values[0]\n",
" pct_eligible_origins_with_good_lcp_change_desktop = cwv_after_desktop['pct_eligible_origins_with_good_lcp'].values[0] - cwv_before_desktop['pct_eligible_origins_with_good_lcp'].values[0]\n",
" pct_eligible_origins_with_good_cls_change_desktop = cwv_after_desktop['pct_eligible_origins_with_good_cls'].values[0] - cwv_before_desktop['pct_eligible_origins_with_good_cls'].values[0]\n",
" pct_eligible_origins_with_good_inp_change_desktop = cwv_after_desktop['pct_eligible_origins_with_good_inp'].values[0] - cwv_before_desktop['pct_eligible_origins_with_good_inp'].values[0]\n",
"\n",
" # 3. Clean up: Delete the temporary table (optional but good practice)\n",
" client.delete_table(f\"{project_id}.temp_dataset.origins\")\n",
"\n",
" return [\n",
" pct_eligible_origins_with_good_cwv_change_mobile,\n",
" pct_eligible_origins_with_good_lcp_change_mobile,\n",
" pct_eligible_origins_with_good_cls_change_mobile,\n",
" pct_eligible_origins_with_good_inp_change_mobile,\n",
" pct_eligible_origins_with_good_cwv_change_desktop,\n",
" pct_eligible_origins_with_good_lcp_change_desktop,\n",
" pct_eligible_origins_with_good_cls_change_desktop,\n",
" pct_eligible_origins_with_good_inp_change_desktop,\n",
" cwv_before_mobile,\n",
" cwv_after_mobile,\n",
" cwv_before_desktop,\n",
" cwv_after_desktop\n",
" ]"
],
"metadata": {
"id": "EkvnjJrKOH5O"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ## Calculate impact minus baseline\n",
"\n",
"# calculate the impact by subracting the baseline from the change\n",
"def calculate_feature_impact(change, baseline):\n",
" cwv_change = change[0] - baseline[0]\n",
" lcp_change = change[1] - baseline[1]\n",
" cls_change = change[2] - baseline[2]\n",
" inp_change = change[3] - baseline[3]\n",
" return {\n",
" 'pct_good_cwv_change': \"{:.2%}\".format(cwv_change),\n",
" 'pct_good_lcp_change': \"{:.2%}\".format(lcp_change),\n",
" 'pct_good_cls_change': \"{:.2%}\".format(cls_change),\n",
" 'pct_good_inp_change': \"{:.2%}\".format(inp_change)\n",
" }"
],
"metadata": {
"id": "gF8s3t5L6Vjg"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ## Calculate impact minus baseline by device\n",
"\n",
"# calculate the impact by subracting the baseline from the change\n",
"def calculate_feature_impact_by_device(change, baseline):\n",
" cwv_change_mobile = change[0] - baseline[0]\n",
" lcp_change_mobile = change[1] - baseline[1]\n",
" cls_change_mobile = change[2] - baseline[2]\n",
" inp_change_mobile = change[3] - baseline[3]\n",
" cwv_change_desktop = change[4] - baseline[4]\n",
" lcp_change_desktop = change[5] - baseline[5]\n",
" cls_change_desktop = change[6] - baseline[6]\n",
" inp_change_desktop = change[7] - baseline[7]\n",
" return {\n",
" 'pct_good_cwv_change_mobile': \"{:.2%}\".format(cwv_change_mobile),\n",
" 'pct_good_lcp_change_mobile': \"{:.2%}\".format(lcp_change_mobile),\n",
" 'pct_good_cls_change_mobile': \"{:.2%}\".format(cls_change_mobile),\n",
" 'pct_good_inp_change_mobile': \"{:.2%}\".format(inp_change_mobile),\n",
" 'pct_good_cwv_change_desktop': \"{:.2%}\".format(cwv_change_desktop),\n",
" 'pct_good_lcp_change_desktop': \"{:.2%}\".format(lcp_change_desktop),\n",
" 'pct_good_cls_change_desktop': \"{:.2%}\".format(cls_change_desktop),\n",
" 'pct_good_inp_change_desktop': \"{:.2%}\".format(inp_change_desktop)\n",
" }"
],
"metadata": {
"id": "1Dh14Oa1Pb2-"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ## Measure feature performance impact minus baseline\n",
"#@markdown\n",
"#@markdown ### Note:\n",
"#@markdown * This helper combines device data.\n",
"def query_cwv_compare_feature_to_baseline(generator_tag, before_date, after_date):\n",
" active_sites = get_sites_with_feature_active(generator_tag, after_date)\n",
" count_active_sites = active_sites.count()\n",
" inactive_sites = get_sites_with_feature_inactive(generator_tag, after_date)\n",
" count_inactive_sites = inactive_sites.count()\n",
"\n",
" # Count feature active sites that were also feature active at the before date.\n",
" active_at_from_date = get_active_sites_also_active_at_before_date(generator_tag, before_date, active_sites)\n",
" count_active_at_from_date = active_at_from_date.count()\n",
"\n",
" # Calculate the percentage of count_active_at_from_date (which might be 0 so use safe divide)\n",
" if count_active_sites['origin'] > 0:\n",
" also_active_at_before = (count_active_at_from_date['origin'] / count_active_sites['origin']) * 100\n",
" else:\n",
" also_active_at_before = 0\n",
"\n",
" # Calculate CWV change\n",
" cwv_changes_active_sites = get_cwv_change_before_after(active_sites, before_date, after_date )\n",
" cwv_changes_inactive_sites = get_cwv_change_before_after(inactive_sites, before_date, after_date )\n",
"\n",
" final_results = calculate_feature_impact(cwv_changes_active_sites, cwv_changes_inactive_sites)\n",
" final_results['feature_active_sites'] = count_active_sites['origin']\n",
" final_results['feature_inactive_sites'] = count_inactive_sites['origin']\n",
" final_results['also_active_at_before'] = \"{:.2%}\".format(also_active_at_before)\n",
"\n",
" return final_results"
],
"metadata": {
"id": "JL0yN8uu76cq"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ## Measure feature performance impact minus baseline\n",
"#@markdown ### Breakout by device\n",
"def query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date):\n",
" active_sites = get_sites_with_feature_active(generator_tag, after_date)\n",
" count_active_sites = active_sites.count()\n",
" inactive_sites = get_sites_with_feature_inactive(generator_tag, after_date)\n",
" count_inactive_sites = inactive_sites.count()\n",
"\n",
" # Count feature active sites that were also feature active at the before date.\n",
" active_at_from_date = get_active_sites_also_active_at_before_date(generator_tag, before_date, active_sites)\n",
" count_active_at_from_date = active_at_from_date.count()\n",
"\n",
" # Calculate the percentage of count_active_at_from_date (which might be 0 so use safe divide)\n",
" if count_active_sites['origin'] > 0:\n",
" also_active_at_before = (count_active_at_from_date['origin'] / count_active_sites['origin']) * 100\n",
" else:\n",
" also_active_at_before = 0\n",
"\n",
" # Calculate CWV change\n",
" cwv_changes_active_sites = get_cwv_change_before_after_by_device(active_sites, before_date, after_date )\n",
" cwv_changes_inactive_sites = get_cwv_change_before_after_by_device(inactive_sites, before_date, after_date )\n",
"\n",
" final_results = calculate_feature_impact_by_device(cwv_changes_active_sites, cwv_changes_inactive_sites)\n",
" final_results['feature_active_sites'] = count_active_sites['origin']\n",
" final_results['feature_inactive_sites'] = count_inactive_sites['origin']\n",
" final_results['also_active_at_before'] = \"{:.2%}\".format(also_active_at_before)\n",
" final_results['before_date'] = before_date\n",
" final_results['after_date'] = after_date\n",
" final_results['generator_tag'] = generator_tag\n",
" final_results['cwv_changes_active_sites'] = cwv_changes_active_sites\n",
" final_results['cwv_changes_inactive_sites'] = cwv_changes_inactive_sites\n",
"\n",
" return final_results"
],
"metadata": {
"id": "UqtzgMvZSpiE"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ## Measure feature performance impact minus baseline\n",
"#@markdown ### Breakout by device\n",
"def query_cwv_compare_feature_to_baseline_by_device_multiple_tags(generator_tag1, generator_tag2, before_date, after_date):\n",
"\n",
" active_sites = get_sites_with_features_active(generator_tag1, generator_tag2, after_date)\n",
" count_active_sites = active_sites.count()\n",
" inactive_sites = get_sites_with_features_inactive(generator_tag1, generator_tag2, after_date)\n",
" count_inactive_sites = inactive_sites.count()\n",
"\n",
" # Count feature active sites that were also feature active at the before date.\n",
" active_at_from_date = get_active_sites_also_active_at_before_date_features(generator_tag1, generator_tag2, before_date, active_sites)\n",
" count_active_at_from_date = active_at_from_date.count()\n",
"\n",
" # Calculate the percentage of count_active_at_from_date (which might be 0 so use safe divide)\n",
" if count_active_sites['origin'] > 0:\n",
" also_active_at_before = (count_active_at_from_date['origin'] / count_active_sites['origin']) * 100\n",
" else:\n",
" also_active_at_before = 0\n",
"\n",
" # Calculate CWV change\n",
" cwv_changes_active_sites = get_cwv_change_before_after_by_device(active_sites, before_date, after_date )\n",
" cwv_changes_inactive_sites = get_cwv_change_before_after_by_device(inactive_sites, before_date, after_date )\n",
"\n",
" final_results = calculate_feature_impact_by_device(cwv_changes_active_sites, cwv_changes_inactive_sites)\n",
" final_results['feature_active_sites'] = count_active_sites['origin']\n",
" final_results['feature_inactive_sites'] = count_inactive_sites['origin']\n",
" final_results['also_active_at_before'] = \"{:.2%}\".format(also_active_at_before)\n",
" final_results['before_date'] = before_date\n",
" final_results['after_date'] = after_date\n",
" final_results['generator_tag1'] = generator_tag1\n",
" final_results['generator_tag2'] = generator_tag2\n",
" final_results['cwv_changes_active_sites'] = cwv_changes_active_sites\n",
" final_results['cwv_changes_inactive_sites'] = cwv_changes_inactive_sites\n",
"\n",
" return final_results"
],
"metadata": {
"id": "zRWWFk1jQOMd"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# Helper use examples"
],
"metadata": {
"id": "tRAH6Wr0moIo"
}
},
{
"cell_type": "code",
"source": [
"elementor_data = query_cwv_for_technology_over_dates('Elementor')\n",
"elementor_data.to_dataframe()"
],
"metadata": {
"id": "whbQMo9ia6HY",
"outputId": "578f5c37-ce5d-4716-f756-552a56c27347",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 304
}
},
"execution_count": null,
"outputs": [
{
"output_type": "error",
"ename": "KeyboardInterrupt",
"evalue": "",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-31-f6231aad8ad2>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0melementor_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mquery_cwv_for_technology_over_dates\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Elementor'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0melementor_data\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_dataframe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m<ipython-input-7-5405016ee9ca>\u001b[0m in \u001b[0;36mquery_cwv_for_technology_over_dates\u001b[0;34m(technology, from_date, to_date)\u001b[0m\n\u001b[1;32m 101\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 102\u001b[0m \u001b[0mquery_job\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mclient\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mquery\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mquery\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 103\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mquery_job\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# Return the results of the query\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 104\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 105\u001b[0m \u001b[0;31m# Restrict the query to on the from and to dates, and to the specified from and to versions\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/job/query.py\u001b[0m in \u001b[0;36mresult\u001b[0;34m(self, page_size, max_results, retry, timeout, start_index, job_retry)\u001b[0m\n\u001b[1;32m 1674\u001b[0m \u001b[0;31m# Since is_job_done() calls jobs.getQueryResults, which is a\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1675\u001b[0m \u001b[0;31m# long-running API, don't delay the next request at all.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1676\u001b[0;31m \u001b[0;32mwhile\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mis_job_done\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1677\u001b[0m \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1678\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/api_core/retry/retry_unary.py\u001b[0m in \u001b[0;36mretry_wrapped_func\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 291\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_initial\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_maximum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmultiplier\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_multiplier\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 292\u001b[0m )\n\u001b[0;32m--> 293\u001b[0;31m return retry_target(\n\u001b[0m\u001b[1;32m 294\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 295\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_predicate\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/api_core/retry/retry_unary.py\u001b[0m in \u001b[0;36mretry_target\u001b[0;34m(target, predicate, sleep_generator, timeout, on_error, exception_factory, **kwargs)\u001b[0m\n\u001b[1;32m 142\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0msleep\u001b[0m \u001b[0;32min\u001b[0m \u001b[0msleep_generator\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 143\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 144\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 145\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0minspect\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0misawaitable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 146\u001b[0m \u001b[0mwarnings\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwarn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_ASYNC_RETRY_WARNING\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/job/query.py\u001b[0m in \u001b[0;36mis_job_done\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1643\u001b[0m \u001b[0;31m# jobs.getQueryResults hangs as long as it can to ensure we\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1644\u001b[0m \u001b[0;31m# know when the query has finished as soon as possible.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1645\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reload_query_results\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mretry\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mretry\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mreload_query_results_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1646\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1647\u001b[0m \u001b[0;31m# Even if the query is finished now according to\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/job/query.py\u001b[0m in \u001b[0;36m_reload_query_results\u001b[0;34m(self, retry, timeout, page_size)\u001b[0m\n\u001b[1;32m 1441\u001b[0m \u001b[0mtransport_timeout\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1442\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1443\u001b[0;31m self._query_results = self._client._get_query_results(\n\u001b[0m\u001b[1;32m 1444\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjob_id\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1445\u001b[0m \u001b[0mretry\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/client.py\u001b[0m in \u001b[0;36m_get_query_results\u001b[0;34m(self, job_id, retry, project, timeout_ms, location, timeout, page_size)\u001b[0m\n\u001b[1;32m 2022\u001b[0m \u001b[0;31m# QueryJob.result()). So we don't need to poll here.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2023\u001b[0m \u001b[0mspan_attributes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m\"path\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mpath\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2024\u001b[0;31m resource = self._call_api(\n\u001b[0m\u001b[1;32m 2025\u001b[0m \u001b[0mretry\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2026\u001b[0m \u001b[0mspan_name\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"BigQuery.getQueryResults\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/cloud/bigquery/client.py\u001b[0m in \u001b[0;36m_call_api\u001b[0;34m(self, retry, span_name, span_attributes, job_ref, headers, **kwargs)\u001b[0m\n\u001b[1;32m 831\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mspan_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mattributes\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mspan_attributes\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mclient\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mjob_ref\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mjob_ref\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 832\u001b[0m ):\n\u001b[0;32m--> 833\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mcall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 834\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 835\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mcall\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/api_core/retry/retry_unary.py\u001b[0m in \u001b[0;36mretry_wrapped_func\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 291\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_initial\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_maximum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmultiplier\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_multiplier\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 292\u001b[0m )\n\u001b[0;32m--> 293\u001b[0;31m return retry_target(\n\u001b[0m\u001b[1;32m 294\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 295\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_predicate\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/api_core/retry/retry_unary.py\u001b[0m in \u001b[0;36mretry_target\u001b[0;34m(target, predicate, sleep_generator, timeout, on_error, exception_factory, **kwargs)\u001b[0m\n\u001b[1;32m 142\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0msleep\u001b[0m \u001b[0;32min\u001b[0m \u001b[0msleep_generator\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 143\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 144\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 145\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0minspect\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0misawaitable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 146\u001b[0m \u001b[0mwarnings\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwarn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_ASYNC_RETRY_WARNING\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/cloud/_http/__init__.py\u001b[0m in \u001b[0;36mapi_request\u001b[0;34m(self, method, path, query_params, data, content_type, headers, api_base_url, api_version, expect_json, _target_object, timeout, extra_api_info)\u001b[0m\n\u001b[1;32m 480\u001b[0m \u001b[0mcontent_type\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"application/json\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 481\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 482\u001b[0;31m response = self._make_request(\n\u001b[0m\u001b[1;32m 483\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmethod\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 484\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/cloud/_http/__init__.py\u001b[0m in \u001b[0;36m_make_request\u001b[0;34m(self, method, url, data, content_type, headers, target_object, timeout, extra_api_info)\u001b[0m\n\u001b[1;32m 339\u001b[0m \u001b[0mheaders\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"User-Agent\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0muser_agent\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 340\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 341\u001b[0;31m return self._do_request(\n\u001b[0m\u001b[1;32m 342\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_object\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 343\u001b[0m )\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/cloud/_http/__init__.py\u001b[0m in \u001b[0;36m_do_request\u001b[0;34m(self, method, url, headers, data, target_object, timeout)\u001b[0m\n\u001b[1;32m 377\u001b[0m \u001b[0;34m:\u001b[0m\u001b[0mreturns\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mThe\u001b[0m \u001b[0mHTTP\u001b[0m \u001b[0mresponse\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 378\u001b[0m \"\"\"\n\u001b[0;32m--> 379\u001b[0;31m return self.http.request(\n\u001b[0m\u001b[1;32m 380\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmethod\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mheaders\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mheaders\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 381\u001b[0m )\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/auth/transport/requests.py\u001b[0m in \u001b[0;36mrequest\u001b[0;34m(self, method, url, data, headers, max_allowed_time, timeout, **kwargs)\u001b[0m\n\u001b[1;32m 539\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 540\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mTimeoutGuard\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mremaining_time\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mguard\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 541\u001b[0;31m response = super(AuthorizedSession, self).request(\n\u001b[0m\u001b[1;32m 542\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 543\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/requests/sessions.py\u001b[0m in \u001b[0;36mrequest\u001b[0;34m(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)\u001b[0m\n\u001b[1;32m 587\u001b[0m }\n\u001b[1;32m 588\u001b[0m \u001b[0msend_kwargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msettings\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 589\u001b[0;31m \u001b[0mresp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprep\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0msend_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 590\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 591\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mresp\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/requests/sessions.py\u001b[0m in \u001b[0;36msend\u001b[0;34m(self, request, **kwargs)\u001b[0m\n\u001b[1;32m 701\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 702\u001b[0m \u001b[0;31m# Send the request\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 703\u001b[0;31m \u001b[0mr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0madapter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 704\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 705\u001b[0m \u001b[0;31m# Total elapsed time of the request (approximately)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/requests/adapters.py\u001b[0m in \u001b[0;36msend\u001b[0;34m(self, request, stream, timeout, verify, cert, proxies)\u001b[0m\n\u001b[1;32m 665\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 666\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 667\u001b[0;31m resp = conn.urlopen(\n\u001b[0m\u001b[1;32m 668\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmethod\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 669\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py\u001b[0m in \u001b[0;36murlopen\u001b[0;34m(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)\u001b[0m\n\u001b[1;32m 787\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 788\u001b[0m \u001b[0;31m# Make the request on the HTTPConnection object\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 789\u001b[0;31m response = self._make_request(\n\u001b[0m\u001b[1;32m 790\u001b[0m \u001b[0mconn\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 791\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py\u001b[0m in \u001b[0;36m_make_request\u001b[0;34m(self, conn, method, url, body, headers, retries, timeout, chunked, response_conn, preload_content, decode_content, enforce_content_length)\u001b[0m\n\u001b[1;32m 534\u001b[0m \u001b[0;31m# Receive the response from the server\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 535\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 536\u001b[0;31m \u001b[0mresponse\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetresponse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 537\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mBaseSSLError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mOSError\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 538\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_raise_timeout\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merr\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0murl\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout_value\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mread_timeout\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/local/lib/python3.10/dist-packages/urllib3/connection.py\u001b[0m in \u001b[0;36mgetresponse\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 505\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 506\u001b[0m \u001b[0;31m# Get the response from http.client.HTTPConnection\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 507\u001b[0;31m \u001b[0mhttplib_response\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetresponse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 508\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 509\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/lib/python3.10/http/client.py\u001b[0m in \u001b[0;36mgetresponse\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 1373\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1374\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1375\u001b[0;31m \u001b[0mresponse\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbegin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1376\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mConnectionError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1377\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/lib/python3.10/http/client.py\u001b[0m in \u001b[0;36mbegin\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 316\u001b[0m \u001b[0;31m# read until we get a non-100 response\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 317\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 318\u001b[0;31m \u001b[0mversion\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatus\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreason\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_read_status\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 319\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mstatus\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mCONTINUE\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 320\u001b[0m \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/lib/python3.10/http/client.py\u001b[0m in \u001b[0;36m_read_status\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 277\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 278\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_read_status\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 279\u001b[0;31m \u001b[0mline\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreadline\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_MAXLINE\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"iso-8859-1\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 280\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mline\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0m_MAXLINE\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 281\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mLineTooLong\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"status line\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/lib/python3.10/socket.py\u001b[0m in \u001b[0;36mreadinto\u001b[0;34m(self, b)\u001b[0m\n\u001b[1;32m 703\u001b[0m \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 704\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 705\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_sock\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrecv_into\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 706\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 707\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_timeout_occurred\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/lib/python3.10/ssl.py\u001b[0m in \u001b[0;36mrecv_into\u001b[0;34m(self, buffer, nbytes, flags)\u001b[0m\n\u001b[1;32m 1301\u001b[0m \u001b[0;34m\"non-zero flags not allowed in calls to recv_into() on %s\"\u001b[0m \u001b[0;34m%\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1302\u001b[0m self.__class__)\n\u001b[0;32m-> 1303\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnbytes\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbuffer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1304\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1305\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrecv_into\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbuffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnbytes\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mflags\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/usr/lib/python3.10/ssl.py\u001b[0m in \u001b[0;36mread\u001b[0;34m(self, len, buffer)\u001b[0m\n\u001b[1;32m 1157\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1158\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mbuffer\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1159\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_sslobj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbuffer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1160\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1161\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_sslobj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mKeyboardInterrupt\u001b[0m: "
]
}
]
},
{
"cell_type": "code",
"source": [
"active_sites = get_sites_with_feature_active('speculation-rules', '2024-11-01')\n",
"active_sites.head(5)\n",
"\n",
"# count active sites\n",
"count_active_sites = active_sites.count()\n",
"print(count_active_sites)"
],
"metadata": {
"id": "XMUE3rbvwpIr"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"get_sites_with_features_active('embed-optimizer', 'optimization-detective', '2024-09-01')"
],
"metadata": {
"id": "tmX9uC1NQ19C"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"inactive_sites = get_sites_with_feature_inactive('speculation-rules', '2024-08-01')\n",
"inactive_sites.head(5)\n",
"\n",
"# count inactive sites\n",
"count_inactive_sites = inactive_sites.count()\n",
"print(count_inactive_sites)"
],
"metadata": {
"id": "Sw5e-p6z479N",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "099caf84-dcf3-4d8f-998f-0ba63ec38c76"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"origin 50000\n",
"dtype: int64\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"sites = get_sites_with_features_inactive( 'embed-optimizer', 'optimization-detective', '2024-09-01')\n",
"sites.head(500)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 669
},
"id": "tOZen3o4Dbw4",
"outputId": "bbebd31d-e2f4-4598-edbc-168e4d6b1e78"
},
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
" origin\n",
"0 https://mrd.cash/\n",
"1 https://www.handipet.org/\n",
"2 https://www.mimasaka-no-kuni.jp/\n",
"3 https://soundcity.com.ar/\n",
"4 https://ciclismoarg.com.ar/\n",
".. ...\n",
"495 https://ivadi.es/\n",
"496 https://meuesquematizado.com.br/\n",
"497 https://safemedicinedrop.com/\n",
"498 https://visittrollaskagi.is/\n",
"499 https://www.plazahyundai.com/\n",
"\n",
"[500 rows x 1 columns]"
],
"text/html": [
"\n",
" <div id=\"df-ee1862df-8ab7-4db4-9950-0909848343fa\" class=\"colab-df-container\">\n",
" <div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>origin</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>https://mrd.cash/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>https://www.handipet.org/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>https://www.mimasaka-no-kuni.jp/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>https://soundcity.com.ar/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>https://ciclismoarg.com.ar/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>495</th>\n",
" <td>https://ivadi.es/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>496</th>\n",
" <td>https://meuesquematizado.com.br/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>497</th>\n",
" <td>https://safemedicinedrop.com/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>498</th>\n",
" <td>https://visittrollaskagi.is/</td>\n",
" </tr>\n",
" <tr>\n",
" <th>499</th>\n",
" <td>https://www.plazahyundai.com/</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>500 rows × 1 columns</p>\n",
"</div>\n",
" <div class=\"colab-df-buttons\">\n",
"\n",
" <div class=\"colab-df-container\">\n",
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ee1862df-8ab7-4db4-9950-0909848343fa')\"\n",
" title=\"Convert this dataframe to an interactive table.\"\n",
" style=\"display:none;\">\n",
"\n",
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
" </svg>\n",
" </button>\n",
"\n",
" <style>\n",
" .colab-df-container {\n",
" display:flex;\n",
" gap: 12px;\n",
" }\n",
"\n",
" .colab-df-convert {\n",
" background-color: #E8F0FE;\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: #1967D2;\n",
" height: 32px;\n",
" padding: 0 0 0 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-convert:hover {\n",
" background-color: #E2EBFA;\n",
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: #174EA6;\n",
" }\n",
"\n",
" .colab-df-buttons div {\n",
" margin-bottom: 4px;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert {\n",
" background-color: #3B4455;\n",
" fill: #D2E3FC;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert:hover {\n",
" background-color: #434B5C;\n",
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
" fill: #FFFFFF;\n",
" }\n",
" </style>\n",
"\n",
" <script>\n",
" const buttonEl =\n",
" document.querySelector('#df-ee1862df-8ab7-4db4-9950-0909848343fa button.colab-df-convert');\n",
" buttonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
"\n",
" async function convertToInteractive(key) {\n",
" const element = document.querySelector('#df-ee1862df-8ab7-4db4-9950-0909848343fa');\n",
" const dataTable =\n",
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
" [key], {});\n",
" if (!dataTable) return;\n",
"\n",
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
" + ' to learn more about interactive tables.';\n",
" element.innerHTML = '';\n",
" dataTable['output_type'] = 'display_data';\n",
" await google.colab.output.renderOutput(dataTable, element);\n",
" const docLink = document.createElement('div');\n",
" docLink.innerHTML = docLinkHtml;\n",
" element.appendChild(docLink);\n",
" }\n",
" </script>\n",
" </div>\n",
"\n",
"\n",
"<div id=\"df-55a2c921-c32c-45d2-b681-54f053a9ac56\">\n",
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-55a2c921-c32c-45d2-b681-54f053a9ac56')\"\n",
" title=\"Suggest charts\"\n",
" style=\"display:none;\">\n",
"\n",
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
" width=\"24px\">\n",
" <g>\n",
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
" </g>\n",
"</svg>\n",
" </button>\n",
"\n",
"<style>\n",
" .colab-df-quickchart {\n",
" --bg-color: #E8F0FE;\n",
" --fill-color: #1967D2;\n",
" --hover-bg-color: #E2EBFA;\n",
" --hover-fill-color: #174EA6;\n",
" --disabled-fill-color: #AAA;\n",
" --disabled-bg-color: #DDD;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-quickchart {\n",
" --bg-color: #3B4455;\n",
" --fill-color: #D2E3FC;\n",
" --hover-bg-color: #434B5C;\n",
" --hover-fill-color: #FFFFFF;\n",
" --disabled-bg-color: #3B4455;\n",
" --disabled-fill-color: #666;\n",
" }\n",
"\n",
" .colab-df-quickchart {\n",
" background-color: var(--bg-color);\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: var(--fill-color);\n",
" height: 32px;\n",
" padding: 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-quickchart:hover {\n",
" background-color: var(--hover-bg-color);\n",
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: var(--button-hover-fill-color);\n",
" }\n",
"\n",
" .colab-df-quickchart-complete:disabled,\n",
" .colab-df-quickchart-complete:disabled:hover {\n",
" background-color: var(--disabled-bg-color);\n",
" fill: var(--disabled-fill-color);\n",
" box-shadow: none;\n",
" }\n",
"\n",
" .colab-df-spinner {\n",
" border: 2px solid var(--fill-color);\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" animation:\n",
" spin 1s steps(1) infinite;\n",
" }\n",
"\n",
" @keyframes spin {\n",
" 0% {\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" border-left-color: var(--fill-color);\n",
" }\n",
" 20% {\n",
" border-color: transparent;\n",
" border-left-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" }\n",
" 30% {\n",
" border-color: transparent;\n",
" border-left-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" border-right-color: var(--fill-color);\n",
" }\n",
" 40% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" }\n",
" 60% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" }\n",
" 80% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" border-bottom-color: var(--fill-color);\n",
" }\n",
" 90% {\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" }\n",
" }\n",
"</style>\n",
"\n",
" <script>\n",
" async function quickchart(key) {\n",
" const quickchartButtonEl =\n",
" document.querySelector('#' + key + ' button');\n",
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
" try {\n",
" const charts = await google.colab.kernel.invokeFunction(\n",
" 'suggestCharts', [key], {});\n",
" } catch (error) {\n",
" console.error('Error during call to suggestCharts:', error);\n",
" }\n",
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
" }\n",
" (() => {\n",
" let quickchartButtonEl =\n",
" document.querySelector('#df-55a2c921-c32c-45d2-b681-54f053a9ac56 button');\n",
" quickchartButtonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
" })();\n",
" </script>\n",
"</div>\n",
"\n",
" </div>\n",
" </div>\n"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "dataframe",
"variable_name": "sites",
"summary": "{\n \"name\": \"sites\",\n \"rows\": 50000,\n \"fields\": [\n {\n \"column\": \"origin\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 50000,\n \"samples\": [\n \"https://lavalamp.com/\",\n \"https://www.ledneonflexarg.com.ar/\",\n \"https://drcinikshop.com/\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"
},
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/e523c247d1e24a05/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"https://mrd.cash/\"],\n [{\n 'v': 1,\n 'f': \"1\",\n },\n\"https://www.handipet.org/\"],\n [{\n 'v': 2,\n 'f': \"2\",\n },\n\"https://www.mimasaka-no-kuni.jp/\"],\n [{\n 'v': 3,\n 'f': \"3\",\n },\n\"https://soundcity.com.ar/\"],\n [{\n 'v': 4,\n 'f': \"4\",\n },\n\"https://ciclismoarg.com.ar/\"],\n [{\n 'v': 5,\n 'f': \"5\",\n },\n\"https://www.freigeist-burger.at/\"],\n [{\n 'v': 6,\n 'f': \"6\",\n },\n\"https://first-clinic.jp/\"],\n [{\n 'v': 7,\n 'f': \"7\",\n },\n\"https://www.suntrupkiasouth.com/\"],\n [{\n 'v': 8,\n 'f': \"8\",\n },\n\"https://ov.taigua.cat/\"],\n [{\n 'v': 9,\n 'f': \"9\",\n },\n\"http://www.piscine-quiberon.fr/\"],\n [{\n 'v': 10,\n 'f': \"10\",\n },\n\"https://ferroluzinversiones.com/\"],\n [{\n 'v': 11,\n 'f': \"11\",\n },\n\"https://www.kumsalsohbet.net/\"],\n [{\n 'v': 12,\n 'f': \"12\",\n },\n\"https://gazetasledcza.pl/\"],\n [{\n 'v': 13,\n 'f': \"13\",\n },\n\"https://biology.tcnj.edu/\"],\n [{\n 'v': 14,\n 'f': \"14\",\n },\n\"https://www.hotel-de-brunville.com/\"],\n [{\n 'v': 15,\n 'f': \"15\",\n },\n\"https://tecnordeste.com.br/\"],\n [{\n 'v': 16,\n 'f': \"16\",\n },\n\"https://bigjoeboreholes.co.za/\"],\n [{\n 'v': 17,\n 'f': \"17\",\n },\n\"https://jornadaalimentacao.com.br/\"],\n [{\n 'v': 18,\n 'f': \"18\",\n },\n\"http://escuelaecologica.com.ar/\"],\n [{\n 'v': 19,\n 'f': \"19\",\n },\n\"http://latablederivegauche.com/\"],\n [{\n 'v': 20,\n 'f': \"20\",\n },\n\"https://www.camping-aure-pyrenees.com/\"],\n [{\n 'v': 21,\n 'f': \"21\",\n },\n\"https://hentai2.org/\"],\n [{\n 'v': 22,\n 'f': \"22\",\n },\n\"https://www.hydoll.jp/\"],\n [{\n 'v': 23,\n 'f': \"23\",\n },\n\"https://turbotoyssa.co.za/\"],\n [{\n 'v': 24,\n 'f': \"24\",\n },\n\"https://aldhafrahs-seha.com/\"],\n [{\n 'v': 25,\n 'f': \"25\",\n },\n\"https://newreach.nl/\"],\n [{\n 'v': 26,\n 'f': \"26\",\n },\n\"https://www.thaisakyant.com/\"],\n [{\n 'v': 27,\n 'f': \"27\",\n },\n\"https://techautonews.com/\"],\n [{\n 'v': 28,\n 'f': \"28\",\n },\n\"https://www.aguanteneuquen.com.ar/\"],\n [{\n 'v': 29,\n 'f': \"29\",\n },\n\"https://lapompeya.com/\"],\n [{\n 'v': 30,\n 'f': \"30\",\n },\n\"https://www.topingatlan.net/\"],\n [{\n 'v': 31,\n 'f': \"31\",\n },\n\"https://poslovnikontakt.com/\"],\n [{\n 'v': 32,\n 'f': \"32\",\n },\n\"https://siksvn.com/\"],\n [{\n 'v': 33,\n 'f': \"33\",\n },\n\"https://mrsillucia.com/\"],\n [{\n 'v': 34,\n 'f': \"34\",\n },\n\"https://kidzville.co.za/\"],\n [{\n 'v': 35,\n 'f': \"35\",\n },\n\"https://dancecentre.co.ke/\"],\n [{\n 'v': 36,\n 'f': \"36\",\n },\n\"https://www.gkcuzice.rs/\"],\n [{\n 'v': 37,\n 'f': \"37\",\n },\n\"https://ajurvedamoterims.lt/\"],\n [{\n 'v': 38,\n 'f': \"38\",\n },\n\"https://app.tokoorder.com/\"],\n [{\n 'v': 39,\n 'f': \"39\",\n },\n\"https://unique-hotels.pl/\"],\n [{\n 'v': 40,\n 'f': \"40\",\n },\n\"https://clinicamiguelangelgarcia.com/\"],\n [{\n 'v': 41,\n 'f': \"41\",\n },\n\"https://blog.alyssachan.space/\"],\n [{\n 'v': 42,\n 'f': \"42\",\n },\n\"https://news.kg/\"],\n [{\n 'v': 43,\n 'f': \"43\",\n },\n\"https://www.canoeatl.com/\"],\n [{\n 'v': 44,\n 'f': \"44\",\n },\n\"https://diskanak.bogorkab.go.id/\"],\n [{\n 'v': 45,\n 'f': \"45\",\n },\n\"https://electrogeo.com/\"],\n [{\n 'v': 46,\n 'f': \"46\",\n },\n\"https://www.infinitioflafayette.com/\"],\n [{\n 'v': 47,\n 'f': \"47\",\n },\n\"https://www.mercedes-benz-kelowna.ca/\"],\n [{\n 'v': 48,\n 'f': \"48\",\n },\n\"http://www.istitutobeatogregorio.it/\"],\n [{\n 'v': 49,\n 'f': \"49\",\n },\n\"https://sportskstars.com/\"],\n [{\n 'v': 50,\n 'f': \"50\",\n },\n\"https://de-lijn.com/\"],\n [{\n 'v': 51,\n 'f': \"51\",\n },\n\"https://viktoria-apartments.cz/\"],\n [{\n 'v': 52,\n 'f': \"52\",\n },\n\"https://natureconceptcm.com/\"],\n [{\n 'v': 53,\n 'f': \"53\",\n },\n\"https://programaahora.misiones.gob.ar/\"],\n [{\n 'v': 54,\n 'f': \"54\",\n },\n\"http://mirupak-irk.ru/\"],\n [{\n 'v': 55,\n 'f': \"55\",\n },\n\"https://www.denieuwedriehoek.be/\"],\n [{\n 'v': 56,\n 'f': \"56\",\n },\n\"https://csm.com.co/\"],\n [{\n 'v': 57,\n 'f': \"57\",\n },\n\"https://elcarpinteroloco.com.ar/\"],\n [{\n 'v': 58,\n 'f': \"58\",\n },\n\"https://dhibrahm.com/\"],\n [{\n 'v': 59,\n 'f': \"59\",\n },\n\"https://o2genes.com/\"],\n [{\n 'v': 60,\n 'f': \"60\",\n },\n\"https://saweratownships.com/\"],\n [{\n 'v': 61,\n 'f': \"61\",\n },\n\"https://www.lbk.be/\"],\n [{\n 'v': 62,\n 'f': \"62\",\n },\n\"https://kino.altmkc.ru/\"],\n [{\n 'v': 63,\n 'f': \"63\",\n },\n\"https://malalsham.com/\"],\n [{\n 'v': 64,\n 'f': \"64\",\n },\n\"https://fav3d.com.ar/\"],\n [{\n 'v': 65,\n 'f': \"65\",\n },\n\"https://seguroautoromaze.com.br/\"],\n [{\n 'v': 66,\n 'f': \"66\",\n },\n\"https://prangthip.ac.th/\"],\n [{\n 'v': 67,\n 'f': \"67\",\n },\n\"https://theraj.in/\"],\n [{\n 'v': 68,\n 'f': \"68\",\n },\n\"http://www.alberguemiguelin.com/\"],\n [{\n 'v': 69,\n 'f': \"69\",\n },\n\"http://www.loevkraj.by/\"],\n [{\n 'v': 70,\n 'f': \"70\",\n },\n\"https://newsmile.com.ua/\"],\n [{\n 'v': 71,\n 'f': \"71\",\n },\n\"https://diary.co.id/\"],\n [{\n 'v': 72,\n 'f': \"72\",\n },\n\"https://24ctnaturaloils.com/\"],\n [{\n 'v': 73,\n 'f': \"73\",\n },\n\"http://dayhocblx.com/\"],\n [{\n 'v': 74,\n 'f': \"74\",\n },\n\"https://mens-reson.com/\"],\n [{\n 'v': 75,\n 'f': \"75\",\n },\n\"https://polsrl.com/\"],\n [{\n 'v': 76,\n 'f': \"76\",\n },\n\"https://higiene-seguridad.com.ar/\"],\n [{\n 'v': 77,\n 'f': \"77\",\n },\n\"https://annahl.sch.id/\"],\n [{\n 'v': 78,\n 'f': \"78\",\n },\n\"https://www.corimonpinturas.com/\"],\n [{\n 'v': 79,\n 'f': \"79\",\n },\n\"https://pappkittiugyved.hu/\"],\n [{\n 'v': 80,\n 'f': \"80\",\n },\n\"https://www.ymaoo.cn/\"],\n [{\n 'v': 81,\n 'f': \"81\",\n },\n\"https://wasabill.pk/\"],\n [{\n 'v': 82,\n 'f': \"82\",\n },\n\"https://catorrent.com/\"],\n [{\n 'v': 83,\n 'f': \"83\",\n },\n\"https://www.herbalmedicine.co.il/\"],\n [{\n 'v': 84,\n 'f': \"84\",\n },\n\"https://intuitiveanswers.com/\"],\n [{\n 'v': 85,\n 'f': \"85\",\n },\n\"https://pokerliveupdates.com/\"],\n [{\n 'v': 86,\n 'f': \"86\",\n },\n\"https://spotyagmarket.com/\"],\n [{\n 'v': 87,\n 'f': \"87\",\n },\n\"https://trainvision.es/\"],\n [{\n 'v': 88,\n 'f': \"88\",\n },\n\"https://eromantik.net/\"],\n [{\n 'v': 89,\n 'f': \"89\",\n },\n\"https://therawellness.us/\"],\n [{\n 'v': 90,\n 'f': \"90\",\n },\n\"http://marutomo-sh.co.jp/\"],\n [{\n 'v': 91,\n 'f': \"91\",\n },\n\"https://www.arepe-mrjardinage.fr/\"],\n [{\n 'v': 92,\n 'f': \"92\",\n },\n\"https://frtbme.hu/\"],\n [{\n 'v': 93,\n 'f': \"93\",\n },\n\"https://modernaestofados.com.br/\"],\n [{\n 'v': 94,\n 'f': \"94\",\n },\n\"https://inhalemd.com/\"],\n [{\n 'v': 95,\n 'f': \"95\",\n },\n\"http://www.r-hajime.jp/\"],\n [{\n 'v': 96,\n 'f': \"96\",\n },\n\"https://popup.co.za/\"],\n [{\n 'v': 97,\n 'f': \"97\",\n },\n\"http://biofaceradiologia.com.br/\"],\n [{\n 'v': 98,\n 'f': \"98\",\n },\n\"https://rttravel.rs/\"],\n [{\n 'v': 99,\n 'f': \"99\",\n },\n\"https://esr.com.ar/\"],\n [{\n 'v': 100,\n 'f': \"100\",\n },\n\"https://www.alfafl.com/\"],\n [{\n 'v': 101,\n 'f': \"101\",\n },\n\"https://www.creditoficoufacil.com.br/\"],\n [{\n 'v': 102,\n 'f': \"102\",\n },\n\"https://deklaroen.be/\"],\n [{\n 'v': 103,\n 'f': \"103\",\n },\n\"https://woodflooringireland.ie/\"],\n [{\n 'v': 104,\n 'f': \"104\",\n },\n\"https://www.maxstuder.ch/\"],\n [{\n 'v': 105,\n 'f': \"105\",\n },\n\"https://www.palmbayford.com/\"],\n [{\n 'v': 106,\n 'f': \"106\",\n },\n\"https://www.svenskahudkliniker.se/\"],\n [{\n 'v': 107,\n 'f': \"107\",\n },\n\"https://www.saudiags.com/\"],\n [{\n 'v': 108,\n 'f': \"108\",\n },\n\"https://www.canadafloorsdepot.com/\"],\n [{\n 'v': 109,\n 'f': \"109\",\n },\n\"https://www.bbobbler.com/\"],\n [{\n 'v': 110,\n 'f': \"110\",\n },\n\"https://encometal.com.ar/\"],\n [{\n 'v': 111,\n 'f': \"111\",\n },\n\"https://www.valenciaforestschool.es/\"],\n [{\n 'v': 112,\n 'f': \"112\",\n },\n\"https://onlineprodajanovatelefonija.rs/\"],\n [{\n 'v': 113,\n 'f': \"113\",\n },\n\"https://evo-land.de/\"],\n [{\n 'v': 114,\n 'f': \"114\",\n },\n\"http://www.eqmarriageservice.com/\"],\n [{\n 'v': 115,\n 'f': \"115\",\n },\n\"https://tuah.com.my/\"],\n [{\n 'v': 116,\n 'f': \"116\",\n },\n\"https://apcainsebt.dz/\"],\n [{\n 'v': 117,\n 'f': \"117\",\n },\n\"https://museumexplorer.co.za/\"],\n [{\n 'v': 118,\n 'f': \"118\",\n },\n\"http://difmazatlan.gob.mx/\"],\n [{\n 'v': 119,\n 'f': \"119\",\n },\n\"https://goodnights.in/\"],\n [{\n 'v': 120,\n 'f': \"120\",\n },\n\"https://catolicanet.net/\"],\n [{\n 'v': 121,\n 'f': \"121\",\n },\n\"https://outdoor-styles.com/\"],\n [{\n 'v': 122,\n 'f': \"122\",\n },\n\"https://hanamaru.my/\"],\n [{\n 'v': 123,\n 'f': \"123\",\n },\n\"https://direitotrabalhista.bravoebonatto.com.br/\"],\n [{\n 'v': 124,\n 'f': \"124\",\n },\n\"https://consultoriosmedicos.com.ar/\"],\n [{\n 'v': 125,\n 'f': \"125\",\n },\n\"https://www.bcsd15.org/\"],\n [{\n 'v': 126,\n 'f': \"126\",\n },\n\"https://fideuadegandia.org/\"],\n [{\n 'v': 127,\n 'f': \"127\",\n },\n\"https://wccoa.net/\"],\n [{\n 'v': 128,\n 'f': \"128\",\n },\n\"https://pizza-show.com/\"],\n [{\n 'v': 129,\n 'f': \"129\",\n },\n\"https://www.michaelsvalleygrill.com/\"],\n [{\n 'v': 130,\n 'f': \"130\",\n },\n\"https://szamosituzep.hu/\"],\n [{\n 'v': 131,\n 'f': \"131\",\n },\n\"http://www.cme1.com.br/\"],\n [{\n 'v': 132,\n 'f': \"132\",\n },\n\"https://calisthenicssa.com.au/\"],\n [{\n 'v': 133,\n 'f': \"133\",\n },\n\"https://stellenboschvineyards.co.za/\"],\n [{\n 'v': 134,\n 'f': \"134\",\n },\n\"https://medicine.uniben.edu/\"],\n [{\n 'v': 135,\n 'f': \"135\",\n },\n\"https://idm.edu.vn/\"],\n [{\n 'v': 136,\n 'f': \"136\",\n },\n\"https://hidraliso-loja.com.br/\"],\n [{\n 'v': 137,\n 'f': \"137\",\n },\n\"https://kidsslides.co.za/\"],\n [{\n 'v': 138,\n 'f': \"138\",\n },\n\"https://www.palmspringsmotors.com/\"],\n [{\n 'v': 139,\n 'f': \"139\",\n },\n\"https://jnvkalumni.in/\"],\n [{\n 'v': 140,\n 'f': \"140\",\n },\n\"https://nsagg.ir/\"],\n [{\n 'v': 141,\n 'f': \"141\",\n },\n\"http://medlive.com.br/\"],\n [{\n 'v': 142,\n 'f': \"142\",\n },\n\"https://www.chakhdhoom.co.uk/\"],\n [{\n 'v': 143,\n 'f': \"143\",\n },\n\"https://majajamoneria.com/\"],\n [{\n 'v': 144,\n 'f': \"144\",\n },\n\"https://jonkvorstfoto.wordpress.com/\"],\n [{\n 'v': 145,\n 'f': \"145\",\n },\n\"https://webacademica.com/\"],\n [{\n 'v': 146,\n 'f': \"146\",\n },\n\"https://onethatch.com/\"],\n [{\n 'v': 147,\n 'f': \"147\",\n },\n\"https://paladin.care/\"],\n [{\n 'v': 148,\n 'f': \"148\",\n },\n\"https://www.aquelakombucha.pt/\"],\n [{\n 'v': 149,\n 'f': \"149\",\n },\n\"https://bazaar.lovedog.tw/\"],\n [{\n 'v': 150,\n 'f': \"150\",\n },\n\"https://s-art.co.jp/\"],\n [{\n 'v': 151,\n 'f': \"151\",\n },\n\"https://www.ayvaliktostcu.com/\"],\n [{\n 'v': 152,\n 'f': \"152\",\n },\n\"https://beithamamtak.co.il/\"],\n [{\n 'v': 153,\n 'f': \"153\",\n },\n\"http://thecarenursinghome.com/\"],\n [{\n 'v': 154,\n 'f': \"154\",\n },\n\"https://betongtrangtribm.vn/\"],\n [{\n 'v': 155,\n 'f': \"155\",\n },\n\"http://wordtsar.ca/\"],\n [{\n 'v': 156,\n 'f': \"156\",\n },\n\"https://www.vitalogikum.at/\"],\n [{\n 'v': 157,\n 'f': \"157\",\n },\n\"https://iswim.bialystok.pl/\"],\n [{\n 'v': 158,\n 'f': \"158\",\n },\n\"https://damattaind.com.br/\"],\n [{\n 'v': 159,\n 'f': \"159\",\n },\n\"https://buchtajeziorak.pl/\"],\n [{\n 'v': 160,\n 'f': \"160\",\n },\n\"https://www.southlanddodgechryslerjeep.com/\"],\n [{\n 'v': 161,\n 'f': \"161\",\n },\n\"https://www.wernerhyundai.com/\"],\n [{\n 'v': 162,\n 'f': \"162\",\n },\n\"https://www.kreat.pe/\"],\n [{\n 'v': 163,\n 'f': \"163\",\n },\n\"https://paulismakeup.co/\"],\n [{\n 'v': 164,\n 'f': \"164\",\n },\n\"https://www.feestweekabcoude.nl/\"],\n [{\n 'v': 165,\n 'f': \"165\",\n },\n\"https://www.vimondmateriaux.com/\"],\n [{\n 'v': 166,\n 'f': \"166\",\n },\n\"https://www.affa.no/\"],\n [{\n 'v': 167,\n 'f': \"167\",\n },\n\"https://casaslimache.cl/\"],\n [{\n 'v': 168,\n 'f': \"168\",\n },\n\"https://heartcore-athletics.com/\"],\n [{\n 'v': 169,\n 'f': \"169\",\n },\n\"https://reflux-gyomor-nyombel.hu/\"],\n [{\n 'v': 170,\n 'f': \"170\",\n },\n\"https://streetcc.com/\"],\n [{\n 'v': 171,\n 'f': \"171\",\n },\n\"https://geolearn.in/\"],\n [{\n 'v': 172,\n 'f': \"172\",\n },\n\"https://digitalrajat.io/\"],\n [{\n 'v': 173,\n 'f': \"173\",\n },\n\"https://www.restaurantemugarra.com/\"],\n [{\n 'v': 174,\n 'f': \"174\",\n },\n\"https://megacercos.com/\"],\n [{\n 'v': 175,\n 'f': \"175\",\n },\n\"https://xn--tattoofashionrdovre-87b.dk/\"],\n [{\n 'v': 176,\n 'f': \"176\",\n },\n\"https://jm3yah.com/\"],\n [{\n 'v': 177,\n 'f': \"177\",\n },\n\"https://kvizprof.hu/\"],\n [{\n 'v': 178,\n 'f': \"178\",\n },\n\"https://ideiaspresentes.com/\"],\n [{\n 'v': 179,\n 'f': \"179\",\n },\n\"https://masivoyretail.com/\"],\n [{\n 'v': 180,\n 'f': \"180\",\n },\n\"https://mielite.com/\"],\n [{\n 'v': 181,\n 'f': \"181\",\n },\n\"https://davidwright.com.au/\"],\n [{\n 'v': 182,\n 'f': \"182\",\n },\n\"https://kinkylondonescorts.co.uk/\"],\n [{\n 'v': 183,\n 'f': \"183\",\n },\n\"https://www.uvvg.ro/\"],\n [{\n 'v': 184,\n 'f': \"184\",\n },\n\"http://www.smakiweroniki.pl/\"],\n [{\n 'v': 185,\n 'f': \"185\",\n },\n\"https://premiumhome.com.ar/\"],\n [{\n 'v': 186,\n 'f': \"186\",\n },\n\"https://jcilmglobal.info/\"],\n [{\n 'v': 187,\n 'f': \"187\",\n },\n\"https://www.projectkesher.org.ua/\"],\n [{\n 'v': 188,\n 'f': \"188\",\n },\n\"https://centre.nikkeiplace.org/\"],\n [{\n 'v': 189,\n 'f': \"189\",\n },\n\"https://cursoreparodeplacasinverter.com/\"],\n [{\n 'v': 190,\n 'f': \"190\",\n },\n\"https://www.stick-and-stone-run.de/\"],\n [{\n 'v': 191,\n 'f': \"191\",\n },\n\"https://www.classifiedmoto.com/\"],\n [{\n 'v': 192,\n 'f': \"192\",\n },\n\"https://oralsinsaosebastiao.com.br/\"],\n [{\n 'v': 193,\n 'f': \"193\",\n },\n\"https://mopptex.com/\"],\n [{\n 'v': 194,\n 'f': \"194\",\n },\n\"http://www.torejaras.com/\"],\n [{\n 'v': 195,\n 'f': \"195\",\n },\n\"https://keysfamilybutchershop.com/\"],\n [{\n 'v': 196,\n 'f': \"196\",\n },\n\"https://www.mattzgarage.at/\"],\n [{\n 'v': 197,\n 'f': \"197\",\n },\n\"https://thehimalayanchocolates.com/\"],\n [{\n 'v': 198,\n 'f': \"198\",\n },\n\"https://alghawasnews.com/\"],\n [{\n 'v': 199,\n 'f': \"199\",\n },\n\"https://drcinikshop.com/\"],\n [{\n 'v': 200,\n 'f': \"200\",\n },\n\"https://gortest.com/\"],\n [{\n 'v': 201,\n 'f': \"201\",\n },\n\"https://www.hyundaiofalbany.com/\"],\n [{\n 'v': 202,\n 'f': \"202\",\n },\n\"https://surgenutrition.in/\"],\n [{\n 'v': 203,\n 'f': \"203\",\n },\n\"https://www.pimas.fr/\"],\n [{\n 'v': 204,\n 'f': \"204\",\n },\n\"https://formulat77.com/\"],\n [{\n 'v': 205,\n 'f': \"205\",\n },\n\"http://www.nittaidai-fc.com/\"],\n [{\n 'v': 206,\n 'f': \"206\",\n },\n\"https://www.slot-labo.com/\"],\n [{\n 'v': 207,\n 'f': \"207\",\n },\n\"https://kaltex.global/\"],\n [{\n 'v': 208,\n 'f': \"208\",\n },\n\"http://www.cannes-petit-train.com/\"],\n [{\n 'v': 209,\n 'f': \"209\",\n },\n\"https://abercor.com.ar/\"],\n [{\n 'v': 210,\n 'f': \"210\",\n },\n\"https://travelua.com.ua/\"],\n [{\n 'v': 211,\n 'f': \"211\",\n },\n\"https://talkbitz.com/\"],\n [{\n 'v': 212,\n 'f': \"212\",\n },\n\"https://goelette.ca/\"],\n [{\n 'v': 213,\n 'f': \"213\",\n },\n\"https://caddiesgolf.co.uk/\"],\n [{\n 'v': 214,\n 'f': \"214\",\n },\n\"https://martynasoul.com/\"],\n [{\n 'v': 215,\n 'f': \"215\",\n },\n\"https://www.mercedes-benz-newmarket.ca/\"],\n [{\n 'v': 216,\n 'f': \"216\",\n },\n\"https://clarissaviaggi.com/\"],\n [{\n 'v': 217,\n 'f': \"217\",\n },\n\"https://www.komsem.de/\"],\n [{\n 'v': 218,\n 'f': \"218\",\n },\n\"https://dive-scoop.com/\"],\n [{\n 'v': 219,\n 'f': \"219\",\n },\n\"https://www.mikewhatleyhonda.com/\"],\n [{\n 'v': 220,\n 'f': \"220\",\n },\n\"https://genderparty.top/\"],\n [{\n 'v': 221,\n 'f': \"221\",\n },\n\"https://logomaker.click/\"],\n [{\n 'v': 222,\n 'f': \"222\",\n },\n\"https://belzy27.pl/\"],\n [{\n 'v': 223,\n 'f': \"223\",\n },\n\"https://almanca-begibegi.com/\"],\n [{\n 'v': 224,\n 'f': \"224\",\n },\n\"https://boulder-zone.pl/\"],\n [{\n 'v': 225,\n 'f': \"225\",\n },\n\"https://www.shinzato-saiyou.jp/\"],\n [{\n 'v': 226,\n 'f': \"226\",\n },\n\"https://www.ihrlungenarzt.at/\"],\n [{\n 'v': 227,\n 'f': \"227\",\n },\n\"https://dalma.com.ar/\"],\n [{\n 'v': 228,\n 'f': \"228\",\n },\n\"https://www.welzijncapelle.nl/\"],\n [{\n 'v': 229,\n 'f': \"229\",\n },\n\"https://manastireabodrog.ro/\"],\n [{\n 'v': 230,\n 'f': \"230\",\n },\n\"https://office-keys.nl/\"],\n [{\n 'v': 231,\n 'f': \"231\",\n },\n\"https://ravipharma.in/\"],\n [{\n 'v': 232,\n 'f': \"232\",\n },\n\"https://www.knittel-entsorgung.com/\"],\n [{\n 'v': 233,\n 'f': \"233\",\n },\n\"https://eakces.eu/\"],\n [{\n 'v': 234,\n 'f': \"234\",\n },\n\"https://kcsir.pl/\"],\n [{\n 'v': 235,\n 'f': \"235\",\n },\n\"https://www.trombetabath.com/\"],\n [{\n 'v': 236,\n 'f': \"236\",\n },\n\"https://golkarpedia.com/\"],\n [{\n 'v': 237,\n 'f': \"237\",\n },\n\"https://ww5.cmovies.co/\"],\n [{\n 'v': 238,\n 'f': \"238\",\n },\n\"https://radiointeractivafm.cl/\"],\n [{\n 'v': 239,\n 'f': \"239\",\n },\n\"https://rendeneralpinefood.it/\"],\n [{\n 'v': 240,\n 'f': \"240\",\n },\n\"https://www.centralvalleyauto.com/\"],\n [{\n 'v': 241,\n 'f': \"241\",\n },\n\"https://dragonwhite.com/\"],\n [{\n 'v': 242,\n 'f': \"242\",\n },\n\"https://ekobites.nl/\"],\n [{\n 'v': 243,\n 'f': \"243\",\n },\n\"https://cadmaxxedtech.com/\"],\n [{\n 'v': 244,\n 'f': \"244\",\n },\n\"https://www.panelectronicsindia.com/\"],\n [{\n 'v': 245,\n 'f': \"245\",\n },\n\"https://institutosanvicente.com/\"],\n [{\n 'v': 246,\n 'f': \"246\",\n },\n\"https://cheaperbytheday1.com/\"],\n [{\n 'v': 247,\n 'f': \"247\",\n },\n\"https://www.vegan-wein.at/\"],\n [{\n 'v': 248,\n 'f': \"248\",\n },\n\"https://www.quirk.mercedesdealer.com/\"],\n [{\n 'v': 249,\n 'f': \"249\",\n },\n\"https://projectcalls.com/\"],\n [{\n 'v': 250,\n 'f': \"250\",\n },\n\"https://www.nightvisions.info/\"],\n [{\n 'v': 251,\n 'f': \"251\",\n },\n\"https://www.abbeyshebburn.co.uk/\"],\n [{\n 'v': 252,\n 'f': \"252\",\n },\n\"https://jobs.angelscarehealth.com/\"],\n [{\n 'v': 253,\n 'f': \"253\",\n },\n\"http://vynuogynas.lt/\"],\n [{\n 'v': 254,\n 'f': \"254\",\n },\n\"https://restauranteyoakewangalfafar.es/\"],\n [{\n 'v': 255,\n 'f': \"255\",\n },\n\"https://armtek.com.tr/\"],\n [{\n 'v': 256,\n 'f': \"256\",\n },\n\"https://whitechartskiing.com/\"],\n [{\n 'v': 257,\n 'f': \"257\",\n },\n\"https://achermannsport.ch/\"],\n [{\n 'v': 258,\n 'f': \"258\",\n },\n\"https://www.morrieswestsalemkia.com/\"],\n [{\n 'v': 259,\n 'f': \"259\",\n },\n\"https://alena-energy.com/\"],\n [{\n 'v': 260,\n 'f': \"260\",\n },\n\"https://gaffarbhaiandsons.com/\"],\n [{\n 'v': 261,\n 'f': \"261\",\n },\n\"https://konggaard.dk/\"],\n [{\n 'v': 262,\n 'f': \"262\",\n },\n\"https://www.dara2you.com/\"],\n [{\n 'v': 263,\n 'f': \"263\",\n },\n\"https://www.udrive.nc/\"],\n [{\n 'v': 264,\n 'f': \"264\",\n },\n\"https://acidosalus.com/\"],\n [{\n 'v': 265,\n 'f': \"265\",\n },\n\"https://www.seisamed.com/\"],\n [{\n 'v': 266,\n 'f': \"266\",\n },\n\"https://www.kagudai.co.jp/\"],\n [{\n 'v': 267,\n 'f': \"267\",\n },\n\"https://bioexo.si/\"],\n [{\n 'v': 268,\n 'f': \"268\",\n },\n\"https://medstore.lt/\"],\n [{\n 'v': 269,\n 'f': \"269\",\n },\n\"https://trevoportasejanelas.com.br/\"],\n [{\n 'v': 270,\n 'f': \"270\",\n },\n\"https://mausoleedinromania.ro/\"],\n [{\n 'v': 271,\n 'f': \"271\",\n },\n\"https://styllbaby.com.br/\"],\n [{\n 'v': 272,\n 'f': \"272\",\n },\n\"https://katana-svard.se/\"],\n [{\n 'v': 273,\n 'f': \"273\",\n },\n\"https://ullevi-vikingen-friidrott.se/\"],\n [{\n 'v': 274,\n 'f': \"274\",\n },\n\"https://sejaazzul.com.br/\"],\n [{\n 'v': 275,\n 'f': \"275\",\n },\n\"http://hotelyunost.com/\"],\n [{\n 'v': 276,\n 'f': \"276\",\n },\n\"https://vol.com.ua/\"],\n [{\n 'v': 277,\n 'f': \"277\",\n },\n\"https://redsol.com.ar/\"],\n [{\n 'v': 278,\n 'f': \"278\",\n },\n\"http://diagnosticosalta.com.ar/\"],\n [{\n 'v': 279,\n 'f': \"279\",\n },\n\"https://farmajindris.cz/\"],\n [{\n 'v': 280,\n 'f': \"280\",\n },\n\"https://www.sonnuoctot.com/\"],\n [{\n 'v': 281,\n 'f': \"281\",\n },\n\"http://www.mtbtestcentral.it/\"],\n [{\n 'v': 282,\n 'f': \"282\",\n },\n\"https://www.arterydirect.com.au/\"],\n [{\n 'v': 283,\n 'f': \"283\",\n },\n\"https://pizzalaguna.cz/\"],\n [{\n 'v': 284,\n 'f': \"284\",\n },\n\"https://www.proshield.co.za/\"],\n [{\n 'v': 285,\n 'f': \"285\",\n },\n\"https://www.balletypilatesterapeutico.com/\"],\n [{\n 'v': 286,\n 'f': \"286\",\n },\n\"https://www.mczgroup.it/\"],\n [{\n 'v': 287,\n 'f': \"287\",\n },\n\"https://www.segurototal.com.br/\"],\n [{\n 'v': 288,\n 'f': \"288\",\n },\n\"https://www.profimorava.cz/\"],\n [{\n 'v': 289,\n 'f': \"289\",\n },\n\"https://ellenguajedeladios.com/\"],\n [{\n 'v': 290,\n 'f': \"290\",\n },\n\"https://www.forestclub.md/\"],\n [{\n 'v': 291,\n 'f': \"291\",\n },\n\"https://lumof.ua/\"],\n [{\n 'v': 292,\n 'f': \"292\",\n },\n\"http://aurapharma.com/\"],\n [{\n 'v': 293,\n 'f': \"293\",\n },\n\"https://www.newsundog.com/\"],\n [{\n 'v': 294,\n 'f': \"294\",\n },\n\"https://nryachtclub.ru/\"],\n [{\n 'v': 295,\n 'f': \"295\",\n },\n\"https://fdia.co.za/\"],\n [{\n 'v': 296,\n 'f': \"296\",\n },\n\"https://www.lobstervanhire.co.uk/\"],\n [{\n 'v': 297,\n 'f': \"297\",\n },\n\"https://codero.com.pl/\"],\n [{\n 'v': 298,\n 'f': \"298\",\n },\n\"https://omegaap.com.br/\"],\n [{\n 'v': 299,\n 'f': \"299\",\n },\n\"http://www.kebunrayasriwijaya.com/\"],\n [{\n 'v': 300,\n 'f': \"300\",\n },\n\"http://dtdb.ou.edu.vn/\"],\n [{\n 'v': 301,\n 'f': \"301\",\n },\n\"http://www.lycee-marechal-leclerc.com/\"],\n [{\n 'v': 302,\n 'f': \"302\",\n },\n\"https://careers.opisok.com/\"],\n [{\n 'v': 303,\n 'f': \"303\",\n },\n\"https://prosein.com.ve/\"],\n [{\n 'v': 304,\n 'f': \"304\",\n },\n\"http://chushin-sws.jp/\"],\n [{\n 'v': 305,\n 'f': \"305\",\n },\n\"https://www.instantsushinantes.com/\"],\n [{\n 'v': 306,\n 'f': \"306\",\n },\n\"https://graftoneverest.co.za/\"],\n [{\n 'v': 307,\n 'f': \"307\",\n },\n\"https://thegreatescape.info/\"],\n [{\n 'v': 308,\n 'f': \"308\",\n },\n\"https://www.localseeds.com.au/\"],\n [{\n 'v': 309,\n 'f': \"309\",\n },\n\"https://www.travelcompass.nl/\"],\n [{\n 'v': 310,\n 'f': \"310\",\n },\n\"https://vueltaballenas.com.ar/\"],\n [{\n 'v': 311,\n 'f': \"311\",\n },\n\"http://www.retropedia.radom.pl/\"],\n [{\n 'v': 312,\n 'f': \"312\",\n },\n\"https://metropharma.in/\"],\n [{\n 'v': 313,\n 'f': \"313\",\n },\n\"https://saifleet.co.za/\"],\n [{\n 'v': 314,\n 'f': \"314\",\n },\n\"https://wasidog.com/\"],\n [{\n 'v': 315,\n 'f': \"315\",\n },\n\"https://efoils.cz/\"],\n [{\n 'v': 316,\n 'f': \"316\",\n },\n\"https://astrologis.rs/\"],\n [{\n 'v': 317,\n 'f': \"317\",\n },\n\"https://femme-amsterdam.nl/\"],\n [{\n 'v': 318,\n 'f': \"318\",\n },\n\"http://gorskiogrod.pl/\"],\n [{\n 'v': 319,\n 'f': \"319\",\n },\n\"https://de-la-ghetto-ft-randy-sensacion-del-bloque.playurbano.com/\"],\n [{\n 'v': 320,\n 'f': \"320\",\n },\n\"https://marcacundinamarca.com/\"],\n [{\n 'v': 321,\n 'f': \"321\",\n },\n\"https://camarcosanjuan.org/\"],\n [{\n 'v': 322,\n 'f': \"322\",\n },\n\"https://helicol.com.co/\"],\n [{\n 'v': 323,\n 'f': \"323\",\n },\n\"https://www.dentsoftware.com/\"],\n [{\n 'v': 324,\n 'f': \"324\",\n },\n\"https://reparacionysacrificio.org/\"],\n [{\n 'v': 325,\n 'f': \"325\",\n },\n\"http://donlandsdiner.com/\"],\n [{\n 'v': 326,\n 'f': \"326\",\n },\n\"https://www.globaldutchies.nl/\"],\n [{\n 'v': 327,\n 'f': \"327\",\n },\n\"http://www.douno-ganka.com/\"],\n [{\n 'v': 328,\n 'f': \"328\",\n },\n\"https://fsspx.hr/\"],\n [{\n 'v': 329,\n 'f': \"329\",\n },\n\"https://sexshopamonet.com.ar/\"],\n [{\n 'v': 330,\n 'f': \"330\",\n },\n\"https://blog.noqta.co/\"],\n [{\n 'v': 331,\n 'f': \"331\",\n },\n\"https://klubcf.cz/\"],\n [{\n 'v': 332,\n 'f': \"332\",\n },\n\"https://www.appartamentirizzante.com/\"],\n [{\n 'v': 333,\n 'f': \"333\",\n },\n\"https://umenicestovat.com/\"],\n [{\n 'v': 334,\n 'f': \"334\",\n },\n\"http://www.centrocommercialemercogliano.net/\"],\n [{\n 'v': 335,\n 'f': \"335\",\n },\n\"https://sgbasesoria.com/\"],\n [{\n 'v': 336,\n 'f': \"336\",\n },\n\"https://www.mecansur.com/\"],\n [{\n 'v': 337,\n 'f': \"337\",\n },\n\"https://vitavalka.cz/\"],\n [{\n 'v': 338,\n 'f': \"338\",\n },\n\"https://santtina.co/\"],\n [{\n 'v': 339,\n 'f': \"339\",\n },\n\"https://rpi.chubut.gov.ar/\"],\n [{\n 'v': 340,\n 'f': \"340\",\n },\n\"https://sabapsiena.cultura.gov.it/\"],\n [{\n 'v': 341,\n 'f': \"341\",\n },\n\"https://qe4ferry.com/\"],\n [{\n 'v': 342,\n 'f': \"342\",\n },\n\"https://www.pc55.nl/\"],\n [{\n 'v': 343,\n 'f': \"343\",\n },\n\"https://www.unlistedsharebrokers.com/\"],\n [{\n 'v': 344,\n 'f': \"344\",\n },\n\"https://www.jollysailororford.co.uk/\"],\n [{\n 'v': 345,\n 'f': \"345\",\n },\n\"https://www.discotecaindiana.com/\"],\n [{\n 'v': 346,\n 'f': \"346\",\n },\n\"https://ekm.feb.usk.ac.id/\"],\n [{\n 'v': 347,\n 'f': \"347\",\n },\n\"https://cora.org.mx/\"],\n [{\n 'v': 348,\n 'f': \"348\",\n },\n\"https://www.driveknoxchevy.com/\"],\n [{\n 'v': 349,\n 'f': \"349\",\n },\n\"https://dakpro.be/\"],\n [{\n 'v': 350,\n 'f': \"350\",\n },\n\"https://lutyensdelhi.in/\"],\n [{\n 'v': 351,\n 'f': \"351\",\n },\n\"https://macsforcancer.com/\"],\n [{\n 'v': 352,\n 'f': \"352\",\n },\n\"https://www.tspot.com/\"],\n [{\n 'v': 353,\n 'f': \"353\",\n },\n\"https://www.griassdi-kaiserwinkl.at/\"],\n [{\n 'v': 354,\n 'f': \"354\",\n },\n\"https://repuesteranacional.com/\"],\n [{\n 'v': 355,\n 'f': \"355\",\n },\n\"https://www.yfdberlin.com/\"],\n [{\n 'v': 356,\n 'f': \"356\",\n },\n\"https://www.hablemosdetodo.gob.ar/\"],\n [{\n 'v': 357,\n 'f': \"357\",\n },\n\"https://www.gaumen-schmaus.at/\"],\n [{\n 'v': 358,\n 'f': \"358\",\n },\n\"https://pattentitle.com/\"],\n [{\n 'v': 359,\n 'f': \"359\",\n },\n\"https://10nengo.com/\"],\n [{\n 'v': 360,\n 'f': \"360\",\n },\n\"https://www.sexitive.com/\"],\n [{\n 'v': 361,\n 'f': \"361\",\n },\n\"https://houserauctioneers.com/\"],\n [{\n 'v': 362,\n 'f': \"362\",\n },\n\"https://e-dikbang.sdmpoldajambi.com/\"],\n [{\n 'v': 363,\n 'f': \"363\",\n },\n\"https://www.equipar.com.mx/\"],\n [{\n 'v': 364,\n 'f': \"364\",\n },\n\"https://www.stanmcnabbchevroletcolumbia.com/\"],\n [{\n 'v': 365,\n 'f': \"365\",\n },\n\"http://sauna-frivol.com/\"],\n [{\n 'v': 366,\n 'f': \"366\",\n },\n\"https://hipergas.com.ar/\"],\n [{\n 'v': 367,\n 'f': \"367\",\n },\n\"https://jpsdistribuciones.com.ar/\"],\n [{\n 'v': 368,\n 'f': \"368\",\n },\n\"https://enroute.aircanada.com/\"],\n [{\n 'v': 369,\n 'f': \"369\",\n },\n\"https://casttube.org/\"],\n [{\n 'v': 370,\n 'f': \"370\",\n },\n\"https://smart-info.blog/\"],\n [{\n 'v': 371,\n 'f': \"371\",\n },\n\"https://www.cat-kyushu.co.jp/\"],\n [{\n 'v': 372,\n 'f': \"372\",\n },\n\"https://dragonheng29.com/\"],\n [{\n 'v': 373,\n 'f': \"373\",\n },\n\"https://restaurantedafazendinha.com.br/\"],\n [{\n 'v': 374,\n 'f': \"374\",\n },\n\"https://www.designs4cnc.in/\"],\n [{\n 'v': 375,\n 'f': \"375\",\n },\n\"https://www.primariasebes.ro/\"],\n [{\n 'v': 376,\n 'f': \"376\",\n },\n\"https://evorich.com.sg/\"],\n [{\n 'v': 377,\n 'f': \"377\",\n },\n\"https://a2zeegifts.com/\"],\n [{\n 'v': 378,\n 'f': \"378\",\n },\n\"https://www.gigighirotti.it/\"],\n [{\n 'v': 379,\n 'f': \"379\",\n },\n\"https://www.pacificoaks.edu/\"],\n [{\n 'v': 380,\n 'f': \"380\",\n },\n\"https://www.teamkids.com.au/\"],\n [{\n 'v': 381,\n 'f': \"381\",\n },\n\"https://www.cysnews.cz/\"],\n [{\n 'v': 382,\n 'f': \"382\",\n },\n\"https://www.countryhyundai.com/\"],\n [{\n 'v': 383,\n 'f': \"383\",\n },\n\"https://armyrun.ca/\"],\n [{\n 'v': 384,\n 'f': \"384\",\n },\n\"https://www.femporn.de/\"],\n [{\n 'v': 385,\n 'f': \"385\",\n },\n\"https://anchin.jp/\"],\n [{\n 'v': 386,\n 'f': \"386\",\n },\n\"https://igors.com.br/\"],\n [{\n 'v': 387,\n 'f': \"387\",\n },\n\"https://www.williamshonda.com/\"],\n [{\n 'v': 388,\n 'f': \"388\",\n },\n\"https://telecomlead.com/\"],\n [{\n 'v': 389,\n 'f': \"389\",\n },\n\"https://cgc.energy/\"],\n [{\n 'v': 390,\n 'f': \"390\",\n },\n\"https://tazatech.in/\"],\n [{\n 'v': 391,\n 'f': \"391\",\n },\n\"https://metalinsider.net/\"],\n [{\n 'v': 392,\n 'f': \"392\",\n },\n\"https://naciya.home.cx.ua/\"],\n [{\n 'v': 393,\n 'f': \"393\",\n },\n\"https://www.winitcq.com/\"],\n [{\n 'v': 394,\n 'f': \"394\",\n },\n\"https://www.daikanyamastudio.jp/\"],\n [{\n 'v': 395,\n 'f': \"395\",\n },\n\"http://www.mbfindia.net/\"],\n [{\n 'v': 396,\n 'f': \"396\",\n },\n\"https://www.palmerstoyota.com/\"],\n [{\n 'v': 397,\n 'f': \"397\",\n },\n\"https://monpetitpotager.fr/\"],\n [{\n 'v': 398,\n 'f': \"398\",\n },\n\"https://benecol.ie/\"],\n [{\n 'v': 399,\n 'f': \"399\",\n },\n\"https://100poundsocial.com/\"],\n [{\n 'v': 400,\n 'f': \"400\",\n },\n\"http://lallunadevalencia.com/\"],\n [{\n 'v': 401,\n 'f': \"401\",\n },\n\"https://www.granjaescuelaguresustraiak.com/\"],\n [{\n 'v': 402,\n 'f': \"402\",\n },\n\"https://zbmk.zp.ua/\"],\n [{\n 'v': 403,\n 'f': \"403\",\n },\n\"https://sananda.com.bd/\"],\n [{\n 'v': 404,\n 'f': \"404\",\n },\n\"https://www.toyotaofcorvallis.com/\"],\n [{\n 'v': 405,\n 'f': \"405\",\n },\n\"https://www.cavenderbuickgmcwest.com/\"],\n [{\n 'v': 406,\n 'f': \"406\",\n },\n\"https://pers-upn.com/\"],\n [{\n 'v': 407,\n 'f': \"407\",\n },\n\"https://copba-cs.org.ar/\"],\n [{\n 'v': 408,\n 'f': \"408\",\n },\n\"https://www.djbutikken.com/\"],\n [{\n 'v': 409,\n 'f': \"409\",\n },\n\"https://www.fleurdelisbeach.com/\"],\n [{\n 'v': 410,\n 'f': \"410\",\n },\n\"https://booking.tankodrommilovice.cz/\"],\n [{\n 'v': 411,\n 'f': \"411\",\n },\n\"https://www.elanceluxuryliving.net/\"],\n [{\n 'v': 412,\n 'f': \"412\",\n },\n\"https://multi-medica.pl/\"],\n [{\n 'v': 413,\n 'f': \"413\",\n },\n\"https://eikenhuis.nl/\"],\n [{\n 'v': 414,\n 'f': \"414\",\n },\n\"https://camboauto.com/\"],\n [{\n 'v': 415,\n 'f': \"415\",\n },\n\"https://www.goudatijdmachine.nl/\"],\n [{\n 'v': 416,\n 'f': \"416\",\n },\n\"https://empirescollection.pk/\"],\n [{\n 'v': 417,\n 'f': \"417\",\n },\n\"https://starlivestudio.com/\"],\n [{\n 'v': 418,\n 'f': \"418\",\n },\n\"https://www.edax.hu/\"],\n [{\n 'v': 419,\n 'f': \"419\",\n },\n\"https://www.u-hajku.cz/\"],\n [{\n 'v': 420,\n 'f': \"420\",\n },\n\"https://rpgifting.com/\"],\n [{\n 'v': 421,\n 'f': \"421\",\n },\n\"https://aestimaasset.com/\"],\n [{\n 'v': 422,\n 'f': \"422\",\n },\n\"https://zsvelkeprilepy.cz/\"],\n [{\n 'v': 423,\n 'f': \"423\",\n },\n\"https://elmundodelespectaculo.es/\"],\n [{\n 'v': 424,\n 'f': \"424\",\n },\n\"https://www.parksmotors.com/\"],\n [{\n 'v': 425,\n 'f': \"425\",\n },\n\"https://wagtailcountrypark.co.uk/\"],\n [{\n 'v': 426,\n 'f': \"426\",\n },\n\"https://www.exhighlow-tribe.com/\"],\n [{\n 'v': 427,\n 'f': \"427\",\n },\n\"https://tangobaires.com/\"],\n [{\n 'v': 428,\n 'f': \"428\",\n },\n\"https://www.tautrimas.com/\"],\n [{\n 'v': 429,\n 'f': \"429\",\n },\n\"https://westlands.amaron.ke/\"],\n [{\n 'v': 430,\n 'f': \"430\",\n },\n\"https://surinmobiliariazapala.com.ar/\"],\n [{\n 'v': 431,\n 'f': \"431\",\n },\n\"https://laboratorioesarte.com.ar/\"],\n [{\n 'v': 432,\n 'f': \"432\",\n },\n\"https://santasaude.santacasadeitapeva.org.br/\"],\n [{\n 'v': 433,\n 'f': \"433\",\n },\n\"https://www.marylandjunkyards.com/\"],\n [{\n 'v': 434,\n 'f': \"434\",\n },\n\"http://www.gdknt.ru/\"],\n [{\n 'v': 435,\n 'f': \"435\",\n },\n\"https://www.restaurant-les1separables.com/\"],\n [{\n 'v': 436,\n 'f': \"436\",\n },\n\"https://mariapolislia.com.ar/\"],\n [{\n 'v': 437,\n 'f': \"437\",\n },\n\"http://pureza.rn.gov.br/\"],\n [{\n 'v': 438,\n 'f': \"438\",\n },\n\"https://hocvan.edu.vn/\"],\n [{\n 'v': 439,\n 'f': \"439\",\n },\n\"http://eshodinshikhi.com/\"],\n [{\n 'v': 440,\n 'f': \"440\",\n },\n\"https://www.mh-ordi.at/\"],\n [{\n 'v': 441,\n 'f': \"441\",\n },\n\"https://iyibilgiler.wordpress.com/\"],\n [{\n 'v': 442,\n 'f': \"442\",\n },\n\"https://restaurantrosas.nl/\"],\n [{\n 'v': 443,\n 'f': \"443\",\n },\n\"https://febi.iainpalopo.ac.id/\"],\n [{\n 'v': 444,\n 'f': \"444\",\n },\n\"https://www.wacker1930.de/\"],\n [{\n 'v': 445,\n 'f': \"445\",\n },\n\"https://metropostnews.com/\"],\n [{\n 'v': 446,\n 'f': \"446\",\n },\n\"https://labastiglia.com/\"],\n [{\n 'v': 447,\n 'f': \"447\",\n },\n\"https://rushd.sch.id/\"],\n [{\n 'v': 448,\n 'f': \"448\",\n },\n\"https://laviklinic.co.il/\"],\n [{\n 'v': 449,\n 'f': \"449\",\n },\n\"https://garaz-nowoczesny.pl/\"],\n [{\n 'v': 450,\n 'f': \"450\",\n },\n\"https://nirvanaandspa.com/\"],\n [{\n 'v': 451,\n 'f': \"451\",\n },\n\"https://navarro.gob.ar/\"],\n [{\n 'v': 452,\n 'f': \"452\",\n },\n\"https://www.griffincdjrvaldosta.com/\"],\n [{\n 'v': 453,\n 'f': \"453\",\n },\n\"https://tempo-tv.com/\"],\n [{\n 'v': 454,\n 'f': \"454\",\n },\n\"https://uranai.news/\"],\n [{\n 'v': 455,\n 'f': \"455\",\n },\n\"https://thebroughton.co.uk/\"],\n [{\n 'v': 456,\n 'f': \"456\",\n },\n\"https://www.sutliffvolkswagen.com/\"],\n [{\n 'v': 457,\n 'f': \"457\",\n },\n\"https://togetheralive.be/\"],\n [{\n 'v': 458,\n 'f': \"458\",\n },\n\"https://batutairenginiams.lt/\"],\n [{\n 'v': 459,\n 'f': \"459\",\n },\n\"https://www.thorntonbrothers.net/\"],\n [{\n 'v': 460,\n 'f': \"460\",\n },\n\"https://activehistoryteacher.com/\"],\n [{\n 'v': 461,\n 'f': \"461\",\n },\n\"https://powervacamerica.com/\"],\n [{\n 'v': 462,\n 'f': \"462\",\n },\n\"https://www.goeters.nl/\"],\n [{\n 'v': 463,\n 'f': \"463\",\n },\n\"https://hostys.uprwssp.org/\"],\n [{\n 'v': 464,\n 'f': \"464\",\n },\n\"https://iptvlovers.com/\"],\n [{\n 'v': 465,\n 'f': \"465\",\n },\n\"https://www.radiologie-luton-reims.fr/\"],\n [{\n 'v': 466,\n 'f': \"466\",\n },\n\"https://g2r.com.br/\"],\n [{\n 'v': 467,\n 'f': \"467\",\n },\n\"https://brilliantasmit.staff.unri.ac.id/\"],\n [{\n 'v': 468,\n 'f': \"468\",\n },\n\"https://bad-bunny.flowhot.cc/\"],\n [{\n 'v': 469,\n 'f': \"469\",\n },\n\"https://dearbillypatterns.com/\"],\n [{\n 'v': 470,\n 'f': \"470\",\n },\n\"https://green-card-news.com/\"],\n [{\n 'v': 471,\n 'f': \"471\",\n },\n\"https://devon-tools.ru/\"],\n [{\n 'v': 472,\n 'f': \"472\",\n },\n\"https://www.radirna.cz/\"],\n [{\n 'v': 473,\n 'f': \"473\",\n },\n\"https://carasnortes.com/\"],\n [{\n 'v': 474,\n 'f': \"474\",\n },\n\"https://prodibio.com/\"],\n [{\n 'v': 475,\n 'f': \"475\",\n },\n\"https://idees-store.gr/\"],\n [{\n 'v': 476,\n 'f': \"476\",\n },\n\"https://serenitytalks.com/\"],\n [{\n 'v': 477,\n 'f': \"477\",\n },\n\"https://www.casadeyemanya.com/\"],\n [{\n 'v': 478,\n 'f': \"478\",\n },\n\"https://greenpuddle.net/\"],\n [{\n 'v': 479,\n 'f': \"479\",\n },\n\"http://www.fluirautomacao.com.br/\"],\n [{\n 'v': 480,\n 'f': \"480\",\n },\n\"https://www.ptplusvip.com/\"],\n [{\n 'v': 481,\n 'f': \"481\",\n },\n\"https://medicoop.com.do/\"],\n [{\n 'v': 482,\n 'f': \"482\",\n },\n\"https://www.benevit.at/\"],\n [{\n 'v': 483,\n 'f': \"483\",\n },\n\"https://slaskiecentrumosteopatii.pl/\"],\n [{\n 'v': 484,\n 'f': \"484\",\n },\n\"https://campervanbruno.es/\"],\n [{\n 'v': 485,\n 'f': \"485\",\n },\n\"https://amancialandrealty.com/\"],\n [{\n 'v': 486,\n 'f': \"486\",\n },\n\"https://andes-travel.cl/\"],\n [{\n 'v': 487,\n 'f': \"487\",\n },\n\"https://www.textilesmontecid.com/\"],\n [{\n 'v': 488,\n 'f': \"488\",\n },\n\"https://infinitybouncepark.com/\"],\n [{\n 'v': 489,\n 'f': \"489\",\n },\n\"http://www.motu-yamamoto.com/\"],\n [{\n 'v': 490,\n 'f': \"490\",\n },\n\"https://www.playground-kids.com.ua/\"],\n [{\n 'v': 491,\n 'f': \"491\",\n },\n\"https://lavoixdelesperance.org/\"],\n [{\n 'v': 492,\n 'f': \"492\",\n },\n\"https://www.gbs-mozaiek.be/\"],\n [{\n 'v': 493,\n 'f': \"493\",\n },\n\"https://semi-sa.com.ar/\"],\n [{\n 'v': 494,\n 'f': \"494\",\n },\n\"https://onlinebingo.co.uk/\"],\n [{\n 'v': 495,\n 'f': \"495\",\n },\n\"https://ivadi.es/\"],\n [{\n 'v': 496,\n 'f': \"496\",\n },\n\"https://meuesquematizado.com.br/\"],\n [{\n 'v': 497,\n 'f': \"497\",\n },\n\"https://safemedicinedrop.com/\"],\n [{\n 'v': 498,\n 'f': \"498\",\n },\n\"https://visittrollaskagi.is/\"],\n [{\n 'v': 499,\n 'f': \"499\",\n },\n\"https://www.plazahyundai.com/\"]],\n columns: [[\"number\", \"index\"], [\"string\", \"origin\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n<div id=\"df-1b621b79-a398-4e35-881e-6fbd513a6864\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-1b621b79-a398-4e35-881e-6fbd513a6864')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n \n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n \n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-1b621b79-a398-4e35-881e-6fbd513a6864 button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n</div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n "
},
"metadata": {},
"execution_count": 13
}
]
},
{
"cell_type": "code",
"source": [
"previously_not_active = get_active_sites_also_active_at_before_date('speculation-rules', '2024-05-01', active_sites)\n",
"previously_not_active.head(5)\n",
"\n",
"# count active sites\n",
"count_previously_not_active = previously_not_active.count()\n",
"print(count_previously_not_active)"
],
"metadata": {
"id": "2f60oM9FTQAU",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "937b5fe1-9d08-47fc-a175-d1ceda81a5c4"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"origin 2960\n",
"dtype: int64\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"cwv_changes_active_sites = get_cwv_change_before_after(active_sites, '2024-03-01', '2024-08-01' )\n",
"print(cwv_changes_active_sites)"
],
"metadata": {
"id": "g7DrwD1Z6kcB",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "f584f309-8303-49e6-ae3a-939f2b864123"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"[0.06232002113378332, 0.06346816102047553, 0.03573906737828336, 0.044142435735356034]\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"cwv_changes_active_sites_by_device = get_cwv_change_before_after_by_device(active_sites, '2024-03-01', '2024-08-01' )\n",
"print(cwv_changes_active_sites_by_device)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Yl99_75aLbMr",
"outputId": "ba349d86-1aa2-4d87-8027-556472c1068c"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"[0.07083488781153219, 0.06909267366979688, 0.03922572086938758, 0.08127807033918977, 0.053894219906421015, 0.05999325206027517, 0.026095263217394526, 0.0027941094172748704]\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"cwv_changes_inactive_sites = get_cwv_change_before_after(inactive_sites, '2024-03-01', '2024-08-01' )\n",
"print(cwv_changes_inactive_sites)"
],
"metadata": {
"id": "Hv4-fMck7pD6",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "370f9c31-68eb-450f-c691-e5b518de9154"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"[0.03842766935669645, 0.03281879625688, 0.016236264348328477, 0.036834856975053176]\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"cwv_changes_inactive_sites_by_device = get_cwv_change_before_after_by_device(inactive_sites, '2024-03-01', '2024-08-01' )\n",
"print(cwv_changes_inactive_sites_by_device)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ReoWFooEOaFJ",
"outputId": "330e1059-3ac9-41e9-8c34-8fd24fe82f6c"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"[0.04525254428354497, 0.03586424508053565, 0.012105841662442773, 0.06688614771352841, 0.030943523718571342, 0.03481560556441132, 0.01572887912168608, 0.0004040350745310173]\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"final_results = calculate_feature_impact(cwv_changes_active_sites, cwv_changes_inactive_sites)\n",
"print(final_results)\n"
],
"metadata": {
"id": "9ZgXf5Je9N4x",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "e66ea769-9bca-4b4d-b327-62aa2624d51b"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change': '2.39%', 'pct_good_lcp_change': '3.06%', 'pct_good_cls_change': '1.95%', 'pct_good_inp_change': '0.73%'}\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"final_results = calculate_feature_impact_by_device(cwv_changes_active_sites_by_device, cwv_changes_inactive_sites_by_device)\n",
"print(final_results)\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "20c210b9-a94d-467d-d65d-ff3410e0fed0",
"id": "9LPwX7qGP3Uw"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '2.56%', 'pct_good_lcp_change_mobile': '3.32%', 'pct_good_cls_change_mobile': '2.71%', 'pct_good_inp_change_mobile': '1.44%', 'pct_good_cwv_change_desktop': '2.30%', 'pct_good_lcp_change_desktop': '2.52%', 'pct_good_cls_change_desktop': '1.04%', 'pct_good_inp_change_desktop': '0.24%'}\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"results = query_cwv_compare_feature_to_baseline( 'speculation-rules', '2024-03-01', '2024-08-01' )\n",
"print(results)"
],
"metadata": {
"id": "1fHDFRQVRpfH",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "88d4fef4-3987-489f-c662-412880e9f134"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change': '2.42%', 'pct_good_lcp_change': '3.06%', 'pct_good_cls_change': '1.98%', 'pct_good_inp_change': '0.82%', 'feature_active_sites': 7203, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%'}\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"results = query_cwv_compare_feature_to_baseline_by_device( 'speculation-rules', '2024-03-01', '2024-08-01' )\n",
"print(results)"
],
"metadata": {
"id": "68oaX4ByWQL5",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "46cbae13-ba83-4ba1-9858-1a018e5ddcbe"
},
"execution_count": null,
"outputs": [
{
"metadata": {
"tags": null
},
"name": "stdout",
"output_type": "stream",
"text": [
"{'pct_good_cwv_change_mobile': '2.60%', 'pct_good_lcp_change_mobile': '3.39%', 'pct_good_cls_change_mobile': '2.75%', 'pct_good_inp_change_mobile': '1.52%', 'pct_good_cwv_change_desktop': '2.39%', 'pct_good_lcp_change_desktop': '2.55%', 'pct_good_cls_change_desktop': '1.09%', 'pct_good_inp_change_desktop': '0.23%', 'feature_active_sites': 7203, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-03-01', 'after_date': '2024-08-01', 'generator_tag': 'speculation-rules', 'cwv_changes_active_sites': [0.07083488781153219, 0.06909267366979688, 0.03922572086938758, 0.08127807033918977, 0.053894219906421015, 0.05999325206027517, 0.026095263217394526, 0.0027941094172748704], 'cwv_changes_inactive_sites': [0.04482710642311871, 0.03515486832303927, 0.011717077581957724, 0.06609143742833012, 0.02997313470714902, 0.034507587443841525, 0.015172879563230546, 0.0005163767055975654]}\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"# show results as a table\n",
"import pandas as pd\n",
"df = pd.DataFrame([results])\n",
"df"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 160
},
"id": "ch7RupSn_2Vp",
"outputId": "b536443f-07ee-425a-c788-be211957564d"
},
"execution_count": null,
"outputs": [
{
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"summary": "{\n \"name\": \"df\",\n \"rows\": 1,\n \"fields\": [\n {\n \"column\": \"pct_good_cwv_change_mobile\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"2.60%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pct_good_lcp_change_mobile\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"3.39%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pct_good_cls_change_mobile\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"2.75%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pct_good_inp_change_mobile\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"1.52%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pct_good_cwv_change_desktop\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"2.39%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pct_good_lcp_change_desktop\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"2.55%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pct_good_cls_change_desktop\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"1.09%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pct_good_inp_change_desktop\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"0.23%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"feature_active_sites\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": null,\n \"min\": 7203,\n \"max\": 7203,\n \"num_unique_values\": 1,\n \"samples\": [\n 7203\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"feature_inactive_sites\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": null,\n \"min\": 50000,\n \"max\": 50000,\n \"num_unique_values\": 1,\n \"samples\": [\n 50000\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"also_active_at_before\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"0.00%\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"before_date\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"2024-03-01\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"after_date\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"2024-08-01\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"generator_tag\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"speculation-rules\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cwv_changes_active_sites\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cwv_changes_inactive_sites\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
"type": "dataframe",
"variable_name": "df"
},
"application/vnd.google.colaboratory.module+javascript": "\n import \"https://ssl.gstatic.com/colaboratory/data_table/54ded79905fc55e4/data_table.js\";\n\n const table = window.createDataTable({\n data: [[{\n 'v': 0,\n 'f': \"0\",\n },\n\"2.60%\",\n\"3.39%\",\n\"2.75%\",\n\"1.52%\",\n\"2.39%\",\n\"2.55%\",\n\"1.09%\",\n\"0.23%\",\n{\n 'v': 7203,\n 'f': \"7203\",\n },\n{\n 'v': 50000,\n 'f': \"50000\",\n },\n\"0.00%\",\n\"2024-03-01\",\n\"2024-08-01\",\n\"speculation-rules\",\n[0.07083488781153219, 0.06909267366979688, 0.03922572086938758, 0.08127807033918977, 0.053894219906421015, 0.05999325206027517, 0.026095263217394526, 0.0027941094172748704],\n[0.04482710642311871, 0.03515486832303927, 0.011717077581957724, 0.06609143742833012, 0.02997313470714902, 0.034507587443841525, 0.015172879563230546, 0.0005163767055975654]]],\n columns: [[\"number\", \"index\"], [\"string\", \"pct_good_cwv_change_mobile\"], [\"string\", \"pct_good_lcp_change_mobile\"], [\"string\", \"pct_good_cls_change_mobile\"], [\"string\", \"pct_good_inp_change_mobile\"], [\"string\", \"pct_good_cwv_change_desktop\"], [\"string\", \"pct_good_lcp_change_desktop\"], [\"string\", \"pct_good_cls_change_desktop\"], [\"string\", \"pct_good_inp_change_desktop\"], [\"number\", \"feature_active_sites\"], [\"number\", \"feature_inactive_sites\"], [\"string\", \"also_active_at_before\"], [\"string\", \"before_date\"], [\"string\", \"after_date\"], [\"string\", \"generator_tag\"], [\"string\", \"cwv_changes_active_sites\"], [\"string\", \"cwv_changes_inactive_sites\"]],\n columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n rowsPerPage: 25,\n helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n suppressOutputScrolling: true,\n minimumWidth: undefined,\n });\n\n function appendQuickchartButton(parentElement) {\n let quickchartButtonContainerElement = document.createElement('div');\n quickchartButtonContainerElement.innerHTML = `\n<div id=\"df-0e402274-b5e4-4cea-b901-fdbdd5094abd\">\n <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0e402274-b5e4-4cea-b901-fdbdd5094abd')\"\n title=\"Suggest charts\"\n style=\"display:none;\">\n \n<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n width=\"24px\">\n <g>\n <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n </g>\n</svg>\n </button>\n \n<style>\n .colab-df-quickchart {\n --bg-color: #E8F0FE;\n --fill-color: #1967D2;\n --hover-bg-color: #E2EBFA;\n --hover-fill-color: #174EA6;\n --disabled-fill-color: #AAA;\n --disabled-bg-color: #DDD;\n }\n\n [theme=dark] .colab-df-quickchart {\n --bg-color: #3B4455;\n --fill-color: #D2E3FC;\n --hover-bg-color: #434B5C;\n --hover-fill-color: #FFFFFF;\n --disabled-bg-color: #3B4455;\n --disabled-fill-color: #666;\n }\n\n .colab-df-quickchart {\n background-color: var(--bg-color);\n border: none;\n border-radius: 50%;\n cursor: pointer;\n display: none;\n fill: var(--fill-color);\n height: 32px;\n padding: 0;\n width: 32px;\n }\n\n .colab-df-quickchart:hover {\n background-color: var(--hover-bg-color);\n box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n fill: var(--button-hover-fill-color);\n }\n\n .colab-df-quickchart-complete:disabled,\n .colab-df-quickchart-complete:disabled:hover {\n background-color: var(--disabled-bg-color);\n fill: var(--disabled-fill-color);\n box-shadow: none;\n }\n\n .colab-df-spinner {\n border: 2px solid var(--fill-color);\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n animation:\n spin 1s steps(1) infinite;\n }\n\n @keyframes spin {\n 0% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n border-left-color: var(--fill-color);\n }\n 20% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 30% {\n border-color: transparent;\n border-left-color: var(--fill-color);\n border-top-color: var(--fill-color);\n border-right-color: var(--fill-color);\n }\n 40% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-top-color: var(--fill-color);\n }\n 60% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n }\n 80% {\n border-color: transparent;\n border-right-color: var(--fill-color);\n border-bottom-color: var(--fill-color);\n }\n 90% {\n border-color: transparent;\n border-bottom-color: var(--fill-color);\n }\n }\n</style>\n\n <script>\n async function quickchart(key) {\n const quickchartButtonEl =\n document.querySelector('#' + key + ' button');\n quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n quickchartButtonEl.classList.add('colab-df-spinner');\n try {\n const charts = await google.colab.kernel.invokeFunction(\n 'suggestCharts', [key], {});\n } catch (error) {\n console.error('Error during call to suggestCharts:', error);\n }\n quickchartButtonEl.classList.remove('colab-df-spinner');\n quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n }\n (() => {\n let quickchartButtonEl =\n document.querySelector('#df-0e402274-b5e4-4cea-b901-fdbdd5094abd button');\n quickchartButtonEl.style.display =\n google.colab.kernel.accessAllowed ? 'block' : 'none';\n })();\n </script>\n</div>`;\n parentElement.appendChild(quickchartButtonContainerElement);\n }\n\n appendQuickchartButton(table);\n ",
"text/html": [
"\n",
" <div id=\"df-7e2e0d92-cc37-439e-931d-2918003059ee\" class=\"colab-df-container\">\n",
" <div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>pct_good_cwv_change_mobile</th>\n",
" <th>pct_good_lcp_change_mobile</th>\n",
" <th>pct_good_cls_change_mobile</th>\n",
" <th>pct_good_inp_change_mobile</th>\n",
" <th>pct_good_cwv_change_desktop</th>\n",
" <th>pct_good_lcp_change_desktop</th>\n",
" <th>pct_good_cls_change_desktop</th>\n",
" <th>pct_good_inp_change_desktop</th>\n",
" <th>feature_active_sites</th>\n",
" <th>feature_inactive_sites</th>\n",
" <th>also_active_at_before</th>\n",
" <th>before_date</th>\n",
" <th>after_date</th>\n",
" <th>generator_tag</th>\n",
" <th>cwv_changes_active_sites</th>\n",
" <th>cwv_changes_inactive_sites</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>2.60%</td>\n",
" <td>3.39%</td>\n",
" <td>2.75%</td>\n",
" <td>1.52%</td>\n",
" <td>2.39%</td>\n",
" <td>2.55%</td>\n",
" <td>1.09%</td>\n",
" <td>0.23%</td>\n",
" <td>7203</td>\n",
" <td>50000</td>\n",
" <td>0.00%</td>\n",
" <td>2024-03-01</td>\n",
" <td>2024-08-01</td>\n",
" <td>speculation-rules</td>\n",
" <td>[0.07083488781153219, 0.06909267366979688, 0.0...</td>\n",
" <td>[0.04482710642311871, 0.03515486832303927, 0.0...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>\n",
" <div class=\"colab-df-buttons\">\n",
"\n",
" <div class=\"colab-df-container\">\n",
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7e2e0d92-cc37-439e-931d-2918003059ee')\"\n",
" title=\"Convert this dataframe to an interactive table.\"\n",
" style=\"display:none;\">\n",
"\n",
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
" </svg>\n",
" </button>\n",
"\n",
" <style>\n",
" .colab-df-container {\n",
" display:flex;\n",
" gap: 12px;\n",
" }\n",
"\n",
" .colab-df-convert {\n",
" background-color: #E8F0FE;\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: #1967D2;\n",
" height: 32px;\n",
" padding: 0 0 0 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-convert:hover {\n",
" background-color: #E2EBFA;\n",
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: #174EA6;\n",
" }\n",
"\n",
" .colab-df-buttons div {\n",
" margin-bottom: 4px;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert {\n",
" background-color: #3B4455;\n",
" fill: #D2E3FC;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert:hover {\n",
" background-color: #434B5C;\n",
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
" fill: #FFFFFF;\n",
" }\n",
" </style>\n",
"\n",
" <script>\n",
" const buttonEl =\n",
" document.querySelector('#df-7e2e0d92-cc37-439e-931d-2918003059ee button.colab-df-convert');\n",
" buttonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
"\n",
" async function convertToInteractive(key) {\n",
" const element = document.querySelector('#df-7e2e0d92-cc37-439e-931d-2918003059ee');\n",
" const dataTable =\n",
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
" [key], {});\n",
" if (!dataTable) return;\n",
"\n",
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
" + ' to learn more about interactive tables.';\n",
" element.innerHTML = '';\n",
" dataTable['output_type'] = 'display_data';\n",
" await google.colab.output.renderOutput(dataTable, element);\n",
" const docLink = document.createElement('div');\n",
" docLink.innerHTML = docLinkHtml;\n",
" element.appendChild(docLink);\n",
" }\n",
" </script>\n",
" </div>\n",
"\n",
"\n",
" <div id=\"id_82e24f8f-0a87-4fe0-8a91-f56fb2b3a9dd\">\n",
" <style>\n",
" .colab-df-generate {\n",
" background-color: #E8F0FE;\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: #1967D2;\n",
" height: 32px;\n",
" padding: 0 0 0 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-generate:hover {\n",
" background-color: #E2EBFA;\n",
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: #174EA6;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-generate {\n",
" background-color: #3B4455;\n",
" fill: #D2E3FC;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-generate:hover {\n",
" background-color: #434B5C;\n",
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
" fill: #FFFFFF;\n",
" }\n",
" </style>\n",
" <button class=\"colab-df-generate\" onclick=\"generateWithVariable('df')\"\n",
" title=\"Generate code using this dataframe.\"\n",
" style=\"display:none;\">\n",
"\n",
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
" width=\"24px\">\n",
" <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
" </svg>\n",
" </button>\n",
" <script>\n",
" (() => {\n",
" const buttonEl =\n",
" document.querySelector('#id_82e24f8f-0a87-4fe0-8a91-f56fb2b3a9dd button.colab-df-generate');\n",
" buttonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
"\n",
" buttonEl.onclick = () => {\n",
" google.colab.notebook.generateWithVariable('df');\n",
" }\n",
" })();\n",
" </script>\n",
" </div>\n",
"\n",
" </div>\n",
" </div>\n"
],
"text/plain": [
" pct_good_cwv_change_mobile pct_good_lcp_change_mobile \\\n",
"0 2.60% 3.39% \n",
"\n",
" pct_good_cls_change_mobile pct_good_inp_change_mobile \\\n",
"0 2.75% 1.52% \n",
"\n",
" pct_good_cwv_change_desktop pct_good_lcp_change_desktop \\\n",
"0 2.39% 2.55% \n",
"\n",
" pct_good_cls_change_desktop pct_good_inp_change_desktop \\\n",
"0 1.09% 0.23% \n",
"\n",
" feature_active_sites feature_inactive_sites also_active_at_before \\\n",
"0 7203 50000 0.00% \n",
"\n",
" before_date after_date generator_tag \\\n",
"0 2024-03-01 2024-08-01 speculation-rules \n",
"\n",
" cwv_changes_active_sites \\\n",
"0 [0.07083488781153219, 0.06909267366979688, 0.0... \n",
"\n",
" cwv_changes_inactive_sites \n",
"0 [0.04482710642311871, 0.03515486832303927, 0.0... "
]
},
"execution_count": 31,
"metadata": {},
"output_type": "execute_result"
}
]
},
{
"cell_type": "markdown",
"source": [
"# Performance by Feature Queries\n",
"\n",
"## Notes\n",
"* `inactive_to_active_percent` Shows the percent of feature-active sites at the after date that __were also feature-active__ at the before date. Ideally this should be 0, meaning none of the sites had the feature ative at the before date.\n"
],
"metadata": {
"id": "cLMOaWak2-e3"
}
},
{
"cell_type": "markdown",
"source": [
"## Speculative Loading"
],
"metadata": {
"id": "KSgLpTio3Dpj"
}
},
{
"cell_type": "code",
"source": [
"latest_dataset = '2024-09-01'"
],
"metadata": {
"id": "6mYeQTz-xkyN"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"#@markdown ### Compare before/after\n",
"#@markdown * `before_date`: a date to analyze where the feature is not enabled\n",
"#@markdown * `after_date`: a date to analyze where the feature is enabled\n",
"#@markdown * `generator`: the generator tag indicating a feature is installed\n",
"#@markdown * `inactive_to_active_percent` Shows the percent of feature-active sites at the after date that were also feature-active at the before date.\n",
"\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'speculation-rules'\n",
"\n",
"results_sl = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_sl)"
],
"metadata": {
"id": "QWxVV0iZ3afy",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "5d398dbc-0184-4425-f635-1e6eee110e03"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '2.04%', 'pct_good_lcp_change_mobile': '2.00%', 'pct_good_cls_change_mobile': '2.00%', 'pct_good_inp_change_mobile': '1.33%', 'pct_good_cwv_change_desktop': '2.25%', 'pct_good_lcp_change_desktop': '2.55%', 'pct_good_cls_change_desktop': '1.59%', 'pct_good_inp_change_desktop': '-0.24%', 'feature_active_sites': 9091, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-03-01', 'after_date': '2024-09-01', 'generator_tag': 'speculation-rules', 'cwv_changes_active_sites': [0.0709573638372546, 0.05973671280401477, 0.03339427392262129, 0.09929041920330417, 0.05443899666326818, 0.06367646788742376, 0.02837772363253499, -0.0026272786983606577], 'cwv_changes_inactive_sites': [0.050594884631586345, 0.03974626020874489, 0.013417786565544043, 0.08595551937012391, 0.031957622667799446, 0.03816732623091601, 0.012432157707554903, -0.00018933025299994188]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Image Prioitizer"
],
"metadata": {
"id": "vYf6nRjU44pU"
}
},
{
"cell_type": "code",
"source": [
"# image-prioritizer\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'image-prioritizer'\n",
"\n",
"results_ip = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_ip)"
],
"metadata": {
"id": "Vrk1yAp15u1C",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "bfe6b1b2-d9ca-422a-8e28-7c9a2e2e9d5b"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '1.83%', 'pct_good_lcp_change_mobile': '2.63%', 'pct_good_cls_change_mobile': '2.02%', 'pct_good_inp_change_mobile': '1.07%', 'pct_good_cwv_change_desktop': '3.63%', 'pct_good_lcp_change_desktop': '3.64%', 'pct_good_cls_change_desktop': '2.66%', 'pct_good_inp_change_desktop': '-0.46%', 'feature_active_sites': 3057, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-04-01', 'after_date': '2024-09-01', 'generator_tag': 'image-prioritizer', 'cwv_changes_active_sites': [0.06355639390247644, 0.066255044344453, 0.025530920421860004, 0.08869432502523766, 0.06324726213202053, 0.06865713642895399, 0.034364047044162294, -0.004424134395824675], 'cwv_changes_inactive_sites': [0.04529243087001833, 0.039932468951196454, 0.005380117484358293, 0.07796061197519022, 0.02694167020240129, 0.03227752930248928, 0.007753865162267282, 0.00015203821501585946]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Embed Optimizer\n",
"\n",
"\n"
],
"metadata": {
"id": "Q0OKwj0847rt"
}
},
{
"cell_type": "code",
"source": [
"# embed-optimizer\n",
"before_date = '2024-07-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'embed-optimizer'\n",
"\n",
"results_eo = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_eo)"
],
"metadata": {
"id": "Deb8H7gl5qYH",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "94442b25-a59c-47bb-f8dd-3c361d36c04e"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '-5.69%', 'pct_good_lcp_change_mobile': '-4.23%', 'pct_good_cls_change_mobile': '-0.57%', 'pct_good_inp_change_mobile': '0.11%', 'pct_good_cwv_change_desktop': '-0.83%', 'pct_good_lcp_change_desktop': '-4.91%', 'pct_good_cls_change_desktop': '0.91%', 'pct_good_inp_change_desktop': '-0.37%', 'feature_active_sites': 4908, 'feature_inactive_sites': 50000, 'also_active_at_before': '5906.68%', 'before_date': '2024-07-01', 'after_date': '2024-09-01', 'generator_tag': 'embed-optimizer', 'cwv_changes_active_sites': [-0.041306793074358306, -0.031142749330665964, -0.009310373802833838, 0.03288217754941303, -0.00780285351372112, -0.04850123311220522, 0.010024549239471159, -0.005046772644276998], 'cwv_changes_inactive_sites': [0.01555233539948292, 0.011153826487459773, -0.0036123086688277795, 0.03178291363875008, 0.0005148192916936134, 0.0005865276079072279, 0.0008919736799152256, -0.0013168643303140826]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Modern Image Formats"
],
"metadata": {
"id": "U0sMVoPs4_Xh"
}
},
{
"cell_type": "code",
"source": [
"# webp-uploads\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'webp-uploads'\n",
"\n",
"results_mi = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_mi)"
],
"metadata": {
"id": "1tNlKpIU5zVS",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "81a9fd42-9de8-41b9-d1bc-a0491b4efb05"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '1.86%', 'pct_good_lcp_change_mobile': '1.26%', 'pct_good_cls_change_mobile': '2.31%', 'pct_good_inp_change_mobile': '1.57%', 'pct_good_cwv_change_desktop': '2.31%', 'pct_good_lcp_change_desktop': '2.14%', 'pct_good_cls_change_desktop': '1.45%', 'pct_good_inp_change_desktop': '0.15%', 'feature_active_sites': 16148, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-01-01', 'after_date': '2024-09-01', 'generator_tag': 'webp-uploads', 'cwv_changes_active_sites': [0.09053611980768622, 0.06681956845397613, 0.04422785761234971, 0.13775801636582619, 0.053067897464070646, 0.055216839004633256, 0.029493570947297654, 0.003979401208744959], 'cwv_changes_inactive_sites': [0.0719599926198663, 0.05425106123086576, 0.021089317940479635, 0.1220972067245063, 0.030003443405208163, 0.033798004781138036, 0.015025429495077125, 0.002452523250483174]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Enhanced Image Sizes"
],
"metadata": {
"id": "xT4NB_OY5Jld"
}
},
{
"cell_type": "code",
"source": [
"# auto-sizes\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'auto-sizes'\n",
"\n",
"results_is = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_is)"
],
"metadata": {
"id": "2dQQhAFC5mnw",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "2018309b-32bc-422e-a2c4-d79746a4740a"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '0.77%', 'pct_good_lcp_change_mobile': '-0.22%', 'pct_good_cls_change_mobile': '2.33%', 'pct_good_inp_change_mobile': '2.50%', 'pct_good_cwv_change_desktop': '3.02%', 'pct_good_lcp_change_desktop': '1.37%', 'pct_good_cls_change_desktop': '3.14%', 'pct_good_inp_change_desktop': '0.18%', 'feature_active_sites': 5633, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-03-01', 'after_date': '2024-09-01', 'generator_tag': 'auto-sizes', 'cwv_changes_active_sites': [0.06442163880919188, 0.048145394148988996, 0.0325715565983854, 0.11997871774408086, 0.06243687002187076, 0.05522724145910157, 0.041617845644969065, 0.001961164045756192], 'cwv_changes_inactive_sites': [0.05675288022675129, 0.05037639877576028, 0.009301400069805288, 0.09498100979108803, 0.03222344332123689, 0.041554902302256336, 0.010225594823027317, 0.00014161616508323593]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Performant Translations"
],
"metadata": {
"id": "SyUbptUIQ-R0"
}
},
{
"cell_type": "code",
"source": [
"# performant-translations\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'performant-translations'\n",
"\n",
"results_is = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_is)"
],
"metadata": {
"id": "RfVL5TDlRA-h"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Web Worker Offloading"
],
"metadata": {
"id": "dFBb2AfYReqb"
}
},
{
"cell_type": "code",
"source": [
"# web-worker-offloading\n",
"before_date = '2024-07-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'web-worker-offloading'\n",
"\n",
"results_is = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_is)"
],
"metadata": {
"id": "Y2GyJeDDRg99",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "2ffbe55d-168c-4906-bfac-e281544e2092"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '0.94%', 'pct_good_lcp_change_mobile': '-1.22%', 'pct_good_cls_change_mobile': '5.04%', 'pct_good_inp_change_mobile': '1.29%', 'pct_good_cwv_change_desktop': '0.55%', 'pct_good_lcp_change_desktop': '-1.35%', 'pct_good_cls_change_desktop': '4.56%', 'pct_good_inp_change_desktop': '0.10%', 'feature_active_sites': 601, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-07-01', 'after_date': '2024-11-01', 'generator_tag': 'web-worker-offloading', 'cwv_changes_active_sites': [0.026853975535168162, 0.0001429762897652842, 0.046814342865609704, 0.04866273605058535, 0.005261578120310084, -0.007205877183732334, 0.039323973650941535, 0.000566951860632936, date device origins pct_eligible_origins_with_good_cwv \\\n",
"1 2024-07-01 mobile 312 0.275229 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"1 0.412844 0.744118 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"1 0.614458 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"2 2024-11-01 mobile 363 0.302083 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"2 0.412987 0.790932 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"2 0.663121 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"0 2024-07-01 desktop 227 0.37788 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"0 0.53211 0.588496 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"0 0.97191 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"3 2024-11-01 desktop 266 0.383142 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"3 0.524904 0.62782 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"3 0.972477 ], 'cwv_changes_inactive_sites': [0.01742855793811049, 0.012363291471110371, -0.003585737185353821, 0.03574913070510788, -0.00026561188097889987, 0.00628024071174782, -0.006276333951685142, -0.00045329309517982974, date device origins pct_eligible_origins_with_good_cwv \\\n",
"1 2024-07-01 mobile 38975 0.390947 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"1 0.536264 0.785201 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"1 0.755696 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"2 2024-11-01 mobile 41797 0.408376 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"2 0.548627 0.781615 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"2 0.791445 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"0 2024-07-01 desktop 31126 0.46937 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"0 0.640656 0.693395 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"0 0.987947 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"3 2024-11-01 desktop 33969 0.469105 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"3 0.646936 0.687118 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"3 0.987494 ]}\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"latest_dataset = '2024-11-01'"
],
"metadata": {
"id": "GnSlKJqWZRFF"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Embed Optimizer plus Optimization detective\n"
],
"metadata": {
"id": "bYE8yA7d_8dv"
}
},
{
"cell_type": "code",
"source": [
"# this will use a slightly different approach because we are looking for sites with two features enabled.\n",
"before_date = '2024-07-01'\n",
"after_date = latest_dataset\n",
"generator_tag1 = 'optimization-detective'\n",
"generator_tag2 = 'embed-optimizer'\n",
"\n",
"results_is = query_cwv_compare_feature_to_baseline_by_device_multiple_tags(generator_tag1, generator_tag2, before_date, after_date)\n",
"print(results_is)\n",
"\n"
],
"metadata": {
"id": "huI-O2YMAAuO",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "6c55fe80-7933-45a6-c675-0ae583e33250"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '0.20%', 'pct_good_lcp_change_mobile': '1.09%', 'pct_good_cls_change_mobile': '2.08%', 'pct_good_inp_change_mobile': '-0.48%', 'pct_good_cwv_change_desktop': '4.39%', 'pct_good_lcp_change_desktop': '2.12%', 'pct_good_cls_change_desktop': '3.70%', 'pct_good_inp_change_desktop': '-0.22%', 'feature_active_sites': 3523, 'feature_inactive_sites': 50000, 'also_active_at_before': '1481.69%', 'before_date': '2024-07-01', 'after_date': '2024-11-01', 'generator_tag1': 'optimization-detective', 'generator_tag2': 'embed-optimizer', 'cwv_changes_active_sites': [0.022084835266597713, 0.026946373942071167, 0.014579789123231879, 0.03380411086302515, 0.04506952879183501, 0.031181994226598397, 0.032788667906710534, -0.0035211267605633756, date device origins pct_eligible_origins_with_good_cwv \\\n",
"0 2024-07-01 mobile 1715 0.260748 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"0 0.363788 0.719718 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"0 0.694842 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"2 2024-11-01 mobile 2066 0.282833 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"2 0.390735 0.734297 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"2 0.728647 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"1 2024-07-01 desktop 1212 0.323051 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"1 0.479897 0.614108 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"1 0.985915 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"3 2024-11-01 desktop 1456 0.36812 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"3 0.511079 0.646897 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"3 0.982394 ], 'cwv_changes_inactive_sites': [0.020120224450941193, 0.016093714735728737, -0.006181671591478977, 0.038600244501898584, 0.0012012495827020686, 0.009982428443944924, -0.004181185090754269, -0.0013615621774009101, date device origins pct_eligible_origins_with_good_cwv \\\n",
"0 2024-07-01 mobile 40668 0.378968 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"0 0.539324 0.77524 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"0 0.733711 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"3 2024-11-01 mobile 43404 0.399088 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"3 0.555418 0.769058 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"3 0.772312 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"1 2024-07-01 desktop 33696 0.468656 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"1 0.643538 0.688438 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"1 0.986965 , date device origins pct_eligible_origins_with_good_cwv \\\n",
"2 2024-11-01 desktop 36528 0.469858 \n",
"\n",
" pct_eligible_origins_with_good_lcp pct_eligible_origins_with_good_cls \\\n",
"2 0.653521 0.684257 \n",
"\n",
" pct_eligible_origins_with_good_inp \n",
"2 0.985604 ]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"# Graveyard"
],
"metadata": {
"id": "BoVOOEMO46eL"
}
},
{
"cell_type": "code",
"source": [
"from google.cloud import bigquery\n",
"import pandas as pd\n",
"\n",
"def analyse_performance_impact_of_feature(generator_tag, after_date=None):\n",
"\n",
" client = bigquery.Client()\n",
"\n",
" # 1. Identify Active and Non-Active Sites\n",
" active_sites_query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING)\n",
" RETURNS STRING\n",
" LANGUAGE js\n",
" AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith('$.{{generator}}'));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
" ''';\n",
"\n",
" SELECT * FROM\n",
" (\n",
" SELECT\n",
" page AS origin,\n",
" client AS device,\n",
" date,\n",
" getFeature(payload) as feature\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = 'WordPress'\n",
" AND date = PARSE_DATE('%Y_%m_%d', '{after_date}' )\n",
" AND is_root_page = TRUE\n",
" )\n",
" WHERE feature != \"\"\n",
" LIMIT 10000\n",
" \"\"\"\n",
"\n",
" non_active_sites_query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING)\n",
" RETURNS STRING\n",
" LANGUAGE js\n",
" AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith('$.{{generator}}'));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
" ''';\n",
"\n",
" SELECT * FROM\n",
" (\n",
" SELECT\n",
" page AS origin,\n",
" client AS device,\n",
" date,\n",
" getFeature(payload) as feature\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = 'WordPress'\n",
" AND date = PARSE_DATE('%Y_%m_%d', '{after_date}' )\n",
" AND is_root_page = TRUE\n",
" )\n",
" WHERE feature = \"\"\n",
" LIMIT 10000\n",
" \"\"\"\n",
"\n",
" active_sites_df = client.query(active_sites_query).to_dataframe()\n",
" non_active_sites_df = client.query(non_active_sites_query).to_dataframe()\n",
"\n",
"\n",
" # 2. Collect CWV Data and Calculate Baselines\n",
" def calculate_cwv_stats(df, active=False):\n",
" cwv_metrics = ['largest_contentful_paint', 'first_input_delay', 'cumulative_layout_shift']\n",
" all_cwv_data = []\n",
"\n",
" for _, row in df.iterrows():\n",
" for month_offset in range(14): # collect data for the past 14 months\n",
" query_date = row['parse_date'] - pd.DateOffset(months=month_offset)\n",
" query_string = f\"\"\"\n",
"\n",
" SELECT origin,\n",
" IF(fast_lcp_count IS NOT NULL, fast_lcp_count / (fast_lcp_count + avg_lcp_count + slow_lcp_count), NULL) AS lcp_pass_rate,\n",
" IF(fast_fid_count IS NOT NULL, fast_fid_count / (fast_fid_count + avg_fid_count + slow_fid_count), NULL) AS fid_pass_rate,\n",
" IF(small_cls_count IS NOT NULL, small_cls_count / (small_cls_count + medium_cls_count + large_cls_count), NULL) AS cls_pass_rate,\n",
" IF(good_inp_count IS NOT NULL, good_inp_count / (good_inp_count + ok_inp_count + poor_inp_count), NULL) AS inp_pass_rate\n",
" FROM `chrome-ux-report.materialized.device_summary`\n",
" WHERE origin = '{row[\"url\"]}'\n",
" AND DATE_TRUNC(date, MONTH) = DATE_TRUNC(DATE('{query_date}'), MONTH)\n",
" \"\"\"\n",
"\n",
" cwv_data = client.query(query_string).to_dataframe()\n",
" if not cwv_data.empty:\n",
" cwv_data['month'] = query_date # include the month for each metric\n",
" cwv_data['active'] = active\n",
" all_cwv_data.append(cwv_data)\n",
"\n",
" return pd.concat(all_cwv_data)\n",
"\n",
" # 3. Identify Feature Activation and Calculate Impact\n",
" active_cwv_stats = calculate_cwv_stats(active_sites_df, active=True)\n",
" non_active_cwv_stats = calculate_cwv_stats(non_active_sites_df)\n",
"\n",
" def calculate_feature_impact(active_cwv_stats):\n",
" impact_data = []\n",
" for url, group in active_cwv_stats.groupby('origin'):\n",
" activation_month = group[group['active'] == True]['month'].min()\n",
" if activation_month is not pd.NaT:\n",
" before_activation = group[group['month'] == activation_month - pd.DateOffset(months=1)]\n",
" after_activation = group[group['month'] == activation_month + pd.DateOffset(months=1)]\n",
"\n",
" if not before_activation.empty and not after_activation.empty:\n",
" impact = after_activation.mean() - before_activation.mean()\n",
" impact['url'] = url\n",
" impact_data.append(impact)\n",
"\n",
" return pd.DataFrame(impact_data)\n",
"\n",
" feature_impact_df = calculate_feature_impact(active_cwv_stats)\n",
"\n",
" return feature_impact_df"
],
"metadata": {
"id": "kD6TC3jpXI2W"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"analyse_performance_impact_of_feature('speculation-rules', '2024_06_01')\n"
],
"metadata": {
"id": "QrL9fImAtqKX"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"id": "rDn6hWyywriY"
}
},
{
"cell_type": "markdown",
"source": [
"## Example using Speculation Rules"
],
"metadata": {
"id": "CT9vfgXOu0UA"
}
},
{
"cell_type": "code",
"source": [
"#@markdown ### Compare speculation rules before/after\n",
"results = query_cwv_compare_feature_to_baseline( 'speculation-rules', '2024-03-01', '2024-06-01' )\n",
"print(results)"
],
"metadata": {
"id": "VBtebScA8k-m"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### Helper to compare feature before / after"
],
"metadata": {
"id": "8o_CYabIgMwI"
}
},
{
"cell_type": "code",
"source": [
"# Restrict the query to on the from and to dates, and without and with a specific generator tag indicating a feature is active\n",
"def query_cwv_compare_dates_by_generator(technology, from_date, to_date, generator):\n",
" \"\"\"\n",
" Queries Core Web Vitals metrics from HTTP Archive.\n",
"\n",
" Args:\n",
" technology (str): The technology to filter by (e.g., 'WordPress').\n",
" from_date (str): The start date in YYYY_MM_DD format.\n",
" to_date (str): The end date in YYYY_MM_DD format.\n",
" generator (str): The generator tag indicating a feature is installed.\n",
" Returns:\n",
" A BigQuery query result object containing CWV data.\n",
" \"\"\"\n",
" # set to_date if not passed\n",
" if not to_date:\n",
" to_date = latest_dataset\n",
"\n",
" # set from date if not passed\n",
" if not from_date:\n",
" from_date = (datetime.strptime(to_date, '%Y_%m_%d').date() - timedelta(days=365)).strftime('%Y_%m_%d')\n",
"\n",
"\n",
" query = f\"\"\"\n",
" CREATE TEMP FUNCTION getFeature(payload STRING)\n",
"RETURNS STRING\n",
"LANGUAGE js\n",
"AS '''\n",
" try {{\n",
" var $ = JSON.parse(payload);\n",
" var almanac = JSON.parse($._almanac);\n",
" var generators = almanac['meta-nodes'].nodes.find(node => node.name == 'generator' && node.content.startsWith('$.{{generator}}'));\n",
" if ( generators.length == 0 ) {{\n",
" return \"\";\n",
" }}\n",
" var content = generators.content;\n",
" return content;\n",
" }} catch (e) {{\n",
" return \"\";\n",
" }}\n",
"''';\n",
"\n",
"CREATE TEMP FUNCTION IS_GOOD(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good / (good + needs_improvement + poor) >= 0.75\n",
" );\n",
"\n",
" CREATE TEMP FUNCTION IS_NON_ZERO(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64)\n",
" RETURNS BOOL\n",
" AS (\n",
" good + needs_improvement + poor > 0\n",
" );\n",
" WITH sites_available_on_from_or_to_date AS (\n",
" SELECT\n",
" page AS origin,\n",
" client AS device,\n",
" date,\n",
" getFeature(payload) as feature\n",
" FROM\n",
" `httparchive.all.pages`,\n",
" UNNEST(technologies) AS technologies\n",
" WHERE\n",
" technologies.technology = '{technology}'\n",
" AND (\n",
" ( date = PARSE_DATE('%Y_%m_%d', '{from_date}') ) OR\n",
" ( date = PARSE_DATE('%Y_%m_%d', '{to_date}' ) )\n",
" )\n",
" AND is_root_page = TRUE\n",
"\n",
" ),\n",
" site_on_to_date_with_feature AS (\n",
" SELECT\n",
" origin,\n",
" device,\n",
" date,\n",
" feature\n",
" FROM sites_available_on_from_or_to_date\n",
" WHERE date = PARSE_DATE('%Y_%m_%d', '{to_date}' )\n",
" AND feature != \"\"\n",
" ),\n",
" sites_on_from_date_without_feature AS (\n",
" SELECT\n",
" origin,\n",
" device,\n",
" date,\n",
" feature\n",
" FROM\n",
" sites_available_on_from_or_to_date\n",
" WHERE\n",
" date = PARSE_DATE('%Y_%m_%d', '{from_date}')\n",
" AND feature = \"\"\n",
" ),\n",
" sites_available_for_both_from_and_to AS (\n",
" SELECT\n",
" sites_available_on_from_or_to_date.origin\n",
" FROM\n",
" sites_available_on_from_or_to_date\n",
" JOIN\n",
" sites_on_from_date_without_feature\n",
" ON (sites_available_on_from_or_to_date.origin = sites_on_from_date_without_feature.origin)\n",
" ),\n",
" cwv_metrics_for_origins AS (\n",
" SELECT\n",
" cwvs.date AS date,\n",
" site_on_to_date_with_feature.origin AS origin,\n",
" # Device phone or tablet as mobile, otherwise desktop.\n",
" IF(cwvs.device = 'phone' OR cwvs.device = 'tablet', 'mobile', cwvs.device) AS device,\n",
" IS_NON_ZERO(fast_inp, avg_inp, slow_inp) AS any_inp,\n",
" IS_GOOD(fast_inp, avg_inp, slow_inp) AS good_inp,\n",
" IS_NON_ZERO(small_cls, medium_cls, large_cls) AS any_cls,\n",
" IS_GOOD(small_cls, medium_cls, large_cls) AS good_cls,\n",
" IS_NON_ZERO(fast_lcp, avg_lcp, slow_lcp) AS any_lcp,\n",
" IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_lcp,\n",
" (IS_GOOD(fast_inp, avg_inp, slow_inp) OR fast_inp IS NULL) AND IS_GOOD(small_cls, medium_cls, large_cls) AND IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_cwv\n",
" FROM\n",
" `chrome-ux-report.materialized.device_summary` AS cwvs\n",
" JOIN\n",
" site_on_to_date_with_feature\n",
" ON (\n",
" site_on_to_date_with_feature.origin = CONCAT(cwvs.origin, '/') AND\n",
" site_on_to_date_with_feature.device = IF(cwvs.device = 'phone' OR cwvs.device = 'tablet', 'mobile', cwvs.device)\n",
" )\n",
" WHERE\n",
" cwvs.date BETWEEN PARSE_DATE('%Y_%m_%d', '{from_date}') AND PARSE_DATE('%Y_%m_%d', '{to_date}' )\n",
" AND cwvs.device IN ('desktop', 'tablet', 'phone')\n",
" # Restrict to sites that are in both dates\n",
" AND CONCAT(cwvs.origin, '/') IN (\n",
" SELECT\n",
" origin\n",
" FROM\n",
" sites_available_for_both_from_and_to\n",
" )\n",
" ),\n",
"\n",
"\n",
" cwvs_by_date AS (\n",
" SELECT\n",
" date,\n",
" device,\n",
" COUNT(DISTINCT origin) AS origins,\n",
" SAFE_DIVIDE(COUNTIF(good_cwv), COUNTIF(any_lcp AND any_cls)) AS pct_eligible_origins_with_good_cwv,\n",
" SAFE_DIVIDE(COUNTIF(good_lcp), COUNTIF(any_lcp)) AS pct_eligible_origins_with_good_lcp,\n",
" SAFE_DIVIDE(COUNTIF(good_cls), COUNTIF(any_cls)) AS pct_eligible_origins_with_good_cls,\n",
" SAFE_DIVIDE(COUNTIF(good_inp), COUNTIF(any_inp)) AS pct_eligible_origins_with_good_inp\n",
" FROM\n",
" cwv_metrics_for_origins\n",
" GROUP BY\n",
" date, device\n",
" ORDER BY\n",
" date, device\n",
" )\n",
" SELECT * FROM cwvs_by_date\n",
" \"\"\"\n",
"\n",
" query_job = client.query(query)\n",
" return query_job.result() # Return the results of the query"
],
"metadata": {
"id": "uVTvoirMaxIe"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# Step By Step testing"
],
"metadata": {
"id": "5R3EB1yo4Y5A"
}
},
{
"cell_type": "code",
"source": [
"generator_tag = 'speculation-rules'\n",
"after_date = '2024-06-01'\n",
"before_date = '2024-03-01'\n",
"\n",
"active_sites = get_sites_with_feature_active(generator_tag, after_date)\n",
"count_active_sites = active_sites.count()\n",
"inactive_sites = get_sites_with_feature_inactive(generator_tag, after_date)\n",
"count_inactive_sites = inactive_sites.count()\n"
],
"metadata": {
"id": "TiX_xxi_iv-H"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Count feature active sites that were also feature active at the before date.\n",
"print(before_date)\n",
"active_at_from_date = get_active_sites_also_active_at_before_date(generator_tag, before_date, active_sites)\n",
"count_active_at_from_date = active_at_from_date.count()\n",
"\n"
],
"metadata": {
"id": "zWiHNoWla0aS"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"count_active_sites['origin']"
],
"metadata": {
"id": "S88ocl0vvgr6"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Calculate the percentage of count_active_at_from_date (which might be 0)\n",
"# Calculate the percentage of count_active_at_from_date (which might be 0 so use safe divide)\n",
"if count_active_sites['origin'] > 0:\n",
" percent_of_active_sites_also_active_at_from_date = (count_active_at_from_date['origin'] / count_active_sites['origin']) * 100\n",
"else:\n",
" percent_of_active_sites_also_active_at_from_date = 0\n",
"\n",
"inactive_to_active_percent = 100 - percent_of_active_sites_also_active_at_from_date;"
],
"metadata": {
"id": "c6_fKLCmvJ9e"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# Calculate CWV change\n",
"cwv_changes_active_sites = get_cwv_change_before_after(active_sites, before_date, after_date )\n",
"cwv_changes_inactive_sites = get_cwv_change_before_after(inactive_sites, before_date, after_date )\n",
"\n",
"final_results = calculate_feature_impact(cwv_changes_active_sites, cwv_changes_inactive_sites)\n",
"final_results['feature_active_sites'] = count_active_sites['origin']\n",
"final_results['feature_inactive_sites'] = count_inactive_sites['origin']\n",
"final_results['inactive_to_active_percent'] = inactive_to_active_percent\n"
],
"metadata": {
"id": "Y4tpTREfa0SN"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"final_results"
],
"metadata": {
"id": "zj2E6bsda0Gy"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"id": "aRkOWFb247PX"
}
},
{
"cell_type": "markdown",
"source": [
"----"
],
"metadata": {
"id": "yN-yPeZa46y_"
}
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"id": "z-1YtLdF46qa"
}
},
{
"cell_type": "markdown",
"source": [
"# Performance by Feature Queries from 2024-03-01"
],
"metadata": {
"id": "nuOOxybxdCma"
}
},
{
"cell_type": "markdown",
"source": [
"## Speculative Loading"
],
"metadata": {
"id": "ZhClysvodCmb"
}
},
{
"cell_type": "code",
"source": [
"#@markdown ### Compare before/after\n",
"#@markdown * `before_date`: a date to analyze where the feature is not enabled\n",
"#@markdown * `after_date`: a date to analyze where the feature is enabled\n",
"#@markdown * `generator`: the generator tag indicating a feature is installed\n",
"\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'speculation-rules'\n",
"\n",
"results_sl2 = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_sl2)"
],
"metadata": {
"id": "mpMzlYEMdCmb",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "9e7f14b5-99fb-4525-bec3-6561a8880f18"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '1.42%', 'pct_good_lcp_change_mobile': '1.30%', 'pct_good_cls_change_mobile': '2.19%', 'pct_good_inp_change_mobile': '0.56%', 'pct_good_cwv_change_desktop': '2.14%', 'pct_good_lcp_change_desktop': '2.37%', 'pct_good_cls_change_desktop': '1.63%', 'pct_good_inp_change_desktop': '-0.25%', 'feature_active_sites': 9091, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-03-01', 'after_date': '2024-09-01', 'generator_tag': 'speculation-rules', 'cwv_changes_active_sites': [0.0709573638372546, 0.05973671280401477, 0.03339427392262129, 0.09929041920330417, 0.05443899666326818, 0.06367646788742376, 0.02837772363253499, -0.0026272786983606577], 'cwv_changes_inactive_sites': [0.0567628587630879, 0.04675857079386847, 0.011511984996228963, 0.09365062821091052, 0.03303288698159201, 0.0399720547958603, 0.012113975978503277, -0.00013179455489875647]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Image Prioitizer"
],
"metadata": {
"id": "hkxRu7zLdCmb"
}
},
{
"cell_type": "code",
"source": [
"# image-prioritizer\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'image-prioritizer'\n",
"\n",
"results_ip2 = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_ip2)"
],
"metadata": {
"id": "TxduuIH2dCmb",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "4a6f3cb0-5dcc-4f7f-f4d0-e23fbee60e13"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '0.07%', 'pct_good_lcp_change_mobile': '0.24%', 'pct_good_cls_change_mobile': '1.99%', 'pct_good_inp_change_mobile': '0.58%', 'pct_good_cwv_change_desktop': '2.98%', 'pct_good_lcp_change_desktop': '1.71%', 'pct_good_cls_change_desktop': '2.74%', 'pct_good_inp_change_desktop': '-0.07%', 'feature_active_sites': 3057, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-03-01', 'after_date': '2024-09-01', 'generator_tag': 'image-prioritizer', 'cwv_changes_active_sites': [0.05372470728308859, 0.045826828366386796, 0.031917168837576115, 0.09848436113575865, 0.0579437376094758, 0.05101553940017034, 0.037424997986087116, 0.00018115942028984477], 'cwv_changes_inactive_sites': [0.05306598031955251, 0.04344795680480029, 0.012023500874824844, 0.09271561656330818, 0.028133587541988136, 0.03390036546028041, 0.01006490094309187, 0.0008955579797447211]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Embed Optimizer"
],
"metadata": {
"id": "pxscfkaxdCmb"
}
},
{
"cell_type": "code",
"source": [
"# embed-optimizer\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'embed-optimizer'\n",
"\n",
"results_eo2 = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_eo2)"
],
"metadata": {
"id": "jzUIkzusdCmb",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "03316c6d-9b7d-49d1-d7ee-b02c3be80927"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '-6.71%', 'pct_good_lcp_change_mobile': '-6.99%', 'pct_good_cls_change_mobile': '0.25%', 'pct_good_inp_change_mobile': '-0.26%', 'pct_good_cwv_change_desktop': '2.01%', 'pct_good_lcp_change_desktop': '-7.08%', 'pct_good_cls_change_desktop': '5.58%', 'pct_good_inp_change_desktop': '0.01%', 'feature_active_sites': 4908, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-03-01', 'after_date': '2024-09-01', 'generator_tag': 'embed-optimizer', 'cwv_changes_active_sites': [-0.008912738178832036, -0.017832398899997826, 0.013302035749933383, 0.0978406351757094, 0.05156835387437664, -0.030477031089655715, 0.06554045194282965, 0.0013325894195166388], 'cwv_changes_inactive_sites': [0.05813776995746334, 0.05203346729111852, 0.01076988011891955, 0.1004048888519854, 0.03146862294181668, 0.04029451169372489, 0.00971873533213996, 0.0012195856464317645]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Modern Images"
],
"metadata": {
"id": "XFGBFa5QdCmb"
}
},
{
"cell_type": "code",
"source": [
"# webp-uploads\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'webp-uploads'\n",
"\n",
"results_mi2 = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_mi2)"
],
"metadata": {
"id": "RSjCFLw9dCmc",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "9b80c1d2-6511-4578-8bb8-1fc31231a5ab"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '1.53%', 'pct_good_lcp_change_mobile': '0.50%', 'pct_good_cls_change_mobile': '2.53%', 'pct_good_inp_change_mobile': '1.01%', 'pct_good_cwv_change_desktop': '2.67%', 'pct_good_lcp_change_desktop': '2.36%', 'pct_good_cls_change_desktop': '2.13%', 'pct_good_inp_change_desktop': '0.14%', 'feature_active_sites': 16148, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-03-01', 'after_date': '2024-09-01', 'generator_tag': 'webp-uploads', 'cwv_changes_active_sites': [0.06732568282636309, 0.0500484400843042, 0.036909719376889694, 0.1028913557761153, 0.05754036788123096, 0.06235835105115772, 0.03277655490922826, 0.001201716738197467], 'cwv_changes_inactive_sites': [0.05201159105926506, 0.04509394431000929, 0.011659601752061821, 0.09280415428856859, 0.030870589794880943, 0.038753358233235335, 0.011429699662780735, -0.00019254586873129753]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Image Sizes"
],
"metadata": {
"id": "GVdW1ccMdCmc"
}
},
{
"cell_type": "code",
"source": [
"# auto-sizes\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'auto-sizes'\n",
"\n",
"results_is2 = query_cwv_compare_feature_to_baseline_by_device(generator_tag, before_date, after_date)\n",
"print(results_is2)"
],
"metadata": {
"id": "9EmGXoLLdCmc",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "ad69451e-c86d-4388-ade8-dd18e0e370c8"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'pct_good_cwv_change_mobile': '0.95%', 'pct_good_lcp_change_mobile': '0.17%', 'pct_good_cls_change_mobile': '2.26%', 'pct_good_inp_change_mobile': '2.43%', 'pct_good_cwv_change_desktop': '2.96%', 'pct_good_lcp_change_desktop': '1.45%', 'pct_good_cls_change_desktop': '3.11%', 'pct_good_inp_change_desktop': '0.13%', 'feature_active_sites': 5633, 'feature_inactive_sites': 50000, 'also_active_at_before': '0.00%', 'before_date': '2024-03-01', 'after_date': '2024-09-01', 'generator_tag': 'auto-sizes', 'cwv_changes_active_sites': [0.06442163880919188, 0.048145394148988996, 0.0325715565983854, 0.11997871774408086, 0.06243687002187076, 0.05522724145910157, 0.041617845644969065, 0.001961164045756192], 'cwv_changes_inactive_sites': [0.054874468344218774, 0.04648435338003126, 0.009980232692361701, 0.09568983354224359, 0.032824792702341954, 0.0406943834871375, 0.01054937525763644, 0.0007026135858518856]}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Optimization Detective plus Embed Optimizer"
],
"metadata": {
"id": "Ow3y6ErZbD6x"
}
},
{
"cell_type": "code",
"source": [
"# embed-optimizer\n",
"before_date = '2024-03-01'\n",
"after_date = latest_dataset\n",
"generator_tag = 'embed-optimizer'\n",
"\n",
"results_eo2 = query_cwv_compare_feature_to_baseline_by_device_multiple(generator_tag, generator_tag2, before_date, after_date)\n",
"print(results_eo2)"
],
"metadata": {
"id": "ic31QLgGbg7u"
},
"execution_count": null,
"outputs": []
}
],
"metadata": {
"colab": {
"provenance": [],
"collapsed_sections": [
"4G2WkwMPzxbT",
"BoVOOEMO46eL",
"5R3EB1yo4Y5A",
"nuOOxybxdCma"
],
"name": "WPP Metrics Tracking colab",
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment