Skip to content

Instantly share code, notes, and snippets.

@DinoChiesa

DinoChiesa/README.md

Created August 5, 2025 20:09
Show Gist options
  • Save DinoChiesa/cce1988b30e263388c85d5a0cb2da306 to your computer and use it in GitHub Desktop.
Save DinoChiesa/cce1988b30e263388c85d5a0cb2da306 to your computer and use it in GitHub Desktop.
Download ZIP
Dashboard for Gemini Code Assist metrics
Raw
README.md

tl;dr

To use this, Just copy and paste the JSON from gca_metrics_dashboard.json into a new Cloud Observability dashboard.

Prerequisites for Code Acceptance Metrics

For the "Code Acceptance Metrics" section of the dashboard to work, you must have the following set up in your Google Cloud project:

  1. Enable Gemini for Google Cloud Logging: Logging must be enabled for your project to collect the necessary data. To do this:

    • Navigate to the Google Cloud console and go to the Admin for Gemini page.

    • In the Settings tab, you will see toggles for logging. You may need to expand the Logging for Gemini for Code Assist section.

    • Ensure that Logging for Gemini Code Assist metadata is enabled. This is essential for the code acceptance and exposure metrics.

    • You can also enable Logging for Gemini Code Assist prompts and responses for more detailed analysis, but it is not strictly required for this dashboard.

  2. End-User Telemetry: Your developers must have the telemetry setting enabled in their IDEs (both the Gemini Code Assist extension setting and the global telemetry setting in their IDE). Without this, the codeExposure and codeAcceptance events will not be generated.

  3. Upgrade Log Bucket for Analytics: The Observability Analytics charts run SQL queries directly against your logs. For this to work, the Cloud Logging bucket that stores your Gemini metadata logs (typically the _Default bucket) must be upgraded to enable Log Analytics. This is a one-time action that enables the SQL interface on the log bucket.

Raw
gca_metrics_dashboard.json
drive.web-frontend_20250731.11_p1
{
"displayName": "Gemini Code Assist Metrics",
"dashboardFilters": [],
"labels": {},
"mosaicLayout": {
"columns": 48,
"tiles": [
{
"height": 4,
"width": 48,
"widget": {
"title": "Gemini Code Assist - Active Users",
"id": "",
"sectionHeader": {
"dividerBelow": true,
"subtitle": "This shows the number of active users across all GCA surfaces broken down by 1D, 7D, and 28D actives."
}
}
},
{
"yPos": 4,
"height": 12,
"width": 29,
"widget": {
"title": "Active Users",
"id": "",
"xyChart": {
"chartOptions": {
"displayHorizontal": false,
"mode": "COLOR",
"showLegend": false
},
"dataSets": [
{
"breakdowns": [],
"dimensions": [],
"legendTemplate": "Daily",
"measures": [],
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"outputFullDuration": false,
"timeSeriesFilter": {
"aggregation": {
"alignmentPeriod": "60s",
"crossSeriesReducer": "REDUCE_SUM",
"groupByFields": [],
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"cloudaicompanion.googleapis.com/usage/responses/daily_active_users\" resource.type=\"cloudaicompanion.googleapis.com/Instance\""
},
"unitOverride": ""
}
},
{
"breakdowns": [],
"dimensions": [],
"legendTemplate": "7 Days",
"measures": [],
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"outputFullDuration": false,
"timeSeriesFilter": {
"aggregation": {
"alignmentPeriod": "60s",
"crossSeriesReducer": "REDUCE_SUM",
"groupByFields": [],
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"cloudaicompanion.googleapis.com/usage/responses/seven_day_active_users\" resource.type=\"cloudaicompanion.googleapis.com/Instance\""
},
"unitOverride": ""
}
},
{
"breakdowns": [],
"dimensions": [],
"legendTemplate": "28 Days",
"measures": [],
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"outputFullDuration": false,
"timeSeriesFilter": {
"aggregation": {
"alignmentPeriod": "60s",
"crossSeriesReducer": "REDUCE_SUM",
"groupByFields": [],
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"cloudaicompanion.googleapis.com/usage/responses/twenty_eight_day_active_users\" resource.type=\"cloudaicompanion.googleapis.com/Instance\""
},
"unitOverride": ""
}
}
],
"thresholds": [],
"yAxis": {
"label": "",
"scale": "LINEAR"
}
}
}
},
{
"yPos": 4,
"xPos": 29,
"height": 12,
"width": 19,
"widget": {
"title": "Activity by User",
"timeSeriesTable": {
"columnSettings": [],
"dataSets": [
{
"timeSeriesQuery": {
"opsAnalyticsQuery": {
"queryHandle": "CgiufAH_ouL_UxIgam9iX3FHNkltTjZPYkJtd0FhbmFoaVFyQmtvWGRER3QaAlVTQObvw_DwEQ",
"sql": "SELECT\n JSON_VALUE(labels.user_id) AS user_id,\n MAX(timestamp) AS last_accessed\nFROM\n `gplasky-canhaz-coffee.global._Default._Default`\n WHERE resource.type=\"cloudaicompanion.googleapis.com/Instance\"\nAND\n JSON_VALUE(labels.user_id) IS NOT NULL\nGROUP BY\n user_id\nORDER BY\n last_accessed DESC"
}
}
}
],
"metricVisualization": "NUMBER"
}
}
},
{
"yPos": 16,
"height": 4,
"width": 48,
"widget": {
"title": "Gemini Code Assist - Code Acceptance Metrics",
"id": "",
"sectionHeader": {
"dividerBelow": true,
"subtitle": "This shows the various code acceptance metrics for GCA."
}
}
},
{
"yPos": 20,
"height": 9,
"width": 48,
"widget": {
"title": "",
"id": "",
"text": {
"content": "| Metric | Calculation method |\n| ------------------------------ | ------- |\n| **code_acceptance_rate** | This rate is calculated by dividing the total count of unique suggestions that were accepted by the total count of unique suggestions that were shown to users within each time period. It represents the percentage of suggestions that users found helpful enough to accept. |\n| **lines_of_code_accepted** | This metric is the total sum of lines from all accepted code suggestions. For each individual suggestion that might have multiple acceptance events (like typing over it), the query first finds the maximum number of lines from that single event before adding it to the grand total for the time period. |\n| **total_suggestions_exposed** | This is a straightforward count of every unique code suggestion that was generated and shown to a user. Each codeExposure event in the metadata logs contributes to this total. |\n| **total_suggestions_accepted** | This is the total count of unique code suggestions that were accepted by a user. The query counts every codeAcceptance event after grouping them to ensure that a single suggestion accepted by a user is only counted once. |",
"format": "MARKDOWN",
"style": {
"backgroundColor": "#FFFFFF",
"fontSize": "FS_LARGE",
"horizontalAlignment": "H_LEFT",
"padding": "P_EXTRA_SMALL",
"pointerLocation": "POINTER_LOCATION_UNSPECIFIED",
"textColor": "#212121",
"verticalAlignment": "V_TOP"
}
}
}
},
{
"yPos": 29,
"height": 16,
"width": 48,
"widget": {
"title": "Code Acceptance Rate",
"id": "",
"xyChart": {
"chartOptions": {
"displayHorizontal": false,
"mode": "COLOR",
"showLegend": false
},
"dataSets": [
{
"breakdowns": [],
"dimensions": [
{
"column": "time_bucket",
"columnType": "TIMESTAMP",
"maxBinCount": 0,
"sortColumn": "time_bucket",
"sortOrder": "SORT_ORDER_ASCENDING",
"timeBinSize": "3600s"
}
],
"legendTemplate": "",
"measures": [
{
"aggregationFunction": {
"parameters": [],
"type": "sum"
},
"column": "code_acceptance_rate"
}
],
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"opsAnalyticsQuery": {
"queryHandle": "",
"savedQueryId": "",
"sql": "WITH Exposures AS (\n SELECT DISTINCT\n -- Truncate the timestamp to the desired bucket size (e.g., DAY, HOUR, WEEK).\n --DATETIME_TRUNC(DATETIME(timestamp, 'America/Los_Angeles'), HOUR) as time_bucket,\n TIMESTAMP_TRUNC(timestamp, HOUR) as time_bucket,\n JSON_VALUE(json_payload.codeExposure.originalRequestId) AS request_id\n FROM\n `gplasky-canhaz-coffee.global._Default._Default`\n WHERE\n resource.type = \"cloudaicompanion.googleapis.com/Instance\"\n AND json_payload.codeExposure.originalRequestId IS NOT NULL\n -- Recommended: Filter the time range here for better performance.\n AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)\n),\n\n-- Create a CTE to find the final number of lines accepted for each suggestion.\n-- This handles cases where a single suggestion has multiple 'acceptance' events.\nAcceptances AS (\n SELECT\n JSON_VALUE(json_payload.codeAcceptance.originalRequestId) AS request_id,\n -- Find the maximum number of lines accepted for this single suggestion\n MAX(CAST(JSON_VALUE(json_payload.codeAcceptance.linesCount) AS INT64)) as final_lines_accepted\n FROM\n `gplasky-canhaz-coffee.global._Default._Default`\n WHERE\n resource.type = \"cloudaicompanion.googleapis.com/Instance\"\n AND json_payload.codeAcceptance.originalRequestId IS NOT NULL\n -- Recommended: Filter the time range here for better performance.\n AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)\n GROUP BY\n request_id\n)\n\n-- Join exposures with acceptances and aggregate results into time buckets.\nSELECT\n e.time_bucket,\n -- Metric 1: Code Acceptance Rate for this time bucket\n SAFE_DIVIDE(\n COUNT(a.request_id),\n COUNT(e.request_id)\n ) AS code_acceptance_rate,\n\n -- Metric 2: Lines of Code Accepted for this time bucket\n SUM(a.final_lines_accepted) AS lines_of_code_accepted,\n\n -- Supporting raw counts for context\n COUNT(e.request_id) AS total_suggestions_exposed,\n COUNT(a.request_id) AS total_suggestions_accepted\nFROM\n Exposures e\nLEFT JOIN\n Acceptances a ON e.request_id = a.request_id\nGROUP BY\n e.time_bucket\nORDER BY\n e.time_bucket ASC;"
},
"outputFullDuration": false,
"unitOverride": ""
}
}
],
"thresholds": [],
"yAxis": {
"label": "",
"scale": "LINEAR"
}
}
}
},
{
"yPos": 45,
"height": 16,
"width": 48,
"widget": {
"title": "Code Acceptance Metrics (By Day)",
"id": "",
"xyChart": {
"chartOptions": {
"displayHorizontal": false,
"mode": "COLOR",
"showLegend": false
},
"dataSets": [
{
"breakdowns": [],
"dimensions": [
{
"column": "time_bucket",
"columnType": "TIMESTAMP",
"maxBinCount": 0,
"sortColumn": "time_bucket",
"sortOrder": "SORT_ORDER_ASCENDING",
"timeBinSize": "86400s"
}
],
"legendTemplate": "",
"measures": [
{
"aggregationFunction": {
"parameters": [],
"type": "sum"
},
"column": "total_suggestions_accepted"
},
{
"aggregationFunction": {
"parameters": [],
"type": "sum"
},
"column": "lines_of_code_accepted"
},
{
"aggregationFunction": {
"parameters": [],
"type": "sum"
},
"column": "total_suggestions_exposed"
}
],
"plotType": "STACKED_BAR",
"targetAxis": "Y1",
"timeSeriesQuery": {
"opsAnalyticsQuery": {
"queryHandle": "",
"savedQueryId": "",
"sql": "WITH Exposures AS (\n SELECT DISTINCT\n -- Truncate the timestamp to the desired bucket size (e.g., DAY, HOUR, WEEK).\n --DATETIME_TRUNC(DATETIME(timestamp, 'America/Los_Angeles'), HOUR) as time_bucket,\n TIMESTAMP_TRUNC(timestamp, DAY) as time_bucket,\n JSON_VALUE(json_payload.codeExposure.originalRequestId) AS request_id\n FROM\n `gplasky-canhaz-coffee.global._Default._Default`\n WHERE\n resource.type = \"cloudaicompanion.googleapis.com/Instance\"\n AND json_payload.codeExposure.originalRequestId IS NOT NULL\n -- Recommended: Filter the time range here for better performance.\n AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)\n),\n\n-- Create a CTE to find the final number of lines accepted for each suggestion.\n-- This handles cases where a single suggestion has multiple 'acceptance' events.\nAcceptances AS (\n SELECT\n JSON_VALUE(json_payload.codeAcceptance.originalRequestId) AS request_id,\n -- Find the maximum number of lines accepted for this single suggestion\n MAX(CAST(JSON_VALUE(json_payload.codeAcceptance.linesCount) AS INT64)) as final_lines_accepted\n FROM\n `gplasky-canhaz-coffee.global._Default._Default`\n WHERE\n resource.type = \"cloudaicompanion.googleapis.com/Instance\"\n AND json_payload.codeAcceptance.originalRequestId IS NOT NULL\n -- Recommended: Filter the time range here for better performance.\n AND timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)\n GROUP BY\n request_id\n)\n\n-- Join exposures with acceptances and aggregate results into time buckets.\nSELECT\n e.time_bucket,\n -- Metric 1: Code Acceptance Rate for this time bucket\n SAFE_DIVIDE(\n COUNT(a.request_id),\n COUNT(e.request_id)\n ) AS code_acceptance_rate,\n\n -- Metric 2: Lines of Code Accepted for this time bucket\n SUM(a.final_lines_accepted) AS lines_of_code_accepted,\n\n -- Supporting raw counts for context\n COUNT(e.request_id) AS total_suggestions_exposed,\n COUNT(a.request_id) AS total_suggestions_accepted\nFROM\n Exposures e\nLEFT JOIN\n Acceptances a ON e.request_id = a.request_id\nGROUP BY\n e.time_bucket\nORDER BY\n e.time_bucket ASC;"
},
"outputFullDuration": false,
"unitOverride": ""
}
}
],
"thresholds": [],
"yAxis": {
"label": "",
"scale": "LINEAR"
}
}
}
},
{
"yPos": 61,
"height": 4,
"width": 48,
"widget": {
"title": "Gemini Code Assist Response Breakdown",
"id": "",
"sectionHeader": {
"dividerBelow": true,
"subtitle": "Shows responses by programming language, the type of request, and the client."
}
}
},
{
"yPos": 65,
"height": 16,
"width": 24,
"widget": {
"title": "Responses by Language",
"id": "",
"xyChart": {
"chartOptions": {
"displayHorizontal": false,
"mode": "COLOR",
"showLegend": false
},
"dataSets": [
{
"breakdowns": [],
"dimensions": [],
"legendTemplate": "",
"measures": [],
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"outputFullDuration": false,
"timeSeriesFilter": {
"aggregation": {
"alignmentPeriod": "60s",
"crossSeriesReducer": "REDUCE_SUM",
"groupByFields": [
"metric.label.\"programming_language\""
],
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"cloudaicompanion.googleapis.com/usage/response_count\" resource.type=\"cloudaicompanion.googleapis.com/Instance\""
},
"unitOverride": ""
}
}
],
"thresholds": [],
"yAxis": {
"label": "",
"scale": "LINEAR"
}
}
}
},
{
"yPos": 65,
"xPos": 24,
"height": 16,
"width": 24,
"widget": {
"title": "Responses by Client",
"id": "",
"xyChart": {
"chartOptions": {
"displayHorizontal": false,
"mode": "COLOR",
"showLegend": false
},
"dataSets": [
{
"breakdowns": [],
"dimensions": [],
"legendTemplate": "",
"measures": [],
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"outputFullDuration": false,
"timeSeriesFilter": {
"aggregation": {
"alignmentPeriod": "60s",
"crossSeriesReducer": "REDUCE_SUM",
"groupByFields": [
"metric.label.\"client_name\""
],
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"cloudaicompanion.googleapis.com/usage/response_count\" resource.type=\"cloudaicompanion.googleapis.com/Instance\""
},
"unitOverride": ""
}
}
],
"thresholds": [],
"yAxis": {
"label": "",
"scale": "LINEAR"
}
}
}
},
{
"yPos": 81,
"height": 16,
"width": 24,
"widget": {
"title": "Responses by Product",
"id": "",
"xyChart": {
"chartOptions": {
"displayHorizontal": false,
"mode": "COLOR",
"showLegend": false
},
"dataSets": [
{
"breakdowns": [],
"dimensions": [],
"legendTemplate": "",
"measures": [],
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"outputFullDuration": false,
"timeSeriesFilter": {
"aggregation": {
"alignmentPeriod": "60s",
"crossSeriesReducer": "REDUCE_SUM",
"groupByFields": [
"metric.label.\"product\""
],
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"cloudaicompanion.googleapis.com/usage/response_count\" resource.type=\"cloudaicompanion.googleapis.com/Instance\""
},
"unitOverride": ""
}
}
],
"thresholds": [],
"yAxis": {
"label": "",
"scale": "LINEAR"
}
}
}
},
{
"yPos": 81,
"xPos": 24,
"height": 16,
"width": 24,
"widget": {
"title": "Responses by Type",
"id": "",
"xyChart": {
"chartOptions": {
"displayHorizontal": false,
"mode": "COLOR",
"showLegend": false
},
"dataSets": [
{
"breakdowns": [],
"dimensions": [],
"legendTemplate": "",
"measures": [],
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"outputFullDuration": false,
"timeSeriesFilter": {
"aggregation": {
"alignmentPeriod": "60s",
"crossSeriesReducer": "REDUCE_SUM",
"groupByFields": [
"metric.label.\"method\""
],
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"cloudaicompanion.googleapis.com/usage/response_count\" resource.type=\"cloudaicompanion.googleapis.com/Instance\""
},
"unitOverride": ""
}
}
],
"thresholds": [],
"yAxis": {
"label": "",
"scale": "LINEAR"
}
}
}
}
]
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment