Created
June 12, 2024 17:03
-
-
Save robrap/477502700b6acbaf8c07d33c47cc6453 to your computer and use it in GitHub Desktop.
Asymmetric JWT Rollout - NR Dashboard details
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ODgxNzh8VklafERBU0hCT0FSRHxkYTozMjE2NDcw Asymmetric JWT Rollout {"name": "Asymmetric JWT Rollout", "pages": [{"name": "Asymmetric JWT Rollout", "widgets": [{"layout": {"column": 1, "height": 2, "row": 1, "width": 12}, "rawConfiguration": {"text": "# README\n---\n## Warning \nDuring a past rollout attempt, we learned about issues. See github issue [Enable asymmetric JWTs everywhere](https://github.com/openedx/edx-drf-extensions/issues/333) for discussion of how to proceed.\n\nDashboard for rollout of [[DEPR]: Symmetric JWTs](https://github.com/openedx/public-engineering/issues/83)."}, "title": "", "visualization": {"id": "viz.markdown"}}, {"layout": {"column": 1, "height": 2, "row": 3, "width": 10}, "rawConfiguration": {"text": "# Asymmetric JWTs Created\n---\n## WARNING:\nWe learned during a past rollout attempt that the errors listed under token verification are not comprehensive, because this assumed use of the shared `jwt_decode_handler`, and there is a lot of custom JWT decoding in the platform. See README above for more links.\n\nWe've added tracking of algorithm errors, which affected `edx-notes-api`, but there could be other errors we weren't aware of."}, "title": "", "visualization": {"id": "viz.markdown"}}, {"layout": {"column": 1, "height": 2, "row": 5, "width": 10}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Log WHERE message = 'Token verification failed.' FACET entity.name SINCE 1 day ago TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Token verification failed count (by app)", "visualization": {"id": "viz.line"}}, {"layout": {"column": 1, "height": 2, "row": 7, "width": 10}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) AS 'Alg error count' FROM TransactionError WHERE error.message like '%alg value%' FACET appName SINCE 1 day ago TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Algorithm error count (by app)", "visualization": {"id": "viz.line"}}, {"layout": {"column": 1, "height": 3, "row": 9, "width": 5}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT sum(create_asymmetric_jwt_count) FROM Transaction WHERE create_asymmetric_jwt_count IS NOT NULL FACET appName TIMESERIES MAX SINCE 1 day ago"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Asymmetric JWTs", "visualization": {"id": "viz.line"}}, {"layout": {"column": 6, "height": 3, "row": 9, "width": 5}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT sum(create_symmetric_jwt_count) FROM Transaction WHERE create_symmetric_jwt_count IS NOT NULL FACET appName SINCE 1 day ago TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Symmetric JWTs", "visualization": {"id": "viz.line"}}, {"layout": {"column": 1, "height": 1, "row": 12, "width": 12}, "rawConfiguration": {"text": "# Ecommerce JWT decoding\n---\nThese custom attributes were in support of landing this ecommerce PR removing the custom jwt decoder from ecommerce: https://github.com/openedx/ecommerce/pull/3943\n\nOnce this has merged, this data should stop coming in and this section of the dashboard can be removed."}, "title": "", "visualization": {"id": "viz.markdown"}}, {"layout": {"column": 1, "height": 3, "row": 13, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE ecom_jwt_decode_standard IS NOT NULL TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Ecommerce standard JWT decoder", "visualization": {"id": "viz.line"}}, {"layout": {"column": 5, "height": 3, "row": 13, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE ecom_jwt_decode_custom IS NOT NULL"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Ecommerce custom JWT decoder", "visualization": {"id": "viz.billboard"}}, {"layout": {"column": 9, "height": 3, "row": 13, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE ecom_jwt_decode_failed IS NOT NULL TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Ecommerce failed JWT decoder", "visualization": {"id": "viz.line"}}, {"layout": {"column": 1, "height": 3, "row": 16, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE ecom_discount_jwt IS NOT NULL SINCE 1 week ago FACET ecom_discount_jwt LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "ecom_discount_jwt count", "visualization": {"id": "viz.billboard"}}, {"layout": {"column": 1, "height": 2, "row": 19, "width": 12}, "rawConfiguration": {"text": "# JWT decoding (all services)\n---\n## WARNING\nThese charts require edx-drf-extensions 8.7.0+. See [New Relic dashboard for edx-drf-extensions versions](https://onenr.io/0VwglzrxERJ).\n\nSee the [edx-drf-extensions 8.7.0 and 8.6.0 changelog](https://onenr.io/0VwglzrxERJ) for a variety of custom attributes that might help in this rollout, and with other JWT decoding clean-up."}, "title": "", "visualization": {"id": "viz.markdown"}}, {"layout": {"column": 1, "height": 3, "row": 21, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction FACET appName, jwt_auth_asymmetric_verified LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Counts of jwt_auth_asymmetric_verified", "visualization": {"id": "viz.table"}}, {"layout": {"column": 5, "height": 3, "row": 21, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction FACET appName, jwt_auth_symmetric_verified LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Counts of jwt_auth_symmetric_verified", "visualization": {"id": "viz.table"}}, {"layout": {"column": 9, "height": 3, "row": 21, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction FACET appName, jwt_auth_verification_failed LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Counts of jwt_auth_verification_failed", "visualization": {"id": "viz.table"}}, {"layout": {"column": 1, "height": 3, "row": 24, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE request_auth_type_guess IS NOT NULL FACET appName LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Services with edx-drf-extensions observability", "visualization": {"id": "viz.table"}}, {"layout": {"column": 5, "height": 3, "row": 24, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction FACET appName, jwt_auth_issuer_verification LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Counts of jwt_auth_issuer_verification", "visualization": {"id": "viz.table"}}]}], "permissions": "PUBLIC_READ_WRITE"} | |
| ODgxNzh8VklafERBU0hCT0FSRHw4Nzk1MDc0 Asymmetric JWT Rollout / Asymmetric JWT Rollout {"name": "Asymmetric JWT Rollout / Asymmetric JWT Rollout", "pages": [{"name": "Asymmetric JWT Rollout", "widgets": [{"layout": {"column": 1, "height": 2, "row": 1, "width": 12}, "rawConfiguration": {"text": "# README\n---\n## Warning \nDuring a past rollout attempt, we learned about issues. See github issue [Enable asymmetric JWTs everywhere](https://github.com/openedx/edx-drf-extensions/issues/333) for discussion of how to proceed.\n\nDashboard for rollout of [[DEPR]: Symmetric JWTs](https://github.com/openedx/public-engineering/issues/83)."}, "title": "", "visualization": {"id": "viz.markdown"}}, {"layout": {"column": 1, "height": 2, "row": 3, "width": 10}, "rawConfiguration": {"text": "# Asymmetric JWTs Created\n---\n## WARNING:\nWe learned during a past rollout attempt that the errors listed under token verification are not comprehensive, because this assumed use of the shared `jwt_decode_handler`, and there is a lot of custom JWT decoding in the platform. See README above for more links.\n\nWe've added tracking of algorithm errors, which affected `edx-notes-api`, but there could be other errors we weren't aware of."}, "title": "", "visualization": {"id": "viz.markdown"}}, {"layout": {"column": 1, "height": 2, "row": 5, "width": 10}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Log WHERE message = 'Token verification failed.' FACET entity.name SINCE 1 day ago TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Token verification failed count (by app)", "visualization": {"id": "viz.line"}}, {"layout": {"column": 1, "height": 2, "row": 7, "width": 10}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) AS 'Alg error count' FROM TransactionError WHERE error.message like '%alg value%' FACET appName SINCE 1 day ago TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Algorithm error count (by app)", "visualization": {"id": "viz.line"}}, {"layout": {"column": 1, "height": 3, "row": 9, "width": 5}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT sum(create_asymmetric_jwt_count) FROM Transaction WHERE create_asymmetric_jwt_count IS NOT NULL FACET appName TIMESERIES MAX SINCE 1 day ago"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Asymmetric JWTs", "visualization": {"id": "viz.line"}}, {"layout": {"column": 6, "height": 3, "row": 9, "width": 5}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT sum(create_symmetric_jwt_count) FROM Transaction WHERE create_symmetric_jwt_count IS NOT NULL FACET appName SINCE 1 day ago TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Symmetric JWTs", "visualization": {"id": "viz.line"}}, {"layout": {"column": 1, "height": 1, "row": 12, "width": 12}, "rawConfiguration": {"text": "# Ecommerce JWT decoding\n---\nThese custom attributes were in support of landing this ecommerce PR removing the custom jwt decoder from ecommerce: https://github.com/openedx/ecommerce/pull/3943\n\nOnce this has merged, this data should stop coming in and this section of the dashboard can be removed."}, "title": "", "visualization": {"id": "viz.markdown"}}, {"layout": {"column": 1, "height": 3, "row": 13, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE ecom_jwt_decode_standard IS NOT NULL TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Ecommerce standard JWT decoder", "visualization": {"id": "viz.line"}}, {"layout": {"column": 5, "height": 3, "row": 13, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE ecom_jwt_decode_custom IS NOT NULL"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Ecommerce custom JWT decoder", "visualization": {"id": "viz.billboard"}}, {"layout": {"column": 9, "height": 3, "row": 13, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "legend": {"enabled": true}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE ecom_jwt_decode_failed IS NOT NULL TIMESERIES MAX"}], "platformOptions": {"ignoreTimeRange": false}, "yAxisLeft": {"zero": true}}, "title": "Ecommerce failed JWT decoder", "visualization": {"id": "viz.line"}}, {"layout": {"column": 1, "height": 3, "row": 16, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE ecom_discount_jwt IS NOT NULL SINCE 1 week ago FACET ecom_discount_jwt LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "ecom_discount_jwt count", "visualization": {"id": "viz.billboard"}}, {"layout": {"column": 1, "height": 2, "row": 19, "width": 12}, "rawConfiguration": {"text": "# JWT decoding (all services)\n---\n## WARNING\nThese charts require edx-drf-extensions 8.7.0+. See [New Relic dashboard for edx-drf-extensions versions](https://onenr.io/0VwglzrxERJ).\n\nSee the [edx-drf-extensions 8.7.0 and 8.6.0 changelog](https://onenr.io/0VwglzrxERJ) for a variety of custom attributes that might help in this rollout, and with other JWT decoding clean-up."}, "title": "", "visualization": {"id": "viz.markdown"}}, {"layout": {"column": 1, "height": 3, "row": 21, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction FACET appName, jwt_auth_asymmetric_verified LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Counts of jwt_auth_asymmetric_verified", "visualization": {"id": "viz.table"}}, {"layout": {"column": 5, "height": 3, "row": 21, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction FACET appName, jwt_auth_symmetric_verified LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Counts of jwt_auth_symmetric_verified", "visualization": {"id": "viz.table"}}, {"layout": {"column": 9, "height": 3, "row": 21, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction FACET appName, jwt_auth_verification_failed LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Counts of jwt_auth_verification_failed", "visualization": {"id": "viz.table"}}, {"layout": {"column": 1, "height": 3, "row": 24, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction WHERE request_auth_type_guess IS NOT NULL FACET appName LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Services with edx-drf-extensions observability", "visualization": {"id": "viz.table"}}, {"layout": {"column": 5, "height": 3, "row": 24, "width": 4}, "rawConfiguration": {"facet": {"showOtherSeries": false}, "nrqlQueries": [{"accountIds": [88178], "query": "SELECT count(*) FROM Transaction FACET appName, jwt_auth_issuer_verification LIMIT MAX"}], "platformOptions": {"ignoreTimeRange": false}}, "title": "Counts of jwt_auth_issuer_verification", "visualization": {"id": "viz.table"}}]}], "permissions": "PUBLIC_READ_WRITE"} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment