Machine Learning Development Environment refers to the suite of tools, infrastructure, and processes that facilitate the development, testing, deployment, and maintenance of machine learning models.
This initial release is part of the first batch of control catalogs produced by the CCC. It is the result of thousands of hours dedicated to exploring different ways of working and collaborating, on top of time spent researching, writing, and reviewing the content. This marks a huge milestone for the CCC and the broader community as further releases will continue to build on this foundation. A huge thanks to everyone who has brought us to this point!
Release Manager - Damien Burks, Citi (damienjburks)
| Capability ID | Capability Title |
|---|---|
| CCC.MLDE.F01 | Managed Notebook Environments |
| CCC.MLDE.F02 | Pre-configured Machine Learning Libraries |
| CCC.MLDE.F03 | Integrated Experiment Management |
| CCC.MLDE.F04 | Model Training and Deployment Integration |
| CCC.MLDE.F05 | Automated Machine Learning (AutoML) Capabilities |
| CCC.MLDE.F06 | GPU/Specialized Hardware Support |
| CCC.MLDE.F07 | Data Pipeline Integration |
| CCC.MLDE.F08 | Model Registry |
| CCC.MLDE.F09 | Collaborative Development Support |
| CCC.MLDE.F10 | Model Monitoring and Drift Detection |
| CCC.MLDE.F11 | Reproducibility Capabilities |
| CCC.MLDE.F12 | Resource Scheduling and Optimization |
| CCC.MLDE.F13 | Security and Compliance Controls |
| CCC.F03 | Access/Activity Logs |
| CCC.F06 | Identity Based Access Control |
| CCC.F08 | Multi-zone Deployment |
| CCC.F09 | Monitoring |
| CCC.F10 | Logging |
| CCC.F14 | API Access |
| CCC.F15 | Cost Management |
| CCC.F16 | Budgeting |
| CCC.F17 | Alerting |
| CCC.F20 | Tagging |
| CCC.F23 | Network Access Rules |
Provides fully managed notebook instances specifically designed for machine learning development, eliminating the need to manage underlying infrastructure.
Offers environments pre-installed with popular machine learning libraries and frameworks such as TensorFlow, PyTorch, and Scikit-learn, optimized for ML tasks.
Facilitates tracking and management of machine learning experiments, including parameters, metrics, and artifacts, within the development environment.
Supports seamless transition from model development to training and deployment, allowing models to be trained and deployed directly from the MLDE.
Offers AutoML functionalities to automatically build, train, and optimize machine learning models with minimal manual intervention.
Provides access to GPU instances and specialized ML acceleration hardware (TPUs, FPGAs) with automated driver and runtime management.
Supports integration with data preparation and feature engineering pipelines, including versioning of datasets and capabilities used in ML experiments.
Provides centralized storage and versioning for trained models, including metadata about training runs, model artifacts, and deployment history.
Enables multiple data scientists to work on the same project with version control integration, shared notebooks, and resource management.
Supports monitoring of deployed models for performance degradation, data drift, and concept drift with automated alerting capabilities.
Provides capability to capture and version all components needed to reproduce an ML experiment, including code, data, and environment configurations.
Supports scheduling and optimization of compute resources for training jobs, including spot instance usage and auto-scaling capabilities.
Provides specific controls for ML workflows including model governance, bias detection, and compliance documentation for regulated industries.
Provides users with the ability to track all requests made to or activities performed on resources for audit purposes.
Provides the ability to determine access to resources based on attributes associated with a user identity.
Provides the ability for the service to be deployed in multiple availability zones or regions to increase availability and fault tolerance.
Provides the ability to continuously observe, track, and analyze the performance, availability, and health of the service resources or applications.
Provides the ability to transmit system events, application activities, and/or user interactions to a logging service
Allows users to interact programmatically with the service and its resources using APIs, SDKs and CLI.
Provides the ability to filter spending and to detect cost anomalies for the service.
Provides the ability to trigger alerts when spending thresholds are approached or exceeded for the service.
Provides the ability to set an alarm based on performance metrics, logs, events or spending thresholds of the service.
Provide the ability to tag a resource to effectively manage and gain insights of the resource.
Ability to control access to the resource by defining network access rules.
| Threat ID | Threat Title |
|---|
| Control ID | Control Title |
|---|---|
| CCC.MLDE.C01 | Define Access Mode for ML Development Environments |
| CCC.MLDE.C03 | Disable Root Access on MLDE Instances |
| CCC.MLDE.C04 | Disable Terminal Access on MLDE Instances |
| CCC.MLDE.C02 | Disable File Downloads on MLDE Instances |
| CCC.MLDE.C05 | Restrict Environment Options on MLDE Instances |
| CCC.MLDE.C06 | Require Automatic Scheduled Upgrades on User-Managed MLDE Instances |
| CCC.MLDE.C07 | Restrict Public IP Access on MLDE Instances |
| CCC.MLDE.C08 | Restrict Virtual Networks for MLDE Instances |
| CCC.C01 | Prevent Unencrypted Requests |
| CCC.C06 | Prevent Deployment in Restricted Regions |
| CCC.C02 | Ensure Data Encryption at Rest for All Stored Data |
| CCC.C03 | Implement Multi-factor Authentication (MFA) for Access |
| CCC.C05 | Prevent Access from Untrusted Entities |
| CCC.C04 | Log All Access and Changes |
Ensure that access to Machine Learning Development Environment (MLDE) resources is strictly defined and controlled. Only authorized users with appropriate permissions can access these environments, mitigating the risk of unauthorized access, data leakage, or service disruption.
Control Family: Identity and Access Management
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.MLDE.TH01 |
| CCC | CCC.TH01 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.AC-3 |
| ISO_27001 | 2013 A.9.1.1 |
| ISO_27001 | 2013 A.9.2.1 |
| NIST_800_53 | AC-2 |
| NIST_800_53 | AC-3 |
| CCM | IAM-01 |
| CCM | IAM-02 |
Prevent users from obtaining root access on MLDE instances to reduce the risk of unauthorized system modifications and potential security breaches.
Control Family: Identity and Access Management
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.MLDE.TH01 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.AC-4 |
| NIST_800_53 | AC-6 |
| CCM | IAM-08 |
| CCM | IAM-12 |
| ISO_27001 | 2013 A.9.2.3 |
Prevent users from accessing the terminal on MLDE instances to limit the risk of unauthorized commands and potential system compromise.
Control Family: Identity and Access Management
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.MLDE.TH01 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.AC-4 |
| NIST_800_53 | AC-6 |
| CCM | IAM-08 |
| ISO_27001 | 2013 A.9.2.3 |
Prevent unauthorized file downloads from MLDE instances to protect sensitive data from being exfiltrated.
Control Family: Data Protection
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.MLDE.TH02 |
| CCC | CCC.TH02 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.DS-5 |
| CCM | DSI-05 |
| CCM | DSI-07 |
| ISO_27001 | 2013 A.13.2.1 |
| NIST_800_53 | SC-7 |
| NIST_800_53 | SC-8 |
Limit the virtual machine and container image options available when creating new MLDE instances to approved and secure configurations.
Control Family: Configuration Management
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.MLDE.TH04 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.IP-1 |
| CCM | TVM-02 |
| ISO_27001 | 2013 A.12.5.1 |
| NIST_800_53 | CM-2 |
Ensure that MLDE instances are kept up-to-date with the latest security patches by enforcing automatic scheduled upgrades.
Control Family: Vulnerability Management
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.MLDE.TH04 |
| CCC | CCC.TH06 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.IP-12 |
| CCM | TVM-01 |
| CCM | TVM-02 |
| ISO_27001 | 2013 A.12.6.1 |
| NIST_800_53 | SI-2 |
Prevent public IP access to MLDE instances to reduce exposure to the internet and enhance security.
Control Family: Network Security
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.MLDE.TH02 |
| CCC | CCC.VPC.TH02 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.AC-3 |
| CCM | SEF-05 |
| ISO_27001 | 2013 A.13.1.1 |
| NIST_800_53 | SC-7 |
Limit the virtual networks that can be used when creating new MLDE instances to ensure they are deployed within approved and secure network environments.
Control Family: Network Security
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.MLDE.TH01 |
| CCC | CCC.TH01 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.AC-4 |
| CCM | IAM-12 |
| ISO_27001 | 2013 A.9.1.2 |
| NIST_800_53 | AC-6 |
Ensure that all communications are encrypted in transit to protect data integrity and confidentiality.
Control Family:
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.TH02 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.DS-02 |
| CCM | IVS-03 |
| CCM | IVS-07 |
| ISO_27001 | 2013 A.13.1.1 |
| NIST_800_53 | SC-8 |
| NIST_800_53 | SC-13 |
Ensure that resources are not provisioned or deployed in geographic regions or cloud availability zones that have been designated as restricted or prohibited, to comply with regulatory requirements and reduce exposure to geopolitical risks.
Control Family:
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.TH03 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.DS-1 |
| CCM | DSI-06 |
| CCM | DSI-08 |
| ISO_27001 | 2013 A.11.1.1 |
| NIST_800_53 | AC-6 |
Ensure that all data stored is encrypted at rest to maintain confidentiality and integrity.
Control Family:
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.TH01 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.DS-1 |
| CCM | DSP-17 |
| NIST_800_53 | SC-13 |
| NIST_800_53 | SC-28 |
Ensure that all sensitive activities require two or more identity factors during authentication to prevent unauthorized access. This may include something you know, something you have, or something you are. In the case of programattically accessible services, such as API endpoints, this includes a combination of API keys or tokens and network restrictions.
Control Family:
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.TH01 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.AC-7 |
| CCM | IAM-03 |
| CCM | IAM-08 |
| ISO_27001 | 2013 A.9.4.2 |
| NIST_800_53 | IA-2 |
Ensure that secure access controls prevent unauthorized access, mitigate risks of data exfiltration, and block misuse of services by adversaries. This includes restricting access based on trust criteria such as IP allowlists, domain restrictions, and tenant isolation.
Control Family:
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.TH01 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | PR.AC-3 |
| CCM | DS-5 |
| ISO_27001 | 2013 A.13.1.3 |
| NIST_800_53 | AC-3 |
Ensure that all access and changes are logged to maintain a detailed audit trail for security and compliance purposes.
Control Family:
| Threat Catalog | Mapped Threats |
|---|---|
| CCC | CCC.TH01 |
| Guideline | Mapped Controls |
|---|---|
| NIST-CSF | DE.AE-3 |
| CCM | LOG-08 |
| NIST_800_53 | AU-2 |
| NIST_800_53 | AU-3 |
| NIST_800_53 | AU-12 |
We would like to acknowledge the following organizations for their valuable contributions to this project: