NOTABLE_CHANGES_21.1.0

# Notable Changes — Ceph Umbrella v21.1.0

## RADOS / OSD / Erasure Coding

* **Optimized ("Fast") Erasure Coding**: a substantial new EC I/O path —
  EC Direct Reads, EC Synchronous Reads, EC OMAP support (recovery of OMAP
  header/entries in EC pools), Optimized EC Deep Scrubbing, and numerous
  correctness fixes to the new path (truncate+write planning, PDW vs RMW
  selection, snapshot scrub handling, peering/recovery corner cases).
  Ships with health warnings for deprecated EC plugins/techniques and for
  Blaum-Roth EC profiles whose `w+1` isn't prime; non-4k-aligned chunk
  sizes are now rejected for optimized EC.
* mon: new optional monitor RocksDB backup/restore mechanism
  (`mon_backup_interval`, `ceph tell mon.* backup`, offline restore via
  `ceph-mon --restore-backup`).
* New **availability score** feature for mon/OSD (tracking + health
  warnings + config toggle), off by default.
* Stretch clusters: tiebreaker mon can now be optional, added a max
  bucket-weight-diff threshold, a health warning for colocated monitors,
  and a restriction on changing the election strategy after stretch mode
  is enabled.
* `mgr/DaemonServer`: `ok-to-upgrade` now aggregates and globally sorts
  OSDs across CRUSH child buckets (fixing inaccurate per-chassis/rack
  ordering), and upgrades can now be scoped to a CRUSH bucket.
* `pg-upmap-primary` support and improved pg-upmap computing speed.
* BlueStore: BlueFS Spillover Cleaner evolution, faster allocation
  recovery, static/runtime object-fragmentation tracking, full-device
  discard on mkfs now default, several spanning-blob/onode-cache-stall/
  reallocation-corruption fixes, and device-type-specific allocation
  policy.
* mClock scheduler refactor.
* Crimson/Seastore: continued heavy investment — erasure coding support
  landed in Crimson (including recovery), PG splitting, a new 2Q cache
  replacement algorithm, broad coroutine conversions, and a long tail of
  correctness/perf fixes across the object-data, LBA/omap-btree, and
  cleaner subsystems.

## RGW

* **Full object dedup** — tech preview.
* **Bucket logging** — new feature: journal-mode logging, EC-pool support,
  admin commands, per-bucket lifecycle/logging performance monitoring.
* **Cloud transition / cloud restore**: restore from Glacier/tape,
  admin CLI for restore list/status, restore-event notifications,
  persistent restore state, per-bucket target options.
* Encryption: added AES-256-GCM SSE support; fixed a GCM key/IV-reuse bug
  on multipart re-upload and a timing side-channel regression in the
  ISA-L GCM tag-comparison path (both security-relevant); CopyObject now
  supported for encrypted objects.
* IAM/Account: `GetAccountSummary`, account-wide `PublicAccessBlock`,
  Admin REST APIs for account quota, S3 Object Ownership (disable ACLs),
  legacy Keystone admin-token requirement removed, Keystone roles
  injected into IAM policy.
* STS/OIDC: `GetCallerIdentity`, JWT/JWKS signature validation, OIDC
  multisite support.
* Multisite: numerous sync-correctness fixes (full-sync infinite loop,
  segfault during startup, forwarded awsv4/UNSIGNED-PAYLOAD requests,
  bucket-sync full-sync locking, zonegroup rename handling).
* D4N (Redis-backed) cache: continued build-out (bucket check_empty,
  connection pooling/pipelining, dirty-entry handling, crash fixes).
* s3-tests vendored directly into the repo (`src/test/rgw/s3-tests`);
  QA suites run them from there instead of an external branch reference.
* Timestamp-based epochs for versioned objects; global CORS rule support;
  rate-limit response with `Retry-After`; standalone RGW build (without
  RadosStore) via a new config-store abstraction — a significant
  architectural refactor of `rgw_zone`/realm-watching code.
* Lua scripting: new postAuth hook, fresh VM per background script
  execution, package-management fixes.

## Dashboard

* Large-scale UI migration to the IBM Carbon design system across most
  pages (RGW, pools, OSD flags, multisite, zone/zonegroup, services,
  upgrade page, and more).
* NVMe/TCP (NVMeoF) management significantly expanded: multi-step
  subsystem/gateway/namespace creation wizards, TLS/mTLS, DHCHAP auth,
  KMIP support, consistency-group CRUD APIs, and many CLI/UI alignment
  fixes.
* SMB cluster management support added.
* RGW role management realigned with Carbon + API routing fixes; global
  RGW tenant "Roles" tab removed in favor of per-account roles.
* CephFS mirroring wizard/entity management UI.
* OAuth2 SSO support (config UI + backend prerequisites).
* Angular upgraded to v19.
* Fixed a dashboard bind-address regression (from a CherryPy-isolation
  refactor) that broke the management gateway whenever its VIP wasn't on
  the same interface as `mon_ip`.

## cephadm / Orchestrator

* Certificate management overhaul: a `certmgr` service now issues/tracks
  TLS certs across mgmt-gateway, monitoring stack, NFS, RGW, and more,
  replacing ad hoc per-service cert handling.
* **SMB/Samba cluster support** — an entirely new managed service type
  (remote-control client, QoS/rate limiting, custom ports, share configs).
* NVMe-oF gateway spec gained many fields (TLS, KMIP, Prometheus
  frequency, namespace limits, group assignment).
* Hardware monitoring (`node-proxy`): new `AtollonSystem` support for
  AMI/Atollon BMCs, temperature/fan-speed reporting, and FCM (NVMe vendor
  log page 0xCA) stats exposed via `ceph orch hardware status`.
* NFS-Ganesha: active-active/keepalived-only ingress mode, TLS
  configuration support, RDMA handling during upgrades.
* Crimson/seastore OSD deployment support added to cephadm and
  ceph-volume.

## CephFS / MDS

* **FSCrypt** (client-side encryption) userspace implementation.
* Multi-threaded cephfs-mirror, plus a blockdiff-based sync mechanism for
  regular-file transfers.
* New pybind/cephfs bindings: `utimensat`, `chownat`, `symlinkat`,
  `readlinkat`, `statxat`, `mkdirat`; non-recursive `rmtree()`/`cptree()`.
* Subvolume group name-mangling support; new subvolume performance/
  utilization metrics; clone-source info retained in `subvolume info`.
* Root-directory fragmentation disallowed; session-reclaim blocklisting
  fix; several shutdown-hang and scrub-pinning fixes.

## ceph-volume

* Migrated internal config/sysinfo representation to dataclasses; device
  detection switched to udev data instead of shelling out to LVM/udevadm
  in several code paths (perf and reliability win).
* Added seastore OSD support; fixed inventory failures on hosts with
  RAID-mirrored system volumes, cdrom devices, and missing `/dev/vg/lv`
  paths.

## Build / Packaging / QA infrastructure

* Rocky Linux 9 and 10 support added across build and QA; CMake bumped to
  require 3.22.1 and gained CMake-4 support; C++ standard raised to C++23.