Skip to content

Instantly share code, notes, and snippets.

@riptl
Created November 9, 2020 12:41
Show Gist options
  • Save riptl/22372ab8493f7db2a7ff037265d4ee2d to your computer and use it in GitHub Desktop.
Save riptl/22372ab8493f7db2a7ff037265d4ee2d to your computer and use it in GitHub Desktop.
Rook v1.4 Ceph v14.2.13 OSD Resize Bug report
apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
name: rook-ceph
namespace: rook-ceph
spec:
cephVersion:
# For the latest ceph images, see https://hub.docker.com/r/ceph/ceph/tags
image: ceph/ceph:v14.2
dataDirHostPath: /var/lib/rook
mon:
count: 3
volumeClaimTemplate:
spec:
storageClassName: pd-topology-aware
resources:
requests:
storage: 10Gi
dashboard:
enabled: true
monitoring:
enabled: true
rulesNamespace: rook-ceph
storage:
storageClassDeviceSets:
- name: set1
count: 3
portable: true
tuneSlowDeviceClass: false
resources:
requests:
cpu: 500m
memory: 1.5Gi
limits:
cpu: 1000m
memory: 2Gi
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 1.8Ti
storageClassName: pd-topology-aware
volumeMode: Block
accessModes:
- ReadWriteOnce
$ kubectl -n rook-ceph logs deploy/rook-ceph-osd-1 -c expand-bluefs
inferring bluefs devices from bluestore path
1 : device size 0x1cd00000000 : own 0x[7ae1470000~a3d710000] = 0xa3d710000 : using 0x3c73d0000(15 GiB) : bluestore has 0xe238580000(905 GiB) available
Expanding DB/WAL...
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.13/rpm/el7/BUILD/ceph-14.2.13/src/os/bluestore/Allocator.cc: In function 'virtual Allocator::SocketHook::~SocketHook()' thread 7fb1df599ec0 time 2020-11-09 12:32:47.294560
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.13/rpm/el7/BUILD/ceph-14.2.13/src/os/bluestore/Allocator.cc: 53: FAILED ceph_assert(r == 0)
ceph version 14.2.13 (1778d63e55dbff6cedb071ab7d367f8f52a8699f) nautilus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x7fb1d5738b27]
2: (()+0x25ccef) [0x7fb1d5738cef]
3: (()+0x3ccd9f) [0x5622aba22d9f]
4: (HybridAllocator::~HybridAllocator()+0x17) [0x5622aba52757]
5: (BlueStore::_close_alloc()+0x42) [0x5622ab8967d2]
6: (BlueStore::_close_db_and_around(bool)+0x2f8) [0x5622ab91a2e8]
7: (BlueStore::cold_close()+0xe) [0x5622ab91a43e]
8: (BlueStore::expand_devices(std::ostream&)+0x41d) [0x5622ab91bb8d]
9: (main()+0x2476) [0x5622ab82aa16]
10: (__libc_start_main()+0xf5) [0x7fb1d23bf555]
11: (()+0x1fad1f) [0x5622ab850d1f]
*** Caught signal (Aborted) **
in thread 7fb1df599ec0 thread_name:ceph-bluestore-
2020-11-09 12:32:47.294 7fb1df599ec0 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.13/rpm/el7/BUILD/ceph-14.2.13/src/os/bluestore/Allocator.cc: In function 'virtual Allocator::SocketHook::~SocketHook()' thread 7fb1df599ec0 time 2020-11-09 12:32:47.294560
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.13/rpm/el7/BUILD/ceph-14.2.13/src/os/bluestore/Allocator.cc: 53: FAILED ceph_assert(r == 0)
ceph version 14.2.13 (1778d63e55dbff6cedb071ab7d367f8f52a8699f) nautilus (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x7fb1d5738b27]
2: (()+0x25ccef) [0x7fb1d5738cef]
3: (()+0x3ccd9f) [0x5622aba22d9f]
4: (HybridAllocator::~HybridAllocator()+0x17) [0x5622aba52757]
5: (BlueStore::_close_alloc()+0x42) [0x5622ab8967d2]
6: (BlueStore::_close_db_and_around(bool)+0x2f8) [0x5622ab91a2e8]
7: (BlueStore::cold_close()+0xe) [0x5622ab91a43e]
8: (BlueStore::expand_devices(std::ostream&)+0x41d) [0x5622ab91bb8d]
9: (main()+0x2476) [0x5622ab82aa16]
10: (__libc_start_main()+0xf5) [0x7fb1d23bf555]
11: (()+0x1fad1f) [0x5622ab850d1f]
ceph version 14.2.13 (1778d63e55dbff6cedb071ab7d367f8f52a8699f) nautilus (stable)
1: (()+0xf630) [0x7fb1d3c05630]
2: (gsignal()+0x37) [0x7fb1d23d3387]
3: (abort()+0x148) [0x7fb1d23d4a78]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x7fb1d5738b76]
5: (()+0x25ccef) [0x7fb1d5738cef]
6: (()+0x3ccd9f) [0x5622aba22d9f]
7: (HybridAllocator::~HybridAllocator()+0x17) [0x5622aba52757]
8: (BlueStore::_close_alloc()+0x42) [0x5622ab8967d2]
9: (BlueStore::_close_db_and_around(bool)+0x2f8) [0x5622ab91a2e8]
10: (BlueStore::cold_close()+0xe) [0x5622ab91a43e]
11: (BlueStore::expand_devices(std::ostream&)+0x41d) [0x5622ab91bb8d]
12: (main()+0x2476) [0x5622ab82aa16]
13: (__libc_start_main()+0xf5) [0x7fb1d23bf555]
14: (()+0x1fad1f) [0x5622ab850d1f]
2020-11-09 12:32:47.294 7fb1df599ec0 -1 *** Caught signal (Aborted) **
in thread 7fb1df599ec0 thread_name:ceph-bluestore-
ceph version 14.2.13 (1778d63e55dbff6cedb071ab7d367f8f52a8699f) nautilus (stable)
1: (()+0xf630) [0x7fb1d3c05630]
2: (gsignal()+0x37) [0x7fb1d23d3387]
3: (abort()+0x148) [0x7fb1d23d4a78]
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x7fb1d5738b76]
5: (()+0x25ccef) [0x7fb1d5738cef]
6: (()+0x3ccd9f) [0x5622aba22d9f]
7: (HybridAllocator::~HybridAllocator()+0x17) [0x5622aba52757]
8: (BlueStore::_close_alloc()+0x42) [0x5622ab8967d2]
9: (BlueStore::_close_db_and_around(bool)+0x2f8) [0x5622ab91a2e8]
10: (BlueStore::cold_close()+0xe) [0x5622ab91a43e]
11: (BlueStore::expand_devices(std::ostream&)+0x41d) [0x5622ab91bb8d]
12: (main()+0x2476) [0x5622ab82aa16]
13: (__libc_start_main()+0xf5) [0x7fb1d23bf555]
14: (()+0x1fad1f) [0x5622ab850d1f]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment