How to run Ceph commands in OpenShift Container Storage OCS / ODF

ceph-toolbox.sh

#!/bin/bash
if [ "$1" == "off" ]; then
    oc patch OCSInitialization/ocsinit -n openshift-storage \
      --type=merge -p='{"spec":{ "enableCephTools": false}}'
    sleep 3
    echo "removing any existing toolbox pod"
    oc delete pods -n openshift-storage -l app=rook-ceph-tools
else
    oc patch OCSInitialization/ocsinit -n openshift-storage \
      --type=merge -p='{"spec":{ "enableCephTools": true}}'

    TOOLS_POD=""
    echo -n "waiting for ceph tools pod to schedule "
    until [ -n "$TOOLS_POD" ]; do
        echo -n "."
        sleep 5
        TOOLS_POD=$(oc get pod -n openshift-storage -l app=rook-ceph-tools -o name)
    done
    echo "$TOOLS_POD"

    echo "waiting for ceph tools pod to startup"
    oc wait $TOOLS_POD --for=condition=Ready --timeout=300s  -n openshift-storage

    echo "connecting to ceph toolbox"
    oc rsh -n openshift-storage $TOOLS_POD
fi

openshift-container-storage-ceph-toolbox.adoc

Ceph Toolbox is not recommended or supported for use with OpenShift Data Foundation, but never the less, sometimes you need a client to troubleshoot with.

$ ./ceph-toolbox.sh
ocsinitialization.ocs.openshift.io/ocsinit patched
waiting for ceph tools pod to schedule .........pod/rook-ceph-tools-565ffdb78c-sf2bf
waiting for ceph tools pod to startup
pod/rook-ceph-tools-565ffdb78c-sf2bf condition met
sh-4.4$
sh-4.4$ ceph osd status
ID  HOST                      USED  AVAIL  WR OPS  WR DATA  RD OPS  RD DATA  STATE
 0  hub-q4jtr-store-1-5nlkv   299G   724G      1      135k      1        0   exists,up
 1  hub-q4jtr-store-2-652pc   299G   724G      0     18.3k      3      105   exists,up
 2  hub-q4jtr-store-3-mtddc   299G   724G      2     18.4k      2      241   exists,up

Here are a couple of troubleshooting commands to get you started.

ceph status
ceph osd status
ceph osd pool ls
ceph df
rados df

Afterwards turn off the toolbox by running ./ceph-toolbox off and deleting the pod.

See also:

• https://github.com/rook/rook/blob/master/deploy/examples/toolbox.yaml

• https://guifreelife.com/blog/2023/04/06/Accessing-Ceph-CLI-with-OpenShift-Data-Foundation/