Skip to content

Instantly share code, notes, and snippets.

View cbodley's full-sized avatar

Casey Bodley cbodley

81 followers · 0 following
  • Red Hat
  • Ann Arbor, MI
View GitHub Profile
@cbodley
cbodley / radosgw.8000.log.md
Last active May 7, 2025 20:16
radosgw objecter log output from ceph api test failure

snippets for client.4539.objecter from https://jenkins.ceph.com/job/ceph-api/95379/artifact/build/out/radosgw.8000.log

startup:

2025-05-07T17:23:51.076+0000 7fe487d0f880 10 client.4539.objecter _maybe_request_map subscribing (onetime) to next osd map
2025-05-07T17:23:51.096+0000 7fe4804ca640 10 client.4539.objecter ms_dispatch2 0x55f4868fa000 osd_map(29..29 src has 1..29)
2025-05-07T17:23:51.096+0000 7fe4804ca640  3 client.4539.objecter handle_osd_map got epochs [29,29] > 0
2025-05-07T17:23:51.096+0000 7fe4804ca640  3 client.4539.objecter handle_osd_map decoding full epoch 29
2025-05-07T17:23:51.096+0000 7fe4804ca640 20 client.4539.objecter dump_active .. 0 homeless
2025-05-07T17:23:51.096+0000 7fe487d0f880 10 client.4539.objecter _op_submit op 0x55f4868b1400
@cbodley
cbodley / get_obj.0
Created April 21, 2025 20:10
versioned GetObject no prefetch
2025-04-21T16:00:42.226-0400 7fa0a0be46c0 2 req 5120578449180881708 0.000000000s s3:get_obj reading permissions
2025-04-21T16:00:42.226-0400 7fa0a0be46c0 20 req 5120578449180881708 0.000000000s s3:get_obj get_obj_state: octx=0x55f10f9e9e40 obj=testbucket:obj state=0x55f10d3799e8 s->prefetch_data=1
2025-04-21T16:00:42.226-0400 7fa0a0be46c0 1 -- 192.168.245.130:0/861247037 --> [v2:192.168.245.130:6800/2079918955,v1:192.168.245.130:6801/2079918955] -- osd_op(unknown.0.0:931 6.0 6:4b6bbc27:::3469a8fe-710c-4381-86b3-8ecc3e3150be.4179.1_obj:head [call version.read in=11b,getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+full_try+supports_pool_eio e19) -- 0x55f10fb7aa80 con 0x55f10d48ac00
2025-04-21T16:00:42.226-0400 7fa0e2c686c0 1 -- 192.168.245.130:0/861247037 <== osd.0 v2:192.168.245.130:6800/2079918955 937 ==== osd_op_reply(931 3469a8fe-710c-4381-86b3-8ecc3e3150be.4179.1_obj [call out=24b,getxattrs out=344b,stat out=16b,read 0~0] v0'0 uv3 ondisk = 0) ==== 317+0+384 (crc 0 0 0) 0x55f10f
@cbodley
cbodley / design.md
Created December 13, 2024 01:28
non-replicated buckets in rgw multisite

use case

in a replicated zonegroup, create a non-replicated bucket for data that doesn't require redundancy. if requirements change, reenable redundancy on the bucket

design

extend the CreateBucketConfiguration xml document so it can specify a non-replicated bucket. when requested by CreateBucket, the bucket is "pinned" to the zone that creates it

when requests on that bucket are sent to other zones, they're redirected to the pinned zone. these redirects prevent object uploads on other zones, and DeleteBucket requests are redirected and processed without ambiguity

@cbodley
cbodley / clang.log
Created October 9, 2024 16:48
asio::spawn() with move-only return value
In file included from main.cc:4:
In file included from /home/cbodley/asio/include/boost/asio.hpp:31:
In file included from /home/cbodley/asio/include/boost/asio/awaitable.hpp:140:
In file included from /home/cbodley/asio/include/boost/asio/impl/awaitable.hpp:29:
In file included from /home/cbodley/asio/include/boost/asio/post.hpp:20:
In file included from /home/cbodley/asio/include/boost/asio/detail/initiate_post.hpp:21:
In file included from /home/cbodley/asio/include/boost/asio/detail/work_dispatcher.hpp:19:
/home/cbodley/asio/include/boost/asio/detail/bind_handler.hpp:161:7: error: call to deleted constructor of 'std::unique_ptr<int>'
161 | arg2_(arg2)
| ^ ~~~~
@cbodley
cbodley / tsan.log
Created May 14, 2024 20:27
signal_co_spawn.cc thread sanitizer output
cbodley@fedora ~/test/signal_co_spawn $ g++ -std=c++20 -fsanitize=thread -lgtest_main -lgtest -I/home/cbodley/ceph/build/boost/include -L/home/cbodley/ceph/build/boost/lib -lboost_system main.cc
In file included from /usr/include/c++/13/latch:38,
from main.cc:3:
In function ‘void std::atomic_thread_fence(memory_order)’,
inlined from ‘boost::asio::detail::std_fenced_block::std_fenced_block(full_t)’ at /home/cbodley/ceph/build/boost/include/boost/asio/detail/std_fenced_block.hpp:43:29:
/usr/include/c++/13/bits/atomic_base.h:144:26: warning: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Wtsan]
144 | { __atomic_thread_fence(int(__m)); }
| ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
In function ‘void std::atomic_thread_fence(memory_order)’,
inlined from ‘boost::asio::detail::std_fenced_block::~std_fenced_block()’ at /home/cbodley/ceph/build/boost/include/boost/asio/detail/std_fenced_block.hpp:49:29:
@cbodley
cbodley / signal_co_spawn.cc
Created May 14, 2024 20:19
#include <chrono>
#include <exception>
#include <latch>
#include <thread>
#include <boost/asio.hpp>
#include <gtest/gtest.h>
using namespace std::chrono_literals;
namespace asio = boost::asio;
@cbodley
cbodley / sal.md
Created May 8, 2024 14:58

sal is overloaded. sal::Bucket/Object act as:

  • backend storage interfaces,
  • handles to generic state, and
  • names

early in request processing, we construct s->object and s->bucket with just their names, leaving other state uninitialized until later. there's no explicit way to know whether a handle is initialized yet. struct rgw_bucket_key and rgw_obj_key should be used instead to represent the names

the generic state in sal::StoreBucket/StoreObject could be separated from sal entirely

@cbodley
cbodley / get_role.log
Created February 22, 2024 14:25
iam:get_role access denied
2024-02-22T09:12:02.204-0500 7f9f78c006c0 20 req 5008242608135058575 0.000000000s iam:get_role inline role policy: { Version: 2012-10-17, Statements: [ { Effect: Allow, Action: [ iam:GetRole ], Resource: [ arn:*:*:*::* ] } ], }
2024-02-22T09:12:02.204-0500 7f9f78c006c0 2 req 5008242608135058575 0.000000000s iam:get_role normalizing buckets and tenants
2024-02-22T09:12:02.204-0500 7f9f78c006c0 2 req 5008242608135058575 0.000000000s iam:get_role init permissions
2024-02-22T09:12:02.204-0500 7f9f78c006c0 2 req 5008242608135058575 0.000000000s iam:get_role recalculating target
2024-02-22T09:12:02.205-0500 7f9f78c006c0 2 req 5008242608135058575 0.001000034s iam:get_role reading permissions
2024-02-22T09:12:02.205-0500 7f9f78c006c0 2 req 5008242608135058575 0.001000034s iam:get_role init op
2024-02-22T09:12:02.205-0500 7f9f78c006c0 10 req 5008242608135058575 0.001000034s iam:get_role cache get: name=default.rgw.meta+roles+RGW11111111111111111role_names.s3-tests-myrole : hit (requested=0x1, cached=0x7)
2024-
@cbodley
cbodley / rgw-iam-accounts.md
Created February 21, 2024 14:54
IAM accounts in Ceph

iam accounts

what is IAM? from https://docs.aws.amazon.com/IAM/latest/APIReference/welcome.html:

AWS Identity and Access Management (IAM) is a web service for securely controlling access to AWS services. With IAM, you can centrally manage users, security credentials such as access keys, and permissions that control which AWS resources users and applications can access.

each aws account has its own set of users and policies. when you log into the aws website, there's an iam dashboard where you can view/manage all of the account's resources and policies. aws also provides a rest api with all of the same functionality

it's that iam rest api that we implement here for ceph. someday the ceph dashboard might expose this stuff too

$ aws iam create-user --user-name Alice
@cbodley
cbodley / data_changes.h
Created January 30, 2024 18:23
pseudo-design for data change set
#pragma once
#include <set>
#include <string>
#include <vector>
#include "common/ceph_time.h"
namespace data_changes {
// A data change event signals the presence of new entries in a bucket index log