wdberkeley

$ bin/kudu cluster ksck -checksum_scan -checksum_snapshot=false -ksck_format=json localhost:7051,localhost:7052,localhost:7053
{
    "master_health": [
        {
            "uuid": "c78e5cede62f48bd9975cc27e0bd4870",
            "address": "localhost:7051",
            "health": "HEALTHY",
            "status": "OK"
        },
        {

wdberkeley

$ bin/kudu cluster ksck -ksck_format=json localhost:7051,localhost:7052,localhost:7053
{
    "master_health": [
        {
            "uuid": "c78e5cede62f48bd9975cc27e0bd4870",
            "address": "localhost:7051",
            "health": "HEALTHY",
            "status": "OK"
        },
        {

wdberkeley

{
    "master_health": [
        {
            "uuid": "c78e5cede62f48bd9975cc27e0bd4870",
            "address": "localhost:7051",
            "health": "HEALTHY",
            "status": "OK"
        },
        {
            "uuid": "7b2c2970d9fa482c8ac7c06546b30323",

wdberkeley

import argparse
import random

# Number of pool slots per TS.
POOL_SLOTS_PER_TS=10

# Maximum number of time steps it takes a move to complete.
MAX_MOVE_STEPS = 5

# Event types.

wdberkeley

# Return the table skew of 'table'.
# Table skew is (# replicas on TS with most replica) - (# replicas on TS with least replica).
# This modifies table so it is sorted by # of replicas, increasing.
def table_skew(table):
    table.sort()
    return table[-1] - table[0]

# Return the next move that should be done to balance table, encoded as (i, j)
# where i is the index of the TS to move from and j is the index of the TS to
# move to.

wdberkeley

#include <regex>
#include <iostream>
#include <string>

using namespace std;

int main() {
        string input;
        regex integer("([[:digit:]]+.[[:digit:]]+.[[:digit:]]+)(-.*)?");
        smatch matches;

wdberkeley

1. Disable authentication

  --rpc_authentication=disabled
  --rpc_encryption=disabled
  
2. Create a table

  > kudu perf loadgen --keep_auto_table --table_num_replicas=3 kudu-master
  Using auto-created table 'loadgen_auto_6579615914f7441b808db3a03ae88aae'

wdberkeley

1. Get a tablet id for a tablet from the table you're interested in. 
   The easiest way to do this is usually to to use the /tablets endpoint of the webui.
   
2. Run the following command to find the blocks for this tablet stored on the tablet server:
   $ kudu local_replica dump rowset --fs_wal_dir=<wal dir> --fs_data_dirs=<data dirs> -metadata-only <tablet id>
   
   You'll see something like
   
   Dumping rowset 0
----------------------------------------------------------------------

wdberkeley

Apache Kudu Scaling

wdberkeley

[ RUN      ] TabletCopyClientTest.TestDownloadBlock
W1205 10:17:49.985105 1999118336 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I1205 10:17:49.985236 1999118336 server_base.cc:229] Could not load existing FS layout: Not found: /private/tmp/kudutest-502/tablet_copy_client-test.TabletCopyClientTest.TestDownloadBlock.1512497869518041-53106/TabletServerTest-fsroot/data-0/instance: No such file or directory (error 2)
I1205 10:17:49.985250 1999118336 server_base.cc:230] Creating new FS layout
I1205 10:17:49.991677 1999118336 fs_manager.cc:567] Generated new instance metadata in path /private/tmp/kudutest-502/tablet_copy_client-test.TabletCopyClientTest.TestDownloadBlock.1512497869518041-53106/TabletServerTest-fsroot/data-0/instance:
uuid: "c0b5ced5c9ca4d75819b8854b477855e"
format_stamp: "Formatted at 2017-12-05 18:17:49 on dhcp-10-16-0-202.pa.cloudera.com"
I1205 10:17:49.991961 1999118336 fs_manager.cc:567]