richardkiss

RocksDB Storage Migration Plan

Overview

We are modifying the chia-blockchain project to allow RocksDB to be optionally used instead of an SQLite database. RocksDB's LSM-tree structure improves upon the SQLite B-tree performance degradation we observe as the coin database grows larger.

TLDR

• prototype suggests RocksDB can be used for consensus-critical data with a simple schema and API
• reduction of reliance on DBWrapper2

richardkiss

#!/usr/bin/env python3
"""
Systematic import detector using is-comparison to find true import sources.
Clean algorithm: try all potential import locations, use 'is' to find the truth.
"""

import inspect
import importlib
from typing import Dict, Set, List, Optional, get_type_hints, get_origin, get_args

richardkiss

Understanding CLVM Compression

Introduction

CLVM compression was designed for straightforward implementation within CLVM itself. To understand how it works, we'll first examine the standard deserialization algorithm implemented in CLVM. See also here https://github.com/Chia-Network/clvm_rs/blob/7e58298b85aed07672678e54b9f28571724814dd/cl/deserialize_w_backrefs.cl

Basic Concepts: CLVM Stack Operations

A CLVM stack is a nil-terminated list of CLVM objects. Stack operations push and pop from the front:

• Push operation: (c new_object old_stack)
• Pop operation: (f stack) returns topmost object

richardkiss

This is a report on the chia-blockchain sqlite3 DB. The concern is that some operations are slow, and get slower over time. The symptom is the conversion from v1 of the DB to v2 never finishes and seems to get slower asymptotically as time goes on. This may affect performance during regular day-to-day operation, since DB conversion is not wholy different from DB operations performed on a day-to-day basis.

Summary

There are six tables and eight indices.

richardkiss

from ctypes import CFUNCTYPE, c_int


Ft = CFUNCTYPE(c_int, c_int, c_int)


def entry_point(ptr_to_function):
    function_pointer = Ft(ptr_to_function)
    result = function_pointer(5000, 2)
    return result

richardkiss

The load_clvm method has been duplicated in multiple places, each version with slightly different semantics and bugs. We present here a set of tiny python wheels that solve many problems with chialisp contracts including:

• distribution of libraries
• distribution of compiled .hex files
• automatic dynamic builds in development mode
• dependency analysis (on library include files)

runtime_builder

richardkiss

from math import comb
from typing import List


Hand = List[int]

      
def hand_from_index(index: int, deck_indices: List[int], hand_size: int) -> Hand:
    hand = []
    deck_size = len(deck_indices)

richardkiss

(mod V0 ;; V0 should be 0

  (include *standard-cl-21*)

  ;; projective point: ((X . Y) . Z)

  (defun-inline x-for-p (POINT) (f (f POINT)))
  (defun-inline y-for-p (POINT) (r (f POINT)))
  (defun-inline z-for-p (POINT) (r POINT))

richardkiss

Add the non-free debian repository

echo 'deb http://ftp.us.debian.org/debian bullseye non-free' > /etc/apt/sources.list.d/non-free.list

Refresh

apt update

Add proxmox-specific headers required for DKMS https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support

richardkiss

(mod arguments

  ; this generator blob will fail if the correct run-time parameters are not included
  
  (defun sha256tree
         (TREE)
         (if (l TREE)
             (sha256 2 (sha256tree (f TREE)) (sha256tree (r TREE)))
             (sha256 1 TREE)
         )