richardkiss

Understanding CLVM Compression

Introduction

CLVM compression was designed for straightforward implementation within CLVM itself. To understand how it works, we'll first examine the standard deserialization algorithm implemented in CLVM. See also here https://github.com/Chia-Network/clvm_rs/blob/7e58298b85aed07672678e54b9f28571724814dd/cl/deserialize_w_backrefs.cl

Basic Concepts: CLVM Stack Operations

A CLVM stack is a nil-terminated list of CLVM objects. Stack operations push and pop from the front:

• Push operation: (c new_object old_stack)
• Pop operation: (f stack) returns topmost object

richardkiss

This is a report on the chia-blockchain sqlite3 DB. The concern is that some operations are slow, and get slower over time. The symptom is the conversion from v1 of the DB to v2 never finishes and seems to get slower asymptotically as time goes on. This may affect performance during regular day-to-day operation, since DB conversion is not wholy different from DB operations performed on a day-to-day basis.

Summary

There are six tables and eight indices.

richardkiss

from ctypes import CFUNCTYPE, c_int


Ft = CFUNCTYPE(c_int, c_int, c_int)


def entry_point(ptr_to_function):
    function_pointer = Ft(ptr_to_function)
    result = function_pointer(5000, 2)
    return result

richardkiss

The load_clvm method has been duplicated in multiple places, each version with slightly different semantics and bugs. We present here a set of tiny python wheels that solve many problems with chialisp contracts including:

• distribution of libraries
• distribution of compiled .hex files
• automatic dynamic builds in development mode
• dependency analysis (on library include files)

runtime_builder

richardkiss

from math import comb
from typing import List


Hand = List[int]

      
def hand_from_index(index: int, deck_indices: List[int], hand_size: int) -> Hand:
    hand = []
    deck_size = len(deck_indices)

richardkiss

(mod V0 ;; V0 should be 0

  (include *standard-cl-21*)

  ;; projective point: ((X . Y) . Z)

  (defun-inline x-for-p (POINT) (f (f POINT)))
  (defun-inline y-for-p (POINT) (r (f POINT)))
  (defun-inline z-for-p (POINT) (r POINT))

richardkiss

Add the non-free debian repository

echo 'deb http://ftp.us.debian.org/debian bullseye non-free' > /etc/apt/sources.list.d/non-free.list

Refresh

apt update

Add proxmox-specific headers required for DKMS https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support

richardkiss

(mod arguments

  ; this generator blob will fail if the correct run-time parameters are not included
  
  (defun sha256tree
         (TREE)
         (if (l TREE)
             (sha256 2 (sha256tree (f TREE)) (sha256tree (r TREE)))
             (sha256 1 TREE)
         )

richardkiss

Block Generators

Block generators in Chia are clvm programs that, when run with particular arguments, produce a list of coin spends. This provides opportunity to do compression by recognizing patterns and encoding.

CLVM Serialization

The standard serialization mechanism of clvm is defined here.

If you review this specification, you'll see that deserialization begins by inspecting the first byte. If the byte is 0xff, the object is a pair. If the byte is 0x00 to 0xfb, the object is an atom whose length is partially or fully encoded by this first byte.

richardkiss

import zlib

from chia.wallet.puzzles.load_clvm import load_clvm
from chia.wallet.puzzles import p2_delegated_puzzle_or_hidden_puzzle as standard_puzzle
from chia.wallet.puzzles.cc_loader import CC_MOD

OFFER_MOD = load_clvm("settlement_payments.clvm")

puzzle_dict = bytes(OFFER_MOD) + bytes(standard_puzzle.MOD) + bytes(CC_MOD)