Dusty Daemon 2024-Q4 Report.md

1) How did you spend your time?

I accomplished two primary things with my time. The first is a large progression towards so called "inter-op" for Splicing with Eclair. Inter-op is when two separate lightning implementations support splicing and are able to perform a splice between the two implementations. Essentially this means having a channel between the Eclair and CLN implementations, then doing a splice over that channel. Once a single splice is completed, numerous edge and corner cases are tested and must be handled robustly by both implementations.

This step is a big deal because once completed splicing officially becomes part of the Lightning specification for BOLTs 2, 7, and 9.

In a test environment I setup Eclair and CLN nodes and formed a channel between them. Then I attempted splicing. Numerous issues and incompatibilities emerged and had to be addressed.

These tests are important to lightning for two reasons:

• Revealing items in the spec that are not documented or under-documented.
• Ensuring the feature works consistently across implementations.

My work on this culminated in 22 commits in this PR: ElementsProject/lightning#7719

Due to these commits and others making me the largest contributor of the Core Lightning release, I was given the honor of naming it 💪.

The work in this PR included

• Supporting rotating funding pubkeys
  • This is an optional feature in the splicing spec that CLN left disabled but available for future use / implementation.
  • Eclair decided to implement this feature and require its use
  • Instead of disabling it in Eclair, I instead did the heavy lifting to implement the feature in CLN ahead of it being needed
  • Now both implementations support rotating funding pubkeys -- this has no use today but will be critical when taproot and gossip v2 are completed later on.
• Numerous subtle differences in protocol implementation
  • Adding a batch_size parameter to commitment_signed since Eclair needs it
  • Removing the transmission of prevtx bytes and instead sending prevtxid. This is because Eclair's max packet size is smaller than the largest potential prevtx bytes.
  • Removing blockhash from multiple messages as they’re no longer needed (and Eclair errored when it was sent).
  • Pre-calculating the final channel balance for a splice. CLN calculated it at the end but Eclair requires it to be calculated at the beginning.
  • Updating of message numbers of splice and splice_ack
• While working on all this, various minor issues were spotted and cleaned up
  • Using the correct encoding for tx_abort
  • Upgrading the warning and error messages for splicing all around. This was quite helpful for knowing what was going wrong with inter-op testing.

There is more work to be done to get official inter-op, but this update was a tremendous amount of progress towards that goal.

I am delighted to get Splice Script beta merged in this PR with 21 commits: ElementsProject/lightning#6980

This is the first stage in a large project to revolutionize the way on-chain transactions are created for Lightning (and potentially beyond). This project completed multiple peer review cycles and had major components adjusted, tweaked, rewritten, and fleshed out to bring it closer towards a mature outcome.

Some of the major components are:

• A script parser
  • Splice Script is a new language that has to be parsed. This large project tokenizes, compiles, and interprets the scripting language. It is a compiler, calculator, and validator that turns the script into a machine friendly json encoding.
• A splice solver
  • This takes the machine interpreted result and can arbitrarily execute any splice. Because it can execute /any/ splice it must be extremely thorough.
  • In addition to executing every possible splice, it must also validate that the splice is done correctly. The open-ended nature of this project makes it quite challenging.
  • The huge benefit of doing it this way is that downstream tools and users become free from the need of doing validation, resulting in more robust easier to user splicing for all.
• Splice script testing framework
  • Testing is critically important

While working on Splice Script, numerous upgrades were needed inside the CLN implementation for the system to use:

• Adding inputs to a PSBT
• Adding custom flags to PSBT when adding inputs or outputs
• Adding channel_ids flag to PSBT to track which channels its associated with
• Adding "string or array" type to the RPC interface.
• Decoding address types for validation in script
• Restructuring splice's witness stack and remote signature to be kept during a restart
• Checking if a specific signature exists in a PSBT
• RPC interface for manually entering stfu (with balance confirmation response), and aborting with tx_abort
• Splice Script revealed the need for a large refactor of splice message ordering to accommodate certain complex splices.
  • ChannelD was refactored to handle this case as well as the adjusted stfu process.
• Other minor changes.

Getting this PR merged is very exciting and sets the foundation for a truly exciting script based future for lightning.

Some smaller but still important PRs were also merged:

• ElementsProject/lightning#7083
  • A new test that does hyper aggressive crashing and restarting of the primary ChannelD
• ElementsProject/lightning#7465
  • Add tracking of the specific transaction output for a given splice
• ElementsProject/lightning#7466
  • Splicing means there can now be /multiple/ closing transactions. Updates log tracking for the multiple tx's.
• ElementsProject/lightning#7816
  • Mac documentation update
• ElementsProject/lightning#7825
  • Code cleanliness fix

In additional to my engineering efforts, I also visited the summit in Tokyo and attended many meetings discussing and reviewing the existing and proposed lightning specification.

2) How did you make use of the money?

The grant covered my living expenses while I worked on this project.

3) What do you plan to work on next quarter?

My primary goals for the next quarter are to:

• Accomplish splicing inter-op with Eclair
  • While not entirely in my control, I suspect we will accomplish this soon!
• Implement the next round of features in Splice Script
  • Having the beta release is awesome -- now that that is merged, it's time to start growing what it can do.
• Support other implementations integrate splicing
  • Many implementations seem to be working on it, which is very exciting. I plan to assist them in integrating in whatever form I can.
• Splicing Spec
  • As the splice specification nears merging, its more important than ever to keep it in the highest quality possible.
  • Small spec questions can result in days of research to confirm we have chosen precisely the correct answers.
  • New implementations often have different opinions on the spec as they start implementing it.
  • It's important to be present for these conversations and delve into all the (usually complex) issues that arise (ex. lightning/bolts#1214)

Inevitably small issue and bugs are found in my existing splicing code & spec that are addressed as well when they come up.