David Buchanan DavidBuchanan314

Awesome Merkle Search Trees

Various MST implementations. Paper: https://inria.hal.science/hal-02303490/document

atproto docs: https://atproto.com/specs/repository

atproto-flavour

https://github.com/bluesky-social/atproto/tree/main/packages/repo/src/mst typescript "reference implementation"

I looked at a JAR file protected using JNIC, version jnic.dev v3.6.0. I haven't written a full-auto deobfuscater yet, but these notes should be useful for anyone reversing it.

The first layer is a LZMA2 compressed .dat file, from which a native library is extracted into a temp dir, and then loaded using System.load.

The sample I looked at had 4 different library versions (for different platforms/architectures), and the script I wrote to extract them looks like this:

import lzma

# from JNICLoader.java

	"""
	iterate thru /export jsonlines, shove them into ~~sqlite~~ rocksdb, then pull them out
	again into a new jsonlines where each line is in /log/audit format for one DID

	expects input data in "out.jsonlines"
	"""

	from tqdm import tqdm
	import json
	import aimrocks

	No ethernet available
	Adding Xtra Brick Ram @0xa000000, Size = 6291264 - 12 cards.


	]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
	]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
	]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
	]]]]]]]]]]] ]]]] ]]]]]]]]]] ]] ]]]] TM
	] ]]]]]]]]] ]]]]]] ]]]]]]]] ]] ]]]]
	]] ]]]]]]] ]]]]]]]] ]]]]]] ] ]] ]]]]

	import unsafe # https://github.com/DavidBuchanan314/unsafe-python

	mem = unsafe.getmem()

	def very_equal(a: int, b: int) -> None:
	addr = id(a)
	mem[addr+24:addr+32] = b.to_bytes(8, "little")

	very_equal(2 + 2, 5)

	from magic_numbers import FORTY_TWO, SIXTY_NINE, FOUR_HUNDRED_AND_TWENTY
	from magic_numbers import ONE_THOUSAND_THREE_HUNDRED_AND_TWELVE

	print(f"{FORTY_TWO = }")
	print(f"{SIXTY_NINE = }")
	print(f"{FOUR_HUNDRED_AND_TWENTY = }")
	print(f"{ONE_THOUSAND_THREE_HUNDRED_AND_TWELVE = }")

	# https://github.com/P-H-C/phc-string-format/blob/master/phc-sf-spec.md

	import re
	import base64

	PHC_STRING_RE = re.compile(r"""
	\$(?P<id>[a-z0-9-]{1,32})
	(\$v=(?P<version>[0-9]+))?
	(\$(?P<params>[a-z0-9-]{1,32}=[a-zA-Z0-9/+.-=,]*))? # NOTE: full param parse happens later, this just checks at least one param exists
	(

	[
	(try_except := (lambda fn, exception_callback:
	type("mycontext", (__import__("contextlib").ContextDecorator,), {
	"__enter__": lambda self: self,
	"__exit__": lambda self, a, b, c: (a and exception_callback(a, b, c)) or True
	})()(fn)()
	)),

	(throw := lambda e:
	(_ for _ in ()).throw(e)

	(
	solve := lambda data: sum(
	abs(a - b) for a, b in
	zip(map(sorted,zip((
	map(int, line.split())
	for line in data.split("\n")
	))))
	),

	example := """\

	#!/bin/bash
	# SPDX-License-Identifier: GPL-2.0-only

	shopt -s extglob

	unshare=0
	keepresolvconf=0

	#!/hint/bash
	# SPDX-License-Identifier: GPL-2.0-only