Clemens Otto cleot

Native Secure Enclave backed ssh keys on MacOS

It turns out that MacOS Tahoe can generate and use secure-enclave backed SSH keys! This replaces projects like https://github.com/maxgoedjen/secretive

There is a shared library /usr/lib/ssh-keychain.dylib that traditionally has been used to add smartcard support to ssh by implementing PKCS11Provider interface. However since recently it also implements SecurityKeyProivder which supports loading keys directly from the secure enclave! SecurityKeyProvider is what is normally used to talk to FIDO2 devices (e.g. libfido2 can be used to talk to your Yubikey). However you can now use it to talk to your Secure Enclave instead!

Shopping List

Minisforum MS-01 Barebones

I was looking for mini PCs with SFP+ and found a lot of fairly expensive small servers that were tempting. Then I got lucky and saw a new product coming out from minisforum, the MS-01, which had everything I needed at a much lower pricepoint.

I went with the 20 core intel i9-13900H but I think any of the three would have been fine for my needs.

ProxMox Cluster - Soup-to-Nutz

aka what i did to get from nothing to done.

note: these are designed to be primarily a re-install guide for myself (writing things down helps me memorize the knowledge), as such don't take any of this on blind faith - some areas are well tested and the docs are very robust, some items, less so). YMMV

Purpose of Proxmox cluster project

Required Outomces of cluster project

Staking on ZFS

Intro

I always staked on ZFS before the merge, using a number of SATA SSDs in a simple stripe configuration, adding more as my space requirements increased. The merge imposed additional load on my disks that meant my setup was no longer appropriate; this sent me down a long road of testing and optimization. Let me say this up front, there are definitely more performant setups for this than ZFS. I've heard of very good results using mdadm and a simple ext4 filesystem (XFS also works). However, there are so many useful features baked into ZFS (compression, snapshots) and the ergonomics are so good that I was compelled to make this work for my (aging) setup.

Benchmark

I settled on a single fio benchmark for comparing my different setups, based on sar/iostat analyses of working setups. It is as follows: sudo fio --name=randrw --rw=randrw --direct=1 --ioengine=libaio --bs=4k --numjobs=8 --rwmixread=20 --size=1G --runtime=600 --group_reporting. This will lay down several fil

Overview

Syncing an Ethereum node is largely reliant on latency and IOPS, I/O Per Second, of the storage. Budget SSDs will struggle to an extent, and some won't be able to sync at all. IOPS can roughly be used as proxy of / predictor for latency. Measuring latency directly is arguably better.

This document aims to snapshot some known good and known bad models.

The drive lists are ordered by interface and then by capacity and alphabetically by vendor name, not by preference. The lists are not exhaustive at all. @mwpastore linked a filterable spreadsheet in comments that has a far greater variety of drives and their characteristics. Filter it by DRAM yes, NAND Type TLC, Form Factor M.2, and desired capacity.

For size, 4TB is a conservative choice which also supports a Fusaka "supernode". The smaller 2TB drive should last an Ethereum full node until at least sometime 2026, with [pre-merge history expiry](http

About

Summarizing the instructions of the pass tool (as seen on its website).

Getting Started

Installation

Execute: $ sudo apt install pass

Activate wsl2

dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
wsl --set-default-version 2

Install and Configure Ubuntu

Proxmox Assign Bind Mount To Unprivileged Container

In order for the LXC container to have full access the proxmox host directory, a subgid is set as owner of a host directory, and an ACL is used to ensure permissions.

Bind Mount dataset to LXC

Add the following line to /etc/pve/lxc/<CT_ID>.conf

mp0:/mount/point/on/host,mp=/mount/point/on/lxc

Create group on host

In the default Proxmox configuration, unpriviliged container subgids will have the prefix "10" followed by the expected 4-digit gid.

Method 1: NFS server on LXC container

Create a priviledged LXC container with:

mount=nfs,nesting=1

apt update

apt install nfs-kernel-server

	# pre-requisite: python3, web3, requests
	# usage: python3 check_no_relays.py

	import os
	from json import JSONDecodeError

	import requests
	from requests.adapters import HTTPAdapter, Retry
	from web3 import Web3