Skip to content

Instantly share code, notes, and snippets.

View alanshaw's full-sized avatar
🌶️
https://storacha.network

ash alanshaw

🌶️
https://storacha.network
View GitHub Profile
@alanshaw
alanshaw / lassie-fetch.sh
Created June 30, 2023 11:17
Lassie fetch from dag.w3s.link
lassie fetch --dag-scope=entity -providers=/dns4/dag.w3s.link/tcp/443/https/p2p/QmUA9D3H7HeCYsirB3KmPSvZh3dNXMZas6Lwgr4fv1HTTp bafybeicuc7vfbxzaperdc6ng2c4ufeulyjpfrmwuxhxrm6cfdamp5hpw5i
@alanshaw
alanshaw / curl.sh
Created May 19, 2023 14:19
curling some dfs carz from dag.w3s.link
$ curl -v -H "Accept: application/vnd.ipld.car;order=dfs" https://dag.w3s.link/ipfs/bafybeibrqc2se2p3k4kfdwg7deigdggamlumemkiggrnqw3edrjosqhvnm --output my.dfs.car
...
> GET /ipfs/bafybeibrqc2se2p3k4kfdwg7deigdggamlumemkiggrnqw3edrjosqhvnm HTTP/2
> Host: dag.w3s.link
> user-agent: curl/7.77.0
> accept: application/vnd.ipld.car;order=dfs
>
...
< HTTP/2 200
< date: Fri, 19 May 2023 14:16:47 GMT
@alanshaw
alanshaw / directory-builder.js
Created March 28, 2023 17:56
Building a directory after uploading individual files async
import { createWriter } from '@ipld/unixfs'
import * as CAR from '@web3-storage/upload-client/car'
import { create } from '@web3-storage/w3up-client'
/**
* @param {import('@web3-storage/w3up-client').Client} client
* @param {import('@web3-storage/w3up-client/types').FileLike} file
*/
async function uploadFile (client, file) {
/** @type {import('@web3-storage/upload-client/types').DirectoryEntryLink[]} */
@alanshaw
alanshaw / create-perma-map-fixture.js
Created March 7, 2023 15:26
Create a fixture for `@perma/map` from a CAR file containing a HAMT sharded directory
import { CarIndexedReader } from '@ipld/car'
import * as pb from '@ipld/dag-pb'
import { UnixFS } from 'ipfs-unixfs'
// usage: node create-perma-map-fixture sharded-dir.car
/** @typedef {{ bitfield: number[], links: Array<Link|Element>, prefix?: string }} Link */
/** @typedef {{ key: string, value: string, prefix: string }} Element */
async function main () {
@alanshaw
alanshaw / upload-large-cars.md
Last active January 11, 2023 18:29
Upload large CAR files to w3up

Install the new CLI:

npm install -g @web3-storage/w3cli

Create and register new space:

w3 space create SPACENAME
@alanshaw
alanshaw / w3-up-server.md
Last active June 25, 2024 10:46
How do you use w3up on the server/in CI where there's no persistent filesystem?

On your local computer:

  1. Create a keypair for the server

    npx ucan-key ed --json

    Note down did and key values!

@alanshaw
alanshaw / agent.js
Last active December 1, 2022 11:26
w3access agent refactor
// Goals:
// 1. Decouple store from agent
// 2. Simplify agent creation
// 3. Agent governs data format not store
// 4. Initialization of agent, not store
// 5. DRY initialization in agent, not repeated in each store impl
/**
* @param {AgentData} [data] Agent data
* @param {object} [options]
@alanshaw
alanshaw / extract-gateway-peers.mjs
Created November 30, 2022 12:38
Extract the gateway peer IDs from the bifrost infra repo
import fs from 'fs'
import path from 'path'
const dir = path.join('ansible', 'inventories', 'bifrost', 'host_vars')
const regex = /ipfs_config_identity_peerid: "(.*)"/ig
async function main () {
const peers = []
for (const f of fs.readdirSync(dir)) {
if (!(f.startsWith('ipfs-bank') && f.endsWith('ipfs.gateway.dwebops.net.yml'))) continue
@alanshaw
alanshaw / react-uploader.jsx
Last active September 28, 2022 20:36
Simplified examples for the W3UI website
import { useState } from 'react'
import { useUploader } from '@w3ui/react-uploader'
export default function Component () {
const { uploader } = useUploader()
const [file, setFile] = useState(null)
const [cid, setCid] = useState('')
const handleUploadSubmit = async e => {
e.preventDefault()
@alanshaw
alanshaw / linkdex.md
Created August 24, 2022 07:50
Linkdex idea

Linkdex

An index mapping block CID to linked block CID.

Characteristics

  • Small - just the link blocks - no data for raw leaves. The majority of files are small files, so the DAGs are not super deep.
  • Root CID independent - it doesn’t matter what the root CID is, we can just iteratively decode each block in a CAR and index the links it has

What is it useful for?