---
title: w3 writes
---
flowchart TD
w3infra --> |upload/add rootCid,carCid|r2-dudewhere
api.web3.storage -->|PUT carCid/carCid.idx|r2-satnav[(R2 satnav \n bucket)]
api.web3.storage -->|PUT carCid/carCid.car|r2-carpark[(R2 carpark \n bucket)]
api.web3.storage -->|PUT carCid/carCid.idx|s3-carpark[(S3 carpark \n bucket)]
api.web3.storage -->|PUT rootCid/carCid|r2-dudewhere[(R2 dudewhere \n bucket)]
---
title: w3s upload
# w3up-client upload sequence
---
sequenceDiagram
Agent ->> w3up: store/add (carCid)
w3up -->> Agent: signed carURL
Agent ->> R2: PUT to carURL
Agent ->> w3up: store/add (indexCarCid)
---
title: w3 indexing pipeline
---
flowchart TD
s3-dotstorage[(S3 dotstorage\nbucket)] -->|ObjectCreated fa:fa-car| bucket-to-indexer(bucket-to-indexer ƛ)
bucket-to-indexer -->|region/bucket/cid/cid.car| indexer-queue[/indexer queue/]
indexer-queue --> indexer(Indexer ƛ)
indexer --> |zQmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn| multihash-queue[/multihash queue/]
multihash-queue --> ipni-advertisment-content(ipni advertisment content ƛ)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import readline from 'node:readline' | |
import cliProgress from 'cli-progress' | |
const COUNT_PER_FILE = 1_000_000 | |
const inputCounts = new Map([ | |
['nft-30.json', 196294], | |
['web3-4.json', 576770] | |
]) | |
function getCountPerFile (file) { | |
return inputCounts.get(file) ?? COUNT_PER_FILE |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import fs from 'fs' | |
import { CarBufferWriter } from '@ipld/car' | |
import { CID } from 'multiformats/cid' | |
import * as json from 'multiformats/codecs/json' | |
import * as raw from 'multiformats/codecs/raw' | |
import { sha256 } from 'multiformats/hashes/sha2' | |
/** | |
* @param {CID} root | |
* @param {Block[]} blocks |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import * as idb from 'idb-keyval' | |
import { CID } from 'multiformats' | |
import { BytesReader } from '@ipld/car/lib/coding' | |
import { createDecoder, asyncIterableReader } from '@ipld/car/lib/decoder.js' | |
export class CarReaderIdb { | |
_version: number; | |
_roots: Array<CID>; | |
constructor (version: number, roots: Array<CID>) { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"openrpc": "1.2.6", | |
"info": { | |
"title": "Lotus RPC API", | |
"version": "1.7.0-dev" | |
}, | |
"methods": [ | |
{ | |
"name": "Filecoin.BeaconGetEntry", | |
"description": "```go\nfunc (s *FullNodeStruct) BeaconGetEntry(p0 context.Context, p1 abi.ChainEpoch) (*types.BeaconEntry, error) {\n\treturn s.Internal.BeaconGetEntry(p0, p1)\n}\n```", |
The gateway hosted at https://ipfs.io allows browsers and tools that speak http
to access content from the IPFS network. It provides the canonical way to address IPFS content via http scheme urls.
A request for:
https://ipfs.io/ipfs/QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN
...will hit the gateway and get forwarded to an IPFS node, which fetches the block for the CID QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN
and returns the content via http.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ipfs swarm connect \ | |
/dns4/cluster0.fsn.dwebops.pub/tcp/4001/p2p/QmUEMvxS2e7iDrereVYc5SWPauXPyNwxcy9BXZrC1QTcHE \ | |
/dns4/cluster1.fsn.dwebops.pub/tcp/4001/p2p/QmNSYxZAiJHeLdkBg38roksAR9So7Y5eojks1yjEcUtZ7i \ | |
/dns4/cluster2.fsn.dwebops.pub/tcp/4001/p2p/QmUd6zHcbkbcs7SMxwLs48qZVX3vpcM8errYS7xEczwRMA \ | |
/dns4/cluster3.fsn.dwebops.pub/tcp/4001/p2p/QmbVWZQhCGrS7DhgLqWbgvdmKN7JueKCREVanfnVpgyq8x \ | |
/dns4/cluster4.fsn.dwebops.pub/tcp/4001/p2p/QmdnXwLrC8p1ueiq2Qya8joNvk3TVVDAut7PrikmZwubtR |
3:06 PM <olizilla> hallo
3:07 PM <olizilla> i have an idea
3:07 PM <olizilla> i was thinking about "reaching out to other ipfs files share services" https://github.com/ipfs-shipyard/ipfs-share-files/issues/52
3:07 PM <olizilla> and the copy for ipfs-share-files
3:08 PM <olizilla> mostly thinking around why do we make these apps when there are community alternatives
3:09 PM <olizilla> to which i figured 1. it's the dweb, there should be 1000+ file sharing services 2. we do it to learn more about how the protocol should work 3. we do it demo whats possible
3:10 PM <olizilla> but if you were an ipfs app developer, you might not want to invest a ton of time into an idea if you thought the IPFS team might then build a similar thing and have their one become the de-facto first choice
3:11 PM <olizilla> but we still need to build out apps and show whats possible
3:11 PM <olizilla> so
NewerOlder