Skip to content

Instantly share code, notes, and snippets.

@yusefnapora

yusefnapora/libp2p.md

Created July 29, 2019 21:00
Show Gist options
  • Save yusefnapora/ed0c97ecf8bc368afededfb38241fb8e to your computer and use it in GitHub Desktop.
Save yusefnapora/ed0c97ecf8bc368afededfb38241fb8e to your computer and use it in GitHub Desktop.
Download ZIP
EthBerlinZwei libp2p resources
Raw
libp2p.md

libp2p

libp2p is a modular framework and protocol suite for developing peer-to-peer applications. Originally part of the IPFS project, libp2p has evolved to meet the needs of a growing community of developers and users.

There are many implementations of libp2p, with go-libp2p being the oldest and most "feature complete". Javascripters can use js-libp2p in Node or browsers, and rustaceans can dig into rust-libp2p.

Other language implementations are in developmement, but if your favorite language isn't quite there yet, the libp2p daemon allows you to run "libp2p as a service" in an external process. Check out the supported language bindings to see if there's already a binding for your language.

Features

  • Ensures secure, multiplexed streams between connected peers. Security and multiplexing protocols are modular, and peers can negotiate supported configurations.
  • Pluggable transport modules allow support for TCP, WebSockets, QUIC & WebRTC (if supported by the language / runtime). Nodes can support multiple transports for interop / network flexibility.
  • Kademlia-like DHT provides peer routing (to lookup network addresses by peer id), content routing (to look up content / service providers by hash key), and a public key lookup service.
  • Extensible PubSub protocol allows broadcast messaging to peers that are subscribed to a topic.
  • Can automatically traverse (some) consumer NATs using UPnP.
  • Local peer discovery using mDNS.
  • Circuit relay protocol allows peers behind NAT boundaries to communicate, using a relay node as intermediary.

Prizes Offered

1. 2.

Getting Started

Some background info before diving in to code:

  • Peer identity is based on public keys, so starting a node requires generating a keypair or loading one from disk.
    • Peer ids are multihashes of the public key. They get encoded to strings in base58, and generally use the SHA2-256 hash algorithm. Since the multihash prefix for SHA2-256 is Qm, libp2p peer ids generally start with Qm, e.g.: QmVT6GYwjeeAF5TR485Yc58S3xRF5EFsZ5YAF4VcP3URHt.
  • The core abstraction for communication between peers is the Stream - a reliable, ordered channel that's multiplexed over some transport connection. Streams are cheap to open once the initial connection is made, so it's okay to open a new stream for a quick request/reply sequence.
  • You can define new libp2p protocols by creating a handler function and registering it with libp2p. When registering your protocol, you give it a string ID, which must be globally unique and should contain a version number, e.g. /my-app/my-protocol/1.0.0.
  • When you open a new stream to a remote peer, you send the protocol ID you want to use. If the remote peer has a handler registered for the protocol, it will be invoked. Protocol handlers take two arguments, the protocol ID and the Stream used for communication.
  • The semantics of your protocols are entirely up to you; there's no built-in RPC framework in the vein of gRPC, etc. The core libp2p protocols generally exchange protobuf messages, prefixed with their length in bytes as an unsigned varint.
  • Network addresses are encoded as multiaddrs. There's a draft spec in progress that has some detail about valid multiaddr constructions.
  • mDNS discovery will return all nearby libp2p peers, so if you use it at a hackathon, don't be surprised if you discover a bunch of peers that don't speak your custom protocol and error out when you try to dial them.

The libp2p docs site is a work in progress, but has some useful info. The concepts section is especially helpful if you're new to libp2p. There are also quick tutorials for go-libp2 and js-libp2p.

Code examples:

Detailed specs are available in the specs repository for most core libp2p wire protocols and abstractions.

Need Help

Check out the libp2p discussion forums to ask questions & find out about new developments in libp2p. Tagging Stack Overflow questions with libp2p is also a good way to get them seen by people in the community. The #libp2p IRC channel on freenode can also be a good venue for a quick question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment