Skip to content

Instantly share code, notes, and snippets.

@carsonfarmer
Last active September 12, 2021 06:42
Show Gist options
  • Save carsonfarmer/e7062ab30ddd4eb55812d64890db20d3 to your computer and use it in GitHub Desktop.
Save carsonfarmer/e7062ab30ddd4eb55812d64890db20d3 to your computer and use it in GitHub Desktop.
Dynamic Data on IPFS: An Introduction to Buckets and Threads

Dynamic Data on IPFS: An Introduction to Textile's Buckets and Threads

This document: https://tinyurl.com/ybk38mfh
Slides from intro presentation: https://tinyurl.com/y8hxgrq2

Description

In this workshop we’re going to cover tools for building engaging apps, for real users, dealing with real data. We’ll focus on JavaScript app building and we’ll show you simple and fast ways to add IPFS, IPNS, and even potentially Filecoin so you can create interoperable & unstoppable data for your users.

What we’ll cover:

We'll cover in introduction to Textile’s Threads, Buckets, and Hub tools. We’ll also cover some command-line (CLI) actions, and Textile’s JavaScript Client(s). To prepare, we encourage you to install a few things before we get started:

  • For the first section, we'll each be connecting to Textile’s Hub via the hub command-line tools. To do this, you’ll need to have the hub cli installed on your computer. You can get the latest releases from here. Make sure you grab the latest release for your platform.
  • The second part of the workshop will be brief, but if you haven't run a Nodejs app before, you should be sure to set up Node on your computer. Mozilla has a good set of instructions. Even if you aren’t a seasoned Nodejs developer, you should be able to follow along with a few CLI commands. We’ll be playing along with the bucket-photo-gallery example in this demo repo.

Getting Started with the Hub

There are a few resources you'll need before you start writing code. First is the CLI tool.

Get the cli tool

You can grab the latest releases from our releases page. Go ahead and download the release archive for your platform, either Linux or Darwin (for Mac**), similar to hub_v1.0.12_darwin-amd64.tar.gz in the top most release. Download, unarchive, and change your directory into it.

Unpack the archive

You can do this step with your OS or in your terminal.

mkdir hub_v1.0.12_darwin-amd64
tar -xvf hub_v1.0.12_darwin-amd64.tar -C hub_v1.0.12_darwin-amd64
cd hub_v1.0.12_darwin-amd64/
ls
# LICENSE   README.md hub       install

Install the tool

If on *nix-like systems, you can install with the install script. Otherwise, either call it from where it is, or move the binary to somewhere on your PATH. Once you have it installed and available, you can simply call it like any other CLI tool.

./install
# Tada! 🎉🎉🎉 or...

Mac Gotcha

On MacOS you will need to confirm that it is okay to run the hub binary before it will run successfully. Let's fix it now. Run the following

hub --help

You will see a warning, click cancel, and then go to ' > System Preferences… > Security & Privacy' where you will be able to click, 'Allow Anyway'.

If you run hub --help again, you might see one more warning, but this time, you should be able to click 'open' and get going.

Using the CLI

hub --help
# The Hub Client.
# 
# Usage:
#   hub [command]
# 
# Available Commands:
#  buck        Manage an object storage bucket
#  destroy     Destroy your account
#  help        Help about any command
#  init        Initialize account
#  keys        API key management
#  login       Login
#  logout      Logout
#  orgs        Org management
#  threads     Thread management
#  whoami      Show current user
#
#Flags:
#      --api string       API target (default "api.textile.io:443")
#  -h, --help             help for hub
#  -s, --session string   User session token
#
#Use "hub [command] --help" for more information about a command.

Using the Hub

The Hub is a portal where teams and individual developers can access IPFS and soon Filecoin resources easily. The Hub makes it simple to manage and update Buckets on IPFS, persist data for your users on IPFS, deploy and scale Threads databases for your app users, and collaborate on all of it with your team!

Once you have your Hub account, you can start using the Hub to host your data on the IPFS network. IPFS hosting is offered using a Textile technology called, Buckets. Buckets offer you dynamic, folder-based, directories that you can use to persist data on IPFS. They are editable, sharable, and come with free URLs.

Another thing you can do with your account on the Hub, is create an app token that will allow you to use the Hub for persisting user Thread data. Think of it as a trust-less service that your app's database can rely on to quickly store and access data on IPFS!

Accounts

To start using remote services such as IPFS pinning, Bucket sharing, and Thread APIs, you need an account on the Hub. Textile provides a simple, password-less account setup. You can create a new account and username with just an email address:

hub init
# Choose a username: carson█
# Enter your email: [email protected]█
# > We sent an email to [email protected]. Please follow the steps provided inside it.
# ✔ Email confirmed
# > Success! Welcome to the Hub. Initialize a new bucket with `hub buck init`.

If you already have an account on the Hub, you can also simply login, using the same auth flow:

hub login
# Enter your username or email: carson

You can always verify that you have an active session on the Hub and that you are using the correct account by running the hub whoami command.

App Keys

You can use the Hub to help scale your applications on IPFS. The Hub allows you to generate API keys that will grant your app access to your APIs. There are two flavors of API key, Account keys that are not ideal for apps since they will grant admin access to your developer account, and User group keys which are designed for apps where you want one set of keys to grant API access for many users. App keys and tokens are covered extensively in our online documentation. Take a look. While very similar to most standard web 2 developer keys/tokens, there are a few important “quicks” to consider when developing apps designed with user-control of data in mind.

Like many API keys, User group keys come with a key and a secret. You never want to share your secret or save it in a place where it may be exposed. However, when you are in development mode, you can create keys that have signing disabled, meaning no secret is required. This is ideal during development or working with your internal team, because it will make your first steps of app development a bit faster. And this is what we’re going to do here today.

Create insecure keys

To create a new user group key using hub key create and selecting the user group option. In a real world setting, you might create the key in an organization rather than your personal account so that you can invite collaborators later. We’re going to use the key in a subsequent step, so keep those handy if you’re following along. You can create your insecure keys during the key generation step, simply select N for requiring signature authentication.

hub keys create
# ✔ user group
# ? Require Signature Authentication (recommended)? [y/N] N █

Use insecure keys

Now, you can use the insecure keys for building your app without having to first setup a user login flow. However, when you are ready to add that step to your app, be sure to use a new set of keys. You can read more about production setup in our online documentation.

Awesome, we now have pretty much everything we need to get started using the Hub. We’ll also want an app to play with. We recommend starting with a new Typescript web app. We recommend using Typescript, as Textile libraries are in a period rapid of development and type detection is valuable during upgrades. But to save you some startup effort later, we also have some really nice demo apps we’ve already created and pushed to our examples GitHub repo, check them out. Break them. Ask questions. Today, we’re going to take a look at the bucket-photo-gallery demo.

Getting Start with Examples

Let’s go ahead and either download or git clone the examples repo (if you haven’t already), and install the dependencies.

git clone https://github.com/textileio/js-examples.git
cd js-examples/bucket-photo-gallery
npm i

Great, now that we have that out of the way, we’ll mostly focus on the anatomy of this web-app. There are a few key points I’d like to cover, which we’ll do ‘live’ together during the session. The two main concepts we need are:

  1. Hub credentials.
  2. User Identity

We’ve already talked a bit about Hub credentials, so let’s briefly outline how they can be used.

Modifying our Credentials Take a look at line 20 in App.tsx. You see the key entry there? That’s where you’ll embed your developer key (do it now). This is the public component of your developer auth information we generated earlier. Since we generated an “insecure” key, this is all that is needed to use and authenticate with the Hub on your behalf. But beware, you don’t want to deploy your app like this to production, otherwise, you’ll leave yourself wide open to users using your APIs, which longer term could cost you! See our Authentication docs for upgrading to production API keys.

Creating Users You app needs some concept of users. And if you recall from above, depending on how we setup our Hub credentials, these users can get IPFS superpowers that you the app developer provide to them. Your application can grant users access to your Hub APIs very easily. When doing so, the Hub can also help you verify that the users are who they claim to be using encryption. The general flow is as follows:

  1. A user attempts to sign-in by providing their public key.
  2. Your app creates a one-time challenge for the users.
  3. The user signs the challenge with their private key to generate credentials.
  4. Your app verifies the credentials.

Below, we will simplify these steps by using the Hub's helpful token generation endpoint that also does credential verification.

Take a look at the getIdentity function in App.tsx. This is where we generate a new, random Identity for our user, or where we load an existing identity from local storage. There are a number of alternative identity flows you can use and explore. For example, our docs on user identity mention using Metamask, or an identity provider such as 3Box/Ceramic, or others.

Next Steps

Now that we have those main components our of the way, you are free to roam. Break things, pull things apart, reuse components here and there. With this demo as a base, I encourage you to explore ways of using Buckets in your next app. Also, remember that the steps we’ve covered here today are just a sprinkling of the docs that are available online already. So be sure to check those out, follow along with this recording and those pages, and remember to ask questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment