Skip to content

Instantly share code, notes, and snippets.

@carlos8f

carlos8f/.privet Secret

Last active August 29, 2015 13:57
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save carlos8f/4dff000a439e884f1fb1 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save carlos8f/4dff000a439e884f1fb1 to your computer and use it in GitHub Desktop.
Download ZIP
privet - secure data housing
Raw
.privet
{
"id": "NiBQ2gFHw2e6LTni",
"author": "carlos",
"fingerprint": "6f:2a:cd:0b:02:19:35:ab:62:65:58:c7:91:be:32:64",
"shasum": "3f4bf339c1e1cdc9385842176ca3167d6e402cab",
"signature": "13f2f0d26f32c578394d34d261ab90e74b01693a",
"grants": {
"carlos8f": "w",
"group/te_developers": "w",
"*": "r"
},
"clock": {
"carlos": 2,
"brian": 1
}
}
{
"id": "carlos/about",
"author": "carlos",
"fingerprint": "6f:2a:cd:0b:02:19:35:ab:62:65:58:c7:91:be:32:64",
"shasum": "3f4bf339c1e1cdc9385842176ca3167d6e402cab",
"signature": "13f2f0d26f32c578394d34d261ab90e74b01693a",
"grants": {"*": "r"},
"clock": {
"carlos": 5
}
}
Raw
alpha.md

privet alpha

  • use RSA 4096 pair, stored at ~/.pvt/id_rsa
  • use AES 256 for encryption
  • some kind of key exchange strategy (bitcoin style, enter "privet address"?)
  • push uploads encrypted headers, encrypted payload, gets back resource url, bucket id + clock
  • server subject to data mining (a necessary evil)
  • to check mail, use server bucket id, last clock (stored in local vector clock)
  • data on server is immutable, but has expire time, max 30 days (except for delete by recipient)
  • id generation scheme TBD. perhaps there are no ids necessary (bucket_id and clock are enough?)
  • reasonable cap on payload size

privet complete

  • fixed random padding on payloads, linking scheme
  • move to NTRU or equivalent. RSA may already be cracked, and performance and bandwidth are factors!
  • built-in diffie-hellman transport security (no SSL needed)
  • timed delete by sender
  • public data option (unencrypted)
  • choosable id for public data
  • advanced keyserver w/ searching
  • instant messaging app or something else fun
  • servers can require pre-registration and signature authentication
  • postage fees
  • bitcoin addresses tied to identities, hook up bitcoin private key to pay postage from wallet
Raw
also.md

see also

Raw
commands.md

commands

pvt identify

  • prompt for name, email
  • send to server public fingerprint + identity + contact info

pvt init

  • creates .pvt folder with:
    • new id (random 16-character hash)
    • sig(id) using private key to serve as auth
    • later contains sha1, sig(old sha1 + separator + new sha1) to create history tree, prevent conflicts

pvt status

  • see how ahead or behind we are
  • generate signature and compare with remote

pvt put [--save] [url]

  • tar and encrypt local dir in tmp
  • generate sig(old sha1 + separator + new sha1)
  • could err if remote copy is newer
  • if no url stored in .pvt, store it
  • if no url specified, use stored one
  • set stored url with pvt set url <url>
  • get current url with pvt get url

pvt share [path]

  • prompt for passphrase, blank to auto-generate
  • put a special version of file or dir at [path] (defaults to .) which can be decrypted with passphrase
  • passphrase can allow downloading decrypted version from the server by passing passphrase a number of ways
  • returns download url, as tarball (if target is dir) or file. GET variable if https, otherwise curl command w/ basic auth

pvt unshare [path]

  • revokes share access to path

pvt fetch [--export] [dest-name]

  • fetches remote dir and places it in dest-name or dir named after id
  • decrypt after transfer using private key
  • supports file or dir subpaths
  • if dest-name is ., need to confirm a prompt to overwrite cwd
  • if --export, removes .pvt folder

pvt sync [-f, --force]

  • put or fetch based on whether newer or older than remote copy

pvt delete

  • delete copy on server
  • need to confirm w/ cli-prompt
  • uses signature to authenticate
Raw
detail.md

how to upload data

  • (optional) ssh-keygen genrsa, and put in ~/.pvt
  • (optional) register the key: pvt register (enter server url, username, real name, email, pubkey nickname)
  • (?) obtain a session key: pvt login|identify (set server url, prove that you have the pubkey, exchange for session token)
  • how to prevent replay attacks relating to session token? sign every request?
  • pvt <dir/file>
    • look up local cache to see if there's a previous version, cancel if same
    • encrypt? (yes)
    • enter recipient ssh pubkey, fingerprint, email, or username
    • confirm server url
    • tar/gzip the dir/file,
      encrypt data using random aes key,
      encrypt headers with same key,
      sign both and put in signature file,
      put aes key in secret file,
      encrypt secret w/ rsa
      not sure how to send headers+data to server
      send session key with it (server should have full pubkey)
      could prompt for OTP at this point
    • server runs hash ring function (should apply salt before)
      on pubkey (from session key) to get bucket id
      increments logical clock
    • pubkey can NOT be determined from /{bucket_id}/{id}
    • on cli, we get back server url (incl. bucket id and logical clock), store in local cache

how to check mail

  • send session key to server
  • server looks up pubkey, runs hash ring function, sends back bucket id
  • cli requests OPTIONS /{bucket_id}/{last_logical_clock} (headers), tries decrypting locally
  • if success, that is one "unread mail"
  • note unread mail in local cache, after "reading" (downloading), remove unread flag
  • increment logical clock to try reading next mail

local cache

  • could be in local .pvt dir or in ~/.pvt
  • use known_hosts file?
  • flat files or leveldb?
  • stores registered server list, session keys, bucket id for each server
  • could store "revisions", i.e.
    local path, server url timestamps, change memos/comments
    for each time we pushed the file/dir
    this way the history isn't kept on the server, can't be traced without knowledge of each bucket/id combo, but could be reassembled via mail check

directory storage

  • could be done with tar (optional or mandatory?)
    how would single-file download from server work? paths inside tar?
  • could be done with multiple uploads and "links" (more server intensive)
Raw
intro.md

emx

  • EMX: Encrypted Message eXchange
  • users don't have to trust servers. servers are merely the custodians of identities (pubkeys), and receive data they can't decrypt or match with identities.
  • protocol is designed for a minimum of info leakage (web of trust is not revealed)
  • security by default. full end-to-end encryption, combined with additional diffie-hellman ephemeral encryption, is the default, but can be disabled.
  • email-like address syntax (user@host). allows people to send you EMX using your email address.
  • optional signature for message verification.
  • no central server, signing authority, or agency required.
  • optionally leverage DNS TXT records for custom EMX addresses
  • in the future, identity could be bitcoin address, and postage could be paid with bitcoin
Raw
mechanism.md

mechanism

  • privets are:
    • encrypted files (or tarballs) identified by id, sha1, fingerprint, and signature
    • immutable, but can be deleted from server
  • peer-to-peer secure cloud storage, dropbox/email replacement, distributed keyserver
  • data hosted on your own (local) server, unless you have an account on another server and push it there
  • ids could be replicated by peers (on a voluntary basis), with url pointer to data, as a "soft replication", avoids burden of replicating other peoples' data
  • easy way for entities to share data and identities securely
  • uses ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub to securely transfer and decode tarballs (piggy-backs on existing keys)
  • much like a scp which does not depend on sshd
  • can encode a passphrase-protected version and serve decoded from server
  • use vector clocks, well-known server public keys and signatures to ensure commits are on server, and prevent overwrite conflicts
  • server doubles as keyserver, with voluntarily-submitted details for pubkeys
  • server stores each revision, fully encrypted, delta compressed
  • privets can have authorized_keys list for collaborations, dropboxes
  • acts as "secure history-keeping remote backup service" and "decentralized secure dropbox"
  • servers can't actually decrypt any of the data, they (blindly) curate it
  • possible replacement for email! encrypt messages for a certain user using their public key
  • could be a hosted secure data service (like mega), pay for each pubkey added, storage
  • needs to have some configuration, namely where to find pubkey on disk (default ~/.ssh, prefs file in ~/.pvt)
  • keyserver part: pvt identify proves to server that you hold private key, cliams username, allows altering public profile, including email and website
  • other users can sign a user's key with pvt sign <username>
  • example: manage secure email box, stored in leveldb, encrypted on server
  • for keyserver, need to check United States Patent 6336186
  • key registration adds a level of accountability, desirable for some providers
  • should be impossible to remove a registered user, for history purposes (hash tree?)
  • keys could be "authenticated" by their claiming users by tying a facebook, twitter, or github account using OAuth
  • keyserver could have virtual groups, like github teams
  • support multiple fingerprints per user: enables workflow where user can use multiple machines, multiple key pairs, but one username, kind of solves "keyserver fossil/plaque"
  • keypair could be stored in ~/.pvt dir, or in local dir .pvt
  • peer-to-peer replication of keys and identities, everyone (could be required to?) run a keyserver
  • could use sendmail to send key-verification emails to key owners
  • i wish these ideas would stop! :)
  • each user could be stored via privets?
  • each user runs a server, so that is the "local backup" of each revision? use leveldb for portability and compression?
  • p2p server could start up a web interface on a local port, to explore data, sign stuff, decrypt or encrypt stuff for a certain user
  • cli could ship with a "canonical" keyserver, with s3 storage? how to deal with storage cost. per-user dropbox api? subscription?
  • cannonical server could have "starter" list of peers to connect to, gossip keys, etc
  • out of the box, could be used for a simple encrypted cloud backup system.
  • for group sharing,
    • new keypair generated on server.
    • members of group can request access token to shared resource (OAuth)
    • access token points to private key on server (and privet id / paths)
    • server can disclose plaintext (over ssl) if valid access token presented
    • how to maintain plausible deniability?
  • alternatively,
    • according to grants in .privet file, additional copies are signcrypted using recipients pubkeys and uploaded
    • server still maintains deniability
    • recipients can download/unencrypt at will using their keys
    • only applies to a single "push", subsequent pushes have new session key
  • forward could be provided by generating new privet-specific keypair upon each push
  • key sharing server: given a list of pubkeys that are trusted with given resource, allow them to fetch private key (securely, for a particular revision) by signature
  • should be future-proof by allowing custom ciphers, multiple options for encryption strategy
  • senders should (or shouldn't?) be able to revoke messages they've sent (delete from server, use case: employee resignation), recipients can also delete
  • should probably use specific, rotatable/refreshable private keys
  • pick a cryptosystem that is good (elliptical?), use lots of bits
  • push should incorporate some kind of hashcash, to curb spam, and "validate" it
  • local cache visualization: ASCII art "git tree graph" of pushed files
  • use email-like pubkey/keyserver aliases: [email protected] would look up TXT record for s8f.org, name EMX (like an MX record), get pointed to emx.s8f.org, and end up at pubkey https://emx.s8f.org/carlos8f.keys (in ssh format)
  • for CC functionality, payload could be encrypted once via AES, and key could be encrypted each time for every recipient. (Then all recipients can decode the AES key, and potentially see how many copies exist on the server. Idea: separate payload storage from headers, so there would only be one copy of the payload, and copies of the headers for each recipient.)
  • consider "supercedes" functionality... a message could supercede a previous one (but not delete old copy), if sent from the same identity?
  • difficulty should be primary way servers can control flow. should be adjustable based on message type (public, private, key). server-verification of identity (can be public or private via shared secret) should be separate and optional. spam is discouraging to engage in because all write operations require proof-of-work.
Raw
order.md

order of keypair detection preference

  1. .pvt/id_rsa(.pub)
  2. .pvt/id_dsa(.pub)
  3. ~/.pvt/id_rsa(.pub)
  4. ~/.pvt/id_dsa(.pub)
  5. ~/.ssh/id_rsa(.pub)
  6. ~/.ssh/id_dsa(.pub)
Raw
pitfalls.md

pitfalls

  • share data with group without giving away deniability
  • an email address "in the process" is a problem (http://secushare.org/PGP#sec-1)
    should be no temptation to send unencrypted things (via email)
  • deep packet inspection could identify nature of communication (PGP mail), therefore set off "red flags" in the network
  • PGP mail has meta info leakage, such as, who you're talking to
  • by including the recipient's public key in the message (PGP without –hidden-recipient), anonymous security is violated
  • many internet protocols still work using invalid certificates for TLS. this opens the possibility of a man-in-the-middle attack using an invalid cert the attacker created, and despite a warning, the communication would still succeed.
  • if it's possible to run a server without using proper TLS, someone will do it (also a OAuth 2.0 problem)
    solution: diffie-hellman exchange should end-to-end encrypt everything including headers
  • PS. Note that protecting the server's private key on the disk using PIN or passwords is futile as the key resides unencrypted in the server's memory. http://nmav.gnutls.org/2011/12/price-to-pay-for-perfect-forward.html
  • Don't use SHA1, it will be compromised
  • Not smart to require RSA or any NIST-recommended elliptic curves!
  • short key IDs can be spoofed - http://www.asheesh.org/note/debian/short-key-ids-are-bad-news.html
  • Man in the middle attack against keyserver if keyserver uses an auth besides the key itself
  • how to do initial key exchange properly. QR code on business card?
  • assume RSA, SHA256, and standard Diffie-Hellman are all worthless. Need new ephemeral/rotated Diffie-Hellman system or something, with signatures (Bitcoin).
  • need to figure out long-term key/identity exchange strategy! protect against active attacker!
  • end-to-end security means malware can travel without detection
Raw
questions.md

questions

  • what is the minimum set of properties for a privet? user-definable or strict?
  • should it use sign-before-encrypt, or encrypt-before-sign? is "signcryption" an option yet?
  • what is the set of acceptable information leakage? what is the minimum possible identity? (anonymous?)
  • lean on TLS or implement native diffie-hellman end-to-end?
  • desktop app or cli?
  • server required or optional for p2p?
  • use elliptic cryptography?
  • use hard-coded schemes or ciphers?
  • efficient storage backend? leveldb? tarballs?
  • should complete version history be available, or an option, or required?
  • store identities as privets? or separately?
  • emails, usernames, fingerprints, or random ids should be used for identity? first come first serve? externally backed, such as DNS TXT record?
  • use tar or blobs/paths/links?
  • build TOR into client?
  • to prevent guessing based on encrypted payload size, consider padding the data to fixed length? spanning system to store large files?
  • should id be chosen by client, or fixed to sha256 of payload? depends on whether it's public or not?
Raw
server.md

server api

OPTIONS /{id}

  • unencrypted http header should contain logical clock, "fingerbyte" (first or last X bytes of fingerprint)
  • encrypted body, not signed
  • capped size to prevent "header bomb"
  • contains http-like headers (content-type, length), could be json
  • optionally contains payload sha1, author pubkey + signature for verification of data

GET /{id}

  • encrypted
  • optional signature (before encryption) in headers
  • arbitrary size

POST /since?clock={logical clock}&fingerbyte={first or last X bytes of fingerprint}

  • fetch list of ids since logical clock, optionally filtered to a fingerbyte
  • allows "checking of mail" without leaving ANY public or private identities on server

method

  • by encrypting headers, secrecy of recipient protected. private key decrypts headers
  • to "check mail", iterate through all headers (added since last check, use vector clock, limited byte separation), and try decrypting
  • could be organized by first (or last) X pubkey fingerprint bytes, for efficiency in lookups
  • first byte doesn't disclose a whole lot
Raw
uses.md

use cases

  • private code repo backups
  • bitcoin wallet
  • secure messaging (i.e., email)
  • media hosting
  • twitter-like microblogging
  • dropbox replacement
  • gist replacement
  • social network
  • key/identity/oauth server
  • distributed database
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment