Skip to content

Instantly share code, notes, and snippets.

@tevador
Last active December 10, 2024 20:03
Show Gist options
  • Save tevador/50160d160d24cfc6c52ae02eb3d17024 to your computer and use it in GitHub Desktop.
Save tevador/50160d160d24cfc6c52ae02eb3d17024 to your computer and use it in GitHub Desktop.

JAMTIS

This document describes a new addressing scheme for Monero.

Chapters 1-2 are intended for general audience.

Chapters 3-7 contain technical specifications.

Table of Contents

1. Introduction

1.1 Why a new address format?

Sometime in 2024, Monero plans to adopt a new transaction protocol called Seraphis [1], which enables much larger ring sizes than the current RingCT protocol. However, due to a different key image construction, Seraphis is not compatible with CryptoNote addresses. This means that each user will need to generate a new set of addresses from their existing private keys. This provides a unique opportunity to vastly improve the addressing scheme used by Monero.

1.2 Current Monero addresses

The CryptoNote-based addressing scheme [2] currently used by Monero has several issues:

  1. Addresses are not suitable as human-readable identifiers because they are long and case-sensitive.
  2. Too much information about the wallet is leaked when scanning is delegated to a third party.
  3. Generating subaddresses requires view access to the wallet. This is why many merchants prefer integrated addresses [3].
  4. View-only wallets need key images to be imported to detect spent outputs [4].
  5. Subaddresses that belong to the same wallet can be linked via the Janus attack [5].
  6. The detection of outputs received to subaddresses is based on a lookup table, which can sometimes cause the wallet to miss outputs [6].

1.3 Jamtis

Jamtis is a new addressing scheme that was developed specifically for Seraphis and tackles all of the shortcomings of CryptoNote addresses that were mentioned above. Additionally, Jamtis incorporates two other changes related to addresses to take advantage of this large upgrade opportunity:

  • A new 16-word mnemonic scheme called Polyseed [7] that will replace the legacy 25-word seed for new wallets.
  • The removal of integrated addresses and payment IDs [8].

2. Features

2.1 Address format

Jamtis addresses, when encoded as a string, start with the prefix xmra and consist of 196 characters. Example of an address: xmra1mj0b1977bw3ympyh2yxd7hjymrw8crc9kin0dkm8d3wdu8jdhf3fkdpmgxfkbywbb9mdwkhkya4jtfn0d5h7s49bfyji1936w19tyf3906ypj09n64runqjrxwp6k2s3phxwm6wrb5c0b6c1ntrg2muge0cwdgnnr7u7bgknya9arksrj0re7whkckh51ik

There is no "main address" anymore - all Jamtis addresses are equivalent to a subaddress.

2.1.1 Recipient IDs

Jamtis introduces a short recipient identifier (RID) that can be calculated for every address. RID consists of 25 alphanumeric characters that are separated by underscores for better readability. The RID for the above address is regne_hwbna_u21gh_b54n0_8x36q. Instead of comparing long addresses, users can compare the much shorter RID. RIDs are also suitable to be communicated via phone calls, text messages or handwriting to confirm a recipient's address. This allows the address itself to be transferred via an insecure channel.

2.2 Light wallet scanning

Jamtis introduces new wallet tiers below view-only wallet. One of the new wallet tiers called "FindReceived" is intended for wallet-scanning and only has the ability to calculate view tags [9]. It cannot generate wallet addresses or decode output amounts.

View tags can be used to eliminate 99.6% of outputs that don't belong to the wallet. If provided with a list of wallet addresses, this tier can also link outputs to those addresses. Possible use cases are:

2.2.1 Wallet component

A wallet can have a "FindReceived" component that stays connected to the network at all times and filters out outputs in the blockchain. The full wallet can thus be synchronized at least 256x faster when it comes online (it only needs to check outputs with a matching view tag).

2.2.2 Third party services

If the "FindReceived" private key is provided to a 3rd party, it can preprocess the blockchain and provide a list of potential outputs. This reduces the amount of data that a light wallet has to download by a factor of at least 256. The third party will not learn which outputs actually belong to the wallet and will not see output amounts.

2.3 Wallet tiers for merchants

Jamtis introduces new wallet tiers that are useful for merchants.

2.3.1 Address generator

This tier is intended for merchant point-of-sale terminals. It can generate addresses on demand, but otherwise has no access to the wallet (i.e. it cannot recognize any payments in the blockchain).

2.3.2 Payment validator

This wallet tier combines the Address generator tier with the ability to also view received payments (including amounts). It is intended for validating paid orders. It cannot see outgoing payments and received change.

2.4 Full view-only wallets

Jamtis supports full view-only wallets that can identify spent outputs (unlike legacy view-only wallets), so they can display the correct wallet balance and list all incoming and outgoing transactions.

2.5 Janus attack mitigation

Janus attack is a targeted attack that aims to determine if two addresses A, B belong to the same wallet. Janus outputs are crafted in such a way that they appear to the recipient as being received to the wallet address B, while secretly using a key from address A. If the recipient confirms the receipt of the payment, the sender learns that they own both addresses A and B.

Jamtis prevents this attack by allowing the recipient to recognize a Janus output.

2.6 Robust output detection

Jamtis addresses and outputs contain an encrypted address tag which enables a more robust output detection mechanism that does not need a lookup table and can reliably detect outputs sent to arbitrary wallet addresses.

3. Notation

3.1 Serialization functions

  1. The function BytesToInt256(x) deserializes a 256-bit little-endian integer from a 32-byte input.
  2. The function Int256ToBytes(x) serialized a 256-bit integer to a 32-byte little-endian output.

3.2 Hash function

The function Hb(k, x) with parameters b, k, refers to the Blake2b hash function [10] initialized as follows:

  • The output length is set to b bytes.
  • Hashing is done in sequential mode.
  • The Personalization string is set to the ASCII value "Monero", padded with zero bytes.
  • If the key k is not null, the hash function is initialized using the key k (maximum 64 bytes).
  • The input x is hashed.

The function SecretDerive is defined as:

SecretDerive(k, x) = H32(k, x)

3.3 Elliptic curves

Two elliptic curves are used in this specification:

  1. Curve25519 - a Montgomery curve. Points on this curve include a cyclic subgroup 𝔾1.
  2. Ed25519 - a twisted Edwards curve. Points on this curve include a cyclic subgroup 𝔾2.

Both curves are birationally equivalent, so the subgroups 𝔾1 and 𝔾2 have the same prime order ℓ = 2252 + 27742317777372353535851937790883648493. The total number of points on each curve is 8ℓ.

3.3.1 Curve25519

Curve25519 is used exclusively for the Diffie-Hellman key exchange [11].

Only a single generator point B is used:

Point Derivation Serialized (hex)
B generator of 𝔾1 0900000000000000000000000000000000000000000000000000000000000000

Private keys for Curve25519 are 32-byte integers denoted by a lowercase letter d. They are generated using the following KeyDerive1(k, x) function:

  1. d = H32(k, x)
  2. d[31] &= 0x7f (clear the most significant bit)
  3. d[0] &= 0xf8 (clear the least significant 3 bits)
  4. return d

All Curve25519 private keys are therefore multiples of the cofactor 8, which ensures that all public keys are in the prime-order subgroup. The multiplicative inverse modulo is calculated as d-1 = 8*(8*d)-1 to preserve the aforementioned property.

Public keys (elements of 𝔾1) are denoted by the capital letter D and are serialized as the x-coordinate of the corresponding Curve25519 point. Scalar multiplication is denoted by a space, e.g. D = d B.

3.3.2 Ed25519

The Edwards curve is used for signatures and more complex cryptographic protocols [12]. The following three generators are used:

Point Derivation Serialized (hex)
G generator of 𝔾2 5866666666666666666666666666666666666666666666666666666666666666
U Hp("seraphis U") 126582dfc357b10ecb0ce0f12c26359f53c64d4900b7696c2c4b3f7dcab7f730
X Hp("seraphis X") 4017a126181c34b0774d590523a08346be4f42348eddd50eb7a441b571b2b613

Here Hp refers to an unspecified hash-to-point function.

Private keys for Ed25519 are 32-byte integers denoted by a lowercase letter k. They are generated using the following function:

KeyDerive2(k, x) = H64(k, x) mod ℓ

Public keys (elements of 𝔾2) are denoted by the capital letter K and are serialized as 256-bit integers, with the lower 255 bits being the y-coordinate of the corresponding Ed25519 point and the most significant bit being the parity of the x-coordinate. Scalar multiplication is denoted by a space, e.g. K = k G.

3.4 Block cipher

The function BlockEnc(s, x) refers to the application of the Twofish [13] permutation using the secret key s on the 16-byte input x. The function BlockDec(s, x) refers to the application of the inverse permutation using the key s.

3.5 Base32 encoding

"Base32" in this specification referes to a binary-to-text encoding using the alphabet xmrbase32cdfghijknpqtuwy01456789. This alphabet was selected for the following reasons:

  1. The order of the characters has a unique prefix that distinguishes the encoding from other variants of "base32".
  2. The alphabet contains all digits 0-9, which allows numeric values to be encoded in a human readable form.
  3. Excludes the letters o, l, v and z for the same reasons as the z-base-32 encoding [14].

4. Wallets

4.1 Wallet parameters

Each wallet consists of two main private keys and a timestamp:

Field Type Description
km private key wallet master key
kvb private key view-balance key
birthday timestamp date when the wallet was created

The master key km is required to spend money in the wallet and the view-balance key kvb provides full view-only access.

The birthday timestamp is important when restoring a wallet and determines the blockchain height where scanning for owned outputs should begin.

4.2 New wallets

4.2.1 Standard wallets

Standard Jamtis wallets are generated as a 16-word Polyseed mnemonic [7], which contains a secret seed value used to derive the wallet master key and also encodes the date when the wallet was created. The key kvb is derived from the master key.

Field Derivation
km BytesToInt256(polyseed_key) mod ℓ
kvb kvb = KeyDerive1(km, "jamtis_view_balance_key")
birthday from Polyseed

4.2.2 Multisignature wallets

Multisignature wallets are generated in a setup ceremony, where all the signers collectively generate the wallet master key km and the view-balance key kvb.

Field Derivation
km setup ceremony
kvb setup ceremony
birthday setup ceremony

4.3 Migration of legacy wallets

Legacy pre-Seraphis wallets define two private keys:

  • private spend key ks
  • private view-key kv

4.3.1 Standard wallets

Legacy standard wallets can be migrated to the new scheme based on the following table:

Field Derivation
km km = ks
kvb kvb = KeyDerive1(km, "jamtis_view_balance_key")
birthday entered manually

Legacy wallets cannot be migrated to Polyseed and will keep using the legacy 25-word seed.

4.3.2 Multisignature wallets

Legacy multisignature wallets can be migrated to the new scheme based on the following table:

Field Derivation
km km = ks
kvb kvb = kv
birthday entered manually

4.4 Additional keys

There are additional keys derived from kvb:

Key Name Derivation Used to
dfr find-received key kfr = KeyDerive1(kvb, "jamtis_find_received_key") scan for received outputs
dua unlock-amounts key kid = KeyDerive1(kvb, "jamtis_unlock_amounts_key") decrypt output amounts
sga generate-address secret sga = SecretDerive(kvb, "jamtis_generate_address_secret") generate addresses
sct cipher-tag secret ket = SecretDerive(sga, "jamtis_cipher_tag_secret") encrypt address tags

The key dfr provides the ability to calculate the sender-receiver shared secret when scanning for received outputs. The key dua can be used to create a secondary shared secret and is used to decrypt output amounts.

The key sga is used to generate public addresses. It has an additional child key sct, which is used to encrypt the address tag.

4.5 Key hierarchy

The following figure shows the overall hierarchy of wallet keys. Note that the relationship between km and kvb only applies to standard (non-multisignature) wallets.

key hierarchy

4.6 Wallet access tiers

Tier Knowledge Off-chain capabilities On-chain capabilities
AddrGen sga generate public addresses none
FindReceived dfr recognize all public wallet addresses eliminate 99.6% of non-owned outputs (up to § 5.3.5), link output to an address (except of change and self-spends)
ViewReceived dfr, dua, sga all view all received except of change and self-spends (up to § 5.3.14)
ViewAll kvb all view all
Master km all all

4.6.1 Address generator (AddrGen)

This wallet tier can generate public addresses for the wallet. It doesn't provide any blockchain access.

4.6.2 Output scanning wallet (FindReceived)

Thanks to view tags, this tier can eliminate 99.6% of outputs that don't belong to the wallet. If provided with a list of wallet addresses, it can also link outputs to those addresses (but it cannot generate addresses on its own). This tier should provide a noticeable UX improvement with a limited impact on privacy. Possible use cases are:

  1. An always-online wallet component that filters out outputs in the blockchain. A higher-tier wallet can thus be synchronized 256x faster when it comes online.
  2. Third party scanning services. The service can preprocess the blockchain and provide a list of potential outputs with pre-calculated spend keys (up to § 5.2.4). This reduces the amount of data that a light wallet has to download by a factor of at least 256.

4.6.3 Payment validator (ViewReceived)

This level combines the tiers AddrGen and FindReceived and provides the wallet with the ability to see all incoming payments to the wallet, but cannot see any outgoing payments and change outputs. It can be used for payment processing or auditing purposes.

4.6.4 View-balance wallet (ViewAll)

This is a full view-only wallet than can see all incoming and outgoing payments (and thus can calculate the correct wallet balance).

4.6.5 Master wallet (Master)

This tier has full control of the wallet.

4.7 Wallet public keys

There are 3 global wallet public keys. These keys are not usually published, but are needed by lower wallet tiers.

Key Name Value
Ks wallet spend key Ks = kvb X + km U
Dua unlock-amounts key Dua = dua B
Dfr find-received key Dfr = dfr Dua

5. Addresses

5.1 Address generation

Jamtis wallets can generate up to 2128 different addresses. Each address is constructed from a 128-bit index j. The size of the index space allows stateless generation of new addresses without collisions, for example by constructing j as a UUID [15].

Each Jamtis address encodes the tuple (K1j, D2j, D3j, tj). The first three values are public keys, while tj is the "address tag" that contains the encrypted value of j.

5.1.1 Address keys

The three public keys are constructed as:

  • K1j = Ks + kuj U + kxj X + kgj G
  • D2j = daj Dfr
  • D3j = daj Dua

The private keys kuj, kxj, kgj and daj are derived as follows:

Keys Name Derivation
kuj spend key extensions kuj = KeyDerive2(sga, "jamtis_spendkey_extension_u" || j)
kxj spend key extensions kxj = KeyDerive2(sga, "jamtis_spendkey_extension_x" || j)
kgj spend key extensions kgj = KeyDerive2(sga, "jamtis_spendkey_extension_g" || j)
daj address keys daj = KeyDerive1(sga, "jamtis_address_privkey" || j)

5.1.2 Address tag

Each address additionally includes an 18-byte tag tj = (j', hj'), which consists of the encrypted value of j:

  • j' = BlockEnc(sct, j)

and a 2-byte "tag hint", which can be used to quickly recognize owned addresses:

  • hj' = H2(sct, "jamtis_address_tag_hint" || j')

5.2 Sending to an address

TODO

5.3 Receiving an output

TODO

5.4 Change and self-spends

TODO

5.5 Transaction size

Jamtis has a small impact on transaction size.

5.5.1 Transactions with 2 outputs

The size of 2-output transactions is increased by 28 bytes. The encrypted payment ID is removed, but the transaction needs two encrypted address tags t~ (one for the recipient and one for the change). Both outputs can use the same value of De.

5.5.2 Transactions with 3 or more outputs

Since there are no "main" addresses anymore, the TX_EXTRA_TAG_PUBKEY field can be removed from transactions with 3 or more outputs.

Instead, all transactions with 3 or more outputs will require one 50-byte tuple (De, t~) per output.

6. Address encoding

6.1 Address structure

An address has the following overall structure:

Field Size (bits) Description
Header 30* human-readable address header (§ 6.2)
K1 256 address key 1
D2 255 address key 2
D3 255 address key 3
t 144 address tag
Checksum 40* (§ 6.3)

* The header and the checksum are already in base32 format

6.2 Address header

The address starts with a human-readable header, which has the following format consisting of 6 alphanumeric characters:

"xmra" <version char> <network type char>

Unlike the rest of the address, the header is never encoded and is the same for both the binary and textual representations. The string is not null terminated.

The software decoding an address shall abort if the first 4 bytes are not 0x78 0x6d 0x72 0x61 ("xmra").

The "xmra" prefix serves as a disambiguation from legacy addresses that start with "4" or "8". Additionally, base58 strings that start with the character x are invalid due to overflow [16], so legacy Monero software can never accidentally decode a Jamtis address.

6.2.1 Version character

The version character is "1". The software decoding an address shall abort if a different character is encountered.

6.2.2 Network type

network char network type
"t" testnet
"s" stagenet
"m" mainnet

The software decoding an address shall abort if an invalid network character is encountered.

6.3 Checksum

The purpose of the checksum is to detect accidental corruption of the address. The checksum consists of 8 characters and is calculated with a cyclic code over GF(32) using the polynomial:

x8 + 3x7 + 11x6 + 18x5 + 5x4 + 25x3 + 21x2 + 12x + 1

The checksum can detect all errors affecting 5 or fewer characters. Arbitrary corruption of the address has a chance of less than 1 in 1012 of not being detected. The reference code how to calculate the checksum is in Appendix A.

6.4 Binary-to-text encoding

An address can be encoded into a string as follows:

address_string = header + base32(data) + checksum

where header is the 6-character human-readable header string (already in base32), data refers to the address tuple (K1, D2, D3, t), encoded in 910 bits, and the checksum is the 8-character checksum (already in base32). The total length of the encoded address 196 characters (=6+182+8).

6.4.1 QR Codes

While the canonical form of an address is lower case, when encoding an address into a QR code, the address should be converted to upper case to take advantage of the more efficient alphanumeric encoding mode.

6.5 Recipient authentication

TODO

7. Test vectors

TODO

References

  1. https://github.com/UkoeHB/Seraphis
  2. https://github.com/monero-project/research-lab/blob/master/whitepaper/whitepaper.pdf
  3. monero-project/meta#299 (comment)
  4. https://www.getmonero.org/resources/user-guides/view_only.html
  5. https://web.getmonero.org/2019/10/18/subaddress-janus.html
  6. monero-project/monero#8138
  7. https://github.com/tevador/polyseed
  8. monero-project/monero#7889
  9. monero-project/research-lab#73
  10. https://eprint.iacr.org/2013/322.pdf
  11. https://cr.yp.to/ecdh/curve25519-20060209.pdf
  12. https://ed25519.cr.yp.to/ed25519-20110926.pdf
  13. https://www.schneier.com/wp-content/uploads/2016/02/paper-twofish-paper.pdf
  14. http://philzimmermann.com/docs/human-oriented-base-32-encoding.txt
  15. https://en.wikipedia.org/wiki/Universally_unique_identifier
  16. https://github.com/monero-project/monero/blob/319b831e65437f1c8e5ff4b4cb9be03f091f6fc6/src/common/base58.cpp#L157

Appendix A: Checksum

# Jamtis address checksum algorithm

# cyclic code based on the generator 3BI5PLC1
# can detect 5 errors up to the length of 994 characters
GEN=[0x1ae45cd581, 0x359aad8f02, 0x61754f9b24, 0xc2ba1bb368, 0xcd2623e3f0]

M = 0xffffffffff

def jamtis_polymod(data):
    c = 1
    for v in data:
        b = (c >> 35)
        c = ((c & 0x07ffffffff) << 5) ^ v
        for i in range(5):
            c ^= GEN[i] if ((b >> i) & 1) else 0
    return c

def jamtis_verify_checksum(data):
    return jamtis_polymod(data) == M

def jamtis_create_checksum(data):
    polymod = jamtis_polymod(data + [0,0,0,0,0,0,0,0]) ^ M
    return [(polymod >> 5 * (7 - i)) & 31 for i in range(8)]

# test/example

CHARSET = "xmrbase32cdfghijknpqtuwy01456789"

addr_test = (
    "xmra1mj0b1977bw3ympyh2yxd7hjymrw8crc9kin0dkm8d3"
    "wdu8jdhf3fkdpmgxfkbywbb9mdwkhkya4jtfn0d5h7s49bf"
    "yji1936w19tyf3906ypj09n64runqjrxwp6k2s3phxwm6wr"
    "b5c0b6c1ntrg2muge0cwdgnnr7u7bgknya9arksrj0re7wh")

addr_data = [CHARSET.find(x) for x in addr_test]
addr_enc = addr_data + jamtis_create_checksum(addr_data)
addr = "".join([CHARSET[x] for x in addr_enc])

print(addr)
print("len =", len(addr))
print("valid =", jamtis_verify_checksum(addr_enc))
@tevador
Copy link
Author

tevador commented Aug 8, 2022

I'd like to propose another change to the specs.

The current key hierarchy is vulnerable because if some of the more powerful keys are compromised (for example, due to a faulty signature implementation), all keys in the hierarchy below are also leaked in the process.

This is the new key heirarchy:

c_0
 |
 +- k_s (private spend key)
 |
 +- c_1
     |
     +- c_2
     |   |
     |   +- k_vs (private view-spent key)
     |   |
     |   +- s_vc (secret view-change key)
     |
     +- c_3
         |
         +- c_4
         |   |
         |   +- d_fr (private find-received X25519 key)
         |   |
         |   +- d_ua (private unlock-amounts X25519 key)
         |   
         +- c_5
             |
             +- s_ga (secret generate-address key)
             |
             +- s_ct (secret cipher-tag key)

This heirachy follows the best practice of using each key only for a single purpose. It introduces intermediate nodes c_0 to c_5, which are only used to generate the portion of the key tree below them.

To generate keys, a 512-bit hash is needed, for example Blake2b. Each of the intermediate nodes is hashed to generate the two values below, e.g.:

(k_s, c_1) = Blake2b("c_0", c_0)
(c_2, c_3) = Blake2b("c_1", c_1)
etc.

Additional changes:

  • Renamed k_vb to k_vs ("view-spent" key). This key now serves only to generate linking tags.
  • Introducing a new symmetric key s_vc "view-change key" that is used to generate the shared secret for self-send e-notes. This avoids using the same key in two different contexts.
  • Renamed k_fr and k_ua to d_fr and d_ua to show that those are private keys in a different format (for X25519 Diffie-Hellman).

The full tree above will apply to newly generated wallets. Legacy standard wallets will keep k_s and set c_1 = k_s. Legacy multisig wallets will keep k_s and set c_1 = k_v.

This structure also naturally generates the individual wallet tiers:

  • "Master" wallet needs c_0
  • "View All" wallet needs c_1 and one public key.
  • "View Received" wallet needs c_3 and one public key.
  • "Find Received" wallet needs d_fr.
  • "Generate Address" wallet needs c_5 and 3 public keys.

@tevador
Copy link
Author

tevador commented Aug 8, 2022

Legacy standard wallets will keep k_s and set c_1 = k_s

Actually, I don't see a reason why the Seraphis spend key would need to be the same as the legacy spend key. It's probably better to set c_0 = k_s. This will also mean that the legacy seed and Polyseed are handled the same way.

@UkoeHB
Copy link

UkoeHB commented Aug 8, 2022

Each of the keys kaj , kua can be individually rejected if it fails the mx25519_invkey function (kua when generating a wallet, kaj when generating an address). Then later if mx25519_invkey(kaj , kua) fails, the shared secret can be recovered by inverting each key individually and performing two scalar multiplications instead of one, because r' G = (kaj * kua)-1 Ke = (kaj)-1 (kua)-1 Ke

It is not necessary to reject k^a_j or k_ua. Here is a no-fail algorithm that falls back to 'searching' for a successful inversion.

mx25519_pubkey enote_ephemeral_pubkey;  //K_e = r * K^3_j = r * k^a_j * k_ua * G
mx25519_privkey reverse_DH_privkey;  //1/(k^a_j * k_ua * modifier)
mx25519_pubkey amount_baked_pubkey;  //r G = modifier * reverse_DH_privkey * K_e
static const mx25519_privkey eight = 8;

std::vector<mx25519_privkey> inversion_keys;
inversion_keys.reserve(2);
inversion_keys.emplace_back(k^a_j);
inversion_keys.emplace_back(k_ua);

while (mx25519_invkey(&reverse_DH_privkey, inversion_keys.data(), inversion_keys.size()) != 0)
{
    inversion_keys.emplace_back(eight);  //add 8 to keys to invert (modifier = 8*8*...*8)
    mx25519_scmul(&impl, &enote_ephemeral_pubkey, &eight, &enote_ephemeral_pubkey);  //K_e = 8 * K_e
}

mx25519_scmul(&impl, &amount_baked_pubkey, &reverse_DH_privkey, &enote_ephemeral_pubkey);

It will take me some time to look at your other recent comments properly.

@tevador
Copy link
Author

tevador commented Aug 9, 2022

while (mx25519_invkey(&reverse_DH_privkey, inversion_keys.data(), inversion_keys.size()) != 0)
{
inversion_keys.emplace_back(eight); //add 8 to keys to invert (modifier = 88...*8)
mx25519_scmul(&impl, &enote_ephemeral_pubkey, &eight, &enote_ephemeral_pubkey); //K_e = 8 * K_e
}

Cool. That works.

Btw, I recommend using MX25519_TYPE_PORTABLE when constructing a transaction, since it's not performance-critical and the portable code is less likely to have bugs. When scanning for incoming transactions, MX25519_TYPE_AUTO should be used.

@UkoeHB
Copy link

UkoeHB commented Aug 21, 2022

Introducing a new symmetric key s_vc "view-change [secret]" that is used to generate the shared secret for self-send e-notes. This avoids using the same key in two different contexts.

Tying self-send access directly to the root view key prevents anyone from setting up a remote scanning service that can discover both self-sends and normal enotes without key images or amounts (which you would be able to do with an s_vc secret). Such a tier would be superior to the find-received tier, from a remote scanning service-provider's point of view, because A) you don't have to store nominal address tags for 1/256 of all enotes for every user (probably around 25-30 bytes per view tag match), B) you don't have to transmit view tag matches to users during scanning (around 230 bytes per match + key images), C) the privacy cost seems pretty small relative to the find-received tier.

We designed the find-received tier so service-providers would NOT feel compelled to use a stronger scanning tier. Currently, the next best tier for a remote scanner is full-view (payment validator is stronger but doesn't make scanning more efficient because you still need to save all view tag matches to find self-sends). It's the big privacy gap between find-received and full-view that is supposed to tip the balance in the cost/benefit analysis for remote scanning service providers.

We must not underestimate the potential long-term damage that an s_vc secret could do. Our first priority is to user privacy, user convenience is secondary.

@tevador
Copy link
Author

tevador commented Aug 21, 2022

If that's your concern then I guess we can set s_vc = c_2. Then providing s_vc would be the same as providing k_vs/k_vb, which is not something we can prevent anyways.

@UkoeHB
Copy link

UkoeHB commented Aug 21, 2022

In that case, you still have a view tier that can find received + spent without amounts. Maybe s_vc = c_1. At that point I question what practical differences your proposal has to the existing scheme. 1) wallet tiers store a little less secret data on average, 2) s_ct no longer derives from s_ga, 3) k_vs is introduced (which could be combined with c_5 and k_m U to identify spent normal outputs, which would be enough to identify self-sends).

Overall, the existing scheme is elegant, concise, and well-optimized for the various design considerations that have come up.

@tevador
Copy link
Author

tevador commented Aug 21, 2022

  1. wallet tiers store a little less secret data on average, 2) s_ct no longer derives from s_ga, 3) k_vs is introduced (which could be combined with c_5 and k_m U to identify spent normal outputs, which would be enough to identify self-sends).

Those are all side-effects. The main purpose of the change was this:

The current key hierarchy is vulnerable because if some of the more powerful keys are compromised (for example, due to a faulty signature implementation), all keys in the hierarchy below are also leaked in the process.

@UkoeHB
Copy link

UkoeHB commented Aug 21, 2022

Ok, well I don't see a way to achieve your purpose without adding key combinations that a remote scanning service could implement to the detriment of user and network privacy.

@tevador
Copy link
Author

tevador commented Aug 22, 2022

If the child key leakage is a "feature" rather than a bug, then I guess we can leave it as it is. But it's probably going to come up again in a future audit of Seraphis.

@UkoeHB
Copy link

UkoeHB commented Sep 4, 2022

@tevador pointed out that if an efficient DLP-solver becomes available at any point in the future, then it will be possible to recover k_s from Seraphis ownership proofs since it is pasted raw onto generator U. To improve forward-secrecy of the protocol, we can make the following changes:

  • when generating an address: K_1 = k^j_x X + k^j_u U + K_1_base; K_1_base = k_vb X + k_s U (with spend_key_extension_x = k^j_x and now also spend_key_extension_u = k^j_u)
  • when creating an enote: Ko = Hn("..x..", q, ...) X + Hn("..u..", q, ...) U (with an additional sender extension added to U)

Now if the U component of a onetime address is exposed:

  • passive observers will not be able to link any two spent or unspent enotes together via comparisons of k_s U ?= k_s U
  • if an enote author makes two enotes for two addresses that happen to have the same root keys (i.e. different address indices), the author won't be able to link those enotes after they are spent assuming they don't know beforehand that the addresses have the same root keys

It is necessary to use domain-separated extensions so you can't subtract the X and U components to get something meaningful.

Note: currently there is no known way to prevent a DLP-solver from breaking membership proofs, so forward-secrecy can't be fully preserved. The DLP-solver may also be able to expose amounts (someone with more expertise in bulletproofs would have to comment).

@UkoeHB
Copy link

UkoeHB commented Sep 7, 2022

Follow-up: you can give forward-secrecy to the membership proof by including sender and address extensions on generator G in the same way as X and U.

A DLP-solver will discover t_k + g_extensions when looking at a Seraphis composition proof. He can then set up a system of equations to solve for t_k given a candidate onetime address Ko. The key point is he can solve for t_k given any Ko. In other words, any Ko that he looks at could be the one actually used for the composition proof.

@j-berman
Copy link

j-berman commented Nov 23, 2022

Considering the stated interested in keeping an account-like abstraction where funds across accounts are not commingled, I think it would be beneficial to give an implementation more thought. Expanding from some thoughts laid out by @rbrunner7 in this github issue...

I see 3 main ways to implement accounts:

  1. Sub-divide the address index.
  2. The wallet keeps track of provisioned accounts locally.
  3. Add byte(s) to the address.

1. Sub-divide the address index.

Pros:

  • Maintains accounts similar to how they're implemented today, recoverable from chain data.

Cons:

2. The wallet keeps track of provisioned accounts locally (without immutably tying the address index to an account)

Pros:

  • Since an address index doesn't immutably denote which account the address is a member of, a user can more easily change what addresses belong to which account without on-chain txs.
    • Example: a user decides they actually are comfortable with commingling funds across accounts, or on the flip side that they want separation of funds going forward.
  • It would still be possible to guess which addresses are tied to which account on restore based on how the user has spent enotes.

Cons:

  • It is possible to have imperfect account restoration/inconsistent account states across wallets. This would likely be the source of user confusion as users lose some accounts on restore, and possibly lead to unintended commingling of funds.

3. Add bytes to the address.

Pros:

  • Maintains accounts recoverable from blockchain data.
  • Maintains a 16 byte address index that can be relied upon for clean, robust random address generation.

Cons:

  • Longer addresses that are already pretty long.
  • More data on chain per output.

I bias toward 3. It could be as little as 2-4 bytes. 4 bytes enables the same number of accounts as today (2^32 = 4,294,967,296 accounts). Perhaps if there was more feedback from users/merchants on this feature it would be more helpful toward making a decision. I'm curious to hear if there are more ideas for implementing the abstraction/if I have any of the above wrong.

@UkoeHB
Copy link

UkoeHB commented Nov 23, 2022

@j-berman thanks for the writeup, I agree those are the possible alternatives.

My preference is to option 1 (sub-divide), with a limit to 2 bytes. Option 1 avoids any changes to the spec and minimizes cross-wallet consistency issues.

There would be one general address handling implementation: custom indexing. When setting up your wallet environment, you choose the address handling implementation to inject. The sub-division approach would then be the default strategy for regular user wallets, and all other wallets would have to define their own approach (or just use a default 'no subdivision' implementation). The indexing strategy can be recorded in wallet files to reduce issues from loading a wallet file into an incompatible environment.

It's true you lose some variance for random address generation, but if all enterprise (i.e. high-volume) wallets are using custom indexing then it shouldn't be a problem.

@j-berman
Copy link

j-berman commented Nov 23, 2022

Ok, I'm sufficiently convinced that keeping the spec as is and not adding bytes to the address is ok.

I would agree that sub-dividing 2 bytes for the account and 14 bytes for the address index is an acceptable standard default behavior to move forward with for regular user wallets. Re-looking at the numbers, it would take 8 billion users to generate 1 billion addresses each for there to be a 0.0001% chance (or ~1 in 1 million chance) that at least 1 user has a collision. That's an exceedingly comfortable margin.

from mpmath import mp

NUM_BYTES = 14
NUM_ADDRESSES_PER_USER = 1_000_000_000
NUM_USERS = 8_000_000_000

num_bits = NUM_BYTES * 8

# https://en.wikipedia.org/wiki/Birthday_attack
H = 2**num_bits
n = NUM_ADDRESSES_PER_USER

# P(n) = 1 - e^(-n^2/2H)
p_of_n = 1 - mp.exp(-n**2/(2 * H))

probability_no_users_have_a_collision = (1 - p_of_n)**NUM_USERS
probability_at_least_1_user_has_collision = 1 - probability_no_users_have_a_collision

print("%.10f%%" % (100 * probability_at_least_1_user_has_collision))
# 0.0000888178%

EDIT: I mixed placement of NUM_USERS and NUM_ADDRESSES_PER_USER in the original calculation. The safety margin is even stronger than I first calculated. Now corrected.

@tevador
Copy link
Author

tevador commented Nov 23, 2022

Option 3 looks like the best solution. Sice the implementation already uses the awkward 18-byte tag, which doesn't fit into one cipher block, it wouldn't be much different from a 20-22 byte tag.

When setting up your wallet environment, you choose the address handling implementation to inject.

While this could be stored in the mnemonic seed (Polyseed has space for 3 custom bits), I think it has a big potential to cause confusion.

@j-berman
Copy link

My preference is option 3 still. I agree it has the least room for causing UX issues downstream and is simplest.

And I lean toward 4 bytes.

2 bytes is a limit of 65,536 accounts; I could imagine a use case where merchants want to segregate funds by customer and having a number that low could cause issues for high-volume merchants. Sure, that may not be most efficient/cost-effective for merchants + those merchants could use a custom indexing approach, but that's more work offloaded to them for the future. 4 bytes at 4,294,967,296 accounts seems most future proof to me, and has the benefit of maintaining feature parity with any existing systems that are built to utilize the current account system and presently have >65,536 accounts generated.

@UkoeHB
Copy link

UkoeHB commented Nov 23, 2022

Option 3 looks like the best solution. Sice the implementation already uses the awkward 18-byte tag, which doesn't fit into one cipher block, it wouldn't be much different from a 20-22 byte tag.

I forgot about this: the entire address index must fit into one cipher block since we have no IV (the address index itself is treated like an IV). So option 3 is off the table.

The problem is two address indices that are [block_size + N] bytes with the first [block_size] bytes being the same will have identical [N] leading bytes in the resulting address tag.

@tevador
Copy link
Author

tevador commented Nov 24, 2022

the entire address index must fit into one cipher block since we have no IV

That's solvable. You can derive a synthetic IV using a secret key and the portion of the tag that doesn't fit in the first block. This will cause any single bit difference to avalanche over the whole encrypted tag.

@UkoeHB
Copy link

UkoeHB commented Nov 24, 2022

You can derive a synthetic IV using a secret key and the portion of the tag that doesn't fit in the first block. This will cause any single bit difference to avalanche over the whole encrypted tag.

How do you recover the synthetic IV if the data it's based on is all ciphered?

@rbrunner7
Copy link

First of I think we should consider that we probably talk about a feature that only a minority of people use. Maybe we can make it more popular than today, but I think the fact stays that if your wallet and your Monero use stays below a certain threshold, you won't juggle with multiple accounts.

To make our already scandalously long addresses longer still by going option 3, and make the protocol more complicated again, for such minority-use feature, if we have two alternatives we can choose from, simply does not make sense to me. The basic scheme is now so simple, so elegant, and you want to keep those klunky indexes?

Option 1 would, once a user decided for a certain indexing scheme, be again pretty rigid, and I ask myself what will happen if somebody that uses accounts gives out an address-generating key to somebody else who might then generate addresses with indexes that won't suit the user and where the generated address would be stuck in whatever account the index used puts it, without a way to "move" it.

@j-berman listed a "con" for option 2, "It is possible to have imperfect account restoration/inconsistent account states across wallets". Please consider the fact that this is already possible today, because a number of wallets support accounts, and the rest does not.

But here we are probably talking about what I call "exception squared": The first exception is that somebody does indeed use accounts, and the exception squared is that this person then restores the same wallet on another wallet app, that today does not support accounts or in the future does something differently than the original wall. This must be pretty rare, and probably the reason why we don't hear complaints regularly.

This all leads me to clearly prefer option 2.

@tevador
Copy link
Author

tevador commented Nov 24, 2022

How do you recover the synthetic IV if the data it's based on is all ciphered?

You decrypt it first.

Encryption

Input: key1 (32 bytes), key2 (32 bytes), acct_id (4 bytes), addr_id (16 bytes)
Output: enc_tag (22 bytes)

  1. p0 = acct_id || addr_id[0:11]
  2. p1 = Zeropad(addr_id[12:15])
  3. iv = Hash("synthetic IV" || key1 || p1)
  4. c0 = Twofish_enc(key2, p0 XOR iv)
  5. c1 = Twofish_enc(key2, p1 XOR c0)
  6. return enc_tag = c0[0:5] || c1

Decryption

Input: key1 (32 bytes), key2 (32 bytes), enc_tag (22 bytes)
Output: acct_id (4 bytes), addr_id (16 bytes)

  1. x1 = Twofish_dec(key2, enc_tag[6:21])
  2. p1 = Zeropad(enc_tag[0:5] XOR x1[0:5])
  3. if p1[4:5] != 0x0000 then fail.
  4. iv = Hash("synthetic IV" || key1 || p1)
  5. p0 = Twofish_dec(key2, enc_tag[0:5] || x1[6:15]) XOR iv
  6. return acct_id = p0[0:3], addr_id = p0[4:15] || p1[0:3]

Any single bit change will completely randomize the tag, e.g.

00000000 e13c7ac8461f1269c3cb977214947464 -> 38df8a3f2c5c9de9090dcfe89eb8a50688478758ecf9
01000000 e13c7ac8461f1269c3cb977214947464 -> e083b9bceeaf994e5a650f1de9d60bc6ebaf7a738dbd
01000000 e13c7ac8461f1269c3cb977214947465 -> be8caa8393bd4290058982936b9c133c645b2ca1244d

@tevador
Copy link
Author

tevador commented Nov 25, 2022

There is a fourth option.

4. Account index encoded in the output public key

Let's consider a 32-bit account index i and a 128-bit address index j.

The address spend key (the first public key in a Jamtis address) would be calculated as:

K1 = Ks + (i / kacct_g + kaddr_g) G + kaddr_x X + kaddr_u U

where Ks is the wallet public spend key and the private key extensions kacct_g, kaddr_g, kaddr_x,kaddr_u are derived from a secret key sga and the address index:

kacct_g = Hq(sga || Tacct_g || j)

kaddr_g = Hq(sga || Taddr_g || j)

kaddr_x = Hq(sga || Taddr_x || j)

kaddr_u = Hq(sga || Taddr_u || j)

One-time output keys have the form of:

Ko = K1 + Ksender

where K1 is the public key of the address the payment was sent to and Ksender is a public key derived by the sender from the shared secret.

When decoding an output, the wallet will decrypt j, derive the key extensions and calculate

Kacct' = Ko - Ksender - Ks - kaddr_g G - kaddr_x X - kaddr_u U

If the result is the identity element E, the payment belongs to the default account i == 0 and nothing else needs to be done. If Kacct' != E, the wallet will need to calculate a 32-bit discrete log of Kacct = kacct_g Kacct' to recover the account index i. This can be done quickly using a 1 MB precomputed table.

Pros:

  • No negative impact on address length or transaction size.
  • Accounts are recoverable from chain data.
  • Keeps 16-byte j for robust random address generation.
  • Transparent for wallets that don't use accounts (if i == 0, it falls back to the current address derivation).

Cons:

  • More CPU cycles/memory needed to recover the account index (roughly 10 ms + 1 MB of memory).

@UkoeHB
Copy link

UkoeHB commented Nov 25, 2022

Ok I agree now that an index bigger than the cipher block size is technically workable. Another method would be a triple-cipher similar to the current double-cipher.

Given 18-byte index, 2-byte MAC [0..17 index || 0..1 MAC]:

  1. a = [0..1 index || cipher[s_ct](2..17 index) || 0..1 MAC]: cipher middle block size bytes (trailing bytes of the index)
  2. b = [0..1 a XOR 16..17 a || 2..19 a]: XOR bytes that hang above the first block size bytes into the leading bytes
  3. c = [cipher[s_ct](0..15 b) || 16..19 b]: cipher leading block size bytes (leading bytes of the index)
  4. d = [0..15 c || 0..3 c XOR 16..19 c]: XOR the leading bytes into the trailing bytes
  5. e = [0..3 d || cipher[s_ct](4..19 d)]: cipher the trailing block size bytes (trailing bytes of the index + MAC)

Then you go in reverse to get the MAC after one decipher + XOR, and the index after two additional deciphers + one XOR.

At the end of the day these 'more bytes for account index' solutions don't address the underlying problem, which is that some wallets want to support accounts (e.g. the CLI because it's a demanded feature) and some wallets want a custom indexing scheme (e.g. exchanges that want to embed payment IDs in the address index). The bottom line is custom indexing means custom implementations will exist that are incompatible with the 'default' account scheme. EDIT: therefore the simplest solution is the best one - don't do anything special, just treat 'address index + account index' as one possible custom interpretation of the index (the default interpretation for wallets).

@tevador
Copy link
Author

tevador commented Nov 26, 2022

I have some updates about the address checksum.

I used the tools published by Bitcoin devs to test almost 6000 different degree-8 BCH generators with a design length of 512. While Jamtis addresses are only 196 characters long, using a generator that is efficient for longer payloads might be useful for future features such as certified addresses or XMR invoices.

Here are the 3 best generators together with generators TMKLTI used by Bitcoin and J3PBP3J1 used by Bitcoin Cash.

Generator Len4 Len5 Len6 Len196 (4 err) Len196 (5 err) Len196 (6 err)
F93A29K1 >512 131 34 0.0000000000000 0.4892974701892 0.9928562124433
J3PBP3J1 >512 130 33 0.0000000000000 0.6926081087057 0.9982059678015
6DP7JN5S >512 130 26 0.0000000000000 0.6686892100567 1.0078263591470
I2EIGB9E >512 130 26 0.0000000000000 0.6686892100567 1.0078263591470
TMKLTI 89 18 8 1134.5894215779 1007.9527885898 1024.1499338882

Len4, Len5 and Len6 are the maximum address lengths which allow 4, 5 and 6 errors to be detected, respectively. The Len196 colums show the probability that 4/5/6 errors will not be detected in a 196-character address. These probabilities are scaled by a factor of 240, so they are directly comparable to the detection probability of hash-based checksums (values less than 1 mean stronger detection capability than a 40-bit hash). Bitcoin has a handicap here because it only uses a degree-6 generator, which is about 1000x weaker.

None of the generators can reliably detect 5 or more errors for Jamtis addresses, but I already expected this, so no surprises here.

The generator F93A29K1 seems to be the best choice, outperforming even the Bitcoin Cash generator. I therefore propose a checksum algorithm based on the python code below. Note that the value of M is only tentative. The choice of M might affect the performance of the checksum against insertion errors, which are irrelevant for fixed-length addresses. Bitcoin devs also developed a tool to search for optimal values of M, but our search space is 1000x times larger and it might be infeasible to do an exhaustive analysis like they did.

# Jamtis address checksum algorithm proposal

GEN = [0x7a46a12681, 0xf48d424822, 0xab58143444, 0x1eb0286888, 0x377244f510]
M = 0xffeffffeff

def jamtis_polymod(data):
    c = 1
    for v in data:
        b = (c >> 35)
        c = ((c & 0x07ffffffff) << 5) ^ v
        for i in range(5):
            c ^= GEN[i] if ((b >> i) & 1) else 0
    return c

def jamtis_verify_checksum(data):
    return jamtis_polymod(data) == M

def jamtis_create_checksum(data):
    polymod = jamtis_polymod(data + [0,0,0,0,0,0,0,0]) ^ M
    return [(polymod >> 5 * (7 - i)) & 31 for i in range(8)]

# test/example

CHARSET = "ybndrfg8ejkmcpqxot1uwis2a345h769"

addr_test = (
    "xmr1majob1977bw3ympyh2yxd7hjymrw8crc9kinodkm8d3"
    "wdu8jdhf3fkdpmgxfkbywbb9mdwkhkya4jtfnod5h7s49bf"
    "yji1936w19tyf39o6ypjo9n64runqjrxwp6k2s3phxwm6wr"
    "b5cob6c1ntrg2mugeocwdgnnr7u7bgknya9arksrjore7wb")

addr_data = [CHARSET.find(x) for x in addr_test]
addr_enc = addr_data + jamtis_create_checksum(addr_data)
addr = "".join([CHARSET[x] for x in addr_enc])

print(addr)
print("len =", len(addr))
print("valid =", jamtis_verify_checksum(addr_enc))

@hbs
Copy link

hbs commented Nov 27, 2022

Glad to see there is an interest for preserving the notion of accounts.

I am convinced that 2 bytes for account (leading to 65536 possible accounts) is too low for many use cases, a minimum of 4 bytes (4 billion accounts) should be considered to at least offer the same number of accounts as of today.

@tevador
Copy link
Author

tevador commented Nov 28, 2022

some wallets want to support accounts (e.g. the CLI because it's a demanded feature) and some wallets want a custom indexing scheme (e.g. exchanges that want to embed payment IDs in the address index)

Option 4 proposed above can support 4 billion accounts without touching the address index. I think we should discourage custom implementations from embedding data into the address index. It's supposed to be a randomly generated unstructured identifier to achieve the best collision resistance.

In fact, if j is just a randomly generated index, there is no need to encrypt it at all. Address generation can work as follows:

  1. j = get_random_bytes(16)
  2. mac = H2(sga || Tmac || j)
  3. (derive private keys based on j)
  4. return the tuple (K1, K2, K3, j, mac)

This removes a dependency on the Twofish block cipher and avoids the convoluted encryption of the address index. I think that's a big plus for a future security audit.

@UkoeHB
Copy link

UkoeHB commented Nov 28, 2022

I think we should discourage custom implementations from embedding data into the address index. It's supposed to be a randomly generated unstructured identifier to achieve the best collision resistance.

The index is not 'supposed to be a randomly generated unstructured identifier' - it's supposed to be a generic tool that enables randomly generating addresses. Making indices cleartext would have long-term privacy costs for people who actually want structured indices (i.e. want them more than the privacy cost).

The more I think about it, the more I dislike the idea of baking account indices into the protocol (whether seraphis itself, or jamtis). It excessively elevates the influence of core wallet design decisions on protocol design. There will always be lobbyists who want more bytes to paste in whatever feature they want. Let's not set a precedent here.

@j-berman
Copy link

It's a feature that appears to be used and very much so appreciated as is. Its optimal implementation would function smoothly across user-facing wallets, even on restore. I think the route that has the highest likelihood of getting there is one where the spec defines an implementation.

I personally like option 4. The primary downside seems like added internal complexity, and it seems it would have no practical impact on UX. The gain is that anyone who wants accounts as they are today, a feature which we know has seen solid usage and appreciation at 4 bytes, would get a smooth UX by default.

Are we aware of any custom subaddress index implementations in the wild today? It seems unlikely there would even be a custom implementation provided accounts are offered out the gate with the same number of bytes people use and expect today.

@UkoeHB
Copy link

UkoeHB commented Nov 28, 2022

The primary downside seems like added internal complexity, and it seems it would have no practical impact on UX.

Just because you can do something, doesn't mean you should. Stuff like this just looks like bloat/cruft to me.

Are we aware of any custom subaddress index implementations in the wild today? It seems unlikely there would even be a custom implementation provided accounts are offered out the gate with the same number of bytes people use and expect today.

No, because subaddresses currently require a look-ahead. However we do have integrated addresses with custom-defined payment IDs. The substitute for payment IDs in jamtis is... custom address indexing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment