Why mAPI is shit

mapi.md

A far from exhaustive list of reasons why mAPI is shit

1. While it's an acceptable bandaid for now, we probably shouldn't just wrap JSON-RPC. It's really slow, and there's no reason to run an HTTP server that accesses another HTTP server. It's pretty silly tbh.

2. Nobody likes C#. Like, nobody.

3. You shouldn't have to go and install an entire postgres database in parallel to a mining node (both of which just love to compete with eachother to consume maximum resources) just to offer people different fee rates. If you had to do something like this, SQLite would be a much more lightweight alternative, however in almost all cases, miners will probably want to just run their own mining consoles (like any other kind of API service provider would) where they manage all this stuff themselves, so in reality, a simple JWT with an exp field, a fee field and a signing secret without any kind of user management whatsoever is probably already sufficient. If you have strong opinions to the contrary about this for some reason but are too retarded to write your own REST API, you're simply not cut out to be a miner, sorry.

4. MinerID was never really implemented properly. The whole thing is kinda just one giant hack, and it's not even protecting anything unless you change it to use a modified merkle root and have some way of safely, easily integrating it into your coinbase, which nobody has bothered to do because it's a major pain in the ass.

5. Why are we signing a JSON string and putting the signature and the string into another JSON object? Every single service on the internet that implements something like this simply signs the entire HTTP request body then puts the signature in a request header so there is absolutely no question as to which exact bytes were signed by which key due to different implementations of JSON serialisation accross different languages.

6. If you have multiple nodes behind a load balancer connected via P2P, you will end up with false negatives where one node gets a transaction, it fails for whatever reason and sends back an error, but in the meantime, it was still sent to the other node via P2P where it actually succeeded. What could a BIP270 wallet possibly do to stay in sync when this happens?

7. Regardless of your setup, you will inevitably also get false postives where mAPI or the node itself shits the bed under high load and TXs get dropped from the mempool because Bitcoin isn't actually ACID compliant with 0-conf, and neither is mAPI.

8. Mixed results is a confusing, half-baked, non-admission that the concept of shared state between mAPI and the node itself is non-existent.

9. Error messages are inconsistent and often not even descriptive of the actual problem at hand.

10. There are no defined error codes, meaning not only do we have to parse English phrases in the response to try and figure out what happened, non-English speakers also have no easy way of understanding what an error message means (so, you know, at least half of the dev community) nor are there sufficient resources for them to go and look it up in their native tongue or ask someone to explain it.

11. There is no defined set of test cases by which you can write a comprehensive handler for all these random English phrases in the error codes meaning you're basically flying blind with each new version and waiting for exciting new error cases to pop up and break your response handler. This is especially fun in statically typed languages being presented an unrecognised response with an ambiguous HTTP code as the cherry on top!

12. In some versions, API calls you would intuitively deem a 200 SUCCESS, such as "yes, your transaction is already in the mempool, you have successfully broadcasted it to me already!" instead return an HTTP error code (which stems from the node software itself, as does everything else that makes Bitcoin a gigantic clusterfuck to work with) but of course, in some other versions, this is no longer the case! 🙃

13. There has been no real adherence to the concept of versioning in the API, with a bunch of random shit simply being lumped into each new version, including breaking changes to how existing responses to existing endpoints are formatted, entire conceptual changes to how things are handled, and of course a healthy peppering of exciting new error messages to break your APIs with no clear explanation provided whatsoever! Yes, you're simply expected to figure out what has changed and how you should handle it through trial and error, and because there's no versioning in the API endpoint itself, you can't actually know which version you're about to be dealing with ahead of time 🙃

14. Okay, so this isn't even close to the shittest code I've seen in the nChain node, but boy is this a fucking lazy hack that should not exist in any supposedly "BIP270-ready" implementation of Bitcoin: https://github.com/bitcoin-sv/bitcoin-sv/blob/98064703540dc085bdee53408ff10644f4b8fe28/src/validation.cpp#L1192

    So, when you broadcast a transaction, there is no way to dellineate between a genuine Missing inputs error due to a double spend higher up the chain of transactions and a transaction that has already been mined and had its subsequent outputs onspent. Like, this one actually blows my mind. There's hacky code, and then there's this absolute masterpiece. Here's how this code actually works in the node software:

    a) You broadcast a transaction

    b) It sits in the mempool

    c) It gets mined

    d) Because there is no absolute finality in Bitcoin, you rebroadcast the same transaction again because there isn't actually a better way to ensure a transaction gets mined than to spam it an infinite number of times until it's either confirmed in a block or you have some indication that it got double spent somewhere higher up the chain of spends before it got to you

    e) The node software will then go "Oh, I don't have these inputs in my UTXO store. They must be spent. Instead of simply erroring out immediately, let's see if I have UTXOs for any of the outputs of this transaction in my UTXO store." It will then return one of two outcomes:

    i) If it does have UTXOs from the outputs of that transaction, you get a txn-already-known message. Cool, it's mined.

    ii) If the UTXOs from the outputs of this transaction have since been spent, you get a Missing inputs error which is indistinguishable from a double spend higher up the chain. What is a BIP270 wallet supposed to do now? Drop the transaction and invalidate all subsequent spends? Call the Get TX endpoint and receive the exact same error? /rant

15. "Just maintain a websocket connection with a miner at all times with zero downtime" is not an adequate solution to #14. BIP270 wallets should be able to go offline at any time and recover gracefully in a miner-agnostic fashion, otherwise there's not much point in using Bitcoin at all.

16. If you drew a Venn diagram of maintainability, memory safety, resource efficiency and concurrency, Rust would be the language right at the centre. Seriously, just use Rust.

17. A lot of the data on chain is just doge.jpg. BSV-WASM will soon support Script++ metaopcodes which can drastically reduce the amount of data that has to be transferred over the wire to spam the network with useless shit. As broadcast is always network-bound, not CPU bound, this is one of many zero-cost way to improve throughput significantly, as the faster we can hang up HTTP sockets, the greater the maximum data throughput we can achieve. Imagine being able to fake gigamegs of spam with only kilomegs of data over the wire! Revolutionary!

18. It is absolutely beyond incompetent that some manager in nChain decided: "Let's force mapi to hit the node with a bunch of requests to perform a lookup on on every single input of every single transaction in a submission batch and run an external fee calculation in mAPI to enforce different fee rates, only to submit these transactions back to the node where it'll perform a bunch of these checks anyway" when a fee rate could simply be added to sendrawtransaction/s as an optional parameter.

There's a lot more shit I could say, but I can't be bothered to write any more right now. I have work to do. /pay @40 if you want me to make this a higher priority, or don't, whatever.

CTO respectors:

Twetch - 21.8 BSV https://twetch.com/t/c7ce12d9fc9b49ef88d5814ebc6e825e7e25041c1e5c90cb6d7b6482ec9e6ac1 Deggen - 1 Egg 4eb03b89a4d18a1583e8a205b1321c83f11dc21dfcf19f4b6aeae94550f85298

/pay @40 2180 TBSV