Skip to content

Instantly share code, notes, and snippets.

@Vishwas1

Vishwas1/SCP-debug.md

Last active May 3, 2019 10:50
Show Gist options
  • Save Vishwas1/f1a84216c167c84cfadfc66948f3dc9f to your computer and use it in GitHub Desktop.
Save Vishwas1/f1a84216c167c84cfadfc66948f3dc9f to your computer and use it in GitHub Desktop.
Download ZIP
Raw
SCP-debug.md

Paxos

Proposor----+---+---------------^---------------+---+-----------------^----^------------
/Acceptor   |   |               |               |   | ACCEPT(s1,X)    |    |
            |   |PREPARE(s1)    |               |   |                 |    v 
Acceptor ---v---|-----------^---|---------------|---v-----------------|------------------
                |           |   |PROMISE(s1)    |                     |    ^ ACCEPTED(s1,X)
                |           |   |               |                     |    |
Acceptor -------v-----------+---+---------------v---------------------v----v--------------

Where: s1 is the slot number and X is the proposed value for the slot.

SCP

  • The protocol goes through four phases: NOMINATE, PREPARE, COMMIT, and EXTERNALIZE.
  • The NOMINATE and PREPARE phases run concurrently (though NOMINATE’s messages are sent earlier and it ends before PREPARE ends).
  • The COMMIT and EXTERNALIZE phases are exclusive, with COMMIT occurring immediately after PREPARE and EXTERNALIZE immediately after COMMIT.

Federated Voting

SCP
	NOMINATION
		SCPNominate
			Vote x
			Accept x
			Confirm x
	BALLOTING
		SCPPrepare
			Vote prepare(b)
			Accept prepare(b)
			Confirm prepare(b)/Vote commit(b)
		SCPCommit	
			Accept commit(b)
			Confirms commit(b)
		SCPExternalize
			Outputs b.value


SCP
	NOMINATION
		SCPNominate
			Vote :
				- I am node N
				- My quorum slice is Q
				- I vote x
			Accept x
			Confirm x
	BALLOTING
		SCPPrepare
			Vote prepare(b)
			Accept prepare(b)
			Confirm prepare(b)/Vote commit(b)
		SCPCommit	
			Accept commit(b)
			Confirms commit(b)
		SCPExternalize
			Outputs b.value



## Phases

* NOMINATE
* BALLOTING
            - PREPARE
            - CONFIRM
            - EXTERNALIZE

### NOMINATE 

**Starting a new slot**

// called to take a position during the next round // uses the state in LedgerManager to derive a starting position HerderImpl::triggerNextLedger(uint32_t ledgerSeqToTrigger) : 713

// fetch data from mPendingTransactions and put into proposedSet for (auto const& m : mPendingTransactions) { for (auto const& pair : m) { for (auto const& tx : pair.second->mTransactions) { proposedSet->add(tx.second); } } }

// remove invalid tx if any proposedSet->trimInvalid(mApp, removed);

// get new slot index // lcl : last closed ledger uint32_t slotIndex = lcl.header.ledgerSeq + 1;

// Inform the item fetcher so queries from other peers about his txSet can be answered. mPendingEnvelopes.addTxSet(txSetHash, slotIndex, proposedSet);

StellarValue newProposedValue(txSetHash, nextCloseTime, emptyUpgradeSteps, 0);

// Nominating with new slot index, new proposed value and proposed set mHerderSCPDriver.nominate(slotIndex, newProposedValue, proposedSet, lcl.header.scpValue);


// attempts to nominate a value (proposedSet) for consensus

bool NominationProtocol::nominate(Value const& value, Value const& previousValue,bool timedout) emitNomination();

 // this is nothing but the previous poposedSet 
 mPreviousValue = previousValue;
 
 // set propsedSet as nominating value
 Value nominatingValue;
 nominatingValue = value;
 
 // increment round number
 mRoundNumber++;
 
 // insert the nominatingValue in the votes list
 mVotes.insert(nominatingValue);
 
 // finally call emitNomination to create SCP message and envelope to broadcast the message
  emitNomination();


**DS fo SCPStatements**

struct SCPStatement { NodeID nodeID; // v uint64 slotIndex; // i union switch (SCPStatementType type) { case SCP_ST_PREPARE: struct { Hash quorumSetHash; // D SCPBallot ballot; // b SCPBallot* prepared; // p SCPBallot* preparedPrime; // p' uint32 nC; // c.n uint32 nH; // h.n } prepare; case SCP_ST_CONFIRM: struct { SCPBallot ballot; // b uint32 nPrepared; // p.n uint32 nCommit; // c.n uint32 nH; // h.n Hash quorumSetHash; // D } confirm; case SCP_ST_EXTERNALIZE: struct { SCPBallot commit; // c uint32 nH; // h.n Hash commitQuorumSetHash; // D used before EXTERNALIZE } externalize; case SCP_ST_NOMINATE: SCPNomination nominate; } pledges; };


**DS for Nomination statement/message**

struct SCPNomination { Hash quorumSetHash; // D Value votes<>; // X Value accepted<>; // Y };

**DS of envelope**

struct SCPEnvelope { SCPStatement statement; Signature signature; };

**Emitting nomination statememnt to peers**

void NominationProtocol::emitNomination()

// we need to create SCP nomination message, also called statement

// initialize statement object SCPStatement st; // assign node id st.nodeID = mSlot.getLocalNode()->getNodeID(); // message type SCP_ST_NOMINATE st.pledges.type(SCP_ST_NOMINATE);

// get quoroum set nom.quorumSetHash = mSlot.getLocalNode()->getQuorumSetHash();

// get votes list for (auto const& v : mVotes) { nom.votes.emplace_back(v); }

// get accepted list for (auto const& a : mAccepted) { nom.accepted.emplace_back(a); }

// creating envelope out of statement SCPEnvelope envelope = mSlot.createEnvelope(st);

// process the envelope : here we attempt to promote some of the votes to accepted // SCP::EnvelopeState NominationProtocol::processEnvelope(SCPEnvelope const& envelope) : NominationProtocol.cpp (322) // calling res = mNominationProtocol.processEnvelope(envelope); internally mSlot.processEnvelope(envelope, true)

// broadcast the envelope // Delegates the emission of an SCPEnvelope to the user of SCP. mSlot.getSCPDriver().emitEnvelope(envelope);

**Nomination process envelope**

SCP::EnvelopeState NominationProtocol::processEnvelope(SCPEnvelope const& envelope) // get statement from envelope auto const& st = envelope.staetement;

// check if this is new statement if (isNewerStatement(st.nodeID, nom))

// record this statement in latest nomination list: mLatestNominations, mStatementsHistory recordEnvelope(envelope);

// attempts to promote some of the votes to accepted

// continue if alreay accepted if (mAccepted.find(v) != mAccepted.end()) { continue; }

// move to acceptedlist if (vl == SCPDriver::kFullyValidatedValue) { mAccepted.emplace(v); mVotes.emplace(v); modified = true; }

// Bump to next phase : Balloting - PREPARE, CONFIRM, EXTERNALIZE mSlot.bumpState(mLatestCompositeCandidate, false);


### BALLOTING

**DS of Ballot**

struct SCPBallot { uint32 counter; // slot number Value value; // proposedSet hash };


### PREPARE  

**Creating a Ballot**

bool BallotProtocol::bumpState(Value const& value, uint32 n)

// should only comes here from SCP_ST_NOMINATE phase if (mPhase != SCP_PHASE_PREPARE && mPhase != SCP_PHASE_CONFIRM) { return false; } // initiliaze Ballaot SCPBallot newb; newb.counter = n; newb.value = value;

// emitCurrentStateStatement();

**Emiting PREPARE**

void BallotProtocol::emitCurrentStateStatement()

// create statement and envelope SCPStatement statement = createStatement(t); SCPEnvelope envelope = mSlot.createEnvelope(statement);

// call to process the envelope // but this time calling mBallotProtocol.processEnvelope(envelope, self); internally mSlot.processEnvelope(envelope, true)

sendLatestEnvelope(); mSlot.getSCPDriver().emitEnvelope(*mLastEnvelopeEmit);


**Ballot process envelope**

SCP::EnvelopeState BallotProtocol::processEnvelope(SCPEnvelope const& envelope, bool self)

// get the statement and nodeID SCPStatement const& statement = envelope.statement; NodeID const& nodeID = statement.nodeID;

// valiidate values of this statement auto validationRes = validateValues(statement);

// only process if phase is not SCP_PHASE_EXTERNALIZE if (mPhase != SCP_PHASE_EXTERNALIZE)

// update mLatestEnvelopes mapping recordEnvelope(envelope); // update the mStatementsHistory mapping recordStatement(envelope)

// advanceSlot(statement);

**Advance Slot**

void BallotProtocol::advanceSlot(SCPStatement const& hint)

// call all these every time. irrespective of phase type. bool didWork = false;

//PREPARE didWork = attemptPreparedAccept(hint) || didWork;
//std::bind(&BallotProtocol::hasPreparedBallot, ballot, _1));

//PREPARE didWork = attemptPreparedConfirmed(hint) || didWork;

//CONFIRM didWork = attemptAcceptCommit(hint) || didWork;

//EXTERNALIZE didWork = attemptConfirmCommit(hint) || didWork;

// if any of the methods return true then broadcast envelope if (didWork) { sendLatestEnvelope();
// mSlot.getSCPDriver().emitEnvelope(*mLastEnvelopeEmit); }




**Accept prepare(b)**

bool BallotProtocol::attemptPreparedAccept(SCPStatement const& hint)

// get candidates auto candidates = getPrepareCandidates(hint);

return setPreparedAccept(ballot); //


set Prepared

bool BallotProtocol::setPreparedAccept(SCPBallot const& ballot)

// update our state bool didWork = setPrepared(ballot);

//
mSlot.getSCPDriver().acceptedBallotPrepared(mSlot.getSlotIndex(), ballot);

// emitting current statement emitCurrentStateStatement();


### COMFIRM PHASE

- Once again we are in `emitCurrentStateStatement()` method where, we will call `mSlot.processEnvelope(envelope, true)`  and then `res = mBallotProtocol.processEnvelope(envelope, self);` to process the new phase.
- In `BallotProtocol::processEnvelope()` method, validation happens and `advanceSlot(statement);` gets called.
- And finally envelop gets broadcasted.

advanceSlot(statement)

// didWork = attemptConfirmCommit(hint) || didWork; // setConfirmCommit(c, h);

**Confirms commit(b)**

bool BallotProtocol::setConfirmCommit(SCPBallot const& c, SCPBallot const& h)

mPhase = SCP_PHASE_EXTERNALIZE;

emitCurrentStateStatement();

mSlot.stopNomination();

mSlot.getSCPDriver().valueExternalized(mSlot.getSlotIndex(), mCommit->value);

### EXTERNALIZE

HerderSCPDriver::valueExternalized(uint64_t slotIndex, Value const& value)

(Link)[https://github.com/stellar/stellar-core/blob/13d36e5bb637fc1b7b7c59bc63599a740bc6f933/src/herder/HerderSCPDriver.cpp#L630]


**Storing value into DB and closing the ledger**

void HerderImpl::valueExternalized(uint64 slotIndex, StellarValue const& value)

// validate bool validated = getSCP().isSlotFullyValidated(slotIndex);

// get the transaction set TxSetFramePtr externalizedSet = mPendingEnvelopes.getTxSet(value.txSetHash);

// save the SCP messages in the database mApp.getHerderPersistence().saveSCPHistory(static_cast(slotIndex),getSCP().getExternalizingState(slotIndex));

// tell the LedgerManager that this value got externalized // LedgerManager will perform the proper action based on its internal // state: apply, trigger catchup, etc LedgerCloseData ledgerData(mHerderSCPDriver.lastConsensusLedgerIndex(), externalizedSet, value);

// close the ledger ledgerClosed();

 
 
 ----------------------------------------

// New ledger starts here HerderImpl::triggerNextLedger(uint32_t ledgerSeqToTrigger)

// last closed ledger auto const& lcl = mLedgerManager.getLastClosedLedgerHeader();

// prepare proposedSet from mPendingTransactions auto proposedSet = std::make_shared(lcl.hash);

for (auto const& m : mPendingTransactions)
{
    for (auto const& pair : m)
    {
        for (auto const& tx : pair.second->mTransactions)
        {
            proposedSet->add(tx.second);
        }
    }
}

// trim Invalid transaction from proposedSet proposedSet->trimInvalid(mApp, removed);

// also remove from mPendingTransactions removeReceivedTxs(removed);

// need to make sure every account that is submitting a tx has enough to pay the fees of all the tx it has submitted in this set. Also checks seq num

if (!proposedSet->checkValid(mApp)) { throw std::runtime_error("wanting to emit an invalid txSet"); }

// get the hash of transaction set, remember tx set hash goes for SCP? auto txSetHash = proposedSet->getContentsHash();

// increatment the slot number uint32_t slotIndex = lcl.header.ledgerSeq + 1;

// Itemfectcher essentially to sync transaction set other peers. mPendingEnvelopes.addTxSet(txSetHash, slotIndex, proposedSet);

// set next closing time uint64_t nextCloseTime = VirtualClock::to_time_t(mApp.getClock().now()); if (nextCloseTime <= lcl.header.scpValue.closeTime) { nextCloseTime = lcl.header.scpValue.closeTime + 1; }

// finally nominate the newProposedValue mHerderSCPDriver.nominate(slotIndex, newProposedValue, proposedSet, lcl.header.scpValue);


NominationProtocol.cpp : Line 456

// attempts to nominate a value for consensus bool NominationProtocol::nominate(Value const& value, Value const& previousValue, bool timedout)


**Nomination Protocol**

- Wallet transaction
- First it goes into `mPendingTransactions`
- `triggerNextLedger` gets called at interval of `5` sec
- `proposedSet` gets populated from `mPendingTransactions`
- get the transactoin set hash
- Prepare `newProposedValue`
- Finnally `nominate`
- Emit nomination `NominationProtocol::emitNomination()`
- STEP*** `Slot::processEnvelope(SCPEnvelope const& envelope, bool self)`

if (envelope.statement.pledges.type() ==
        SCPStatementType::SCP_ST_NOMINATE)
    {
        res = mNominationProtocol.processEnvelope(envelope);
    }
    else
    {
        res = mBallotProtocol.processEnvelope(envelope, self);
    }


**Ballot Protocol**

**Prepare**
- Now ballot protocol starts
- ` res = mBallotProtocol.processEnvelope(envelope, self);`
- `BallotProtocol::processEnvelope(SCPEnvelope const& envelope, bool self)`
- `BallotProtocol::bumpToBallot(SCPBallot const& ballot, bool check)`
- Create statement with SCP_ST_PREPARE : `SCPStatement statement = createStatement(SCP_ST_PREPARE);`
- Create envelopve with that statement : `SCPEnvelope envelope = mSlot.createEnvelope(statement);` Goto STEP***
- After that decide which phase to call in `BallotProtocol::advanceSlot(SCPStatement const& hint)` method
- Now do _Accept prepare(b)_ : `BallotProtocol::setPreparedAccept(SCPBallot const& ballot)` and emit this statement finnaly : `emitCurrentStateStatement()`
- Now do _Confirm prepare(b)_: `BallotProtocol::attemptPreparedConfirmed(SCPStatement const& hint)` and emit this statement finnaly : `emitCurrentStateStatement()`

**Commit**

- Now do _Accept commit(b)_ :`BallotProtocol::setAcceptCommit(SCPBallot const& c, SCPBallot const& h)` and emit this statement finnaly : `emitCurrentStateStatement()`
- `mSlot.getSCPDriver().acceptedCommit(mSlot.getSlotIndex(), h);`
- Finally _Confirms commit(b)_ : `BallotProtocol::setConfirmCommit(SCPBallot const& c, SCPBallot const& h)`
   - set externalize phase" ` mPhase = SCP_PHASE_EXTERNALIZE;` and emit the statement emitCurrentStateStatement();
   
**Externalize**

- ` mSlot.getSCPDriver().valueExternalized(mSlot.getSlotIndex(), mCommit->value);`
   - Validate this slot `bool validated = getSCP().isSlotFullyValidated(slotIndex);`
   - Fetch trasnsactionset with to this slot : `TxSetFramePtr externalizedSet = mPendingEnvelopes.getTxSet(value.txSetHash);`
   - Time to finally close the ledger `LedgerManagerImpl::closeLedger(LedgerCloseData const& ledgerData)`
   - Get all transactions that was agreed upon during consensus `vector<TransactionFramePtr> txs = ledgerData.getTxSet()->sortForApply();`
   - Process seqNum and fee `processFeesSeqNums(txs, ltx);`
   - Store transactions in database `applyTransactions(txs, ltx, txResultSet);`
   	- Iterate over the transactions
   	- For each Transaction, iterrate over Operations and call `doApply()`

[Architecutre](https://github.com/stellar/stellar-core/blob/master/docs/architecture.md)

## Major Components

**SCP**: This component is fully abstracted from the rest of the system. It receives candidate black-box values and signals when these values have reached consensus by the network (called externalizing a value). [More](https://github.com/stellar/stellar-core/tree/master/src/scp).

**Herder**: Is responsible for interfacing between SCP and the rest of stellar-core. Herder provides SCP with concrete implementations of the methods SCP uses to communicate with peers, to compare values, to determine whether values contain valid signatures, an so forth. [More](https://github.com/stellar/stellar-core/tree/master/src/herder)

**Overlay**: connects to and keeps track of the peers this node knows about and is connected to. It floods messages and fetches from peers the data that is needed to accomplish consensus. [More](https://github.com/stellar/stellar-core/blob/master/src/overlay/readme.md)

**Ledger**: applies the transaction set that is externalized by SCP. It also forwards the externalization event to other components: it submits the changed ledger entries to the bucket list, triggers the publishing of history, and informs the overlay system to update its map of flooded messages. Ledger also triggers the history system’s catching-up routine when it detects that this node has fallen behind of the rest of the network. [More](https://github.com/stellar/stellar-core/blob/master/src/ledger/readme.md)

**History**: publishes transaction and ledger entries to off-site permanent storage for auditing, and as a source of catch-up data for other nodes. When this node falls behind, the history system fetches catch-up data and submits it to Ledger twice: first to verify its security, then to apply it. [More](https://github.com/stellar/stellar-core/blob/master/src/history/readme.md)

**BucketList**: stores ledger entries on disk arranged for hashing and block-catch-up. BucketList coordinates the hashing and deduplicating of buckets by multiple background threads. [More]()

**Transactions**: implements all the various transaction types. [More](https://github.com/stellar/stellar-core/blob/master/src/transactions/readme.md) [More](https://www.stellar.org/developers/guides/concepts/transactions.html)





https://www.stellar.org/developers/stellar-core/#supporting-code-directories

https://www.stellar.org/developers/stellar-core/

void LedgerManagerImpl::applyTransactions(std::vector& txs, AbstractLedgerTxn& ltx, TransactionResultSet& txResultSet)

-----------------------

- SOCI - The C++ Database Access Library
- https://en.cppreference.com/w/cpp/memory/unique_ptr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment