nsDocShell as an example client of the nsIHttpChannel API

necko-walkthrough.md

nsDocShell as an example client of the nsIHttpChannel API

by Stephen Workman

1. nsDocShell::LoadURI(string)
   • create nsIURI from string
2. nsDocShell::LoadURI(nsIURI)
   • creates 2 nsIInputStream for read response from; passes them with URI to ...
3. nsDocShell::InternalLoad
4. nsDocShell::DoURILoad
   • opens the nsIChannel for the URI (NS_NewChannel)
   • if "http:", it will be an nsIHttpChannel
5. nsDocShell::DoChannelLoad
6. nsURILoader::OpenURI
   • passes an nsIStreamListener pointer, 'loader' to nsURILoader::OpenChannel - it creates an nsDocumentOpenInfo object, which implements nsIStreamListener, i.e. has OnStartRequest, OnStopRequest, OnDataAvailable, the three functions in which channel responses are received asynchronously.
7. nsHttpChannel::AsyncOpen
   • called from OpenURI; OpenChannel isn't named the best, since the opening happens in the context of OpenURI, its calling function.

This is where it hits Necko code. But the interface for clients of Necko is important to consider:

• SEND REQUEST
  • URI helps creates channel
  • setup channel (headers, request data, response callback...)
  • channel->AsyncOpen.
• RECEIVE RESPONSE
  • get a callback to each of these:
    • nsIStreamListener::OnStartRequest (header info)
    • nsIStreamListener::OnDataAvailable (data in single or multiple chunks)
    • nsIStreamListener::OnStopRequest (no more data from http)

Presently, this all happens on the main thread, in a non-blocking fashion: make your request on the main thread, then carry on and get the async response later, also on the main thread. I know there is work to have this kind of async channel access on other threads too.

Then in Necko Http code (still on the main thread for now):

8. nsHttpChannel::AsyncOpen
9. nsHttpChannel::BeginConnect()
   • creates nsHttpConnectionInfo object for the channel
   • checks if we're parodying or not
   • fires off the DNS prefetch request (dispatched to DNS thread pool)
   • some other things
10. nsHttpChannel::Connect
    • might to a SpeculativeConnect (pre open TCP socket)
11. nsHttpChannel::ContinueConnect
    • some cache stuff
12. nsHttpChannel::SetupTransaction
    • creates new nsHttpTransaction, and Inits it with mRequestHead (the request headers) and mUploadStream (which was created from the request data in channel setup)
    • gets an nsIAsyncInputStream (for the response; corresponds to the nsPipeInputStream for the response stream pipe)
    • passes it to nsInputStreamPump
13. nsHttpChannel::gHttpHandler->InitiateTransaction (called from Connect)
    • This is the global nsHttpHandler object, which adds the transaction to the nsHttpConnectionMgr (one of these per nsHttpHandler).
14. nsHttpConnectionMgr::PostEvent
    • creates an nsConnEvent with params including the handler function, nsHttpConnectionMgr::OnMsgNewTransaction, and the recently created nsHttpTransaction.
    • dispatches the nsConnEvent to the socket thread

Back to the context of nsHttpChannel::ContinueConnect:

15. nsInputStreamPump->AsyncRead
    • this pump calls nsIAsyncInputStream::AsyncWait (the input for the response stream pipe created with the nsHttpTransaction, i.e. nsPipeInputStream::AsyncWait), with target thread set to the current thread, i.e. main. So, subsequent callbacks will be dispatched to the main thread.
16. nsPipeInputStream::AsyncWait
    • sets the callback to be used later for a response
    • if a target is specified (in this case, the main thread), callback is proxied via an nsInputStreamReadyEvent, which is created now and may be called later
    • Otherwise, the callback would be called directly, when the socket is readable

Et voila, the transaction has been posted to the socket thread, and the main thread continues on, unblocked from network IO.

So, on the socket thread...

The event queue works around to nsHttpConnectionMgr::OnMsgNewTransaction (with the nsHttpTransaction passed as a param - Remember the event was posted earlier by InitiateTransaction from the main thread).

17. nsHttpConnectionMgr::OnMsgNewTransaction
18. nsHttpConnectionMgr::ProcessNewTransaction
    • (Notice the check that we're on gSocketThread)
    • Gets conn info (i.e. hostname and port) from the transaction, and then gets or creates a connection entry from the connection table, an nsClassHashtable<nsCStringHashKey, nsConnectionEntry> called mCT in nsHttpConnectionMgr.
    • If a connection entry already exists matching the required conn info, then that one will be used; otherwise a new one is created (MakeNewConnection - creates socket etc.).
    • Note: nsConnectionEntry has a single nsHttpConnectionInfo object attached, a pending queue of nsHttpTransactions, and 3 arrays for connections:
      • active nsHttpConnections
      • idle nsHttpConnections
      • nsHalfOpenSockets
19. nsHttpConnectionMgr::TryDispatchTransaction
    • There is a series of decisions about whether DispatchTransaction is called, along with good code comments: best to read the code for more detail.
    • Basically, if a connection is available...
20. nsHttpConnectionMgr::DispatchTransaction
21. nsHttpConnectionMgr::DispatchAbstractTransaction
    • the transaction is given an indirect reference to the connection (for use later, when the socket has received data from the far end).
22. nsHttpConnection::Activate
    • this connection is passed the transaction
23. nsHttpConnection::OnOutputStreamReady
24. nsHttpConnection::OnSocketWritable
    • tries to write the request data from the current transaction (mTransaction)
    • tells the transaction to now wait (``ResumeRecv`)
25. nsHttpConnection::ResumeRecv
26. nsHttpTransaction::ReadSegments
    • ReadRequestSegment is passed to mRequestStream->ReadSegments - this function pointer is called and used to read the request bytes, which in turn calls ...
27. nsHttpConnection::OnReadSegment
    • passes bytes read from request to mSocketOut->Write

Back to the context of OnSocketWritable:

28. nsIAsyncInputStream::AsyncWait (i.e. mSocketIn)
    • sets the callback and waits.

The HTTP request is now written to the socket, which has a callback to the nsHttpConnection...

Note: from what I can tell, there are some cases where the transaction is queued up to be written to the socket later if it's not writable now, or in the case of pipelining or SPDY where it's done in batches. But I'm not an expert in either of these things.

Once the socket is readable (more async behavior), nsHttpConnection::OnInputStreamReady is called on the socket thread.

29. nsHttpConnection::OnInputStreamReady
30. nsHttpConnection::OnSocketReadable
31. nsHttpTransaction::WriteSegments
32. nsHttpConnection::OnWriteSegment
    • passes bytes from mSocketIn->Read to the transaction's pipe
33. nsPipeOutputStream::WriteSegments
34. nsPipe::AdvanceWriteCursor
35. nsPipeInputStream::OnInputReadable
    • so now the response is hitting the other end of the pipe, saying it has bytes to read
36. nsPipeEvents::NotifyInputReady
    • puts the callback object into an nsPipeEvents object. Once this object goes out of scope, mCallback->OnInputStreamReady is called.
    • Note: this callback may be a proxy object, nsInputStreamReadyEvent: it is a runnable that dispatches itself to a previously set target thread, and calls its internal mCallback->OnInputStreamReady function.

Remember that nsPipeInputStream::AsyncWait was called earlier, after the transaction was initially created and posted to the connection manager on the socket thread. And in that function it created a proxy callback because it wished to have OnInputStreamReady called on the main thread.

Back on the main thread:

37. nsInputStreamPump::OnInputStreamReady
    • This function in turn calls nsInputStreamPump::OnStateStart, nsInputStreamPump::OnStateTransfer and nsInputStreamPump::OnStateStop.
    • These functions call nsIStreamListener::OnStartRequest, nsIStreamListener::OnDataAvailable and nsIStreamListener::OnStopRequest respectively.

And that brings us back to the nsHttpChannel API and nsDocShell gets its data.