valenting · August 30, 2018 18:13
diff --git a/baku-chat.txt b/baku-chat.txt

 (17:26:18) baku: ping
 (17:28:00) baku: when you have 1 minute, can you tell me more about this alt-data and nsICacheInfoChannel ?
 (17:28:22) valentin: sure...
 (17:28:37) valentin: what do you want to know? :)
 (17:29:04) baku: it seems that, when I create a channel, I can ask for a particular alt-data
 (17:29:16) baku: and if that is in cache, I read it directly, instead of doing the networking
 (17:29:17) baku: am I right?
 (17:30:06) valentin: yes. You call the preferAltDataType(type) ... if it exists you will get the alt-data contents, and GetAlternativeDataType will return that type
 (17:30:51) valentin: otherwise you get what the cached server response... or we may not even use the cache... in which case it's also the raw server response :)
 (17:30:57) baku: and if I say: setPreferAltDataType('foo') and 'foo' doesn't exist, necko creates 'foo' for the next request?
 (17:31:20) valentin: no. In order to create the foo type, you need to call OpenAlternativeOutputStream('foo')
 (17:31:23) valentin: on the channel
 (17:31:32) baku: ok I see.
 (17:31:43) baku: what's the difference between that 'foo' and what is in the necko cache?
 (17:32:56) valentin: well, the 'foo' alt-data will return whatever bytes you write into the alternativeOutputStream
 (17:34:14) baku: ok... it seems something useful for ServiceWorkers mainly.
 (17:35:11) valentin: yup... we're looking into using it for wasm as well...
 (17:35:31) valentin: I have a bug for being able to specify alt-data types by contentType https://bugzilla.mozilla.org/show_bug.cgi?id=1487100
 (17:37:49) baku: yeah, luke asked me something related to that bug from a fetch() point of view.
 (17:38:01) baku: the problem is that a fetch() produces a Response object
 (17:38:33) baku: a response already has a initialized fetch operation (nsIChannel has been already created, AsyncOpen2() called, and so on)
 (17:39:11) valentin: yeah, I talked to him about what he needs to be able to use alt-data...
 (17:39:29) baku: so... if we want to call  WebAssembly.compileStreaming(response), maybe it's too late to ask for alt-data.
 (17:39:34) baku: in particular the issue is:
 (17:39:41) baku: fetch('a.wasm').then(response => { var r1 = response.clone(); r1.blob().then(() => { "here the blob" }); var r2 = response; WebAssembly.compileStreaming(r2); });
 (17:39:52) baku: response should be cloneable....
 (17:40:11) baku: and for the .blob(), we don't want the alt-data, I guess.
 (17:40:55) valentin: does the response already contain the content? or just the inputStream?
 (17:41:38) baku: no content, just the inputStream. _but_ from a FetchDriver point of view, the fetch() promise is resolved at OnStartRequest
 (17:41:53) valentin: I'm thinking the Fetch shouldn't know anything about alt-data... but we could add a fastpath to compileStreaming
 (17:42:20) baku: currently it knows, because of ServiceWorkers
 (17:42:29) baku: if the InternalResponse has a preferrerAltData flag, that is used.
 (17:42:35) baku: but that doesn't matter here.
 (17:43:31) baku: a fastpath would mean that, if the Response is used in WebAssembly.compileStreaming(), we don't use its inputStream, but we start a new nsIChannel with a preferrer alt data?
 (17:44:31) baku: the problem I see here is that a Response() can be generated in different ways. fetch() is just 1 of them.
 (17:44:38) valentin: right...
 (17:45:51) valentin: so... technically we could at some point say that we want the alt-data inputStream or http response inputStream from the channel
 (17:46:43) valentin: the reason we have the api we do now is e10s... you can't decide in onStartRequest if you want one or the other... because by the time you decide in the content process... the parent channel may already be in onStopRequest
 (17:47:43) baku: do you mind to move this conversation in a public channel? maybe luke wants to follow the discussion
 (17:47:45) valentin: I didn't think about this earlier... but in onStartRequest we know if we've got alt-data or not.
 (17:47:57) valentin: sure... let's do it
 (17:48:02) baku: #necko
 (17:48:04) valentin: cool
diff --git a/necko-chat.txt b/necko-chat.txt
 (17:48:36) baku: valentin: related to our previous discussion, could we have multiple alternative data? or just 1?
 (17:49:08) valentin: baku: just one type
 (17:49:26) valentin: if it's really needed, we could do multiple, but it would be a lot of work
 (17:49:34) baku: valentin: just asking
 (17:50:05) valentin: log of our private chat: https://gist.github.com/valenting/7cdedffd75c078a9b0f31297864670ff
 (17:50:06) baku: valentin: so... if at onStartRequest() we know if we have alt-data, we could create a Response object that keeps a reference to that alt-data and at the same time continues as it currently does.
 (17:50:57) valentin: yes, what we could do is deliver the http Response as usual, but since the channel has a ref to the cache entry, it could open the alt-data without creating a new channel/request
 (17:51:02) baku: valentin: than we can change Response::GetBody(nsIInputStream**) to ask for alt-data. if alt-data is know, the internal response nsInputStream is ignored, and, instead, we use alt-data inputStream.
 (17:51:33) valentin: something like that, yes
 (17:51:48) baku: valentin: right. We should keep the internal inputStream because Response can be cloned and used elsewhere, but this approach seems to work.
 (17:53:08) valentin: yes... sounds good
 (17:53:21) baku: valentin: the only concern is that, this approach is not nice from a performance point of view, because the nsIHttpChannel is still retrieving data from the necko cache or from the network.
 (17:54:05) valentin: so, it would make the alt-data use case a bit slower... because it adds at least 1 IPC RTT
 (17:54:38) valentin: child gets response, and knows we have alt-data, makes call to parent to get alt-data... parent opens alt-data and sends stream to child
 (17:55:07) baku: well no, this doesn't work.
 (17:55:32) baku: the current setup is: fetch(foo).then(response...
 (17:55:49) baku: when that fetch(foo) is resolved, we are at OnStartRequest()
 (17:55:58) valentin: yes
 (17:55:58) baku: the response object is created but we don't know yet how it will be used.
 (17:56:37) baku: the response object has, internally, an nsIInputStream which is populated with the data coming from necko.
 (17:56:38) valentin: what I'm suggesting is using the Response, to get an alt-data pseudo-Response instead
 (17:57:19) valentin: otherwise we do have to decide before making the request what we want to use it for :) 
 (17:57:47) baku: ok
 (17:58:16) valentin: we could also make it deliver both HTTP and alt-data... if we wanted to
 (17:58:24) baku: so, just to be sure, Response code is basically untouched. It needs to store the alt-data when OnStartRequest is executed (and maybe keep something alive)
 (17:59:01) baku: Wasm.compileStreaming(response) calls the internal response method to retrieve a new alt-data pseudo Response from the original Response.
 (18:00:09) valentin: I think so...
 (18:00:19) baku: perfect
 (18:01:37) valentin: we would need to make some changes to allow for both... but it seems doable...
 bagder baku 
 (18:02:13) valentin: baku: we should open a bug one we decide how exactly we want this to work
 (18:05:35) baku: ok
 (18:09:22) luke: baku, valentin: sounds good from what i could understand :)  so in FetchUtil::StreamResponseToJS(), where we GetUnfilteredBody()/SetBodyUsed(), is this where we'd see there was alt-data present and ask for the new pseudo-response?
 luke lukegb_ 
 (18:13:34) valentin: luke: yes... I think something like that could work... 
 (18:13:55) baku: luke: right. It's depend how the alt-data should be retrieved, but probably we could skip the creation of that pseudo-response (maybe)
 (18:14:04) luke: valentin, baku: cool, and at that point, would we have an nsIInputStream for the raw content, alt-data, or both?
 (18:14:25) baku: luke: valentin: I'm thinking of a Response::GetUnfilteredBody(nsIInputStream**, const nsAString& alternativeData)
 (18:15:16) baku: and if alternativeData matches what OnStartRequest knows, maybe we can create a nsIInputStream that retrieves the alt-data directly, somehow.
 luke lukegb_ 
 luke lukegb_ 
 (18:17:20) luke: baku, valentin: and could that GetUnfilteredBody() work without any a priori nsICacheInfoChannel::preferAlternativeDataType() ?
 luke lukegb_ 
 (18:17:45) luke: other than maybe some flag to say "i'll possibly take whatever you have"
 (18:17:46) valentin: luke: technically yes.
 (18:18:03) valentin: baku: luke: it depends on how we want it... the nsHttpChannel could deliver the altdata input stream at any time
 (18:18:18) luke: well that sounds lovely
 (18:18:40) valentin: we would need a flag to always open the altdata stream, or to keep the channel alive so we could get the stream later
 (18:19:23) luke: valentin: right.  and if we want to minimize the possible overhead, we could have a flag/arg to only do this for the combination of: fetch()+response.contentType='application/wasm'
 (18:19:45) valentin: yes
 (18:19:49) luke: that sounds awesome
 (18:20:31) luke: b/c then fetch() could unconditionally set that flag and FetchUtil::StreamResponseToJS() would just pluck out the wasm alt-data if it was there and i can go nuts from there :)
 (18:22:58) luke: should this go in a separate bug or just as an update to bug 1487100?
 (18:23:05) luke: oops, baku, valentin: ^
 luke lukegb_ 
 (18:23:43) baku: I would split the Fetch API part and the necko part.
 (18:24:04) valentin: luke: it think the necko bits can go in 1487100... I don't intend to overengineer the alt-data API too much...
 (18:24:20) baku: I guess we need a bug to have the possibility to retrieve an alt-data somehow at any time and to have the alt-data availability at OnStartRequest, from necko.

	(17:26:18) baku: ping
	(17:28:00) baku: when you have 1 minute, can you tell me more about this alt-data and nsICacheInfoChannel ?
	(17:28:22) valentin: sure...
	(17:28:37) valentin: what do you want to know? :)
	(17:29:04) baku: it seems that, when I create a channel, I can ask for a particular alt-data
	(17:29:16) baku: and if that is in cache, I read it directly, instead of doing the networking
	(17:29:17) baku: am I right?
	(17:30:06) valentin: yes. You call the preferAltDataType(type) ... if it exists you will get the alt-data contents, and GetAlternativeDataType will return that type
	(17:30:51) valentin: otherwise you get what the cached server response... or we may not even use the cache... in which case it's also the raw server response :)
	(17:30:57) baku: and if I say: setPreferAltDataType('foo') and 'foo' doesn't exist, necko creates 'foo' for the next request?
	(17:31:20) valentin: no. In order to create the foo type, you need to call OpenAlternativeOutputStream('foo')
	(17:31:23) valentin: on the channel
	(17:31:32) baku: ok I see.
	(17:31:43) baku: what's the difference between that 'foo' and what is in the necko cache?
	(17:32:56) valentin: well, the 'foo' alt-data will return whatever bytes you write into the alternativeOutputStream
	(17:34:14) baku: ok... it seems something useful for ServiceWorkers mainly.
	(17:35:11) valentin: yup... we're looking into using it for wasm as well...
	(17:35:31) valentin: I have a bug for being able to specify alt-data types by contentType https://bugzilla.mozilla.org/show_bug.cgi?id=1487100
	(17:37:49) baku: yeah, luke asked me something related to that bug from a fetch() point of view.
	(17:38:01) baku: the problem is that a fetch() produces a Response object
	(17:38:33) baku: a response already has a initialized fetch operation (nsIChannel has been already created, AsyncOpen2() called, and so on)
	(17:39:11) valentin: yeah, I talked to him about what he needs to be able to use alt-data...
	(17:39:29) baku: so... if we want to call WebAssembly.compileStreaming(response), maybe it's too late to ask for alt-data.
	(17:39:34) baku: in particular the issue is:
	(17:39:41) baku: fetch('a.wasm').then(response => { var r1 = response.clone(); r1.blob().then(() => { "here the blob" }); var r2 = response; WebAssembly.compileStreaming(r2); });
	(17:39:52) baku: response should be cloneable....
	(17:40:11) baku: and for the .blob(), we don't want the alt-data, I guess.
	(17:40:55) valentin: does the response already contain the content? or just the inputStream?
	(17:41:38) baku: no content, just the inputStream. _but_ from a FetchDriver point of view, the fetch() promise is resolved at OnStartRequest
	(17:41:53) valentin: I'm thinking the Fetch shouldn't know anything about alt-data... but we could add a fastpath to compileStreaming
	(17:42:20) baku: currently it knows, because of ServiceWorkers
	(17:42:29) baku: if the InternalResponse has a preferrerAltData flag, that is used.
	(17:42:35) baku: but that doesn't matter here.
	(17:43:31) baku: a fastpath would mean that, if the Response is used in WebAssembly.compileStreaming(), we don't use its inputStream, but we start a new nsIChannel with a preferrer alt data?
	(17:44:31) baku: the problem I see here is that a Response() can be generated in different ways. fetch() is just 1 of them.
	(17:44:38) valentin: right...
	(17:45:51) valentin: so... technically we could at some point say that we want the alt-data inputStream or http response inputStream from the channel
	(17:46:43) valentin: the reason we have the api we do now is e10s... you can't decide in onStartRequest if you want one or the other... because by the time you decide in the content process... the parent channel may already be in onStopRequest
	(17:47:43) baku: do you mind to move this conversation in a public channel? maybe luke wants to follow the discussion
	(17:47:45) valentin: I didn't think about this earlier... but in onStartRequest we know if we've got alt-data or not.
	(17:47:57) valentin: sure... let's do it
	(17:48:02) baku: #necko
	(17:48:04) valentin: cool
	(17:48:36) baku: valentin: related to our previous discussion, could we have multiple alternative data? or just 1?
	(17:49:08) valentin: baku: just one type
	(17:49:26) valentin: if it's really needed, we could do multiple, but it would be a lot of work
	(17:49:34) baku: valentin: just asking
	(17:50:05) valentin: log of our private chat: https://gist.github.com/valenting/7cdedffd75c078a9b0f31297864670ff
	(17:50:06) baku: valentin: so... if at onStartRequest() we know if we have alt-data, we could create a Response object that keeps a reference to that alt-data and at the same time continues as it currently does.
	(17:50:57) valentin: yes, what we could do is deliver the http Response as usual, but since the channel has a ref to the cache entry, it could open the alt-data without creating a new channel/request
	(17:51:02) baku: valentin: than we can change Response::GetBody(nsIInputStream**) to ask for alt-data. if alt-data is know, the internal response nsInputStream is ignored, and, instead, we use alt-data inputStream.
	(17:51:33) valentin: something like that, yes
	(17:51:48) baku: valentin: right. We should keep the internal inputStream because Response can be cloned and used elsewhere, but this approach seems to work.
	(17:53:08) valentin: yes... sounds good
	(17:53:21) baku: valentin: the only concern is that, this approach is not nice from a performance point of view, because the nsIHttpChannel is still retrieving data from the necko cache or from the network.
	(17:54:05) valentin: so, it would make the alt-data use case a bit slower... because it adds at least 1 IPC RTT
	(17:54:38) valentin: child gets response, and knows we have alt-data, makes call to parent to get alt-data... parent opens alt-data and sends stream to child
	(17:55:07) baku: well no, this doesn't work.
	(17:55:32) baku: the current setup is: fetch(foo).then(response...
	(17:55:49) baku: when that fetch(foo) is resolved, we are at OnStartRequest()
	(17:55:58) valentin: yes
	(17:55:58) baku: the response object is created but we don't know yet how it will be used.
	(17:56:37) baku: the response object has, internally, an nsIInputStream which is populated with the data coming from necko.
	(17:56:38) valentin: what I'm suggesting is using the Response, to get an alt-data pseudo-Response instead
	(17:57:19) valentin: otherwise we do have to decide before making the request what we want to use it for :)
	(17:57:47) baku: ok
	(17:58:16) valentin: we could also make it deliver both HTTP and alt-data... if we wanted to
	(17:58:24) baku: so, just to be sure, Response code is basically untouched. It needs to store the alt-data when OnStartRequest is executed (and maybe keep something alive)
	(17:59:01) baku: Wasm.compileStreaming(response) calls the internal response method to retrieve a new alt-data pseudo Response from the original Response.
	(18:00:09) valentin: I think so...
	(18:00:19) baku: perfect
	(18:01:37) valentin: we would need to make some changes to allow for both... but it seems doable...
	bagder baku
	(18:02:13) valentin: baku: we should open a bug one we decide how exactly we want this to work
	(18:05:35) baku: ok
	(18:09:22) luke: baku, valentin: sounds good from what i could understand :) so in FetchUtil::StreamResponseToJS(), where we GetUnfilteredBody()/SetBodyUsed(), is this where we'd see there was alt-data present and ask for the new pseudo-response?
	luke lukegb_
	(18:13:34) valentin: luke: yes... I think something like that could work...
	(18:13:55) baku: luke: right. It's depend how the alt-data should be retrieved, but probably we could skip the creation of that pseudo-response (maybe)
	(18:14:04) luke: valentin, baku: cool, and at that point, would we have an nsIInputStream for the raw content, alt-data, or both?
	(18:14:25) baku: luke: valentin: I'm thinking of a Response::GetUnfilteredBody(nsIInputStream**, const nsAString& alternativeData)
	(18:15:16) baku: and if alternativeData matches what OnStartRequest knows, maybe we can create a nsIInputStream that retrieves the alt-data directly, somehow.
	luke lukegb_
	luke lukegb_
	(18:17:20) luke: baku, valentin: and could that GetUnfilteredBody() work without any a priori nsICacheInfoChannel::preferAlternativeDataType() ?
	luke lukegb_
	(18:17:45) luke: other than maybe some flag to say "i'll possibly take whatever you have"
	(18:17:46) valentin: luke: technically yes.
	(18:18:03) valentin: baku: luke: it depends on how we want it... the nsHttpChannel could deliver the altdata input stream at any time
	(18:18:18) luke: well that sounds lovely
	(18:18:40) valentin: we would need a flag to always open the altdata stream, or to keep the channel alive so we could get the stream later
	(18:19:23) luke: valentin: right. and if we want to minimize the possible overhead, we could have a flag/arg to only do this for the combination of: fetch()+response.contentType='application/wasm'
	(18:19:45) valentin: yes
	(18:19:49) luke: that sounds awesome
	(18:20:31) luke: b/c then fetch() could unconditionally set that flag and FetchUtil::StreamResponseToJS() would just pluck out the wasm alt-data if it was there and i can go nuts from there :)
	(18:22:58) luke: should this go in a separate bug or just as an update to bug 1487100?
	(18:23:05) luke: oops, baku, valentin: ^
	luke lukegb_
	(18:23:43) baku: I would split the Fetch API part and the necko part.
	(18:24:04) valentin: luke: it think the necko bits can go in 1487100... I don't intend to overengineer the alt-data API too much...
	(18:24:20) baku: I guess we need a bug to have the possibility to retrieve an alt-data somehow at any time and to have the alt-data availability at OnStartRequest, from necko.