-
The equivalent of specification's
media.load()
in Webkit isHTMLMediaElement::load()
and is located inSource/WebCore/html/HTMLMediaElement.cpp
. -
When in specifications says about
discarding pending events and callbacks for the media element
, the Webkit callscancelPendingEventsAndCallbacks()
to perform these operations. It's located inSource/WebCore/html/HTMLMediaElement.cpp
. -
The
networkState
in specification has four states:- NETWORK_EMPTY: The element has not yet been initialized. All attributes are in their initial states.
- NETWORK_IDLE: The element's resource selection algorithm is active and has selected a resource, but it is not actually using the network at this time.
- NETWORK_LOADING: The user agent is actively trying to download data.
- NETWORK_NO_SOURCE: The element's resource selection algorithm is active, but it has not yet found a resource to use.
The same states has Webkit,
enum NetworkState { NETWORK_EMPTY, NETWORK_IDLE, NETWORK_LOADING, NETWORK_NO_SOURCE };
, as described inSource/WebCore/html/HTMLMediaElement.h
. More states are added in thenetworkState
inSource/WebCore/platform/graphics/MediaPlayer.h
which are the followingenum NetworkState { Empty, Idle, Loading, Loaded, FormatError, NetworkError, DecodeError };
. -
The
readyState
in specification has five states:- HAVE_NOTHING: No information regarding the media resource is available. No data for the current playback position is available. Media elements whose networkState attribute are set to NETWORK_EMPTY are always in the HAVE_NOTHING state.
- HAVE_METADATA: Enough of the resource has been obtained that the duration of the resource is available. In the case of a video element, the dimensions of the video are also available. The API will no longer throw an exception when seeking. No media data is available for the immediate current playback position.
- HAVE_CURRENT_DATA: Data for the immediate current playback position is available, but either not enough data is available that the user agent could successfully advance the current playback position in the direction of playback at all without immediately reverting to the HAVE_METADATA state, or there is no more data to obtain in the direction of playback. For example, in video this corresponds to the user agent having data from the current frame, but not the next frame, when the current playback position is at the end of the current frame; and to when playback has ended.
- HAVE_FUTURE_DATA: Data for the immediate current playback position is available, as well as enough data for the user agent to advance the current playback position in the direction of playback at least a little without immediately reverting to the HAVE_METADATA state, and the text tracks are ready. For example, in video this corresponds to the user agent having data for at least the current frame and the next frame when the current playback position is at the instant in time between the two frames, or to the user agent having the video data for the current frame and audio data to keep playing at least a little when the current playback position is in the middle of a frame. The user agent cannot be in this state if playback has ended, as the current playback position can never advance in this case.
- HAVE_ENOUGH_DATA: All the conditions described for the HAVE_FUTURE_DATA state are met, and, in addition, either of the following conditions is also true:
- The user agent estimates that data is being fetched at a rate where the current playback position, if it were to advance at the effective playback rate, would not overtake the available data before playback reaches the end of the media resource.
- The user agent has entered a state where waiting longer will not result in further data being obtained, and therefore nothing would be gained by delaying playback any further. (For example, the buffer might be full.)
In practice, the difference between HAVE_METADATA and HAVE_CURRENT_DATA is negligible. Really the only time the difference is relevant is when painting a video element onto a canvas, where it distinguishes the case where something will be drawn (HAVE_CURRENT_DATA or greater) from the case where nothing is drawn (HAVE_METADATA or less). Similarly, the difference between HAVE_CURRENT_DATA (only the current frame) and HAVE_FUTURE_DATA (at least this frame and the next) can be negligible (in the extreme, only one frame). The only time that distinction really matters is when a page provides an interface for "frame-by-frame" navigation.
The states in Webkit is
enum ReadyState { HaveNothing, HaveMetadata, HaveCurrentData, HaveFutureData, HaveEnoughData };
, as described inSource/WebCore/platform/graphics/MediaPlayer.h
. -
When in the specifications says
queue a task to fire a simple event
in the Webkit the methodscheduleEvent
is called with the name of the event. For example, if the specification saysqueue a task to fire a simple event named abort at the media element
, in Webkit will be likescheduleEvent(eventNames().abortEvent);
. All the events names that Webkit uses can be found inSource/WebCore/dom/EventNames.h
. -
The Media Element Load Algorith (MELA) is called when the load methods is called. It performs some initializations of several global variables of Media Element, like
networkState
,readyState
,paused
,seeking
, etc. Then it calls the Resource Selection Algorithm (RSA). -
The equivalent of specification's
set the media element's delaying-the-load-event flag to true
in Webkit issetShouldDelayLoadEvent(true);
and located inSource/WebCore/html/HTMLMediaElement.cpp
. -
The most important methods in Webkit which are used by the
HTMLMediaElement::load()
are the following:prepareForLoad()
: It is mainly the MELA.configureMediaControls()
:
loadInternal()
:selectMediaResource()
: It is mainly the RSA.loadResource(const KURL& initialURL, ContentType& contentType, const String& keySystem)
: It is mainly the Resource Fetch Algorithm (RFA).m_player->load
:MediaPlayer::load(const KURL& url, const ContentType& contentType, const String& keySystem)
. InSource/WebCore/platform/graphics/MediaPlayer.cpp
.MediaPlayer::loadWithNextMediaEngine(MediaPlayerFactory* current)
: InSource/WebCore/platform/graphics/MediaPlayer.cpp
.-
m_private->load
: There are some choises regarding the player:- GStreamer then it will be
MediaPlayerPrivateGStreamer::load(const String& url)
. InSource/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
. - AVFoundation then it will be
MediaPlayerPrivateAVFoundation::load(const String& url)
. InSource/WebCore/platform/graphics/avfoundation/MediaPlayerPrivateAVFoundation.cpp
. - BlackBerry then it will be
MediaPlayerPrivateBlackBerry::load(const String& url)
. InSource/WebCore/platform/graphics/blackberry/MediaPlayerPrivateBlackBerry.cpp
. - QTKit (for QuickTime in Mac) then it will be
MediaPlayerPrivateQTKit::load(const String& url)
. InSource/WebCore/platform/graphics/mac/MediaPlayerPrivateQTKit.mm
. - QuickTimeVisualContext (for QuickTime in Windows) then it will be
MediaPlayerPrivateQuickTimeVisualContext::load(const String& url)
. InSource/WebCore/platform/graphics/win/MediaPlayerPrivateQuickTimeVisualContext.cpp
. - Qt (Qt from Nokia) then it will be
MediaPlayerPrivateQt::load(const String& url)
. InSource/WebCore/platform/graphics/qt/MediaPlayerPrivateQt.cpp
.
Because all the above have the same functionality, I will replace their player's name with a more generic one, like Generic. So for example we have the
MediaPlayerPrivateGeneric::load(const String& url)
.- MediaPlayerPrivateGeneric::commitLoad(): In
Source/WebCore/platform/graphics/generic/MediaPlayerPrivateGeneric.cpp
- GStreamer then it will be
-
loadNextSourceChild()
: This method is called when the mode is not attribute in order to use the source elements.mediaLoadingFailed(MediaPlayer::NetworkState error)
: InSource/WebCore/html/HTMLMediaElement.cpp
.mediaEngineError(PassRefPtr<MediaError> err)
: Send an error event. Set the networkState to NETWORK_EMPTY and send an emptied event. It is called only inmediaLoadingFailed
. InSource/WebCore/html/HTMLMediaElement.cpp
.
prepareToPlay()
:
-
The Webkit method for checking if the Media Element has a
src
attribute isfastHasAttribute
. -
In Webkit there is another state for loading,
LoadState
. The states areenum LoadState { WaitingForSource, LoadingFromSrcAttr, LoadingFromSourceElement };
. -
Progress event in Webkit.
void HTMLMediaElement::changeNetworkStateFromLoadingToIdle() { m_progressEventTimer.stop(); if (hasMediaControls() && m_player->didLoadingProgress()) mediaControls()->bufferingProgressed(); // Schedule one last progress event so we guarantee that at least one is fired // for files that load very quickly. scheduleEvent(eventNames().progressEvent); scheduleEvent(eventNames().suspendEvent); m_networkState = NETWORK_IDLE; } void HTMLMediaElement::progressEventTimerFired(Timer<HTMLMediaElement>*) { ASSERT(m_player); if (m_networkState != NETWORK_LOADING) return; double time = WTF::currentTime(); double timedelta = time - m_previousProgressTime; if (m_player->didLoadingProgress()) { scheduleEvent(eventNames().progressEvent); m_previousProgressTime = time; m_sentStalledEvent = false; if (renderer()) renderer()->updateFromElement(); if (hasMediaControls()) mediaControls()->bufferingProgressed(); } else if (timedelta > 3.0 && !m_sentStalledEvent) { scheduleEvent(eventNames().stalledEvent); m_sentStalledEvent = true; setShouldDelayLoadEvent(false); } }
-
The relationship between the networkState of
HTMLMediaElement
and the networkState ofMediaPlayer
can be seen in the methodHTMLMediaElement::setNetworkState(MediaPlayer::NetworkState state)
located inSource/WebCore/html/HTMLMediaElement.cpp
. Next following the source of this method along with some methods related.void HTMLMediaElement::setNetworkState(MediaPlayer::NetworkState state) { LOG(Media, "HTMLMediaElement::setNetworkState(%d) - current state is %d", static_cast<int>(state), static_cast<int>(m_networkState)); if (state == MediaPlayer::Empty) { // Just update the cached state and leave, we can't do anything. m_networkState = NETWORK_EMPTY; return; } if (state == MediaPlayer::FormatError || state == MediaPlayer::NetworkError || state == MediaPlayer::DecodeError) { mediaLoadingFailed(state); return; } if (state == MediaPlayer::Idle) { if (m_networkState > NETWORK_IDLE) { changeNetworkStateFromLoadingToIdle(); setShouldDelayLoadEvent(false); } else { m_networkState = NETWORK_IDLE; } } if (state == MediaPlayer::Loading) { if (m_networkState < NETWORK_LOADING || m_networkState == NETWORK_NO_SOURCE) startProgressEventTimer(); m_networkState = NETWORK_LOADING; } if (state == MediaPlayer::Loaded) { if (m_networkState != NETWORK_IDLE) changeNetworkStateFromLoadingToIdle(); m_completelyLoaded = true; } if (hasMediaControls()) mediaControls()->updateStatusDisplay(); } void HTMLMediaElement::changeNetworkStateFromLoadingToIdle() { m_progressEventTimer.stop(); if (hasMediaControls() && m_player->didLoadingProgress()) mediaControls()->bufferingProgressed(); // Schedule one last progress event so we guarantee that at least one is fired // for files that load very quickly. scheduleEvent(eventNames().progressEvent); scheduleEvent(eventNames().suspendEvent); m_networkState = NETWORK_IDLE; } void HTMLMediaElement::mediaLoadingFailed(MediaPlayer::NetworkState error) { stopPeriodicTimers(); // If we failed while trying to load a <source> element, the movie was never parsed, and there are more // <source> children, schedule the next one if (m_readyState < HAVE_METADATA && m_loadState == LoadingFromSourceElement) { if (m_currentSourceNode) m_currentSourceNode->scheduleErrorEvent(); else LOG(Media, "HTMLMediaElement::setNetworkState - error event not sent, <source> was removed"); if (havePotentialSourceChild()) { LOG(Media, "HTMLMediaElement::setNetworkState - scheduling next <source>"); scheduleNextSourceChild(); } else { LOG(Media, "HTMLMediaElement::setNetworkState - no more <source> elements, waiting"); waitForSourceChange(); } return; } if (error == MediaPlayer::NetworkError && m_readyState >= HAVE_METADATA) mediaEngineError(MediaError::create(MediaError::MEDIA_ERR_NETWORK)); else if (error == MediaPlayer::DecodeError) mediaEngineError(MediaError::create(MediaError::MEDIA_ERR_DECODE)); else if ((error == MediaPlayer::FormatError || error == MediaPlayer::NetworkError) && m_loadState == LoadingFromSrcAttr) noneSupported(); updateDisplayState(); if (hasMediaControls()) { mediaControls()->reset(); mediaControls()->reportedError(); } logMediaLoadRequest(document()->page(), String(), stringForNetworkState(error), false); }
-
An important method is the
potentiallyPlaying
. Next following the source and some depended methods. It's in theSource/WebCore/html/HTMLMediaElement.cpp
.bool HTMLMediaElement::potentiallyPlaying() const { // "pausedToBuffer" means the media engine's rate is 0, but only because it had to stop playing // when it ran out of buffered data. A movie is this state is "potentially playing", modulo the // checks in couldPlayIfEnoughData(). bool pausedToBuffer = m_readyStateMaximum >= HAVE_FUTURE_DATA && m_readyState < HAVE_FUTURE_DATA; return (pausedToBuffer || m_readyState >= HAVE_FUTURE_DATA) && couldPlayIfEnoughData() && !isBlockedOnMediaController(); } bool HTMLMediaElement::couldPlayIfEnoughData() const { return !paused() && !endedPlayback() && !stoppedDueToErrors() && !pausedForUserInteraction(); } bool HTMLMediaElement::endedPlayback() const { double dur = duration(); if (!m_player || std::isnan(dur)) return false; // 4.8.10.8 Playing the media resource // A media element is said to have ended playback when the element's // readyState attribute is HAVE_METADATA or greater, if (m_readyState < HAVE_METADATA) return false; // and the current playback position is the end of the media resource and the direction // of playback is forwards, Either the media element does not have a loop attribute specified, // or the media element has a current media controller. double now = currentTime(); if (m_playbackRate > 0) return dur > 0 && now >= dur && (!loop() || m_mediaController); // or the current playback position is the earliest possible position and the direction // of playback is backwards if (m_playbackRate < 0) return now <= 0; return false; } bool HTMLMediaElement::stoppedDueToErrors() const { if (m_readyState >= HAVE_METADATA && m_error) { RefPtr<TimeRanges> seekableRanges = seekable(); if (!seekableRanges->contain(currentTime())) return true; } return false; }
-
Stop method in Webkit, source
Source/WebCore/html/HTMLMediaElement.cpp
.void HTMLMediaElement::stop() { LOG(Media, "HTMLMediaElement::stop"); if (m_isFullscreen) exitFullscreen(); m_inActiveDocument = false; userCancelledLoad(); // Stop the playback without generating events m_playing = false; setPausedInternal(true); if (renderer()) renderer()->updateFromElement(); stopPeriodicTimers(); cancelPendingEventsAndCallbacks(); } void HTMLMediaElement::setPausedInternal(bool b) { m_pausedInternal = b; updatePlayState(); } void HTMLMediaElement::updatePlayState() { if (!m_player) return; if (m_pausedInternal) { if (!m_player->paused()) m_player->pause(); refreshCachedTime(); m_playbackProgressTimer.stop(); if (hasMediaControls()) mediaControls()->playbackStopped(); return; } bool shouldBePlaying = potentiallyPlaying(); bool playerPaused = m_player->paused(); LOG(Media, "HTMLMediaElement::updatePlayState - shouldBePlaying = %s, playerPaused = %s", boolString(shouldBePlaying), boolString(playerPaused)); if (shouldBePlaying) { setDisplayMode(Video); invalidateCachedTime(); if (playerPaused) { if (!m_isFullscreen && isVideo() && document() && document()->page() && document()->page()->chrome()->requiresFullscreenForVideoPlayback()) enterFullscreen(); // Set rate, muted before calling play in case they were set before the media engine was setup. // The media engine should just stash the rate and muted values since it isn't already playing. m_player->setRate(m_playbackRate); m_player->setMuted(m_muted); m_player->play(); } if (hasMediaControls()) mediaControls()->playbackStarted(); startPlaybackProgressTimer(); m_playing = true; } else { // Should not be playing right now if (!playerPaused) m_player->pause(); refreshCachedTime(); m_playbackProgressTimer.stop(); m_playing = false; double time = currentTime(); if (time > m_lastSeekTime) addPlayedRange(m_lastSeekTime, time); if (couldPlayIfEnoughData()) prepareToPlay(); if (hasMediaControls()) mediaControls()->playbackStopped(); } updateMediaController(); if (renderer()) renderer()->updateFromElement(); }
-
When the progress events are sent in Webkit Media element code,
Source/WebCore/html/HTMLMediaElement.cpp
void HTMLMediaElement::changeNetworkStateFromLoadingToIdle() { m_progressEventTimer.stop(); if (hasMediaControls() && m_player->bytesLoaded() != m_previousProgress) mediaControls()->bufferingProgressed(); // Schedule one last progress event so we guarantee that at least one is fired // for files that load very quickly. scheduleEvent(eventNames().progressEvent); m_networkState = NETWORK_IDLE; } void HTMLMediaElement::progressEventTimerFired(Timer<HTMLMediaElement>*) { ASSERT(m_player); if (m_networkState != NETWORK_LOADING) return; unsigned progress = m_player->bytesLoaded(); double time = WTF::currentTime(); double timedelta = time - m_previousProgressTime; if (progress == m_previousProgress) { if (timedelta > 3.0 && !m_sentStalledEvent) { scheduleEvent(eventNames().stalledEvent); m_sentStalledEvent = true; setShouldDelayLoadEvent(false); } } else { scheduleEvent(eventNames().progressEvent); m_previousProgress = progress; m_previousProgressTime = time; m_sentStalledEvent = false; if (renderer()) renderer()->updateFromElement(); if (hasMediaControls()) mediaControls()->bufferingProgressed(); } } void HTMLMediaElement::startProgressEventTimer() { if (m_progressEventTimer.isActive()) return; m_previousProgressTime = WTF::currentTime(); m_previousProgress = 0; // 350ms is not magic, it is in the spec! m_progressEventTimer.startRepeating(0.350); }
-
Seeking in Webkit Media element code,
Source/WebCore/html/HTMLMediaElement.cpp
void HTMLMediaElement::seek(float time, ExceptionCode& ec) { LOG(Media, "HTMLMediaElement::seek(%f)", time); // 4.8.9.9 Seeking // 1 - If the media element's readyState is HAVE_NOTHING, then raise an INVALID_STATE_ERR exception. if (m_readyState == HAVE_NOTHING || !m_player) { ec = INVALID_STATE_ERR; return; } // If the media engine has been told to postpone loading data, let it go ahead now. if (m_preload < MediaPlayer::Auto && m_readyState < HAVE_FUTURE_DATA) prepareToPlay(); // Get the current time before setting m_seeking, m_lastSeekTime is returned once it is set. refreshCachedTime(); float now = currentTime(); // 2 - If the element's seeking IDL attribute is true, then another instance of this algorithm is // already running. Abort that other instance of the algorithm without waiting for the step that // it is running to complete. // Nothing specific to be done here. // 3 - Set the seeking IDL attribute to true. // The flag will be cleared when the engine tells us the time has actually changed. m_seeking = true; // 5 - If the new playback position is later than the end of the media resource, then let it be the end // of the media resource instead. time = min(time, duration()); // 6 - If the new playback position is less than the earliest possible position, let it be that position instead. float earliestTime = m_player->startTime(); time = max(time, earliestTime); // Ask the media engine for the time value in the movie's time scale before comparing with current time. This // is necessary because if the seek time is not equal to currentTime but the delta is less than the movie's // time scale, we will ask the media engine to "seek" to the current movie time, which may be a noop and // not generate a timechanged callback. This means m_seeking will never be cleared and we will never // fire a 'seeked' event. #if !LOG_DISABLED float mediaTime = m_player->mediaTimeForTimeValue(time); if (time != mediaTime) LOG(Media, "HTMLMediaElement::seek(%f) - media timeline equivalent is %f", time, mediaTime); #endif time = m_player->mediaTimeForTimeValue(time); // 7 - If the (possibly now changed) new playback position is not in one of the ranges given in the // seekable attribute, then let it be the position in one of the ranges given in the seekable attribute // that is the nearest to the new playback position. ... If there are no ranges given in the seekable // attribute then set the seeking IDL attribute to false and abort these steps. RefPtr<TimeRanges> seekableRanges = seekable(); // Short circuit seeking to the current time by just firing the events if no seek is required. // Don't skip calling the media engine if we are in poster mode because a seek should always // cancel poster display. bool noSeekRequired = !seekableRanges->length() || (time == now && displayMode() != Poster); #if ENABLE(MEDIA_SOURCE) // Always notify the media engine of a seek if the source is not closed. This ensures that the source is // always in a flushed state when the 'seeking' event fires. if (m_sourceState != SOURCE_CLOSED) noSeekRequired = false; #endif if (noSeekRequired) { if (time == now) { scheduleEvent(eventNames().seekingEvent); scheduleTimeupdateEvent(false); scheduleEvent(eventNames().seekedEvent); } m_seeking = false; return; } time = seekableRanges->nearest(time); if (m_playing) { if (m_lastSeekTime < now) addPlayedRange(m_lastSeekTime, now); } m_lastSeekTime = time; m_sentEndEvent = false; #if ENABLE(MEDIA_SOURCE) if (m_sourceState == SOURCE_ENDED) setSourceState(SOURCE_OPEN); #endif // 8 - Set the current playback position to the given new playback position m_player->seek(time); // 9 - Queue a task to fire a simple event named seeking at the element. scheduleEvent(eventNames().seekingEvent); // 10 - Queue a task to fire a simple event named timeupdate at the element. scheduleTimeupdateEvent(false); // 11-15 are handled, if necessary, when the engine signals a readystate change. } PassRefPtr<TimeRanges> HTMLMediaElement::seekable() const { return m_player ? m_player->seekable() : TimeRanges::create(); }
-
Pause in Webkit Media element code,
Source/WebCore/html/HTMLMediaElement.cpp
void HTMLMediaElement::pause() { LOG(Media, "HTMLMediaElement::pause()"); if (userGestureRequiredForRateChange() && !ScriptController::processingUserGesture()) return; pauseInternal(); } void HTMLMediaElement::pauseInternal() { LOG(Media, "HTMLMediaElement::pauseInternal"); // 4.8.10.9. Playing the media resource if (!m_player || m_networkState == NETWORK_EMPTY) scheduleLoad(MediaResource); m_autoplaying = false; if (!m_paused) { m_paused = true; scheduleTimeupdateEvent(false); scheduleEvent(eventNames().pauseEvent); } updatePlayState(); }
- Ekioh HTML5 TV Browser - http://www.ekioh.com/html5tvbrowser
- Arora - https://code.google.com/p/arora/, https://github.com/Arora/arora
- Webkit driver - https://code.google.com/p/webkitdriver/
- dwb - http://portix.bitbucket.org/dwb/
- Zetakey Webkit Browser - http://www.zetakey.com/browser.php
- Qupzilla - http://www.qupzilla.com/
- surf - http://surf.suckless.org/
- Espial TV Browser - http://www.espial.com/products/evo_browser/
- Ghost.py - https://github.com/jeanphix/Ghost.py
- Easily embedding WebKit into your EFL application - http://www.politreco.com/2010/10/easily-embedding-webkit-into-your-efl-application/
- European WebKit hackathon wrap-up - http://blogs.adobe.com/webplatform/2012/10/01/european-webkit-hackathon-wrap-up/
- WebKit Documentation - http://arunpatole.com/blog/2011/webkit-documentation/
- Getting Started With the WebKit Layout Code - http://blogs.adobe.com/webplatform/2013/01/21/getting-started-with-the-webkit-layout-code/