Using server-side SMIL playlists

Imagine you have several movie clips that you want to stream in a particular order and in addition you only want to play parts of some of these movies. How to do it? Use the SMIL based server-side playlists supported by each of the major streaming formats: RealNetworks, Windows Media and QuickTime.

Another application is to decompose each media item into its constituent parts. In this example a video has been decomposed into a number of sequences. Each sequence is then decomposed into a number of scenes. Each scene could be further decomposed into a number of shots.

The W3C's Synchronised Multimedia Integration Language (SMIL) can be used to achieve this for each of the streaming formats all of which are supported by EUCS. SMIL is very similar to XHTML - unsurprisingly as they are both XML based languages

1. RealNetworks

Real have integrated SMIL capabilities into RealPlayer since 1998. This is not the time and place for a SMIL tutorial: just use the code sample below as a guide.

1.1 What to do?

  1. Author your .smil file containing the SMIL 2.0 timing elements and attributes.
  2. Place the .smil file in your RealServer file space on boombox.ucs.e.dac.uk.
  3. Reference it as any other streaming content. For example:

    Introduction to Edinburgh 

    <a href="http://boombox.ucs.ed.ac.uk/ramgen/domsdemo/vplaylist1.smil">
Introduction to Edinburgh
</a>

1.2 Sample .smil file

This SMIL 2.0 file defines the order in which three extracts from the same video clip are played. After an initial delay of 3 seconds, the first extract is played from 63 to 92 seconds of the video's inherent time line. After a further delay of 2 seconds the next extract is played from 42 to 62 seconds of the video's time line. Finally, after a further pause of 2 seconds the final extract is played from the start to 41 seconds along the video's time line. 

<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
<!-- (c) University of Edinburgh 2003 All Rights Reserved -->
 <head>
  <meta name="author" content="The University of Edinburgh" />
  <meta name="title" content="Introduction to Edinburgh" />
  <meta name="copyright" content="(c) 2001 The University of Edinburgh" />
   <!-- Define the layout for the SMIL presentation -->
  <layout>
    <!-- Presentation window is 320x240 with white background -->
    <root-layout id="main" width="320" height="240" background-color="#ffffff" />
	<!-- Define the regions used by the presentation -->
	<region id="screen" width="320" height="240" left="0" top="0" 
	z-index="2" fit="hidden" />
  </layout>
 </head>
 <body>
 <!-- A sequential group to hold our videos -->
   <seq id="sq1" begin="3s">
   <!-- Call each of our video segments in turn -->
     <video id="vid1" region="screen" src="v2003-1.rm" clipBegin="63s" 
     clipEnd="90s" fill="remove" />
     <video id="vid2" region="screen" src="v2003-1.rm" begin="2s"
     clipBegin="135s" clipEnd="149s" fill="remove" />
     <video id="vid3" region="screen" src="v2003-1.rm" begin="2s"
     clipBegin="0s" clipEnd="41s" fill="freeze" />
   </seq>
  </body>
</smil>

1.3 Timing attributes

The begin timing attribute is used to add pauses for both sequences and movies. The clipBegin and clipEnd attributes are used to precisely define each part of a video or audio file's internal timeline that you want to play. 

<!-- Add  an initial delay of 3 seconds -->
<seq begin="3s" >
  <!-- Call each of our video segments in turn -->
  <video id="vid1" region="screen" src="v2003-1.rm" clipBegin="63s"
  clipEnd="90s" fill="remove" />
  <-- Begin the second video extract 2 seconds after the first -->
  <video id="vid2" region="screen" src="v2003-1.rm" begin="2s"
  clipBegin="42s" clipEnd="62s" fill="remove" />
...

The same begin attribute can also be used to add pauses between sequences. For example: 

<seq id="sq1">
	<video id="vid1" src="my_video2.rm" />
	<video id="vid2" src="my_video2.rm" />
</seq>
>!-- Start 2nd sequence 5 seconds after previous -->
<seq id="sq2" begin="3s >
	<video id="vid3" src="my_video3.rm" />
	<video id="vid4" src="my_video4.rm" />
</seq>

1.3.1 Tip

The attribute fill="remove" is used to remove the video clip from view once it finishes and to present a blank screen before the next clip starts.

1.4 Further information on SMIL

For further information on the extensive support for SMIL 2.0 in the RealONE player see the RealNetworks Production Guide

2. Windows Media

Media Services 9 for Windows 2003 Server supports the use of SMIL 2.0 based playlists contained in .wsx files.

2.1 What to do?

  1. Author your .wsx playlist file containing the SMIL 2.0 timing elements and attributes. Note that you do not specify SMIL regions for the video to play within.
  2. Create a .wvx metafile file that references it on your WWW server. See local details of how to author Windows Media metafiles
  3. Reference this .wvx metafile like any other streaming content. For example:

    Introduction to Edinburgh 

    <a href="http://webhelp.ucs.ed.ac.uk/domsdemo/wmedia/vplaylist1.wvx">
Introduction to Edinburgh
</a>

2.1.1 Tip

When developing server-side playlists ensure that on the Publishing Point used, you have "Cache Expiration" set to "Always Expire" for the Properties/Cache/Proxy and that client and server caching is disabled under Properties/General.

2.2 Sample .wsx file

<?wsx version="1.0"?>
<smil>
  <seq id="sq1">
   <media id="vid1" src="v2003-1.wmv" clipBegin="63s" clipEnd="90s" />
   <media id="vid2" src="v2003-1.wmv" clipBegin="42s" clipEnd="62s" />
   <media id="vid3" src="v2003-1.wmv" clipBegin="0s" clipEnd="41s" />
  <seq>
</smil>

The <seq> container is not strictly necessary in this example as the default SMIL container is sequential. We suggest you use it for consistency and clarity.

2.2.1 Alternative .wsx file using the excl element

SMIL allows more than one way to express the same timing and sychronisation of media. The following example uses an exclusuve group to provide exactly the same results as the previous. 

<?wsx version="1.0"?>
<smil>
  <excl id="sq1">
   <media id="vid1" src="v2003-1.wmv" clipBegin="63s" clipEnd="90s"
   begin="0s" />
   <media id="vid2" src="v2003-1.wmv" clipBegin="2:15" clipEnd="02:29"
   begin="vid1.end" />
   <media id="vid3" src="v2003-1.wmv" clipBegin="0s" clipEnd="41s"
   begin="vid2.end" />
  <seq>
</excl>

Unlike in a sequential seqgroup, clips in an exclusive exclgroup do not automatically play one after the other; we have to provide explicit begin values. In this case we begin clips vid2 and vid3 after the preceding clip with the id has ended. Using this mechanism we can define any order in which the clips play. Play this example which implements the .wsx file shown below. 

<?wsx version="1.0"?>
<smil>
  <excl id="sq1">
   <media id="vid1" src="v2003-1.wmv" clipBegin="63s" clipEnd="90s"
   begin="vid2.end" />
   <media id="vid2" src="v2003-1.wmv" clipBegin="02:15" clipEnd="02:29.5"
   begin="0s" />
   <media id="vid3" src="v2003-1.wmv" clipBegin="0s" clipEnd="41s"
   begin="vid1.end" />
  <seq>
</excl>

2.2.2 Using HTML+TIME playlists

Windows Media server can also parse HTML pages (.htm) containing SMIL 2 timing and sychronisation elements and attributes as if they were normal .wsx playlist files. This HTML+TIME playlist example provides exactly the same functionality as the previous example. 

<html xmlns:t = "urn:schemas-microsoft-com:time">
<head>
<?IMPORT namespace="t" implementation="#default#time2"?>
</head>
<body>
  <t:excl>
   <t:media id="vid1" src="v2003-1.wmv" begin="vid2.end" clipBegin="63s" clipEnd="90s" />
   <t:media id="vid2" src="v2003-1.wmv" begin="0s" clipBegin="02:15" clipEnd="02:29.5" />
   <t:media id="vid3" src="v2003-1.wmv" begin="vid1.end" clipBegin="0s" clipEnd="41s" />
  </t:excl>
</body>
</html>

See MSDN for information of how to author HTML+TIME and details of its implementation in IE5.5 and I6

2.3 Timing attributes

The clipBegin and clipEnd attributes are used to precisely define each part of a video or audio file's internal timeline. The syntax is exactly the same as for the previous SMIL example for RealNetworks.

2.3.1 Warning!

The SMIL 2.0 implementation does not appear to work as documented. This issue has been posted to the WMTalk discussion list. A response is awaited.

For example, using the begin attribute with either the <seq> or <media/> elements does not produce the expected timing behaviour. For <media/> element(s) any <begin> value is simply ignored. For the <seq> element(s) the first media element in the sequence is immediately played for the duration of the begin attribute value e.g. 3s and then it is played as part of the sequence. Subsequent clips play ignoring any begin attributes.

2.3.2 Tip

If the clipBegin attribute is used with a video file, the element will actually begin on the nearest key frame prior to the clipBegin attribute value. For example, if a video file has key frames every 5 seconds, and the clipBegin attribute value is 12s, the video will actually begin at 10 seconds. This is done so that the first frame that appears is a complete frame. If a video file were to start playing back on any other frame, the user would see an incomplete picture until the next key frame.

When encoding material for streaming some care will need to be taken on deciding the interval for key frames. The general rule is to insert key frames approx. every 10 seconds for streaming content. This would mean that the clipBegin attribute would only be accurate to within a range of 0 to 10 seconds. A bette basis would be to inesert keyframes at the same rate as the FPS of the encoding; for 25fps this would require a keyframe every 25 frames to give an accuracy of 0-1 second for clipBegin

2.4 Further information

For further information on Media Sevices 9 Playlist syntax see the Server-Side Playlists on the Microsoft Developers Network (MSDN) website.

2.5 Using HTMLview with WM9 Player

One of the new features of WM9 player for XP that is installed as part of the EUCS managed desktop is the ability to view HTML pages within the player and control media content embedded within the HTMLusing the media player's controls. for example:

Physical Education

Since WMP9 uses an embedded version of IE6 these HTML pages can contain XHTML+TIME to add timing and synchronisation. For example:

Physical Education

3. QuickTime

QuickTime player v4.1 and up supports the use of SMIL 1.0 based playlists. The process for creating them is sligthly more complex than with Real .smil or Windows Media .wsx playlist files.

3.1 What to do

  1. Create a .smil file that refers to the QuickTime video/audio clips you want to play or the parts of them that you want to play using the clip-beginand clip-end attributes.
  2. Save this .smil file as a .mov file.
  3. Transfer this .mov file to your WWW server and write a .qtl (QuickTime Link) file that calls it. Keeping both the .mov and the .qtl fiels together in the same directory is a good idea. See local documentation for details of writing .qtl files.
  4. Reference this .qtl metafile like any other streaming content. For example:

    Introduction to Edinburgh 

    <a href="http://webhelp.ucs.ed.ac.uk/domsdemo/qtime/vplaylist1.qtl">
Introduction to Edinburgh
</a>

3.1.1 Warning!

Apple's implementation of SMIL 1.0 in the QuickTime player appears not to comply with the functionality documented by Apple in "QuickTime for the Web" QuickTime Developer Series:

3.2 Sample .smil (.mov) file

Here is a basic SMIL 1.0 file for use with QuickTime that demonstrates the use of the clip-beginand clip-end attributes. Note that the begin attribute is not used. 

SMILtext<smil xmlns:qt="http://www.apple.com/quicktime/resources/smilextensions" 
qt:autoplay="true" qt:chapter-mode="all" qt:immediate-instantiation="true">
<!-- (c) University of Edinburgh 2003 All Rights reserved -->
<!-- Start of the SMIL document head -->
  <head>
   <layout>
    <root-layout id="main" width="320" height="240" background-color="white" />
	 <region id="screen" width="320" height="240" left="0" top="0" z-index="1" />
    </layout>
  </head>
  <body>
   <seq id="sq1">
    <video id="vid1" region="screen" src="rtsp://pilton.ucs.ed.ac.uk/domsdemo/v2003-1.mov"
     clip-begin="npt=00:00:63" clip-end="npt=00:00:90" qt:chapter="1 Financial Centre" />
    <video id="vid2" region="screen" src="rtsp://pilton.ucs.ed.ac.uk/domsdemo/v2003-1.mov" 
    clip-begin="npt=00:02:15" clip-end="npt=00:02:29" qt:chapter="2 Hogmanay Celebrations" />
    <video id="vid3" region="screen" src="rtsp://pilton.ucs.ed.ac.uk/domsdemo/v2003-1.mov"
    clip-begin="npt=00:00:00" clip-end="npt=00:00:41" qt:chapter="3 Architectural Heritage" />
   </seq>
  </body>
</smil>

This plays the same video extracts as in the RealNetworks and Windows Media examples. We have used the qt:chapter QuickTime extension to provide a popup-menu in the movie controller.

3.3 Timing attributes

The clip-begin and clip-end attributes are used to precisely define each part of a video or audio file's internal timeline. Note that the syntax is different to the previous SMIL examples for RealNetworks and Windows Media.

3.3.1 Tips

3.4 Further information

For further information on the use of SMIL 1.0 with QuickTime player see the QuickTime and SMIL section in Apple's Developer Connection.