
MythStream v0.18
=================================================================
Written by : Eric Giesselbach 2003,2007
Home       : http://home.kabelfoon.nl/~moongies/streamtuned.html
License    : GNU GPL
=================================================================

More recent documentation can be found at:
  http://home.kabelfoon.nl/~moongies/howto/index.html

1.  Intro
2.  Changes
3.  Registered keys
4.  GUI Modes
5.  Parsing www, xml, rss, presenting interactive menus
6a.  Recording streams
6b.  Downloading files
7.  Configuring streams
8.  Play a list of stream items (like a list of music videos)
8.  Creating a web repository ("webservice")
10.  WorldWideMediaProject.com playlist
11. Tips
12. MythStream in MythTV Integration issue
13. Wishes&Bugs


Intro
=====
MythStream is an unofficial MythTV plugin that plays Internet audio and video
streams. The larger part of MythStream is built around the idea that enjoying
streams is more than just playing your favorite top 5 streams. There is a
whole world of content to explore. MythStream uses external parsers to connect
to stream index sites, playlists and XML feeds so you can "browse" streams
without configuring every stream url explicitly.

Support for several playlists and feeds (Icecast, Shoutcast, podcasts,
WorldWideMediaProject, BBC, ...) is included, other sources can be supported
by adding parsers in the form of PERL scripts.

The same scripts can be used to create interactive menu's. At the moment
this is only used to dynamically query websites but other possibilities - such
as giving feedback to site or stream provider - exist.

Stream urls are stored in the mythconverg.streams table (default repository). It is
also possible to manage stream url's in files or through a webservice.

MythStream can record streams, but recording support is very basic.

MythStream v018 changes
=======================

solved:
- high CPU usage during video play (reported on some ATI video card systems)
- direct lirc/keyboard input to mplayer interfered with myth key events
- screen redraw issues while switching views
- keeping video aspect ratio

new:
- TV-off operation facilitated by speech synthesizer (espeak or festival)
- fullscreen command now toggles preview, fullscreen and fullscreen zoom mode
- faster interaction - mplayer stream init skipped when handler is set. If this
  breaks your custom parser let me know.
- playlists: create a playlist by marking streams in a folder, video switches to
  fullscreen or fullscreen zoom depending on the previous zoom state.
- documentation - and it only took 4 years.
- prompting for user input, e.g. search term (<:name::value:> tag in url)
- downloads are listed in a special download folder (deleting item = removing file)
- FUZZY_DL handler matches downloads by name to prevent multiple downloads from dynamic url
- share downloads between frontends
- add, edit, delete streams on the fly (press E)
- asx.ram.rm.pls.m3u are now handled as stream meta files (set in mythstream.pro)
- key 3,9 for pageup,pagedown in stream item list, press 0 to change storage
- small, big font nodes added to stream-ui.xml in default themes

bugs:
- setup window segfault (MythTV/MythStream class name conflict)
- escaping special chars on parser command line (Damien Braillard)
- explicit handler=default by parsers prevented automatic meta file handling
- video pause caused switch to preview window

new parsers:
- youtube, dailymotion (download/play, direct play requires mplayer svn > 2007aug19)
- tvlistings.co.uk, not included in demo storage, see parser directory.

updated parsers:
- omroep (site change)
- all interactive parsers (bugfix)
- shoutcastTV
- apple movie trailer parser works again
- some parsers use stream item name in arg 3


Registered keys
===============
Probably mapped on your remote control:
  Browse folders and streams      cursor keys
  Start stream or harvester       [RETURN]
  Stop stream                     [END, ESC]
  Pause                           P
  Fullscreen toggle               F, [RETURN] when video active
  Volume down                     [
  Volume up                       ]
  decrease audio delay            -
  increase audio delay            +
  Get item information            I    (view stream item properties, Podcast meta data)
  Skip player forward             >    (if supported, e.g. Podcast files)
  Skip player backward            <    (if supported, e.g. Podcast files)
  Load tarball stream storage     9
  Load default mythstream storage 0
  Load configured storage index   1..8 (if configured)

Guess not mapped on your remote control:
  Mark a stream                   M
  Store marked streams            Y
  Show player dump                D

Functions present, but key mapping removed:
  Start recording                 R
  Stop recording                  S
  Stop all recordings             Q


GUI Modes
=========
MythStream has five different "modes":

- browse mode    : Browsing folders and stream items. Folders are listed on
                   the bottom of the display, items are listed from top to
                   bottom. The current folder and stream are highlighted.
- play mode      : Active during stream play. Stream details like name, codec,
                   buffer, stability are shown. The top right LED is green.
                   A Flashing LED means the player is buffering.
- harvester mode : Shown are URL fetch status or harvested URL's. The bottom
                   right LED is red. A flashing LED means the harvester is
                   receiving data (fetching URL's). URL's likely to be
                   streams are prefixed with ~.
- Action mode    : Several actions that require feedback present that feedback
                   using the MythStream display. This mode always has "Cancel"
                   as last option.
- Info mode      : Displays (stream) item details. Most information is displayed
                   in the display prefixed with "I"-icons. Items "harvested" from
                   XML or other data may have embedded html or multiline text. This
                   content is presented as an entry with a dedicated icon. Selecting
                   this entry opens a separate viewer.


Parsing www, xml, rss, presenting interactive menus
===================================================
MythStream parses playlists and xml feeds through external PERL scripts.
For non-PERL parser support, see the alt_parsers directory

Normally, when you select an url, MythStream attempts to play that url using mplayer. If
this url failes to play, the harvester will start a download on that url (using a cache and
if-modified header). The downloaded data is then "harvested" for (stream) urls. If the data is
HTML, XML or some other plain text format containing url's, the result may be (depending on
the used parser) a list of stream items that is displayed in the MythStream "harvester mode".

The parsers are PERL scripts located in the $HOME/.MythStream/parsers directory:
- default.pl   (the default parser)
- icecast.pl   (parsing icecast xml stream list)
- shoutcast.pl (parsing shoutcast stream list, by R. de Vos)
- podcast.pl   (parsing podcast rss feeds)
- rdfcast.pl   (parsing podcasts using rdf format)
- apple.pl     (parsing iTunes movie trailers, by Michael Knoll)
- example.pl   ("template"-script to use for building your own script)
- omroep/*     (interactive menu for uitzendinggemist.nl (Dutch))
- wwmp/*       (interactive menu for worldwidemediaproject.com)
- shoutcast/*  (interactive menu for www.shoutcast.com)
- youtube/*    (youtube rss feed reader)

Some parsers are also available as executables: see the separate README in the alt_parser directory.

Enter the parser filename (without extension) in the "handler" field of the stream item to tell
MythStream to use this parser for the stream item. If you leave the handler property empty,
MythStream will start mplayer when selecting the stream item. When mplayer exits, the harvester
will start, using the default parser. If a parser is set, this parser will be invoked immediately
when selecting the stream item, skipping the mplayer step. 

If a parser is set the harvester retrieves data from the url and passes this data to the parser. 
If you want your parser to get the data itself, prefix the parser name in the handler field with 
an asterisk (*parsername). The harvester then skips the data retrieval and calls the parser directly, 
giving the stream url as parameter.

The icecast parser can parse the XML dump from http://dir.xiph.org/yp.xml. The podcast (and
rdfcast) parsers can parse podcast xml feeds. Stream items returned by these scripts can contain
additional information provided by the xml feed, like embedded html. This information is displayed
while in "info mode" (key I). If a stream item contains html or multiline text a separate text/html
viewer can be started from info mode.

The harvester defaults to the "catch all" parser default.pl. This parser will detect every url,
but cannot detect relations between stream name and stream url other than the html hyperlink
(<a href=[stream url]>[stream name]</a>).

If you have a playlist that is not parsed properly by an existing script, just create a new one.
The existing scripts offer the information needed (if you are familiar with PERL or programming
in general). Mail the script if you want it included in the tarball (always appreciated, I have
to review it first though). Note the optional handler/viewer properties output by the scripts. The
list of possible values in this release is limited (STREAM_DL/inline,text,html) but will grow.

Mutistage parsers can be used to create interactive menu's. Such a parser returns encoded state
in stream urls. When the user selects an entry, the fake url with state information is passed to
the parser, enabling it to determine the next action based on state and user selection.
  Example: wwmp/menu.pl



6a. RECORDING STREAMS
=====================

MythStream is based on StreamTuned (see same webpage you got MythStream from). StreamTuned supports
recording, MythStream does not - at least not in a very usable way. If you insist, you can try to
record your stream using the information below. Not tested in a long time...

Simultaneous recordings are possible. Scheduled, running and finished recordings
are listed in the dedicated folder "recordings". Every entry has an icon indicating
it's current status. Recording only works when the plugin is running. And it often doesn't...

By default, streams are recorded to files in the directory
 $HOME/.mythtv/mythstream/recordings

Replace the directory recordings by a symlink if you want to store the
files elsewhere (warning: following code deletes all recordings):
  cd $HOME/.mythtv/mythstream
  cp -dpR recordings/* /whatever/path/to/whatever/directory/
  rm -Rf recordings
  ln -s /whatever/path/to/whatever/directory recordings

Note that if the stream entry is not a valid stream (i.e. isn't an
url mplayer will play) the harvester will not assist by "harvesting" the
url that failed to record. Use the "store stream" folder during stream
play (use cursor left or right) to store the actual stream url and use that
entry for recording.


To schedule a recording:

  A recording is scheduled by placing a special formed entry in the folder
  "recordings".

  The name of the entry must have the format:
    REC[text] yyyymmdd hhmm HHMM [stream name]
    
    yyyymmdd = start date
    hhmm     = start time
    HHMM     = stop time (if stop < start the recording ends the next day)

  Examples:
    REC0 20040314 1653 1753 Easy Belgium
    REC0 2004-03-14 16:53 17:53 Easy Belgium

  The parts [text] and [stream name] are used to make the entry unique and
    sensible, they do not affect scheduling.
  The "stream url" field contains the file to write the stream to. This
    target file will be played when playing the recording.
  The "stream description" field contains the URL to record.
  Changing target file name or stream url will not affect running
    recordings.

  Errors in scheduling information are reported in the GUI upon commit.


Removing and updating recordings:

  The entries in the "recordings" folder affect files on harddisk in this way:
  1. When a recording starts, the target file (in the "Stream url"-field) will
     be overwritten.
  2. When recording fails, the target file will be deleted if it is empty
  3. When a scheduled, running or finished recording is deleted, the target
     file will be deleted too.
  4. It is possible to change the stop-time of a running recoring
   
The special meaning of a recording entry is related to the special
"recordings" folder. When a entry is moved out of the recordings folder,
the entry is treated as a normal stream entry. It still points to the file,
but if you delete the stream entry, the target file will remain on disk.


playing recordings:

To play a recording browse to it and press [RETURN]. Note that depending
on the stream format, mplayer can/cannot handle a growing stream file (that
is recorded at that moment). The player exists early if it can't. This will
not affect the recording.

Tip: use mencoder to transform the recorded stream to another (seekable)
     format after recording.



6b. DOWNLOADING A/V FILES
=========================
Audio and video files (mp3, flv, mpg, ...) can be downloaded.
Streams cannot be downloaded, see 6a for recording streams.

The parser decides wether a file should be downloaded or played directly. Some parsers
ask the user what to do with the file.

Examples:
- The dailymotion/feed parser asks the user to download or play directly
- The dailymotion/playlist parser plays the file directly without asking. The playlist
  parser is used to play playlists built from RSS feeds (of e.g. Music Videos), asking
  the user would break the playlist.
- The podcast parser downloads first - the proper behaviour for a podcast client

Files are stored in $HOME/.mythtv/mythstream/downloads/<host>/<url path + query>

Replace the directory downloads by a symlink if you want to store the
files elsewhere:
  cd $HOME/.mythtv/mythstream
  cp -dpR downloads/* /whatever/path/to/whatever/directory/ 
  rm -Rf downloads
  ln -s /whatever/path/to/whatever/directory downloads


Configuring streams and stream storages
=======================================
In the configuration module you can manage streams in a repository and
create / switch between repositories. Use configuration module from a
PC running mythfrontend. It was not designed (while designing it in
StreamTuned) to be used with a remote control as input device.

Managing streams : click on a stream on the left. The stream details are
                   presented on the right. Clicking "remove" removes the
                   stream, clicking "update" commits any editing. Note that
                   if you edited the folder name, an update moves the
                   stream entry to a new or existing folder.

Managing storages: Here you can create new storages or stream repositories.
                   See below.

Switching storage: Change the "connected storage" and then choose between
                   "load storage" or "overwrite storage".

                   Use "M" and "Y"
                   copy and paste in browser mode to append items to a 
                   storage.


8.  Play a list of stream items (like a list of music videos)
=============================================================
To create a playlist from a folder of stream items, just mark (m-key)
the items you want to include, then start the first of the marked items.

It is possible to create a playlist from a folder previously created in
the setup module. However, this feature was added with RSS feeds in mind.

The demo playlist includes RSS feeds from DailyMotion created from custom
queries. press 1 in the GUI and check the folder "Music Video". Press I to
see the stream item fields, press 0 to return to your storage in the mythtv
database.




The playlist will not "wrap", if you start an item somewhere down the list,
the items above it will not be played.

Video's will automatically switch to fullscreen five seconds into the video.

The playlist stops running when:
- the end of the playlist has been reached
- one of the stream items fails to play
- one of the stream items produces a new stream item list (parser action)

1) if the playlist stops because of a failed stream, try the failed stream
   again. The playlist will resume from this position.
2) if a new stream item results in a new (parser generated) stream item list,
   blame the parser. It shouldn't happen. Or maybe it should, but then you
   shouldn't use the list to build a playlist from.


   


Creating a web repository ("webservice")
========================================
1. Copy misc/base.php and misc/streamtuned.php to a proper directory under
   the Apache document root.
2. Create a database repository
3. Edit base.php to configure access to the database
4. Change password in streamtuned.php
5. Enter repository properties in MythStream configuration
   -> php.ini must have global vars enabled.
   -> to test repository availability use a browser to open
      http://<path>/streamtuned.php?command=list&pass=<your password>


WorldWideMediaProject playlist
==============================
The wwmp_pls parser lets you load your personal WWMP-playlist in MythStream.
This item is not included in the release database because it depends on a
link to the site that's unique for each user.

To load your playlist:

1) copy the Playlist URL from your user account profile on the
    World Wide Media Project site.
2) add this stream item in MythStream:
      Stream Name   : My WWMP Favorites
      Stream URL    : [your Playlist URL]
      Stream Handler: wwmp/wwmp_pls
3) select the new stream item in MythStream.



Tips
====
- See INSTALL if you have problems playing (Realmedia) streams
- When harvesting a website, try to filter stream url's using
  "filter" to remove (suspected) non-stream url's from the list.
- You can override the default parser in the harvester, and use a perl script.
  see the parsing topic.
- Create a web repository for remote access to your stream list
- The usage of mplayer is configured in player.xml, you can add
  custom command line parameters for mplayer in that file. 
- If you have problems playing a stream then start that stream directly with
  MPlayer
- MPlayer output can be "grepped" and displayed in the GUI by defining
  properties and corresponding regular expressions in the file player.xml.
  If you want to display specific MPlayer output edit the player.xml file:
   - every MPlayer output line is treated as [label]:[value]
   - use item StreamCustomEventX with X in {0..9} 



MythStream in MythTV integration issue
======================================
The reason for MythStream's "unofficial" status is that MythTV's lead developer believes
a separate stream plugin like MythStream shouldn't exist. Streaming should be integrated
in MythMusic*. This MythMusic integration was the most important original requirement
for MythStream code to make it to official status, and I haven't heard of a change in
requirements since. (*reference: http://mythtv.beirdo.ca/ircLog/channel/1/2005-10-08)

Although I would be happy to see MythStream code integrated in MythMusic, that action
alone can't replace MythStream. Internet streaming is about more than playing music, and
about more than playing my favorite radio station.

When I listen to music, I don't care where the music comes from, as long as it just plays
with acceptable quality. When I stream news or a documentary I *do* care where it comes
from, most of the time I searched for it explicitly, and I wouldn't like to miss it solely
based on the fact that it happened to be a video stream instead of audio only. And if the
content is good enough, I can live with less than 100% stream reliability.

I also don't expect my music player to be a browser for unknown content. Some presets are
good enough. But if I want to enjoy the huge and ever changing world of streams I need a
tuning dial. This way I don't have to bring my stream url's to MythStream. I turn the dial,
and MythStream delivers the streams.

What I would like to do is to adapt MythStream to label reliable Internet streams for usage
in MythMusic or MythVideo. These plugins can then be changed to simply play these "preset"
streams. It might even be possible to add streams with schedule information to the existing
TV channels. It's *a* good way to use streaming content, and I would be happy to help make
that possible.

But I will still need that tuning dial. And I don't have the answer where to put it other
than the place it's now. I believe the proper solution starts with an effort to integrate
MythStream functionallity into MythTV. Starting with the claim that MythStream is useless
as a plugin and then see if some of it's functionallity can land in MythMusic is not a
project where I want to devote time to.


Wishlist
========
- auto handling defined stream metafiles
- Visualisations
- Continue streaming after exiting mythstream option
- Valid Myth key events are also passed to mplayer
