NAME
    tv_grab_eu_epgdata - Grab TV listings for parts of Europe.

SYNOPSIS
    tv_grab_eu_epgdata --help

    tv_grab_eu_epgdata --version

    tv_grab_eu_epgdata --capabilities

    tv_grab_eu_epgdata --description

    tv_grab_eu_epgdata [--config-file FILE] [--days N] [--offset N]
    [--output FILE] [--quiet] [--debug]

    tv_grab_eu_epgdata --configure [--config-file FILE]

    tv_grab_eu_epgdata --configure-api [--stage NAME] [--config-file FILE]
    [--output FILE]

    tv_grab_eu_epgdata --list-channels [--config-file FILE] [--output FILE]
    [--quiet] [--debug]

DESCRIPTION
    Output TV and listings in XMLTV format for many stations available in
    Europe.

    First you must run tv_grab_eu_epgdata --configure to choose which
    stations you want to receive.

    Then running tv_grab_eu_epgdata with no arguments will get a listings
    for the stations you chose for five days including today.

    This is a commercial grabber. The data service, www.epgdata.com, plans
    on granting test accounts to people for the rest of 2007. Mail
    service@epgdata.com if you want a Test PIN. The future of this grabber
    is a little bit unclear as of now; it's likely you'll have to pay a
    small sum to use it (beginning 2008).

OPTIONS
    --configure Prompt for which stations to download and write the
    configuration file.

    --config-file FILE Set the name of the configuration file, the default
    is ~/.xmltv/tv_grab_eu_epgdata.conf. This is the file written by
    --configure and read when grabbing.

    --gui OPTION Use this option to enable a graphical interface to be used.
    OPTION may be 'Tk', or left blank for the best available choice.
    Additional allowed values of OPTION are 'Term' for normal terminal
    output (default) and 'TermNoProgressBar' to disable the use of
    Term::ProgressBar.

    --output FILE When grabbing, write output to FILE rather than standard
    output.

    --days N When grabbing, grab N days rather than 5.

    --offset N Start grabbing at today + N days.

    --quiet Suppress the progress-bar normally shown on standard error. This
    option does not do anything right now.

    --debug Provide more information on progress to stderr to help in
    debugging. This option does not do anything right now.

    --list-channels Output a list of all channels that data is available
    for. The list is in xmltv-format.

    --version Show the version of the grabber.

    --help Print a help message and exit.

ERROR HANDLING
    N/A

ENVIRONMENT VARIABLES
    The environment variable HOME can be set to change where configuration
    files are stored. All configuration is stored in $HOME/.xmltv/. On
    Windows, it might be necessary to set HOME to a path without spaces in
    it.

SUPPORTED CHANNELS
    For a list of supported channels, see the channel_ids file distributed
    with this grabber. If additional channels are available, you will
    receive a warning when you run --configure. In the future, it might be
    possible to download an updated channel_ids file from the internet.

COMPATIBILITY
    The channel ids used in this grabber aim to be mostly possible with
    other grabbers, eg tv_grab_de_prisma and some other grabbers for other
    countries. NOTE: Retaining compatibility was not always possible or
    practicable. You can get a list of channel ids using --list-channels

AUTHOR
    Michael Haas, laga -at- laga -dot- ath -dot - cx. This documentation is
    copied from tv_grab_se_swedb by Mattias Holmlund, which in turn was
    copied from tv_grab_uk by Ed Avis. Parts of the code are copied from
    tv_grab_se_swedb and tv_grab_na_dd (in XMLTV 0.5.45) as well as various
    other sources.

BUGS
    There's no proper support for channels with locally different schedules.
    For example, if your EPG package is a German one, you'll get the EPG
    schedule for Germany even if you preferred the Swiss schedule which is
    also available in the data (for some channels at least).

    The EPG package code is not properly detected during the configure stage
    and 'y' is assumed. The package code determines which channel_?.xml is
    loaded.

    Timezones are not handled correctly. Currently, you have to enter your
    time zone manually during the configure step. You have to do this every
    time your time zone changes, eg for daylight saving time ("Sommerzeit"
    and "Normalzeit" for my fellow Germans). I'll try to have this fixed for
    the next XMLTV release. Please see this thread for further discussion
    and some additional issues:
    http://thread.gmane.org/gmane.comp.tv.xmltv.devel/7919 FYI: you can
    modify the time zone directly in the config file which is usually
    located at ~/.xmltv/tv_grab_eu_epgdata.conf or ~/.mythtv/FOO.xmltv where
    FOO is the name of your video source in MythTV.

    If the data source gives us data for one day, they'll also cover a part
    of the following day. Maybe this should be fixed. Please note: data is
    not overlapping! So if we want to get data for today, we might as well
    grab yesterday because that'll give us EPG till ~5am for today.

    I'm sure this list is not complete. Let me know if you encounter
    additional problems. =cut

    use strict; use LWP::Simple qw($ua getstore); use Archive::Zip; use
    File::Temp qw/ tempdir /; use XML::Twig;

    use XMLTV; use XMLTV::Options qw/ParseOptions/; use
    XMLTV::Configure::Writer; use XMLTV::Supplement qw/GetSupplement/;

    # deal with umlauts use HTML::Entities;

    use XMLTV::Memoize; XMLTV::Memoize::check_argv('getstore');

    # set user agent $ua->agent("xmltv/$XMLTV::VERSION");

    our $tmp= tempdir( CLEANUP => 1 ) . '/'; #our $tmp = "/tmp/foobarbaz/";

    # set up XML::Twig our $epg= new XML::Twig( twig_handlers => { data =>
    \&printepg } ); our $channels = new XML::Twig( twig_handlers => { data
    => \&printchannels } ); our %genre; our $genre = new XML::Twig(
    twig_handlers => { data => \&makegenrehash } );

    # build a hash: epgdata.com channel id -> xmltv channel id my $chanids =
    GetSupplement( 'tv_grab_eu_epgdata', 'channel_ids' );

    our %chanid; my @lines = split( /[\n\r]+/, $chanids ); foreach my $line
    (@lines) { if ($line !~ '^#') { my @chanid_array = split(';', $line);
    chomp($chanid_array[1]); $chanid{$chanid_array[0]}= $chanid_array[1]
    unless $line =~ '^#'; } }

    my( $opt, $conf ) = ParseOptions( { grabber_name =>
    "tv_grab_eu_epgdata", capabilities => [qw/baseline manualconfig tkconfig
    apiconfig cache/], stage_sub => \&config_stage, listchannels_sub =>
    \&list_channels, version => '$Id: tv_grab_eu_epgdata.in,v 1.3 2007/11/04
    17:35:41 mihaas Exp $', description => "Parts of Europe (commercial)
    (www.epgdata.com)", } );

    our $pin = $conf->{pin}->[0]; die 'Sorry, your PIN is not defined. Run
    tv_grab_eu_epgdata --configure to fix this.' unless defined($pin);

    our $tz = $conf->{tz}->[0]; # die 'Sorry, time zone is not defined. Run
    tv_grab_eu_epgdata --configure to fix this.' unless defined($tz); #
    Oops. Looks like the line below will result in a warning # telling us
    that we're declaring $tz twice. However, this does not seem to # be an
    issue. our $tz = '+0100' unless defined($tz);

    sub config_stage { # shamelessly stolen from
    http://xmltv.org/wiki/howtowriteagrabber.html

        my( $stage, $conf ) = @_;
        # Sample stage_sub that only needs a single stage.
        die "Unknown stage $stage" if $stage ne "start";

        my $result;
        my $configwriter = new XMLTV::Configure::Writer( OUTPUT => \$result,
                                                         encoding => 'ISO-8859-1' );
        $configwriter->start( { grabber => 'tv_grab_eu_epgdata' } );
        $configwriter->write_string( {
            id => 'pin', 
            title => [ [ 'Enter your PIN for epgdata.com', 'en' ] ],
            description => [ 
            [ 'This alphanumeric string is used for authentication with epgdata.com. 
            Ask service@epgdata.com for a test PIN (before 2007 ends)',
                'en' ] ],
            default => '',
        } );
        $configwriter->write_string( {
            id => 'tz', 
            title => [ [ 'Time zone for your EPG data', 'en' ] ],
            description => [ 
            [ 'Enter the time offset from UTC here. Think of it as your time zone. 
            For example: during winter in Germany, you should enter "+0100". During summer, use "+0200". (without quotation marks) ',
                'en' ] ],
            default => '+0100',
        } );
    
        $configwriter->end( 'select-channels' );
        return $result;
    }

    # construct writer object # taken from tv_grab_na_dd (XMLTV 0.4.45) #
    XMLTV::Options does not redirect stdout properly for us # XML::Twig
    probably messes it up, I don't know. :/ my %w_args; if (defined
    $opt->{output}) { my $fh = new IO::File(">$opt->{output}"); die "ERROR:
    cannot write to $opt->{output}: $!" if not defined $fh; $w_args{OUTPUT}
    = $fh; } $w_args{encoding} = 'ISO-8859-1';

    our $writer = new XMLTV::Writer(%w_args);

    downloadepg(); prepareinclude($conf); # it looks like we can also
    extract the language from the file # name of the epg data our @xmlfiles
    = glob($tmp . "*_*_??_q?.xml"); processxml();

    sub downloadepg { my $days = $opt->{days}; my $offset = $opt->{offset};
    my $i='0'; # we've got to start counting at 0 # if we did "$i <= $days",
    we'd end up with one zip file too much while ( $i < $days) { my
    $dataoffset = $i +$offset; my $baseurl="http://www.epgdata.com"; my
    $url=$baseurl . '/index.php?action=sendPackage&iOEM=&pin=' . $pin .
    '&dayOffset=' . $dataoffset . '&dataType=xml'; getstore($url, $tmp .
    "epgzip" . $dataoffset); # This doesn't seem to work correctly. # It
    doesn't fail even if the PIN is wrong. #unless (getstore($url, $tmp .
    "epgzip" . $dataoffset) == 200) { #die "Couldn't download epg file\n";
    #} $i++; } # FIXME: we can easily create a list of files earlier in this
    function my @zipfiles=(glob($tmp . 'epgzip*')); unzip(@zipfiles); }

    # for simplicity's sake, always call with $conf as argument at least sub
    prepareinclude { my ( $conf, $opt ) = @_; my
    $baseurl="http://www.epgdata.com"; my $pin = $conf->{pin}->[0]; my
    $includeurl=$baseurl . "/index.php?action=sendInclude&iOEM=&pin=" . $pin
    . "&dataType=xml"; getstore($includeurl, $tmp . "includezip"); # This
    doesn't seem to work correctly. # It doesn't fail even if the PIN is
    wrong. # unless (getstore($includeurl, $tmp . "includezip") == 200) { #
    die "Couldn't download include file\n"; # } my @zipfiles=( $tmp .
    "includezip"); unzip(@zipfiles) }

    sub unzip { foreach my $zipfile (@_) { my $zip = Archive::Zip->new(
    $zipfile ); my @filelist = $zip->memberNames; foreach my $ext (("\.dtd",
    "\.xml")) { foreach my $filename (@filelist) { # we only care about .dtd
    and .xml right now my $xmlfile=$filename if $filename =~ /$ext/;
    $zip->extractMember( $xmlfile, $tmp . $xmlfile ) if defined $xmlfile; }
    } } }

    sub processxml { $writer->start({ 'generator-info-name' =>
    'tv_grab_eu_epgdata' }); $genre->parsefile($tmp . 'genre.xml');
    $channels->parsefile($tmp . 'channel_' . findchannelcode($xmlfiles[0],
    $tmp) . '.xml'); foreach my $xmlfile (@xmlfiles) {
    $epg->parsefile($xmlfile); } $writer->end(); }

    sub makegenrehash { my( $twig, $genre)= @_; my $genreid =
    $genre->first_child('g0')->text; my $genrename =
    decode_entities($genre->first_child('g1')->text); $genre{$genreid}=
    $genrename; $twig->purge; }

    sub printepg { my( $twig, $sendung)= @_; my $internalchanid =
    $sendung->first_child('d2')->text; my $internalregionid =
    $sendung->first_child('d3')->text; our $chanid; if (defined
    $main::chanid{$internalchanid}) { $chanid =
    $main::chanid{$internalchanid}; } else { $chanid = $internalchanid; #
    FIXME: not sure if this is correct. # Maybe we should behave differently
    if we encounter an unknown ID, # but this ought to be OK for now } #
    alright, let's try this: # push the channel ids we want to grab in an
    array # http://effectiveperl.blogspot.com/ my %configuredchannels = map
    { $_, 1 } @{$conf->{channel}}; # does the channel we're currently
    processing exist in the hash? # BTW: this is not a lot more efficient in
    our case than looping over a list # but a few seconds are better than
    nothing :) if($configuredchannels{$chanid} && $internalregionid == '0')
    { my $title = decode_entities($sendung->first_child('d19')->text); my
    $subtitle = decode_entities($sendung->first_child('d20')->text); my
    $desc = decode_entities($sendung->first_child('d23')->text); my $start =
    $sendung->first_child('d4')->text; my $internalgenreid =
    $sendung->first_child('d25')->text; my $rating =
    $sendung->first_child('d30')->text; my $wide_aspect =
    $sendung->first_child('d29')->text; # black and white? my $bw_colour =
    $sendung->first_child('d11')->text; my $stereo_audio =
    $sendung->first_child('d27')->text; my $dolby_audio =
    $sendung->first_child('d28')->text; # I was told that technics_hd is
    supposed to exist # However, it's not listed in qy.dtd # my $hd_video =
    $sendung->first_child('XXX')->text;

                $start =~ s/-//g;
                $start =~ s/://g;
                $start =~ s/ //g;
                our %prog = ("channel" => $chanid, "start" => "$start $tz",
                    "title" => [ [ $title ] ]);

                 if ( length($subtitle) > 0 ) {
                    push @{$prog{'sub-title'}}, [$subtitle];
                }
            
                if (exists $genre{$internalgenreid} ) {
                    push @{$prog{'category'}}, [$genre{$internalgenreid}];
                }
            
                if (length($desc) > 0 ) {
                    push @{$prog{'desc'}}, [$desc];
                }

                # star-rating: the data source seems to say <d30>0</d30> 
                # if they mean "unknown"
                # valid values seem to be 1 to 5
                # FIXME: when I did a quick grep, '2' didn't show up
                # is this intentional or just a coincidence?

                if ( $rating gt 0 ) {
                    $prog{'star-rating'} = ["$rating/5"];
                }

                if ($wide_aspect == 1 ) {
                    $prog{'video'}->{'aspect'} = '16:9';
                }

                if ($bw_colour == 1 ) {
                    $prog{'video'}->{'colour'} = '0';

                }

                # check for dolby first
                # not sure if dolby_audio and stereo_audio can be true 
                # simultaneously in the source data, but it's better to be 
                # on the safe side.
                # If stereo_audio is false, is it safe to assume the programme
                # will be broadcast in mono?
                # I mean, this is the 21th century, right?
                # Also, what does dolby mean in this context? 
                # How does it apply to analog broadcasts?
                if ($dolby_audio == 1) {
                     $prog{'audio'}->{'stereo'} = 'dolby';
                }
                elsif ($stereo_audio == 1) {
                    $prog{'audio'}->{'stereo'} = 'stereo';
                }

                $writer->write_programme(\%main::prog);
            
        }
      $twig->purge;
    }

    # we need to extract some information from the xml filename supplied #
    by epgdata.com # the last letter tells us which channel_?.xml we need

    sub findchannelcode { # let's just use the first xml file name for that
    # thanks to Dagmar for the regexp my ($filename, $tmp) = @_; $filename
    =~ s/.*(.)\.xml$/$1/;; return $filename; }

    # this is called as a handler for the channels twig # which is in turn
    called by processxml() sub printchannels { my( $twig, $sendung)= @_; my
    $internalchanid = $sendung->first_child('ch4')->text; our $chanid; if
    (defined $main::chanid{$internalchanid}) { $chanid =
    $main::chanid{$internalchanid}; } else { $chanid = $internalchanid; #
    FIXME: not sure if this is correct. # Maybe we should just return if we
    don't know the channel id } my $name =
    decode_entities($sendung->first_child('ch0')->text); foreach my $channel
    (@{$conf->{channel}}) { if($channel eq $chanid) { my %ch = (id =>
    $chanid, 'display-name' => [ [ $name ] ]); $writer->write_channel(\%ch);
    } } }

    # this list all _available_ channels # used for --configure #
    independent from printchannels which will print list of configured
    channels sub list_channels { my ( $conf, $opt ) = @_;
    prepareinclude($conf, $opt); # borrowed from
    http://www.xmltwig.com/xmltwig/ex_fm1 # FIXME: must not hardcode package
    code! $channels->parsefile($tmp . 'channel_y.xml'); my $channel_list=
    $channels->root; my @channels= $channel_list->children;

        my $xmltv_channel_list = "<tv generator-info-name=\"tv_grab_eu_epgdata\">\n";

        foreach my $channel  (@channels) {
            my $internalchanid = $channel->first_child('ch4')->text;
            our $chanid;
            if (defined $main::chanid{$internalchanid}) {
                $chanid = $main::chanid{$internalchanid};
            }
            else {
                $chanid = $internalchanid;
                warn "New channel with ID $internalchanid found. Please update channel_ids file!"
            }
 
            my $name = $channel->first_child('ch0')->text;
        $xmltv_channel_list = <<END;
        $xmltv_channel_list
        <channel id="$chanid">
        <display-name>$name</display-name> 
        </channel>
    END
         }
         $xmltv_channel_list = $xmltv_channel_list . "</tv>";
         return $xmltv_channel_list
    }
