« back to all changes in this revision
Viewing changes to grab/eu_epgdata/tv_grab_eu_epgdata
- browse files at revision 14
- compare with another revision
- download diff
- download tarball
- view history from revision 14
- Committer: Bazaar Package Importer
- Author(s): Stefan Lesicnik
- Date: 2008-08-07 07:25:45 UTC
- mfrom: (1.2.10 upstream) (4.1.1 lenny)
- Revision ID: james.westby@ubuntu.com-20080807072545-ttu7eljrarkzon1p
Tags: 0.5.52-1ubuntu1
* Merge from debian unstable, remaining changes: (LP: #255450)
- Fixes multiple broken grabbers (LP: #193703)
- Update 06_grab_no.dpatch
* Deleted 07_grab_pt.patch from debian/patches as it is now in Debian.
- Fixes multiple broken grabbers (LP: #193703)
- Update 06_grab_no.dpatch
* Deleted 07_grab_pt.patch from debian/patches as it is now in Debian.
- files added:
- debian/README.source
- files removed:
- debian/patches/07_grab_pt.dpatch
- files modified:
- ChangeLog
added
removed
grab/eu_epgdata/tv_grab_eu_epgdata
1
#!/usr/bin/perl -w
2
3
=pod
4
5
=head1 NAME
6
7
tv_grab_eu_epgdata - Grab TV listings for parts of Europe.
8
9
=head1 SYNOPSIS
10
11
tv_grab_eu_epgdata --help
12
tv_grab_eu_epgdata
13
tv_grab_eu_epgdata --version
14
15
tv_grab_eu_epgdata --capabilities
16
17
tv_grab_eu_epgdata --description
18
19
20
tv_grab_eu_epgdata [--config-file FILE]
21
[--days N] [--offset N]
22
[--output FILE] [--quiet] [--debug]
23
24
tv_grab_eu_epgdata --configure [--config-file FILE]
25
26
tv_grab_eu_epgdata --configure-api [--stage NAME]
27
[--config-file FILE]
28
[--output FILE]
29
30
tv_grab_eu_epgdata --list-channels [--config-file FILE]
31
[--output FILE] [--quiet] [--debug]
32
33
tv_grab_eu_epgdata --preferredmethod
34
35
=head1 DESCRIPTION
36
37
Output TV and listings in XMLTV format for many stations
38
available in Europe.
39
40
First you must run B<tv_grab_eu_epgdata --configure> to choose which stations
41
you want to receive.
42
43
Then running B<tv_grab_eu_epgdata> with no arguments will get a listings for
44
the stations you chose for five days including today.
45
46
This is a commercial grabber.
47
Go to http://wiki.xmltv.org/index.php/EU_epgdata to sign up or
48
send an e-mail to service@epgdata.com for further information.
49
It's also possible to ask for a test account.
50
51
=head1 OPTIONS
52
53
B<--configure> Prompt for which stations to download and write the
54
configuration file.
55
56
B<--config-file FILE> Set the name of the configuration file, the
57
default is B<~/.xmltv/tv_grab_eu_epgdata.conf>. This is the file written by
58
B<--configure> and read when grabbing.
59
60
B<--gui OPTION> Use this option to enable a graphical interface to be used.
61
OPTION may be 'Tk', or left blank for the best available choice.
62
Additional allowed values of OPTION are 'Term' for normal terminal output
63
(default) and 'TermNoProgressBar' to disable the use of Term::ProgressBar.
64
65
B<--output FILE> When grabbing, write output to FILE rather than
66
standard output.
67
68
B<--days N> When grabbing, grab N days rather than 5.
69
70
B<--offset N> Start grabbing at today + N days.
71
72
B<--quiet> Do not show status messages.
73
74
B<--debug> Provide more information on progress to stderr to help in
75
debugging.
76
77
B<--list-channels> Output a list of all channels that data is available
78
for. The list is in xmltv-format.
79
80
B<--version> Show the version of the grabber.
81
82
B<--help> Print a help message and exit.
83
84
B<--preferredmethod> Shows the preferred method for downloading data
85
(see http://xmltv.org/wiki/xmltvcapabilities.html)
86
=head1 ENVIRONMENT VARIABLES
87
88
The environment variable HOME can be set to change where configuration
89
files are stored. All configuration is stored in $HOME/.xmltv/. On Windows,
90
it might be necessary to set HOME to a path without spaces in it.
91
92
=head1 SUPPORTED CHANNELS
93
94
For a list of supported channels, see the channel_ids file distributed with this grabber.
95
If additional channels are available, you will receive a warning when you run --configure.
96
97
Once I am aware that new channels are available, the channel_ids file will be updated and
98
this grabber will automatically fetch an updated copy.
99
100
=head1 COMPATIBILITY
101
102
The channel ids used in this grabber aim to be mostly possible with other grabbers, eg
103
tv_grab_de_prisma and some other grabbers for other countries.
104
NOTE: Retaining compatibility was not always possible or practicable.
105
You can get a list of channel ids using --list-channels
106
107
=head1 AUTHOR
108
109
Michael Haas, laga -at- laga -dot- ath -dot - cx. This documentation is copied
110
from tv_grab_se_swedb by Mattias Holmlund, which in turn was copied from tv_grab_uk by Ed Avis.
111
Parts of the code are copied from tv_grab_se_swedb and tv_grab_na_dd (in XMLTV 0.5.45) as well
112
as various other sources.
113
114
=head1 BUGS
115
116
There's no proper support for channels with locally different schedules. For example,
117
if your EPG package is a German one, you'll get the EPG schedule for Germany
118
even if you preferred the Swiss schedule which is also available in the data (for some channels at least).
119
120
Timezones are not handled correctly. Currently, you have to enter your
121
time zone manually during the configure step. You have to do this every
122
time your time zone changes, eg for daylight saving time
123
("Sommerzeit" and "Normalzeit" for my fellow Germans).
124
I'll try to have this fixed for the next XMLTV release.
125
Please see this thread for further discussion and some additional issues:
126
http://thread.gmane.org/gmane.comp.tv.xmltv.devel/7919
127
FYI: you can modify the time zone directly in the config file which is
128
usually located at ~/.xmltv/tv_grab_eu_epgdata.conf or
129
~/.mythtv/FOO.xmltv where FOO is the name of your video source in MythTV.
130
131
If the data source gives us data for one day, they'll also cover a part of the following day.
132
Maybe this should be fixed. Please note: data is not overlapping! So if we want to get data for
133
today, we might as well grab yesterday because that'll give us EPG till ~5am for today.
134
135
I'm sure this list is not complete. Let me know if you encounter additional problems.
136
137
=cut
138
139
140
use strict;
141
use warnings;
142
use LWP::Simple qw($ua getstore);
143
use Archive::Zip;
144
use File::Temp qw/ tempdir /;
145
use XML::Twig;
146
147
use XMLTV;
148
use XMLTV::Options qw/ParseOptions/;
149
use XMLTV::Configure::Writer;
150
use XMLTV::Supplement qw/GetSupplement/;
151
use HTTP::Request::Common;
152
153
# deal with umlauts
154
use HTML::Entities;
155
156
# to parse expiry dates
157
use Date::Format;
158
159
use XMLTV::Memoize; XMLTV::Memoize::check_argv('getstore');
160
161
# set user agent
162
$ua->agent("xmltv/$XMLTV::VERSION");
163
164
our $tmp= tempdir( CLEANUP => 1 ) . '/';
165
#our $tmp= tempdir() . '/';
166
# set up XML::Twig
167
our $epg= new XML::Twig( twig_handlers => { data => \&printepg } );
168
our $channels = new XML::Twig( twig_handlers => { data => \&printchannels } );
169
our %genre;
170
our $genre = new XML::Twig( twig_handlers => { data => \&makegenrehash } );
171
172
# build a hash: epgdata.com channel id -> xmltv channel id
173
my $chanids = GetSupplement( 'tv_grab_eu_epgdata', 'channel_ids' );
174
175
our $channelgroup;
176
our $expiry_date;
177
178
our %chanid;
179
my @lines = split( /[\n\r]+/, $chanids );
180
foreach my $line (@lines) {
181
if ($line !~ '^#') {
182
my @chanid_array = split(';', $line);
183
chomp($chanid_array[1]);
184
$chanid{$chanid_array[0]}= $chanid_array[1] unless $line =~ '^#';
185
}
186
}
187
188
my( $opt, $conf ) = ParseOptions( {
189
grabber_name => "tv_grab_eu_epgdata",
190
capabilities => [qw/baseline manualconfig tkconfig apiconfig cache preferredmethod/],
191
stage_sub => \&config_stage,
192
listchannels_sub => \&list_channels,
193
version => '$Id: tv_grab_eu_epgdata,v 1.24 2008/06/06 10:38:42 mihaas Exp $',
194
description => "Parts of Europe (commercial) (www.epgdata.com)",
195
preferredmethod => "allatonce",
196
} );
197
198
my $pin = $conf->{pin}->[0];
199
die 'Sorry, your PIN is not defined. Run tv_grab_eu_epgdata --configure to fix this.\n' unless $pin;
200
201
our $tz = $conf->{tz}->[0];
202
if (not $tz) {
203
$tz = '+0100';
204
warn 'Time zone is not set. Run tv_grab_eu_epgdata --configure to fix this.\n' unless $opt->{quiet};
205
}
206
207
208
sub config_stage {
209
# shamelessly stolen from http://xmltv.org/wiki/howtowriteagrabber.html
210
211
my( $stage, $conf ) = @_;
212
# Sample stage_sub that only needs a single stage.
213
die "Unknown stage $stage" if $stage ne "start";
214
215
my $result;
216
my $configwriter = new XMLTV::Configure::Writer( OUTPUT => \$result,
217
encoding => 'ISO-8859-1' );
218
$configwriter->start( { grabber => 'tv_grab_eu_epgdata' } );
219
$configwriter->write_string( {
220
id => 'pin',
221
title => [ [ 'Enter your PIN for epgdata.com', 'en' ] ],
222
description => [
223
[ 'This alphanumeric string is used for authentication with epgdata.com.
224
Go to http://wiki.xmltv.org/index.php/EU_epgdata to sign up or
225
send an e-mail to service@epgdata.com for further information',
226
'en' ] ],
227
default => '',
228
} );
229
$configwriter->write_string( {
230
id => 'tz',
231
title => [ [ 'Time zone for your EPG data', 'en' ] ],
232
description => [
233
[ 'Enter the time offset from UTC here. Think of it as your time zone.
234
For example: during winter in Germany, you should enter "+0100". During summer, use "+0200". (without quotation marks) ',
235
'en' ] ],
236
default => '+0100',
237
} );
238
239
$configwriter->end( 'select-channels' );
240
return $result;
241
}
242
243
244
245
# construct writer object
246
# taken from tv_grab_na_dd (XMLTV 0.4.45)
247
# XMLTV::Options does not redirect stdout properly for us
248
# XML::Twig probably messes it up, I don't know. :/
249
my %w_args;
250
if (defined $opt->{output}) {
251
my $fh = new IO::File(">$opt->{output}");
252
die "ERROR: cannot write to $opt->{output}: $!" if not defined $fh;
253
$w_args{OUTPUT} = $fh;
254
}
255
$w_args{encoding} = 'ISO-8859-1';
256
257
our $writer = new XMLTV::Writer(%w_args);
258
259
260
our @xmlfiles = downloadepg($opt->{days},$opt->{offset},$pin);
261
prepareinclude($conf,$opt);
262
# it looks like we can also extract the language from the file
263
# name of the epg data
264
processxml(@xmlfiles);
265
266
267
sub downloadepg {
268
my $days = shift;
269
my $offset = shift;
270
my $pin = shift;
271
my $i='0';
272
my @filenames;
273
# we've got to start counting at 0
274
# if we did "$i <= $days", we'd end up with one zip file too much
275
while ( $i < $days) {
276
my $dataoffset = $i +$offset;
277
my $baseurl="http://www.epgdata.com";
278
my $url=$baseurl . '/index.php?action=sendPackage&iOEM=&pin=' . $pin . '&dayOffset=' . $dataoffset . '&dataType=xml';
279
# get file name from content-disposition header
280
my $response = $ua->request(GET $url);
281
my $filename = $response->{"_headers"}{"content-disposition"};
282
$expiry_date = $response->{"_headers"}{"x-epgdata-timeout"};
283
$channelgroup = $response->{"_headers"}{"x-epgdata-channelgroup"};
284
$filename =~ s/^.*=//;
285
if ($response->{"_headers"}{"x-epgdata-packageavailable"} == 1) {
286
warn "Downloading zip file for day ", $dataoffset + 1, "\n" unless $opt->{quiet};
287
open(F,">$tmp" . $filename);
288
print F $response->content;
289
close(F);
290
push @filenames, $tmp . $filename;
291
}
292
else {
293
warn "No more zip files available for download\n" unless $opt->{quiet};
294
return unzip(@filenames);
295
}
296
$i++;
297
}
298
return unzip(@filenames);
299
}
300
301
# for simplicity's sake, always call with $conf as argument at least
302
sub prepareinclude {
303
my ( $conf, $opt ) = @_;
304
my $baseurl="http://www.epgdata.com";
305
my $pin = $conf->{pin}->[0];
306
my $includeurl=$baseurl . "/index.php?action=sendInclude&iOEM=&pin=" . $pin
307
. "&dataType=xml";
308
warn "Downloading include zip file\n" unless $opt->{quiet};
309
getstore($includeurl, $tmp . "includezip");
310
# make sure we know $channelgroup
311
downloadepg('1','0',$pin);
312
warn "Your PIN will expire around ", time2str("%C", $expiry_date),"\n" unless $opt->{quiet};
313
my @zipfiles=( $tmp . "includezip");
314
unzip(@zipfiles)
315
}
316
317
# returns list of *.xml files
318
sub unzip {
319
my @xmlfilelist;
320
foreach my $zipfile (@_) {
321
warn "Extracting *.dtd and *.xml from ", $zipfile, "\n" if $opt->{debug};
322
my $zip = Archive::Zip->new( $zipfile );
323
my @filelist = $zip->memberNames;
324
foreach my $filename (@filelist) {
325
# we only care about .dtd and .xml right now
326
my $isdtd = 1 if $filename =~ /\.dtd/;
327
my $isxml = 1 if $filename =~ /\.xml/;
328
$zip->extractMember( $filename, $tmp . $filename ) if ($isdtd or $isxml);
329
push @xmlfilelist, ( $tmp . $filename) if $isxml;
330
}
331
}
332
return @xmlfilelist;
333
}
334
335
336
sub processxml {
337
$writer->start({ 'generator-info-name' => 'tv_grab_eu_epgdata' });
338
$genre->parsefile($tmp . 'genre.xml');
339
$channels->parsefile($tmp . 'channel_' . $channelgroup . '.xml');
340
foreach my $xmlfile (@_) {
341
warn "Processing ". $xmlfile, "\n" if $opt->{debug};
342
$epg->parsefile($xmlfile);
343
}
344
$writer->end();
345
}
346
347
348
sub makegenrehash {
349
my( $twig, $genre)= @_;
350
my $genreid = $genre->first_child('g0')->text;
351
my $genrename = decode_entities($genre->first_child('g1')->text);
352
$genre{$genreid}= $genrename;
353
$twig->purge;
354
}
355
356
357
358
sub printepg {
359
my( $twig, $sendung)= @_;
360
my $internalchanid = $sendung->first_child('d2')->text;
361
my $internalregionid = $sendung->first_child('d3')->text;
362
our $chanid;
363
if (defined $main::chanid{$internalchanid}) {
364
$chanid = $main::chanid{$internalchanid};
365
}
366
else {
367
$chanid = $internalchanid;
368
# FIXME: not sure if this is correct.
369
# Maybe we should behave differently if we encounter an unknown ID,
370
# but this ought to be OK for now
371
}
372
# alright, let's try this:
373
# push the channel ids we want to grab in an array
374
# http://effectiveperl.blogspot.com/
375
my %configuredchannels = map { $_, 1 } @{$conf->{channel}};
376
# does the channel we're currently processing exist in the hash?
377
# BTW: this is not a lot more efficient in our case than looping over a list
378
# but a few seconds are better than nothing :)
379
if($configuredchannels{$chanid} && $internalregionid == '0') {
380
my $title = decode_entities($sendung->first_child('d19')->text);
381
my $subtitle = decode_entities($sendung->first_child('d20')->text);
382
my $desc = decode_entities($sendung->first_child('d23')->text);
383
my $start = $sendung->first_child('d4')->text;
384
my $internalgenreid = $sendung->first_child('d25')->text;
385
my $rating = $sendung->first_child('d30')->text;
386
my $wide_aspect = $sendung->first_child('d29')->text;
387
388
my $series_id = $sendung->first_child('d1')->text;
389
390
# people
391
my $presenter = decode_entities($sendung->first_child('d34')->text);
392
my $studio_guest = decode_entities($sendung->first_child('d35')->text);
393
my $director = decode_entities($sendung->first_child('d36')->text);
394
my $actor = decode_entities($sendung->first_child('d37')->text);
395
396
397
# black and white?
398
my $bw_colour = $sendung->first_child('d11')->text;
399
my $stereo_audio = $sendung->first_child('d27')->text;
400
my $dolby_audio = $sendung->first_child('d28')->text;
401
# I was told that technics_hd is supposed to exist
402
# However, it's not listed in qy.dtd
403
# my $hd_video = $sendung->first_child('XXX')->text;
404
405
$start =~ s/-//g;
406
$start =~ s/://g;
407
$start =~ s/ //g;
408
our %prog = ("channel" => $chanid, "start" => "$start $tz",
409
"title" => [ [ $title ] ]);
410
411
if ( length($subtitle) > 0 ) {
412
push @{$prog{'sub-title'}}, [$subtitle];
413
}
414
415
416
if (exists $genre{$internalgenreid} ) {
417
push @{$prog{'category'}}, [$genre{$internalgenreid}];
418
}
419
420
if (length($desc) > 0 ) {
421
push @{$prog{'desc'}}, [$desc];
422
}
423
424
# people
425
if ( length($actor)>0 ) {
426
push @{$prog{'credits'}{'actor'}}, $actor;
427
}
428
429
if ( length($director)>0 ) {
430
push @{$prog{'credits'}{'director'}}, $director;
431
}
432
433
if ( length($studio_guest)>0) {
434
push @{$prog{'credits'}{'guest'}}, $studio_guest;
435
}
436
437
if ( length($presenter)>0) {
438
push @{$prog{'credits'}{'presenter'}}, $presenter;
439
}
440
441
442
# star-rating: the data source seems to say <d30>0</d30>
443
# if they mean "unknown"
444
# valid values seem to be 1 to 5
445
# 2 is never used
446
if ( $rating gt 0 ) {
447
$prog{'star-rating'} = ["$rating/5"];
448
}
449
450
if ($wide_aspect == 1 ) {
451
$prog{'video'}->{'aspect'} = '16:9';
452
}
453
454
if ($bw_colour == 1 ) {
455
$prog{'video'}->{'colour'} = '0';
456
457
}
458
459
$prog{'episode-num'} = [ [$series_id, 'http://fix.me/' ] ];
460
461
# check for dolby first
462
# not sure if dolby_audio and stereo_audio can be true
463
# simultaneously in the source data, but it's better to be
464
# on the safe side.
465
# If stereo_audio is false, is it safe to assume the programme
466
# will be broadcast in mono?
467
# I mean, this is the 21th century, right?
468
# Also, what does dolby mean in this context?
469
# How does it apply to analog broadcasts?
470
if ($dolby_audio == 1) {
471
$prog{'audio'}->{'stereo'} = 'dolby';
472
}
473
elsif ($stereo_audio == 1) {
474
$prog{'audio'}->{'stereo'} = 'stereo';
475
}
476
477
$writer->write_programme(\%main::prog);
478
479
}
480
$twig->purge;
481
}
482
483
484
485
# this is called as a handler for the channels twig
486
# which is in turn called by processxml()
487
sub printchannels {
488
my( $twig, $sendung)= @_;
489
my $internalchanid = $sendung->first_child('ch4')->text;
490
our $chanid;
491
if (defined $main::chanid{$internalchanid}) {
492
$chanid = $main::chanid{$internalchanid};
493
}
494
else {
495
$chanid = $internalchanid;
496
# FIXME: not sure if this is correct.
497
# Maybe we should just return if we don't know the channel id
498
}
499
my $name = decode_entities($sendung->first_child('ch0')->text);
500
foreach my $channel (@{$conf->{channel}}) {
501
if($channel eq $chanid) {
502
my %ch = (id => $chanid, 'display-name' => [ [ $name ] ]);
503
$writer->write_channel(\%ch);
504
}
505
}
506
}
507
508
# this lists all _available_ channels
509
# used for --configure
510
# independent from printchannels which will print list of configured channels
511
sub list_channels {
512
my ( $conf, $opt ) = @_;
513
prepareinclude($conf, $opt);
514
# borrowed from http://www.xmltwig.com/xmltwig/ex_fm1
515
$channels->parsefile($tmp . 'channel_' . $channelgroup . '.xml');
516
my $channel_list= $channels->root;
517
my @channels= $channel_list->children;
518
519
my $xmltv_channel_list = "<tv generator-info-name=\"tv_grab_eu_epgdata\">\n";
520
521
foreach my $channel (@channels) {
522
my $internalchanid = $channel->first_child('ch4')->text;
523
our $chanid;
524
if (defined $main::chanid{$internalchanid}) {
525
$chanid = $main::chanid{$internalchanid};
526
}
527
else {
528
$chanid = $internalchanid;
529
warn "New channel with ID $internalchanid found. Please update chann
530
el_ids file!" unless $opt->{quiet};
531
}
532
533
my $name = decode_entities($channel->first_child('ch0')->text);
534
$xmltv_channel_list = <<END;
535
$xmltv_channel_list
536
<channel id="$chanid">
537
<display-name>$name</display-name>
538
</channel>
539
END
540
}
541
$xmltv_channel_list = $xmltv_channel_list . "</tv>";
542
return $xmltv_channel_list
543
}
544
Loggerhead is a web-based interface for Breezy
Version: 2.0.1