« back to all changes in this revision
Viewing changes to grab/eu_epgdata/tv_grab_eu_epgdata.in
- browse files at revision 14
- compare with another revision
- download diff
- download tarball
- view history from revision 14
- Committer: Bazaar Package Importer
- Author(s): Stefan Lesicnik
- Date: 2008-08-07 07:25:45 UTC
- mfrom: (1.2.10 upstream) (4.1.1 lenny)
- Revision ID: james.westby@ubuntu.com-20080807072545-ttu7eljrarkzon1p
Tags: 0.5.52-1ubuntu1
* Merge from debian unstable, remaining changes: (LP: #255450)
- Fixes multiple broken grabbers (LP: #193703)
- Update 06_grab_no.dpatch
* Deleted 07_grab_pt.patch from debian/patches as it is now in Debian.
- Fixes multiple broken grabbers (LP: #193703)
- Update 06_grab_no.dpatch
* Deleted 07_grab_pt.patch from debian/patches as it is now in Debian.
- files added:
- debian/README.source
- files removed:
- debian/patches/07_grab_pt.dpatch
- files modified:
- ChangeLog
added
removed
grab/eu_epgdata/tv_grab_eu_epgdata.in
1
#!/usr/bin/perl -w
2
3
=pod
4
5
=head1 NAME
6
7
@@name - Grab TV listings for parts of Europe.
8
9
=head1 SYNOPSIS
10
11
@@name --help
12
13
@@name --version
14
15
@@name --capabilities
16
17
@@name --description
18
19
20
@@name [--config-file FILE]
21
[--days N] [--offset N]
22
[--output FILE] [--quiet] [--debug]
23
24
@@name --configure [--config-file FILE]
25
26
@@name --configure-api [--stage NAME]
27
[--config-file FILE]
28
[--output FILE]
29
30
@@name --list-channels [--config-file FILE]
31
[--output FILE] [--quiet] [--debug]
32
33
=head1 DESCRIPTION
34
35
Output TV and listings in XMLTV format for many stations
36
available in Europe.
37
38
First you must run B<@@name --configure> to choose which stations
39
you want to receive.
40
41
Then running B<@@name> with no arguments will get a listings for
42
the stations you chose for five days including today.
43
44
This is a commercial grabber. The data service, www.epgdata.com,
45
plans on granting test accounts to people for the rest of 2007.
46
Mail service@epgdata.com if you want a Test PIN.
47
The future of this grabber is a little bit unclear as of now;
48
it's likely you'll have to pay a small sum to use it (beginning 2008).
49
50
=head1 OPTIONS
51
52
B<--configure> Prompt for which stations to download and write the
53
configuration file.
54
55
B<--config-file FILE> Set the name of the configuration file, the
56
default is B<~/.xmltv/@@name.conf>. This is the file written by
57
B<--configure> and read when grabbing.
58
59
B<--gui OPTION> Use this option to enable a graphical interface to be used.
60
OPTION may be 'Tk', or left blank for the best available choice.
61
Additional allowed values of OPTION are 'Term' for normal terminal output
62
(default) and 'TermNoProgressBar' to disable the use of Term::ProgressBar.
63
64
B<--output FILE> When grabbing, write output to FILE rather than
65
standard output.
66
67
B<--days N> When grabbing, grab N days rather than 5.
68
69
B<--offset N> Start grabbing at today + N days.
70
71
B<--quiet> Suppress the progress-bar normally shown on standard error. This option does not do anything right now.
72
73
B<--debug> Provide more information on progress to stderr to help in
74
debugging. This option does not do anything right now.
75
76
B<--list-channels> Output a list of all channels that data is available
77
for. The list is in xmltv-format.
78
79
B<--version> Show the version of the grabber.
80
81
B<--help> Print a help message and exit.
82
83
=head1 ERROR HANDLING
84
85
N/A
86
87
=head1 ENVIRONMENT VARIABLES
88
89
The environment variable HOME can be set to change where configuration
90
files are stored. All configuration is stored in $HOME/.xmltv/. On Windows,
91
it might be necessary to set HOME to a path without spaces in it.
92
93
=head1 SUPPORTED CHANNELS
94
95
For a list of supported channels, see the channel_ids file distributed with this grabber.
96
If additional channels are available, you will receive a warning when you run --configure.
97
In the future, it might be possible to download an updated channel_ids file from the internet.
98
99
=head1 COMPATIBILITY
100
101
The channel ids used in this grabber aim to be mostly possible with other grabbers, eg
102
tv_grab_de_prisma and some other grabbers for other countries.
103
NOTE: Retaining compatibility was not always possible or practicable.
104
You can get a list of channel ids using --list-channels
105
106
=head1 AUTHOR
107
108
Michael Haas, laga -at- laga -dot- ath -dot - cx. This documentation is copied
109
from tv_grab_se_swedb by Mattias Holmlund, which in turn was copied from tv_grab_uk by Ed Avis.
110
Parts of the code are copied from tv_grab_se_swedb and tv_grab_na_dd (in XMLTV 0.5.45) as well
111
as various other sources.
112
113
=head1 BUGS
114
115
There's no proper support for channels with locally different schedules. For example,
116
if your EPG package is a German one, you'll get the EPG schedule for Germany
117
even if you preferred the Swiss schedule which is also available in the data (for some channels at least).
118
119
The EPG package code is not properly detected during the configure stage and 'y' is assumed.
120
The package code determines which channel_?.xml is loaded.
121
122
Timezones are not handled correctly. Currently, you have to enter your
123
time zone manually during the configure step. You have to do this every
124
time your time zone changes, eg for daylight saving time
125
("Sommerzeit" and "Normalzeit" for my fellow Germans).
126
I'll try to have this fixed for the next XMLTV release.
127
Please see this thread for further discussion and some additional issues:
128
http://thread.gmane.org/gmane.comp.tv.xmltv.devel/7919
129
FYI: you can modify the time zone directly in the config file which is
130
usually located at ~/.xmltv/tv_grab_eu_epgdata.conf or
131
~/.mythtv/FOO.xmltv where FOO is the name of your video source in MythTV.
132
133
If the data source gives us data for one day, they'll also cover a part of the following day.
134
Maybe this should be fixed. Please note: data is not overlapping! So if we want to get data for
135
today, we might as well grab yesterday because that'll give us EPG till ~5am for today.
136
137
I'm sure this list is not complete. Let me know if you encounter additional problems.
138
=cut
139
140
141
use strict;
142
use LWP::Simple qw($ua getstore);
143
use Archive::Zip;
144
use File::Temp qw/ tempdir /;
145
use XML::Twig;
146
147
use XMLTV;
148
use XMLTV::Options qw/ParseOptions/;
149
use XMLTV::Configure::Writer;
150
use XMLTV::Supplement qw/GetSupplement/;
151
152
# deal with umlauts
153
use HTML::Entities;
154
155
use XMLTV::Memoize; XMLTV::Memoize::check_argv('getstore');
156
157
# set user agent
158
$ua->agent("xmltv/$XMLTV::VERSION");
159
160
our $tmp= tempdir( CLEANUP => 1 ) . '/';
161
#our $tmp = "/tmp/foobarbaz/";
162
163
# set up XML::Twig
164
our $epg= new XML::Twig( twig_handlers => { data => \&printepg } );
165
our $channels = new XML::Twig( twig_handlers => { data => \&printchannels } );
166
our %genre;
167
our $genre = new XML::Twig( twig_handlers => { data => \&makegenrehash } );
168
169
# build a hash: epgdata.com channel id -> xmltv channel id
170
my $chanids = GetSupplement( '@@name', 'channel_ids' );
171
172
our %chanid;
173
my @lines = split( /[\n\r]+/, $chanids );
174
foreach my $line (@lines) {
175
if ($line !~ '^#') {
176
my @chanid_array = split(';', $line);
177
chomp($chanid_array[1]);
178
$chanid{$chanid_array[0]}= $chanid_array[1] unless $line =~ '^#';
179
}
180
}
181
182
my( $opt, $conf ) = ParseOptions( {
183
grabber_name => "@@name",
184
capabilities => [qw/baseline manualconfig tkconfig apiconfig cache/],
185
stage_sub => \&config_stage,
186
listchannels_sub => \&list_channels,
187
version => '$Id: tv_grab_eu_epgdata.in,v 1.8 2008/02/02 20:15:20 mihaas Exp $',
188
description => "@@desc",
189
} );
190
191
our $pin = $conf->{pin}->[0];
192
die 'Sorry, your PIN is not defined. Run @@name --configure to fix this.' unless defined($pin);
193
194
our $tz = $conf->{tz}->[0];
195
# die 'Sorry, time zone is not defined. Run @@name --configure to fix this.' unless defined($tz);
196
# Oops. Looks like the line below will result in a warning
197
# telling us that we're declaring $tz twice. However, this does not seem to
198
# be an issue.
199
$tz = '+0100' unless defined($tz);
200
201
202
sub config_stage {
203
# shamelessly stolen from http://xmltv.org/wiki/howtowriteagrabber.html
204
205
my( $stage, $conf ) = @_;
206
# Sample stage_sub that only needs a single stage.
207
die "Unknown stage $stage" if $stage ne "start";
208
209
my $result;
210
my $configwriter = new XMLTV::Configure::Writer( OUTPUT => \$result,
211
encoding => 'ISO-8859-1' );
212
$configwriter->start( { grabber => '@@name' } );
213
$configwriter->write_string( {
214
id => 'pin',
215
title => [ [ 'Enter your PIN for epgdata.com', 'en' ] ],
216
description => [
217
[ 'This alphanumeric string is used for authentication with epgdata.com.
218
Ask service@epgdata.com for a test PIN (before 2007 ends)',
219
'en' ] ],
220
default => '',
221
} );
222
$configwriter->write_string( {
223
id => 'tz',
224
title => [ [ 'Time zone for your EPG data', 'en' ] ],
225
description => [
226
[ 'Enter the time offset from UTC here. Think of it as your time zone.
227
For example: during winter in Germany, you should enter "+0100". During summer, use "+0200". (without quotation marks) ',
228
'en' ] ],
229
default => '+0100',
230
} );
231
232
$configwriter->end( 'select-channels' );
233
return $result;
234
}
235
236
237
238
# construct writer object
239
# taken from tv_grab_na_dd (XMLTV 0.4.45)
240
# XMLTV::Options does not redirect stdout properly for us
241
# XML::Twig probably messes it up, I don't know. :/
242
my %w_args;
243
if (defined $opt->{output}) {
244
my $fh = new IO::File(">$opt->{output}");
245
die "ERROR: cannot write to $opt->{output}: $!" if not defined $fh;
246
$w_args{OUTPUT} = $fh;
247
}
248
$w_args{encoding} = 'ISO-8859-1';
249
250
our $writer = new XMLTV::Writer(%w_args);
251
252
253
downloadepg();
254
prepareinclude($conf);
255
# it looks like we can also extract the language from the file
256
# name of the epg data
257
our @xmlfiles = glob($tmp . "*_*_??_q?.xml");
258
processxml();
259
260
261
sub downloadepg {
262
my $days = $opt->{days};
263
my $offset = $opt->{offset};
264
my $i='0';
265
# we've got to start counting at 0
266
# if we did "$i <= $days", we'd end up with one zip file too much
267
while ( $i < $days) {
268
my $dataoffset = $i +$offset;
269
my $baseurl="http://www.epgdata.com";
270
my $url=$baseurl . '/index.php?action=sendPackage&iOEM=&pin=' . $pin . '&dayOffset=' . $dataoffset . '&dataType=xml';
271
getstore($url, $tmp . "epgzip" . $dataoffset);
272
# This doesn't seem to work correctly.
273
# It doesn't fail even if the PIN is wrong.
274
#unless (getstore($url, $tmp . "epgzip" . $dataoffset) == 200) {
275
#die "Couldn't download epg file\n";
276
#}
277
$i++;
278
}
279
# FIXME: we can easily create a list of files earlier in this function
280
my @zipfiles=(glob($tmp . 'epgzip*'));
281
unzip(@zipfiles);
282
}
283
284
# for simplicity's sake, always call with $conf as argument at least
285
sub prepareinclude {
286
my ( $conf, $opt ) = @_;
287
my $baseurl="http://www.epgdata.com";
288
my $pin = $conf->{pin}->[0];
289
my $includeurl=$baseurl . "/index.php?action=sendInclude&iOEM=&pin=" . $pin
290
. "&dataType=xml";
291
getstore($includeurl, $tmp . "includezip");
292
# This doesn't seem to work correctly.
293
# It doesn't fail even if the PIN is wrong.
294
# unless (getstore($includeurl, $tmp . "includezip") == 200) {
295
# die "Couldn't download include file\n";
296
# }
297
my @zipfiles=( $tmp . "includezip");
298
unzip(@zipfiles)
299
}
300
301
302
sub unzip {
303
foreach my $zipfile (@_) {
304
my $zip = Archive::Zip->new( $zipfile );
305
my @filelist = $zip->memberNames;
306
foreach my $ext (("\.dtd", "\.xml")) {
307
foreach my $filename (@filelist) {
308
# we only care about .dtd and .xml right now
309
my $xmlfile=$filename if $filename =~ /$ext/;
310
$zip->extractMember( $xmlfile, $tmp . $xmlfile ) if defined $xmlfile;
311
}
312
}
313
}
314
}
315
316
317
sub processxml {
318
$writer->start({ 'generator-info-name' => '@@name' });
319
$genre->parsefile($tmp . 'genre.xml');
320
$channels->parsefile($tmp . 'channel_' . findchannelcode($xmlfiles[0], $tmp) . '.xml');
321
foreach my $xmlfile (@xmlfiles) {
322
$epg->parsefile($xmlfile);
323
}
324
$writer->end();
325
}
326
327
328
sub makegenrehash {
329
my( $twig, $genre)= @_;
330
my $genreid = $genre->first_child('g0')->text;
331
my $genrename = decode_entities($genre->first_child('g1')->text);
332
$genre{$genreid}= $genrename;
333
$twig->purge;
334
}
335
336
337
338
sub printepg {
339
my( $twig, $sendung)= @_;
340
my $internalchanid = $sendung->first_child('d2')->text;
341
my $internalregionid = $sendung->first_child('d3')->text;
342
our $chanid;
343
if (defined $main::chanid{$internalchanid}) {
344
$chanid = $main::chanid{$internalchanid};
345
}
346
else {
347
$chanid = $internalchanid;
348
# FIXME: not sure if this is correct.
349
# Maybe we should behave differently if we encounter an unknown ID,
350
# but this ought to be OK for now
351
}
352
# alright, let's try this:
353
# push the channel ids we want to grab in an array
354
# http://effectiveperl.blogspot.com/
355
my %configuredchannels = map { $_, 1 } @{$conf->{channel}};
356
# does the channel we're currently processing exist in the hash?
357
# BTW: this is not a lot more efficient in our case than looping over a list
358
# but a few seconds are better than nothing :)
359
if($configuredchannels{$chanid} && $internalregionid == '0') {
360
my $title = decode_entities($sendung->first_child('d19')->text);
361
my $subtitle = decode_entities($sendung->first_child('d20')->text);
362
my $desc = decode_entities($sendung->first_child('d23')->text);
363
my $start = $sendung->first_child('d4')->text;
364
my $internalgenreid = $sendung->first_child('d25')->text;
365
my $rating = $sendung->first_child('d30')->text;
366
my $wide_aspect = $sendung->first_child('d29')->text;
367
368
my $series_id = $sendung->first_child('d1')->text;
369
370
# people
371
my $presenter = decode_entities($sendung->first_child('d34')->text);
372
my $studio_guest = decode_entities($sendung->first_child('d35')->text);
373
my $director = decode_entities($sendung->first_child('d36')->text);
374
my $actor = decode_entities($sendung->first_child('d37')->text);
375
376
377
# black and white?
378
my $bw_colour = $sendung->first_child('d11')->text;
379
my $stereo_audio = $sendung->first_child('d27')->text;
380
my $dolby_audio = $sendung->first_child('d28')->text;
381
# I was told that technics_hd is supposed to exist
382
# However, it's not listed in qy.dtd
383
# my $hd_video = $sendung->first_child('XXX')->text;
384
385
$start =~ s/-//g;
386
$start =~ s/://g;
387
$start =~ s/ //g;
388
our %prog = ("channel" => $chanid, "start" => "$start $tz",
389
"title" => [ [ $title ] ]);
390
391
if ( length($subtitle) > 0 ) {
392
push @{$prog{'sub-title'}}, [$subtitle];
393
}
394
395
396
if (exists $genre{$internalgenreid} ) {
397
push @{$prog{'category'}}, [$genre{$internalgenreid}];
398
}
399
400
if (length($desc) > 0 ) {
401
push @{$prog{'desc'}}, [$desc];
402
}
403
404
# people
405
if ( length($actor)>0 ) {
406
push @{$prog{'credits'}{'actor'}}, $actor;
407
}
408
409
if ( length($director)>0 ) {
410
push @{$prog{'credits'}{'director'}}, $director;
411
}
412
413
if ( length($studio_guest)>0) {
414
push @{$prog{'credits'}{'guest'}}, $studio_guest;
415
}
416
417
if ( length($presenter)>0) {
418
push @{$prog{'credits'}{'presenter'}}, $presenter;
419
}
420
421
422
# star-rating: the data source seems to say <d30>0</d30>
423
# if they mean "unknown"
424
# valid values seem to be 1 to 5
425
# FIXME: when I did a quick grep, '2' didn't show up
426
# is this intentional or just a coincidence?
427
428
if ( $rating gt 0 ) {
429
$prog{'star-rating'} = ["$rating/5"];
430
}
431
432
if ($wide_aspect == 1 ) {
433
$prog{'video'}->{'aspect'} = '16:9';
434
}
435
436
if ($bw_colour == 1 ) {
437
$prog{'video'}->{'colour'} = '0';
438
439
}
440
441
$prog{'episode-num'} = [ [$series_id, 'http://fix.me/' ] ];
442
443
# check for dolby first
444
# not sure if dolby_audio and stereo_audio can be true
445
# simultaneously in the source data, but it's better to be
446
# on the safe side.
447
# If stereo_audio is false, is it safe to assume the programme
448
# will be broadcast in mono?
449
# I mean, this is the 21th century, right?
450
# Also, what does dolby mean in this context?
451
# How does it apply to analog broadcasts?
452
if ($dolby_audio == 1) {
453
$prog{'audio'}->{'stereo'} = 'dolby';
454
}
455
elsif ($stereo_audio == 1) {
456
$prog{'audio'}->{'stereo'} = 'stereo';
457
}
458
459
$writer->write_programme(\%main::prog);
460
461
}
462
$twig->purge;
463
}
464
465
466
467
468
469
# we need to extract some information from the xml filename supplied
470
# by epgdata.com
471
# the last letter tells us which channel_?.xml we need
472
473
sub findchannelcode {
474
# let's just use the first xml file name for that
475
# thanks to Dagmar for the regexp
476
my ($filename, $tmp) = @_;
477
$filename =~ s/.*(.)\.xml$/$1/;;
478
return $filename;
479
}
480
481
# this is called as a handler for the channels twig
482
# which is in turn called by processxml()
483
sub printchannels {
484
my( $twig, $sendung)= @_;
485
my $internalchanid = $sendung->first_child('ch4')->text;
486
our $chanid;
487
if (defined $main::chanid{$internalchanid}) {
488
$chanid = $main::chanid{$internalchanid};
489
}
490
else {
491
$chanid = $internalchanid;
492
# FIXME: not sure if this is correct.
493
# Maybe we should just return if we don't know the channel id
494
}
495
my $name = decode_entities($sendung->first_child('ch0')->text);
496
foreach my $channel (@{$conf->{channel}}) {
497
if($channel eq $chanid) {
498
my %ch = (id => $chanid, 'display-name' => [ [ $name ] ]);
499
$writer->write_channel(\%ch);
500
}
501
}
502
}
503
504
# this lists all _available_ channels
505
# used for --configure
506
# independent from printchannels which will print list of configured channels
507
sub list_channels {
508
my ( $conf, $opt ) = @_;
509
prepareinclude($conf, $opt);
510
# borrowed from http://www.xmltwig.com/xmltwig/ex_fm1
511
# FIXME: must not hardcode package code!
512
$channels->parsefile($tmp . 'channel_y.xml');
513
my $channel_list= $channels->root;
514
my @channels= $channel_list->children;
515
516
my $xmltv_channel_list = "<tv generator-info-name=\"@@name\">\n";
517
518
foreach my $channel (@channels) {
519
my $internalchanid = $channel->first_child('ch4')->text;
520
our $chanid;
521
if (defined $main::chanid{$internalchanid}) {
522
$chanid = $main::chanid{$internalchanid};
523
}
524
else {
525
$chanid = $internalchanid;
526
warn "New channel with ID $internalchanid found. Please update chann
527
el_ids file!"
528
}
529
530
my $name = $channel->first_child('ch0')->text;
531
$xmltv_channel_list = <<END;
532
$xmltv_channel_list
533
<channel id="$chanid">
534
<display-name>$name</display-name>
535
</channel>
536
END
537
}
538
$xmltv_channel_list = $xmltv_channel_list . "</tv>";
539
return $xmltv_channel_list
540
}
541
Loggerhead is a web-based interface for Breezy
Version: 2.0.1