« back to all changes in this revision
Viewing changes to lib/LWP/UserAgent.pm
- browse files at revision 3
- compare with another revision
- download diff
- download tarball
- view history from revision 3
- Committer: Bazaar Package Importer
- Author(s): Jay Bonci
- Date: 2005-02-13 18:45:32 UTC
- mfrom: (2.1.2 hoary)
- Revision ID: james.westby@ubuntu.com-20050213184532-67qvopi4wre3010u
Tags: 5.803-4
* Make GET/POST/HEAD symlinks (Closes: #294597)
* lwp-requests now honors -b when dumping links (Closes: #294595)
- Thanks to giuseppe bonacci for the patch
* Moved symlinks to a libwww-perl.links file
* lwp-requests now honors -b when dumping links (Closes: #294595)
- Thanks to giuseppe bonacci for the patch
* Moved symlinks to a libwww-perl.links file
added
removed
lib/LWP/UserAgent.pm
1
# $Id: UserAgent.pm,v 2.1 2001/12/11 21:11:29 gisle Exp $
2
3
1
package LWP::UserAgent;
2
3
# $Id: UserAgent.pm,v 2.33 2004/09/16 09:28:22 gisle Exp $
4
4
5
use strict;
5
6
=head1 NAME
7
8
LWP::UserAgent - A WWW UserAgent class
9
10
=head1 SYNOPSIS
11
12
require LWP::UserAgent;
13
my $ua = LWP::UserAgent->new(env_proxy => 1,
14
keep_alive => 1,
15
timeout => 30,
16
);
17
18
$response = $ua->get('http://search.cpan.org/');
19
20
# or:
21
22
$request = HTTP::Request->new('GET', 'http://search.cpan.org/');
23
# and then one of these:
24
$response = $ua->request($request); # or
25
$response = $ua->request($request, '/tmp/sss'); # or
26
$response = $ua->request($request, \&callback, 4096);
27
28
sub callback { my($data, $response, $protocol) = @_; .... }
29
30
=head1 DESCRIPTION
31
32
The C<LWP::UserAgent> is a class implementing a World-Wide Web
33
user agent in Perl. It brings together the HTTP::Request,
34
HTTP::Response and the LWP::Protocol classes that form the rest of the
35
core of libwww-perl library. For simple uses this class can be used
36
directly to dispatch WWW requests, alternatively it can be subclassed
37
for application-specific behaviour.
38
39
In normal use the application creates a C<LWP::UserAgent> object, and then
40
configures it with values for timeouts, proxies, name, etc. It then
41
creates an instance of C<HTTP::Request> for the request that
42
needs to be performed. This request is then passed to one of the UserAgent's
43
request() methods, which dispatches it using the relevant protocol,
44
and returns a C<HTTP::Response> object.
45
46
There are convenience methods for sending the most common request
47
types; get(), head() and post().
48
49
The basic approach of the library is to use HTTP style communication
50
for all protocol schemes, i.e. you even receive an C<HTTP::Response>
51
object for gopher or ftp requests. In order to achieve even more
52
similarity to HTTP style communications, gopher menus and file
53
directories are converted to HTML documents.
54
55
The send_request(), simple_request() and request() methods can process
56
the content of the response in one of three ways: in core, into a
57
file, or into repeated calls to a subroutine. You choose which one by
58
the kind of value passed as the second argument.
59
60
The in core variant simply stores the content in a scalar 'content'
61
attribute of the response object and is suitable for small HTML
62
replies that might need further parsing. This variant is used if the
63
second argument is missing (or is undef).
64
65
The filename variant requires a scalar containing a filename as the
66
second argument to the request method and is suitable for large WWW
67
objects which need to be written directly to the file without
68
requiring large amounts of memory. In this case the response object
69
returned from the request method will have an empty content attribute.
70
If the request fails, then the content might not be empty, and the
71
file will be untouched.
72
73
The subroutine variant requires a reference to callback routine as the
74
second argument to the request method and it can also take an optional
75
chuck size as the third argument. This variant can be used to
76
construct "pipe-lined" processing, where processing of received
77
chuncks can begin before the complete data has arrived. The callback
78
function is called with 3 arguments: the data received this time, a
79
reference to the response object and a reference to the protocol
80
object. The response object returned from the request method will
81
have empty content. If the request fails, then the the callback
82
routine is not called, and the response->content might not be empty.
83
84
The request can be aborted by calling die() in the callback
85
routine. The die message will be available as the "X-Died" special
86
response header field.
87
88
The library also allows you to use a subroutine reference as
89
content in the request object. This subroutine should return the
90
content (possibly in pieces) when called. It should return an empty
91
string when there is no more content.
92
93
=head1 METHODS
94
95
The following methods are available:
96
97
=over 4
98
99
=cut
100
101
102
6
use vars qw(@ISA $VERSION);
103
7
104
8
require LWP::MemberMixin;
105
9
@ISA = qw(LWP::MemberMixin);
106
$VERSION = sprintf("%d.%03d", q$Revision: 2.1 $ =~ /(\d+)\.(\d+)/);
10
$VERSION = sprintf("%d.%03d", q$Revision: 2.33 $ =~ /(\d+)\.(\d+)/);
107
11
108
12
use HTTP::Request ();
109
13
use HTTP::Response ();
124
28
};
125
29
}
126
30
127
=item $ua = LWP::UserAgent->new( %options );
128
129
This class method constructs a new C<LWP::UserAgent> object and
130
returns a reference to it.
131
132
Key/value pair arguments may be provided to set up the initial state
133
of the user agent. The following options correspond to attribute
134
methods described below:
135
136
KEY DEFAULT
137
----------- --------------------
138
agent "libwww-perl/#.##"
139
from undef
140
timeout 180
141
use_eval 1
142
parse_head 1
143
max_size undef
144
cookie_jar undef
145
conn_cache undef
146
protocols_allowed undef
147
protocols_forbidden undef
148
requests_redirectable ['GET', 'HEAD']
149
150
The followings option are also accepted: If the C<env_proxy> option is
151
passed in an has a TRUE value, then proxy settings are read from
152
environment variables. If the C<keep_alive> option is passed in, then
153
a C<LWP::ConnCache> is set up (see conn_cache() method below). The
154
keep_alive value is a number and is passed on as the total_capacity
155
for the connection cache. The C<keep_alive> option also has the
156
effect of loading and enabling the new experimental HTTP/1.1 protocol
157
module.
158
159
=cut
31
160
32
161
33
sub new
162
34
{
174
46
my $parse_head = delete $cnf{parse_head};
175
47
$parse_head = 1 unless defined $parse_head;
176
48
my $max_size = delete $cnf{max_size};
49
my $max_redirect = delete $cnf{max_redirect};
50
$max_redirect = 7 unless defined $max_redirect;
177
51
my $env_proxy = delete $cnf{env_proxy};
178
52
179
53
my $cookie_jar = delete $cnf{cookie_jar};
205
79
}
206
80
207
81
my $self = bless {
208
from => $from,
209
timeout => $timeout,
210
use_eval => $use_eval,
211
parse_head => $parse_head,
212
max_size => $max_size,
213
proxy => undef,
214
no_proxy => [],
215
protocols_allowed => $protocols_allowed,
216
protocols_forbidden => $protocols_forbidden,
82
from => $from,
83
def_headers => undef,
84
timeout => $timeout,
85
use_eval => $use_eval,
86
parse_head => $parse_head,
87
max_size => $max_size,
88
max_redirect => $max_redirect,
89
proxy => {},
90
no_proxy => [],
91
protocols_allowed => $protocols_allowed,
92
protocols_forbidden => $protocols_forbidden,
217
93
requests_redirectable => $requests_redirectable,
218
94
}, $class;
219
95
253
129
}
254
130
255
131
256
=item $ua->send_request($request, $arg [, $size])
257
258
This method dispatches a single WWW request on behalf of a user, and
259
returns the response received. The request is sent off unmodified,
260
without passing it through C<prepare_request()>.
261
262
The C<$request> should be a reference to a C<HTTP::Request> object
263
with values defined for at least the method() and uri() attributes.
264
265
If C<$arg> is a scalar it is taken as a filename where the content of
266
the response is stored.
267
268
If C<$arg> is a reference to a subroutine, then this routine is called
269
as chunks of the content is received. An optional C<$size> argument
270
is taken as a hint for an appropriate chunk size.
271
272
If C<$arg> is omitted, then the content is stored in the response
273
object itself.
274
275
=cut
276
277
132
sub send_request
278
133
{
279
134
my($self, $request, $arg, $size) = @_;
281
136
282
137
my($method, $url) = ($request->method, $request->uri);
283
138
284
local($SIG{__DIE__}); # protect agains user defined die handlers
139
local($SIG{__DIE__}); # protect against user defined die handlers
285
140
286
141
# Check that we have a METHOD and a URL first
287
142
return _new_response($request, &HTTP::Status::RC_BAD_REQUEST, "Method missing")
298
153
my $proxy = $self->_need_proxy($url);
299
154
if (defined $proxy) {
300
155
$scheme = $proxy->scheme;
301
} else {
156
}
157
else {
302
158
$scheme = $url->scheme;
303
159
}
304
160
309
165
# into class LWP::Protocol::nogo.
310
166
my $x;
311
167
if($x = $self->protocols_allowed) {
312
if(grep $_ eq $scheme, @$x) {
168
if(grep lc($_) eq $scheme, @$x) {
313
169
LWP::Debug::trace("$scheme URLs are among $self\'s allowed protocols (@$x)");
314
} else {
170
}
171
else {
315
172
LWP::Debug::trace("$scheme URLs aren't among $self\'s allowed protocols (@$x)");
316
173
require LWP::Protocol::nogo;
317
174
$protocol = LWP::Protocol::nogo->new;
318
175
}
319
} elsif ($x = $self->protocols_forbidden) {
320
if(grep $_ eq $scheme, @$x) {
176
}
177
elsif ($x = $self->protocols_forbidden) {
178
if(grep lc($_) eq $scheme, @$x) {
321
179
LWP::Debug::trace("$scheme URLs are among $self\'s forbidden protocols (@$x)");
322
180
require LWP::Protocol::nogo;
323
181
$protocol = LWP::Protocol::nogo->new;
324
} else {
182
}
183
else {
325
184
LWP::Debug::trace("$scheme URLs aren't among $self\'s forbidden protocols (@$x)");
326
185
}
327
186
}
332
191
$protocol = eval { LWP::Protocol::create($scheme, $self) };
333
192
if ($@) {
334
193
$@ =~ s/ at .* line \d+.*//s; # remove file/line number
335
return _new_response($request, &HTTP::Status::RC_NOT_IMPLEMENTED, $@);
194
my $response = _new_response($request, &HTTP::Status::RC_NOT_IMPLEMENTED, $@);
195
if ($scheme eq "https") {
196
$response->message($response->message . " (Crypt::SSLeay not installed)");
197
$response->content_type("text/plain");
198
$response->content(<<EOT);
199
LWP will support https URLs if the Crypt::SSLeay module is installed.
200
More information at <http://www.linpro.no/lwp/libwww-perl/README.SSL>.
201
EOT
202
}
203
return $response;
336
204
}
337
205
}
338
206
349
217
};
350
218
if ($@) {
351
219
$@ =~ s/ at .* line \d+.*//s; # remove file/line number
352
$response =
353
HTTP::Response->new(&HTTP::Status::RC_INTERNAL_SERVER_ERROR,
354
$@);
220
$response = _new_response($request,
221
&HTTP::Status::RC_INTERNAL_SERVER_ERROR,
222
$@);
355
223
}
356
} else {
224
}
225
else {
357
226
$response = $protocol->request($request, $proxy,
358
227
$arg, $size, $timeout);
359
228
# XXX: Should we die unless $response->is_success ???
366
235
}
367
236
368
237
369
=item $ua->prepare_request($request)
370
371
This method modifies given C<HTTP::Request> object by setting up
372
various headers based on the attributes of the $ua. The headers
373
affected are; C<User-Agent>, C<From>, C<Range> and C<Cookie>.
374
375
The return value is the $request object passed in.
376
377
=cut
378
379
238
sub prepare_request
380
239
{
381
240
my($self, $request) = @_;
382
241
$self->_request_sanity_check($request);
383
242
384
243
# Extract fields that will be used below
385
my ($agent, $from, $cookie_jar, $max_size) =
386
@{$self}{qw(agent from cookie_jar max_size)};
244
my ($agent, $from, $cookie_jar, $max_size, $def_headers) =
245
@{$self}{qw(agent from cookie_jar max_size def_headers)};
387
246
388
247
# Set User-Agent and From headers if they are defined
389
248
$request->init_header('User-Agent' => $agent) if $agent;
395
254
}
396
255
$cookie_jar->add_cookie_header($request) if $cookie_jar;
397
256
257
if ($def_headers) {
258
for my $h ($def_headers->header_field_names) {
259
$request->init_header($h => [$def_headers->header($h)]);
260
}
261
}
262
398
263
return($request);
399
264
}
400
265
401
266
402
=item $ua->simple_request($request, [$arg [, $size]])
403
404
This method dispatches a single WWW request on behalf of a user, and
405
returns the response received. If differs from C<send_request()> by
406
automatically calling the C<prepare_request()> method before the
407
request is sent.
408
409
The arguments are the same as for C<send_request()>.
410
411
=cut
412
413
267
sub simple_request
414
268
{
415
269
my($self, $request, $arg, $size) = @_;
419
273
}
420
274
421
275
422
=item $ua->request($request, $arg [, $size])
423
424
Process a request, including redirects and security. This method may
425
actually send several different simple requests.
426
427
The arguments are the same as for C<send_request()> and
428
C<simple_request()>.
429
430
=cut
431
432
276
sub request
433
277
{
434
278
my($self, $request, $arg, $size, $previous) = @_;
445
289
"Unknown code $code"));
446
290
447
291
if ($code == &HTTP::Status::RC_MOVED_PERMANENTLY or
448
$code == &HTTP::Status::RC_MOVED_TEMPORARILY) {
449
450
# Make a copy of the request and initialize it with the new URI
292
$code == &HTTP::Status::RC_FOUND or
293
$code == &HTTP::Status::RC_SEE_OTHER or
294
$code == &HTTP::Status::RC_TEMPORARY_REDIRECT)
295
{
451
296
my $referral = $request->clone;
452
297
298
# These headers should never be forwarded
299
$referral->remove_header('Host', 'Cookie');
300
301
if ($referral->header('Referer') &&
302
$request->url->scheme eq 'https' &&
303
$referral->url->scheme eq 'http')
304
{
305
# RFC 2616, section 15.1.3.
306
LWP::Debug::trace("https -> http redirect, suppressing Referer");
307
$referral->remove_header('Referer');
308
}
309
310
if ($code == &HTTP::Status::RC_SEE_OTHER ||
311
$code == &HTTP::Status::RC_FOUND)
312
{
313
my $method = uc($referral->method);
314
unless ($method eq "GET" || $method eq "HEAD") {
315
$referral->method("GET");
316
$referral->content("");
317
$referral->remove_content_headers;
318
}
319
}
320
453
321
# And then we update the URL based on the Location:-header.
454
my($referral_uri) = $response->header('Location');
322
my $referral_uri = $response->header('Location');
455
323
{
456
324
# Some servers erroneously return a relative URL for redirects,
457
325
# so make it absolute if it not already is.
458
326
local $URI::ABS_ALLOW_RELATIVE_SCHEME = 1;
459
327
my $base = $response->base;
328
$referral_uri = "" unless defined $referral_uri;
460
329
$referral_uri = $HTTP::URI_CLASS->new($referral_uri, $base)
461
330
->abs($base);
462
331
}
463
464
332
$referral->url($referral_uri);
465
$referral->remove_header('Host', 'Cookie');
466
467
return $response unless $self->redirect_ok($referral);
468
469
# Check for loop in the redirects
333
334
# Check for loop in the redirects, we only count
470
335
my $count = 0;
471
336
my $r = $response;
472
337
while ($r) {
473
if (++$count > 13 ||
474
$r->request->url->as_string eq $referral_uri->as_string) {
338
if (++$count > $self->{max_redirect}) {
475
339
$response->header("Client-Warning" =>
476
"Redirect loop detected");
340
"Redirect loop detected (max_redirect = $self->{max_redirect})");
477
341
return $response;
478
342
}
479
343
$r = $r->previous;
480
344
}
481
345
346
return $response unless $self->redirect_ok($referral, $response);
482
347
return $self->request($referral, $arg, $size, $response);
483
348
484
} elsif ($code == &HTTP::Status::RC_UNAUTHORIZED ||
349
}
350
elsif ($code == &HTTP::Status::RC_UNAUTHORIZED ||
485
351
$code == &HTTP::Status::RC_PROXY_AUTHENTICATION_REQUIRED
486
352
)
487
353
{
522
388
if ($@ =~ /^Can\'t locate/) {
523
389
$response->header("Client-Warning" =>
524
390
"Unsupported authentication scheme '$scheme'");
525
} else {
391
}
392
else {
526
393
$response->header("Client-Warning" => $@);
527
394
}
528
395
next CHALLENGE;
529
396
}
530
397
}
398
unless ($class->can("authenticate")) {
399
$response->header("Client-Warning" =>
400
"Unsupported authentication scheme '$scheme'");
401
next CHALLENGE;
402
}
531
403
return $class->authenticate($self, $proxy, $challenge, $response,
532
404
$request, $arg, $size);
533
405
}
536
408
return $response;
537
409
}
538
410
539
#---------------------------------------------------------------------------
411
412
#
540
413
# Now the shortcuts...
541
542
=item $ua->get($url, Header => Value,...);
543
544
This is a shortcut for C<$ua-E<gt>request(HTTP::Request::Common::GET(
545
$url, Header =E<gt> Value,... ))>. See
546
L<HTTP::Request::Common|HTTP::Request::Common>.
547
548
=item $ua->post($url, \%formref, Header => Value,...);
549
550
This is a shortcut for C<$ua-E<gt>request( HTTP::Request::Common::POST(
551
$url, \%formref, Header =E<gt> Value,... ))>. Note that the form
552
reference is optional, and can be either a hashref (C<\%formdata> or C<{
553
'key1' => 'val2', 'key2' => 'val2', ...
554
}>) or an arrayref (C<\@formdata> or
555
C<['key1' => 'val2', 'key2' => 'val2', ...]>). See
556
L<HTTP::Request::Common|HTTP::Request::Common>.
557
558
=item $ua->head($url, Header => Value,...);
559
560
This is a shortcut for C<$ua-E<gt>request( HTTP::Request::Common::HEAD(
561
$url, Header =E<gt> Value,... ))>. See
562
L<HTTP::Request::Common|HTTP::Request::Common>.
563
564
=item $ua->put($url, Header => Value,...);
565
566
This is a shortcut for C<$ua-E<gt>request( HTTP::Request::Common::PUT(
567
$url, Header =E<gt> Value,... ))>. See
568
L<HTTP::Request::Common|HTTP::Request::Common>.
569
570
=cut
571
414
#
572
415
sub get {
573
require HTTP::Request::Common;
574
return shift->request( HTTP::Request::Common::GET( @_ ) );
416
require HTTP::Request::Common;
417
my($self, @parameters) = @_;
418
my @suff = $self->_process_colonic_headers(\@parameters,1);
419
return $self->request( HTTP::Request::Common::GET( @parameters ), @suff );
575
420
}
576
421
422
577
423
sub post {
578
require HTTP::Request::Common;
579
return shift->request( HTTP::Request::Common::POST( @_ ) );
424
require HTTP::Request::Common;
425
my($self, @parameters) = @_;
426
my @suff = $self->_process_colonic_headers(\@parameters,2);
427
return $self->request( HTTP::Request::Common::POST( @parameters ), @suff );
580
428
}
581
429
430
582
431
sub head {
583
require HTTP::Request::Common;
584
return shift->request( HTTP::Request::Common::HEAD( @_ ) );
585
}
586
587
sub put {
588
require HTTP::Request::Common;
589
return shift->request( HTTP::Request::Common::PUT( @_ ) );
590
}
591
592
593
#---------------------------------------------------------------------------
432
require HTTP::Request::Common;
433
my($self, @parameters) = @_;
434
my @suff = $self->_process_colonic_headers(\@parameters,1);
435
return $self->request( HTTP::Request::Common::HEAD( @parameters ), @suff );
436
}
437
438
439
sub _process_colonic_headers {
440
# Process :content_cb / :content_file / :read_size_hint headers.
441
my($self, $args, $start_index) = @_;
442
443
my($arg, $size);
444
for(my $i = $start_index; $i < @$args; $i += 2) {
445
next unless defined $args->[$i];
446
447
#printf "Considering %s => %s\n", $args->[$i], $args->[$i + 1];
448
449
if($args->[$i] eq ':content_cb') {
450
# Some sanity-checking...
451
$arg = $args->[$i + 1];
452
Carp::croak("A :content_cb value can't be undef") unless defined $arg;
453
Carp::croak("A :content_cb value must be a coderef")
454
unless ref $arg and UNIVERSAL::isa($arg, 'CODE');
455
456
}
457
elsif ($args->[$i] eq ':content_file') {
458
$arg = $args->[$i + 1];
459
460
# Some sanity-checking...
461
Carp::croak("A :content_file value can't be undef")
462
unless defined $arg;
463
Carp::croak("A :content_file value can't be a reference")
464
if ref $arg;
465
Carp::croak("A :content_file value can't be \"\"")
466
unless length $arg;
467
468
}
469
elsif ($args->[$i] eq ':read_size_hint') {
470
$size = $args->[$i + 1];
471
# Bother checking it?
472
473
}
474
else {
475
next;
476
}
477
splice @$args, $i, 2;
478
$i -= 2;
479
}
480
481
# And return a suitable suffix-list for request(REQ,...)
482
483
return unless defined $arg;
484
return $arg, $size if defined $size;
485
return $arg;
486
}
487
488
489
#
594
490
# This whole allow/forbid thing is based on man 1 at's way of doing things.
595
596
=item $ua->protocols_allowed( ); # to read
597
598
=item $ua->protocols_allowed( \@protocols ); # to set
599
600
This reads (or sets) this user-agent's list of procotols that
601
C<$ua-E<gt>request> and C<$ua-E<gt>simple_request> will exclusively
602
allow.
603
604
For example: C<$ua-E<gt>protocols_allowed( [ 'http', 'https'] );>
605
means that this user agent will I<allow only> those protocols,
606
and attempts to use this user-agent to access URLs with any other
607
schemes (like "ftp://...") will result in a 500 error.
608
609
To delete the list, call:
610
C<$ua-E<gt>protocols_allowed(undef)>
611
612
By default, an object has neither a protocols_allowed list, nor
613
a protocols_forbidden list.
614
615
Note that having a protocols_allowed
616
list causes any protocols_forbidden list to be ignored.
617
618
=item $ua->protocols_forbidden( ); # to read
619
620
=item $ua->protocols_forbidden( \@protocols ); # to set
621
622
This reads (or sets) this user-agent's list of procotols that
623
C<$ua-E<gt>request> and C<$ua-E<gt>simple_request> will I<not> allow.
624
625
For example: C<$ua-E<gt>protocols_forbidden( [ 'file', 'mailto'] );>
626
means that this user-agent will I<not> allow those protocols, and
627
attempts to use this user-agent to access URLs with those schemes
628
will result in a 500 error.
629
630
To delete the list, call:
631
C<$ua-E<gt>protocols_forbidden(undef)>
632
633
=item $ua->is_protocol_supported($scheme)
634
635
You can use this method to test whether this user-agent object supports the
636
specified C<scheme>. (The C<scheme> might be a string (like 'http' or
637
'ftp') or it might be an URI object reference.)
638
639
Whether a scheme is supported, is determined by $ua's protocols_allowed or
640
protocols_forbidden lists (if any), and by the capabilities
641
of LWP. I.e., this will return TRUE only if LWP supports this protocol
642
I<and> it's permitted for this particular object.
643
644
=cut
645
491
#
646
492
sub is_protocol_supported
647
493
{
648
494
my($self, $scheme) = @_;
649
495
if (ref $scheme) {
650
496
# assume we got a reference to an URI object
651
497
$scheme = $scheme->scheme;
652
} else {
498
}
499
else {
653
500
Carp::croak("Illegal scheme '$scheme' passed to is_protocol_supported")
654
501
if $scheme =~ /\W/;
655
502
$scheme = lc $scheme;
657
504
658
505
my $x;
659
506
if(ref($self) and $x = $self->protocols_allowed) {
660
return 0 unless grep $_ eq $scheme, @$x;
661
} elsif (ref($self) and $x = $self->protocols_forbidden) {
662
return 0 if grep $_ eq $scheme, @$x;
507
return 0 unless grep lc($_) eq $scheme, @$x;
508
}
509
elsif (ref($self) and $x = $self->protocols_forbidden) {
510
return 0 if grep lc($_) eq $scheme, @$x;
663
511
}
664
512
665
local($SIG{__DIE__}); # protect agains user defined die handlers
513
local($SIG{__DIE__}); # protect against user defined die handlers
666
514
$x = LWP::Protocol::implementor($scheme);
667
515
return 1 if $x and $x ne 'LWP::Protocol::nogo';
668
516
return 0;
669
517
}
670
518
671
#---------------------------------------------------------------------------
672
673
=item $ua->requests_redirectable( ); # to read
674
675
=item $ua->requests_redirectable( \@requests ); # to set
676
677
This reads or sets the object's list of request names that
678
C<$ua-E<gt>redirect_ok(...)> will allow redirection for. By
679
default, this is C<['GET', 'HEAD']>, as per RFC 2068. To
680
change to include 'POST', consider:
681
682
push @{ $ua->requests_redirectable }, 'POST';
683
684
=cut
685
519
686
520
sub protocols_allowed { shift->_elem('protocols_allowed' , @_) }
687
521
sub protocols_forbidden { shift->_elem('protocols_forbidden' , @_) }
688
522
sub requests_redirectable { shift->_elem('requests_redirectable', @_) }
689
523
690
#---------------------------------------------------------------------------
691
692
=item $ua->redirect_ok($prospective_request)
693
694
This method is called by request() before it tries to follow a
695
redirection to the request in $prospective_request. This
696
should return a true value if this redirection is
697
permissible.
698
699
The default implementation will return FALSE unless the method
700
is in the object's C<requests_redirectable> list,
701
FALSE if the proposed redirection is to a "file://..."
702
URL, and TRUE otherwise.
703
704
Subclasses might want to override this.
705
706
(This method's behavior in previous versions was simply to return
707
TRUE for anything except POST requests).
708
709
=cut
710
524
711
525
sub redirect_ok
712
526
{
713
# RFC 2068, section 10.3.2 and 10.3.3 say:
527
# RFC 2616, section 10.3.2 and 10.3.3 say:
714
528
# If the 30[12] status code is received in response to a request other
715
529
# than GET or HEAD, the user agent MUST NOT automatically redirect the
716
530
# request unless it can be confirmed by the user, since this might
719
533
# Note that this routine used to be just:
720
534
# return 0 if $_[1]->method eq "POST"; return 1;
721
535
722
my($self, $request) = @_;
723
my $method = $request->method;
536
my($self, $new_request, $response) = @_;
537
my $method = $response->request->method;
724
538
return 0 unless grep $_ eq $method,
725
539
@{ $self->requests_redirectable || [] };
726
540
727
if($request->url->scheme eq 'file') {
728
LWP::Debug::trace("Can't redirect to a file:// URL!");
541
if ($new_request->url->scheme eq 'file') {
542
$response->header("Client-Warning" =>
543
"Can't redirect to a file:// URL!");
729
544
return 0;
730
545
}
731
546
734
549
}
735
550
736
551
737
=item $ua->credentials($netloc, $realm, $uname, $pass)
738
739
Set the user name and password to be used for a realm. It is often more
740
useful to specialize the get_basic_credentials() method instead.
741
742
=cut
743
744
552
sub credentials
745
553
{
746
554
my($self, $netloc, $realm, $uid, $pass) = @_;
747
@{ $self->{'basic_authentication'}{$netloc}{$realm} } = ($uid, $pass);
555
@{ $self->{'basic_authentication'}{lc($netloc)}{$realm} } =
556
($uid, $pass);
748
557
}
749
558
750
559
751
=item $ua->get_basic_credentials($realm, $uri, [$proxy])
752
753
This is called by request() to retrieve credentials for a Realm
754
protected by Basic Authentication or Digest Authentication.
755
756
Should return username and password in a list. Return undef to abort
757
the authentication resolution atempts.
758
759
This implementation simply checks a set of pre-stored member
760
variables. Subclasses can override this method to e.g. ask the user
761
for a username/password. An example of this can be found in
762
C<lwp-request> program distributed with this library.
763
764
=cut
765
766
560
sub get_basic_credentials
767
561
{
768
562
my($self, $realm, $uri, $proxy) = @_;
769
563
return if $proxy;
770
564
771
my $host_port = $uri->host_port;
565
my $host_port = lc($uri->host_port);
772
566
if (exists $self->{'basic_authentication'}{$host_port}{$realm}) {
773
567
return @{ $self->{'basic_authentication'}{$host_port}{$realm} };
774
568
}
777
571
}
778
572
779
573
780
=item $ua->agent([$product_id])
781
782
Get/set the product token that is used to identify the user agent on
783
the network. The agent value is sent as the "User-Agent" header in
784
the requests. The default is the string returned by the _agent()
785
method (see below).
786
787
If the $product_id ends with space then the C<_agent> string is
788
appended to it.
789
790
The user agent string should be one or more simple product identifiers
791
with an optional version number separated by the "/" character.
792
Examples are:
793
794
$ua->agent('Checkbot/0.4 ' . $ua->_agent);
795
$ua->agent('Checkbot/0.4 '); # same as above
796
$ua->agent('Mozilla/5.0');
797
$ua->agent(""); # don't identify
798
799
=item $ua->_agent
800
801
Returns the default agent identifier. This is a string of the form
802
"libwww-perl/#.##", where "#.##" is substitued with the version numer
803
of this library.
804
805
=cut
806
807
574
sub agent {
808
575
my $self = shift;
809
576
my $old = $self->{agent};
815
582
$old;
816
583
}
817
584
818
sub _agent { "libwww-perl/$LWP::VERSION" }
819
820
821
=item $ua->from([$email_address])
822
823
Get/set the Internet e-mail address for the human user who controls
824
the requesting user agent. The address should be machine-usable, as
825
defined in RFC 822. The from value is send as the "From" header in
826
the requests. Example:
827
828
$ua->from('gaas@cpan.org');
829
830
The default is to not send a "From" header.
831
832
=item $ua->timeout([$secs])
833
834
Get/set the timeout value in seconds. The default timeout() value is
835
180 seconds, i.e. 3 minutes.
836
837
=item $ua->cookie_jar([$cookie_jar_obj])
838
839
Get/set the cookie jar object to use. The only requirement is that
840
the cookie jar object must implement the extract_cookies($request) and
841
add_cookie_header($response) methods. These methods will then be
842
invoked by the user agent as requests are sent and responses are
843
received. Normally this will be a C<HTTP::Cookies> object or some
844
subclass.
845
846
The default is to have no cookie_jar, i.e. never automatically add
847
"Cookie" headers to the requests.
848
849
Shortcut: If a reference to a plain hash is passed in as the
850
$cookie_jar_object, then it is replaced with an instance of
851
C<HTTP::Cookies> that is initalized based on the hash. This form also
852
automatically loads the C<HTTP::Cookies> module. It means that:
853
854
$ua->cookie_jar({ file => "$ENV{HOME}/.cookies.txt" });
855
856
is really just a shortcut for:
857
858
require HTTP::Cookies;
859
$ua->cookie_jar(HTTP::Cookies->new(file => "$ENV{HOME}/.cookies.txt"));
860
861
=item $ua->conn_cache([$cache_obj])
862
863
Get/set the I<LWP::ConnCache> object to use.
864
865
=item $ua->parse_head([$boolean])
866
867
Get/set a value indicating wether we should initialize response
868
headers from the E<lt>head> section of HTML documents. The default is
869
TRUE. Do not turn this off, unless you know what you are doing.
870
871
=item $ua->max_size([$bytes])
872
873
Get/set the size limit for response content. The default is C<undef>,
874
which means that there is no limit. If the returned response content
875
is only partial, because the size limit was exceeded, then a
876
"Client-Aborted" header will be added to the response.
877
878
=cut
879
880
sub timeout { shift->_elem('timeout', @_); }
881
sub from { shift->_elem('from', @_); }
882
sub parse_head { shift->_elem('parse_head',@_); }
883
sub max_size { shift->_elem('max_size', @_); }
585
586
sub _agent { "libwww-perl/$LWP::VERSION" }
587
588
sub timeout { shift->_elem('timeout', @_); }
589
sub from { shift->_elem('from', @_); }
590
sub parse_head { shift->_elem('parse_head', @_); }
591
sub max_size { shift->_elem('max_size', @_); }
592
sub max_redirect { shift->_elem('max_redirect', @_); }
593
884
594
885
595
sub cookie_jar {
886
596
my $self = shift;
896
606
$old;
897
607
}
898
608
609
sub default_headers {
610
my $self = shift;
611
my $old = $self->{def_headers} ||= HTTP::Headers->new;
612
if (@_) {
613
$self->{def_headers} = shift;
614
}
615
return $old;
616
}
617
618
sub default_header {
619
my $self = shift;
620
return $self->default_headers->header(@_);
621
}
622
623
899
624
sub conn_cache {
900
625
my $self = shift;
901
626
my $old = $self->{conn_cache};
910
635
$old;
911
636
}
912
637
638
913
639
# depreciated
914
640
sub use_eval { shift->_elem('use_eval', @_); }
915
641
sub use_alarm
920
646
}
921
647
922
648
923
=item $ua->clone;
924
925
Returns a copy of the LWP::UserAgent object
926
927
=cut
928
929
930
649
sub clone
931
650
{
932
651
my $self = shift;
944
663
}
945
664
946
665
947
948
949
=item $ua->mirror($url, $file)
950
951
Get and store a document identified by a URL, using If-Modified-Since,
952
and checking of the Content-Length. Returns a reference to the
953
response object.
954
955
=cut
956
957
666
sub mirror
958
667
{
959
668
my($self, $url, $file) = @_;
980
689
unlink($tmpfile);
981
690
die "Transfer truncated: " .
982
691
"only $file_length out of $content_length bytes received\n";
983
} elsif (defined $content_length and $file_length > $content_length) {
692
}
693
elsif (defined $content_length and $file_length > $content_length) {
984
694
unlink($tmpfile);
985
695
die "Content-length mismatch: " .
986
696
"expected $content_length bytes, got $file_length\n";
987
} else {
697
}
698
else {
988
699
# OK
989
700
if (-e $file) {
990
701
# Some dosish systems fail to rename if the target exists
999
710
utime $lm, $lm, $file;
1000
711
}
1001
712
}
1002
} else {
713
}
714
else {
1003
715
unlink($tmpfile);
1004
716
}
1005
717
return $response;
1006
718
}
1007
719
1008
=item $ua->proxy(...)
1009
1010
Set/retrieve proxy URL for a scheme:
1011
1012
$ua->proxy(['http', 'ftp'], 'http://proxy.sn.no:8001/');
1013
$ua->proxy('gopher', 'http://proxy.sn.no:8001/');
1014
1015
The first form specifies that the URL is to be used for proxying of
1016
access methods listed in the list in the first method argument,
1017
i.e. 'http' and 'ftp'.
1018
1019
The second form shows a shorthand form for specifying
1020
proxy URL for a single access scheme.
1021
1022
=cut
1023
720
1024
721
sub proxy
1025
722
{
1035
732
return $old;
1036
733
}
1037
734
1038
=item $ua->env_proxy()
1039
1040
Load proxy settings from *_proxy environment variables. You might
1041
specify proxies like this (sh-syntax):
1042
1043
gopher_proxy=http://proxy.my.place/
1044
wais_proxy=http://proxy.my.place/
1045
no_proxy="localhost,my.domain"
1046
export gopher_proxy wais_proxy no_proxy
1047
1048
Csh or tcsh users should use the C<setenv> command to define these
1049
environment variables.
1050
1051
On systems with case-insensitive environment variables there exists a
1052
name clash between the CGI environment variables and the C<HTTP_PROXY>
1053
environment variable normally picked up by env_proxy(). Because of
1054
this C<HTTP_PROXY> is not honored for CGI scripts. The
1055
C<CGI_HTTP_PROXY> environment variable can be used instead.
1056
1057
=cut
1058
735
1059
736
sub env_proxy {
1060
737
my ($self) = @_;
1078
755
}
1079
756
}
1080
757
1081
=item $ua->no_proxy($domain,...)
1082
1083
Do not proxy requests to the given domains. Calling no_proxy without
1084
any domains clears the list of domains. Eg:
1085
1086
$ua->no_proxy('localhost', 'no', ...);
1087
1088
=cut
1089
758
1090
759
sub no_proxy {
1091
760
my($self, @no) = @_;
1124
793
undef;
1125
794
}
1126
795
796
1127
797
sub _new_response {
1128
798
my($request, $code, $message) = @_;
1129
799
my $response = HTTP::Response->new($code, $message);
1130
800
$response->request($request);
1131
801
$response->header("Client-Date" => HTTP::Date::time2str(time));
802
$response->header("Client-Warning" => "Internal response");
803
$response->header("Content-Type" => "text/plain");
804
$response->content("$code $message\n");
1132
805
return $response;
1133
806
}
1134
807
808
1135
809
1;
1136
810
811
__END__
812
813
=head1 NAME
814
815
LWP::UserAgent - Web user agent class
816
817
=head1 SYNOPSIS
818
819
require LWP::UserAgent;
820
821
my $ua = LWP::UserAgent->new;
822
$ua->timeout(10);
823
$ua->env_proxy;
824
825
my $response = $ua->get('http://search.cpan.org/');
826
827
if ($response->is_success) {
828
print $response->content; # or whatever
829
}
830
else {
831
die $response->status_line;
832
}
833
834
=head1 DESCRIPTION
835
836
The C<LWP::UserAgent> is a class implementing a web user agent.
837
C<LWP::UserAgent> objects can be used to dispatch web requests.
838
839
In normal use the application creates an C<LWP::UserAgent> object, and
840
then configures it with values for timeouts, proxies, name, etc. It
841
then creates an instance of C<HTTP::Request> for the request that
842
needs to be performed. This request is then passed to one of the
843
request method the UserAgent, which dispatches it using the relevant
844
protocol, and returns a C<HTTP::Response> object. There are
845
convenience methods for sending the most common request types: get(),
846
head() and post(). When using these methods then the creation of the
847
request object is hidden as shown in the synopsis above.
848
849
The basic approach of the library is to use HTTP style communication
850
for all protocol schemes. This means that you will construct
851
C<HTTP::Request> objects and receive C<HTTP::Response> objects even
852
for non-HTTP resources like I<gopher> and I<ftp>. In order to achieve
853
even more similarity to HTTP style communications, gopher menus and
854
file directories are converted to HTML documents.
855
856
=head1 CONSTRUCTOR METHODS
857
858
The following constructor methods are available:
859
860
=over 4
861
862
=item $ua = LWP::UserAgent->new( %options )
863
864
This method constructs a new C<LWP::UserAgent> object and returns it.
865
Key/value pair arguments may be provided to set up the initial state.
866
The following options correspond to attribute methods described below:
867
868
KEY DEFAULT
869
----------- --------------------
870
agent "libwww-perl/#.##"
871
from undef
872
conn_cache undef
873
cookie_jar undef
874
default_headers HTTP::Headers->new
875
max_size undef
876
max_redirect 7
877
parse_head 1
878
protocols_allowed undef
879
protocols_forbidden undef
880
requests_redirectable ['GET', 'HEAD']
881
timeout 180
882
883
The following additional options are also accepted: If the
884
C<env_proxy> option is passed in with a TRUE value, then proxy
885
settings are read from environment variables (see env_proxy() method
886
below). If the C<keep_alive> option is passed in, then a
887
C<LWP::ConnCache> is set up (see conn_cache() method below). The
888
C<keep_alive> value is passed on as the C<total_capacity> for the
889
connection cache.
890
891
=item $ua->clone
892
893
Returns a copy of the LWP::UserAgent object.
894
895
=back
896
897
=head1 ATTRIBUTES
898
899
The settings of the configuration attributes modify the behaviour of the
900
C<LWP::UserAgent> when it dispatches requests. Most of these can also
901
be initialized by options passed to the constructor method.
902
903
The following attributes methods are provided. The attribute value is
904
left unchanged if no argument is given. The return value from each
905
method is the old attribute value.
906
907
=over
908
909
=item $ua->agent
910
911
=item $ua->agent( $product_id )
912
913
Get/set the product token that is used to identify the user agent on
914
the network. The agent value is sent as the "User-Agent" header in
915
the requests. The default is the string returned by the _agent()
916
method (see below).
917
918
If the $product_id ends with space then the _agent() string is
919
appended to it.
920
921
The user agent string should be one or more simple product identifiers
922
with an optional version number separated by the "/" character.
923
Examples are:
924
925
$ua->agent('Checkbot/0.4 ' . $ua->_agent);
926
$ua->agent('Checkbot/0.4 '); # same as above
927
$ua->agent('Mozilla/5.0');
928
$ua->agent(""); # don't identify
929
930
=item $ua->_agent
931
932
Returns the default agent identifier. This is a string of the form
933
"libwww-perl/#.##", where "#.##" is substituted with the version number
934
of this library.
935
936
=item $ua->from
937
938
=item $ua->from( $email_address )
939
940
Get/set the e-mail address for the human user who controls
941
the requesting user agent. The address should be machine-usable, as
942
defined in RFC 822. The C<from> value is send as the "From" header in
943
the requests. Example:
944
945
$ua->from('gaas@cpan.org');
946
947
The default is to not send a "From" header. See the default_headers()
948
method for the more general interface that allow any header to be defaulted.
949
950
=item $ua->cookie_jar
951
952
=item $ua->cookie_jar( $cookie_jar_obj )
953
954
Get/set the cookie jar object to use. The only requirement is that
955
the cookie jar object must implement the extract_cookies($request) and
956
add_cookie_header($response) methods. These methods will then be
957
invoked by the user agent as requests are sent and responses are
958
received. Normally this will be a C<HTTP::Cookies> object or some
959
subclass.
960
961
The default is to have no cookie_jar, i.e. never automatically add
962
"Cookie" headers to the requests.
963
964
Shortcut: If a reference to a plain hash is passed in as the
965
$cookie_jar_object, then it is replaced with an instance of
966
C<HTTP::Cookies> that is initialized based on the hash. This form also
967
automatically loads the C<HTTP::Cookies> module. It means that:
968
969
$ua->cookie_jar({ file => "$ENV{HOME}/.cookies.txt" });
970
971
is really just a shortcut for:
972
973
require HTTP::Cookies;
974
$ua->cookie_jar(HTTP::Cookies->new(file => "$ENV{HOME}/.cookies.txt"));
975
976
=item $ua->default_headers
977
978
=item $ua->default_headers( $headers_obj )
979
980
Get/set the headers object that will provide default header values for
981
any requests sent. By default this will be an empty C<HTTP::Headers>
982
object. Example:
983
984
$ua->default_headers->push_header('Accept-Language' => "no, en");
985
986
=item $ua->default_header( $field )
987
988
=item $ua->default_header( $field => $value )
989
990
This is just a short-cut for $ua->default_headers->header( $field =>
991
$value ). Example:
992
993
$ua->default_header('Accept-Language' => "no, en");
994
995
=item $ua->conn_cache
996
997
=item $ua->conn_cache( $cache_obj )
998
999
Get/set the C<LWP::ConnCache> object to use. See L<LWP::ConnCache>
1000
for details.
1001
1002
=item $ua->credentials( $netloc, $realm, $uname, $pass )
1003
1004
Set the user name and password to be used for a realm. It is often more
1005
useful to specialize the get_basic_credentials() method instead.
1006
1007
=item $ua->max_size
1008
1009
=item $ua->max_size( $bytes )
1010
1011
Get/set the size limit for response content. The default is C<undef>,
1012
which means that there is no limit. If the returned response content
1013
is only partial, because the size limit was exceeded, then a
1014
"Client-Aborted" header will be added to the response. The content
1015
might end up longer than C<max_size> as we abort once appending a
1016
chunk of data makes the length exceed the limit. The "Content-Length"
1017
header, if present, will indicate the length of the full content and
1018
will normally not be the same as C<< length($res->content) >>.
1019
1020
=item $ua->max_redirect
1021
1022
=item $ua->max_redirect( $n )
1023
1024
This reads or sets the object's limit of how many times it will obey
1025
redirection responses in a given request cycle.
1026
1027
By default, the value is 7. This means that if you call request()
1028
method and the response is a redirect elsewhere which is in turn a
1029
redirect, and so on seven times, then LWP gives up after that seventh
1030
request.
1031
1032
=item $ua->parse_head
1033
1034
=item $ua->parse_head( $boolean )
1035
1036
Get/set a value indicating whether we should initialize response
1037
headers from the E<lt>head> section of HTML documents. The default is
1038
TRUE. Do not turn this off, unless you know what you are doing.
1039
1040
=item $ua->protocols_allowed
1041
1042
=item $ua->protocols_allowed( \@protocols )
1043
1044
This reads (or sets) this user agent's list of protocols that the
1045
request methods will exclusively allow. The protocol names are case
1046
insensitive.
1047
1048
For example: C<$ua-E<gt>protocols_allowed( [ 'http', 'https'] );>
1049
means that this user agent will I<allow only> those protocols,
1050
and attempts to use this user agent to access URLs with any other
1051
schemes (like "ftp://...") will result in a 500 error.
1052
1053
To delete the list, call: C<$ua-E<gt>protocols_allowed(undef)>
1054
1055
By default, an object has neither a C<protocols_allowed> list, nor a
1056
C<protocols_forbidden> list.
1057
1058
Note that having a C<protocols_allowed> list causes any
1059
C<protocols_forbidden> list to be ignored.
1060
1061
=item $ua->protocols_forbidden
1062
1063
=item $ua->protocols_forbidden( \@protocols )
1064
1065
This reads (or sets) this user agent's list of protocols that the
1066
request method will I<not> allow. The protocol names are case
1067
insensitive.
1068
1069
For example: C<$ua-E<gt>protocols_forbidden( [ 'file', 'mailto'] );>
1070
means that this user agent will I<not> allow those protocols, and
1071
attempts to use this user agent to access URLs with those schemes
1072
will result in a 500 error.
1073
1074
To delete the list, call: C<$ua-E<gt>protocols_forbidden(undef)>
1075
1076
=item $ua->requests_redirectable
1077
1078
=item $ua->requests_redirectable( \@requests )
1079
1080
This reads or sets the object's list of request names that
1081
C<$ua-E<gt>redirect_ok(...)> will allow redirection for. By
1082
default, this is C<['GET', 'HEAD']>, as per RFC 2616. To
1083
change to include 'POST', consider:
1084
1085
push @{ $ua->requests_redirectable }, 'POST';
1086
1087
=item $ua->timeout
1088
1089
=item $ua->timeout( $secs )
1090
1091
Get/set the timeout value in seconds. The default timeout() value is
1092
180 seconds, i.e. 3 minutes.
1093
1094
The requests is aborted if no activity on the connection to the server
1095
is observed for C<timeout> seconds. This means that the time it takes
1096
for the complete transaction and the request() method to actually
1097
return might be longer.
1098
1099
=back
1100
1101
=head2 Proxy attributes
1102
1103
The following methods set up when requests should be passed via a
1104
proxy server.
1105
1106
=over
1107
1108
=item $ua->proxy(\@schemes, $proxy_url)
1109
1110
=item $ua->proxy($scheme, $proxy_url)
1111
1112
Set/retrieve proxy URL for a scheme:
1113
1114
$ua->proxy(['http', 'ftp'], 'http://proxy.sn.no:8001/');
1115
$ua->proxy('gopher', 'http://proxy.sn.no:8001/');
1116
1117
The first form specifies that the URL is to be used for proxying of
1118
access methods listed in the list in the first method argument,
1119
i.e. 'http' and 'ftp'.
1120
1121
The second form shows a shorthand form for specifying
1122
proxy URL for a single access scheme.
1123
1124
=item $ua->no_proxy( $domain, ... )
1125
1126
Do not proxy requests to the given domains. Calling no_proxy without
1127
any domains clears the list of domains. Eg:
1128
1129
$ua->no_proxy('localhost', 'no', ...);
1130
1131
=item $ua->env_proxy
1132
1133
Load proxy settings from *_proxy environment variables. You might
1134
specify proxies like this (sh-syntax):
1135
1136
gopher_proxy=http://proxy.my.place/
1137
wais_proxy=http://proxy.my.place/
1138
no_proxy="localhost,my.domain"
1139
export gopher_proxy wais_proxy no_proxy
1140
1141
csh or tcsh users should use the C<setenv> command to define these
1142
environment variables.
1143
1144
On systems with case insensitive environment variables there exists a
1145
name clash between the CGI environment variables and the C<HTTP_PROXY>
1146
environment variable normally picked up by env_proxy(). Because of
1147
this C<HTTP_PROXY> is not honored for CGI scripts. The
1148
C<CGI_HTTP_PROXY> environment variable can be used instead.
1149
1150
=back
1151
1152
=head1 REQUEST METHODS
1153
1154
The methods described in this section are used to dispatch requests
1155
via the user agent. The following request methods are provided:
1156
1157
=over
1158
1159
=item $ua->get( $url )
1160
1161
=item $ua->get( $url , $field_name => $value, ... )
1162
1163
This method will dispatch a C<GET> request on the given $url. Further
1164
arguments can be given to initialize the headers of the request. These
1165
are given as separate name/value pairs. The return value is a
1166
response object. See L<HTTP::Response> for a description of the
1167
interface it provides.
1168
1169
Fields names that start with ":" are special. These will not
1170
initialize headers of the request but will determine how the response
1171
content is treated. The following special field names are recognized:
1172
1173
:content_file => $filename
1174
:content_cb => \&callback
1175
:read_size_hint => $bytes
1176
1177
If a $filename is provided with the C<:content_file> option, then the
1178
response content will be saved here instead of in the response
1179
object. If a callback is provided with the C<:content_cb> option then
1180
this function will be called for each chunk of the response content as
1181
it is received from the server. If neither of these options are
1182
given, then the response content will accumulate in the response
1183
object itself. This might not be suitable for very large response
1184
bodies. Only one of C<:content_file> or C<:content_cb> can be
1185
specified. The content of unsuccessful responses will always
1186
accumulate in the response object itself, regardless of the
1187
C<:content_file> or C<:content_cb> options passed in.
1188
1189
The C<:read_size_hint> option is passed to the protocol module which
1190
will try to read data from the server in chunks of this size. A
1191
smaller value for the C<:read_size_hint> will result in a higher
1192
number of callback invocations.
1193
1194
The callback function is called with 3 arguments: a chunk of data, a
1195
reference to the response object, and a reference to the protocol
1196
object. The callback can abort the request by invoking die(). The
1197
exception message will show up as the "X-Died" header field in the
1198
response returned by the get() function.
1199
1200
=item $ua->head( $url )
1201
1202
=item $ua->head( $url , $field_name => $value, ... )
1203
1204
This method will dispatch a C<HEAD> request on the given $url.
1205
Otherwise it works like the get() method described above.
1206
1207
=item $ua->post( $url, \%form )
1208
1209
=item $ua->post( $url, \@form )
1210
1211
=item $ua->post( $url, \%form, $field_name => $value, ... )
1212
1213
This method will dispatch a C<POST> request on the given $url, with
1214
%form or @form providing the key/value pairs for the fill-in form
1215
content. Additional headers and content options are the same as for
1216
the get() method.
1217
1218
This method will use the POST() function from C<HTTP::Request::Common>
1219
to build the request. See L<HTTP::Request::Common> for a details on
1220
how to pass form content and other advanced features.
1221
1222
=item $ua->mirror( $url, $filename )
1223
1224
This method will get the document identified by $url and store it in
1225
file called $filename. If the file already exists, then the request
1226
will contain an "If-Modified-Since" header matching the modification
1227
time of the file. If the document on the server has not changed since
1228
this time, then nothing happens. If the document has been updated, it
1229
will be downloaded again. The modification time of the file will be
1230
forced to match that of the server.
1231
1232
The return value is the the response object.
1233
1234
=item $ua->request( $request )
1235
1236
=item $ua->request( $request, $content_file )
1237
1238
=item $ua->request( $request, $content_cb )
1239
1240
=item $ua->request( $request, $content_cb, $read_size_hint )
1241
1242
This method will dispatch the given $request object. Normally this
1243
will be an instance of the C<HTTP::Request> class, but any object with
1244
a similar interface will do. The return value is a response object.
1245
See L<HTTP::Request> and L<HTTP::Response> for a description of the
1246
interface provided by these classes.
1247
1248
The request() method will process redirects and authentication
1249
responses transparently. This means that it may actually send several
1250
simple requests via the simple_request() method described below.
1251
1252
The request methods described above; get(), head(), post() and
1253
mirror(), will all dispatch the request they build via this method.
1254
They are convenience methods that simply hides the creation of the
1255
request object for you.
1256
1257
The $content_file, $content_cb and $read_size_hint all correspond to
1258
options described with the get() method above.
1259
1260
You are allowed to use a CODE reference as C<content> in the request
1261
object passed in. The C<content> function should return the content
1262
when called. The content can be returned in chunks. The content
1263
function will be invoked repeatedly until it return an empty string to
1264
signal that there is no more content.
1265
1266
=item $ua->simple_request( $request )
1267
1268
=item $ua->simple_request( $request, $content_file )
1269
1270
=item $ua->simple_request( $request, $content_cb )
1271
1272
=item $ua->simple_request( $request, $content_cb, $read_size_hint )
1273
1274
This method dispatches a single request and returns the response
1275
received. Arguments are the same as for request() described above.
1276
1277
The difference from request() is that simple_request() will not try to
1278
handle redirects or authentication responses. The request() method
1279
will in fact invoke this method for each simple request it sends.
1280
1281
=item $ua->is_protocol_supported( $scheme )
1282
1283
You can use this method to test whether this user agent object supports the
1284
specified C<scheme>. (The C<scheme> might be a string (like 'http' or
1285
'ftp') or it might be an URI object reference.)
1286
1287
Whether a scheme is supported, is determined by the user agent's
1288
C<protocols_allowed> or C<protocols_forbidden> lists (if any), and by
1289
the capabilities of LWP. I.e., this will return TRUE only if LWP
1290
supports this protocol I<and> it's permitted for this particular
1291
object.
1292
1293
=back
1294
1295
=head2 Callback methods
1296
1297
The following methods will be invoked as requests are processed. These
1298
methods are documented here because subclasses of C<LWP::UserAgent>
1299
might want to override their behaviour.
1300
1301
=over
1302
1303
=item $ua->prepare_request( $request )
1304
1305
This method is invoked by simple_request(). Its task is to modify the
1306
given $request object by setting up various headers based on the
1307
attributes of the user agent. The return value should normally be the
1308
$request object passed in. If a different request object is returned
1309
it will be the one actually processed.
1310
1311
The headers affected by the base implementation are; "User-Agent",
1312
"From", "Range" and "Cookie".
1313
1314
=item $ua->redirect_ok( $prospective_request, $response )
1315
1316
This method is called by request() before it tries to follow a
1317
redirection to the request in $response. This should return a TRUE
1318
value if this redirection is permissible. The $prospective_request
1319
will be the request to be sent if this method returns TRUE.
1320
1321
The base implementation will return FALSE unless the method
1322
is in the object's C<requests_redirectable> list,
1323
FALSE if the proposed redirection is to a "file://..."
1324
URL, and TRUE otherwise.
1325
1326
=item $ua->get_basic_credentials( $realm, $uri, $isproxy )
1327
1328
This is called by request() to retrieve credentials for documents
1329
protected by Basic or Digest Authentication. The arguments passed in
1330
is the $realm provided by the server, the $uri requested and a boolean
1331
flag to indicate if this is authentication against a proxy server.
1332
1333
The method should return a username and password. It should return an
1334
empty list to abort the authentication resolution attempt. Subclasses
1335
can override this method to prompt the user for the information. An
1336
example of this can be found in C<lwp-request> program distributed
1337
with this library.
1338
1339
The base implementation simply checks a set of pre-stored member
1340
variables, set up with the credentials() method.
1341
1137
1342
=back
1138
1343
1139
1344
=head1 SEE ALSO
1140
1345
1141
See L<LWP> for a complete overview of libwww-perl5. See F<lwp-request> and
1142
F<lwp-mirror> for examples of usage.
1346
See L<LWP> for a complete overview of libwww-perl5. See L<lwpcook>
1347
and the scripts F<lwp-request> and F<lwp-download> for examples of
1348
usage.
1349
1350
See L<HTTP::Request> and L<HTTP::Response> for a description of the
1351
message objects dispatched and received. See L<HTTP::Request::Common>
1352
and L<HTML::Form> for other ways to build request objects.
1353
1354
See L<WWW::Mechanize> and L<WWW::Search> for examples of more
1355
specialized user agents based on C<LWP::UserAgent>.
1143
1356
1144
1357
=head1 COPYRIGHT
1145
1358
1146
Copyright 1995-2001 Gisle Aas.
1359
Copyright 1995-2004 Gisle Aas.
1147
1360
1148
1361
This library is free software; you can redistribute it and/or
1149
1362
modify it under the same terms as Perl itself.
1150
1151
=cut
1152
1153
Loggerhead is a web-based interface for Breezy
Version: 2.0.1