~ubuntu-branches/ubuntu/intrepid/perl-doc-html/intrepid

<div id="breadCrumbs"><a href="../index.html">Home</a> > <a href="../index-modules-A.html">Core modules</a> > <a href="../index-modules-F.html">F</a> > File::Fetch</div>

<div id="contentBody"><div class="title_container"><div class="page_title">File::Fetch</div></div><ul><li><a href="#NAME">NAME</a><li><a href="#SYNOPSIS">SYNOPSIS</a><li><a href="#DESCRIPTION">DESCRIPTION</a><li><a href="#ACCESSORS">ACCESSORS</a><li><a href="#METHODS">METHODS</a><ul><li><a href="#%24ff-%3d-File%3a%3aFetch-%3enew(-uri-%3d%3e-'http%3a%2f%2fsome.where.com%2fdir%2ffile.txt'-)%3b">$ff = File::Fetch->new( uri => '<a href="http://some.where.com/dir/file.txt">http://some.where.com/dir/file.txt</a>' );</a><li><a href="#%24ff-%3efetch(-%5bto-%3d%3e-%2fmy%2foutput%2fdir%2f%5d-)">$ff->fetch( [to => /my/output/dir/] )</a><li><a href="#%24ff-%3eerror(%5bBOOL%5d)">$ff->error([BOOL])</a></ul><li><a href="#HOW-IT-WORKS">HOW IT WORKS</a><li><a href="#GLOBAL-VARIABLES">GLOBAL VARIABLES</a><ul><li><a href="#%24File%3a%3aFetch%3a%3aFROM_EMAIL">$File::Fetch::FROM_EMAIL</a><li><a href="#%24File%3a%3aFetch%3a%3aUSER_AGENT">$File::Fetch::USER_AGENT</a><li><a href="#%24File%3a%3aFetch%3a%3aFTP_PASSIVE">$File::Fetch::FTP_PASSIVE</a><li><a href="#%24File%3a%3aFetch%3a%3aTIMEOUT">$File::Fetch::TIMEOUT</a><li><a href="#%24File%3a%3aFetch%3a%3aWARN">$File::Fetch::WARN</a><li><a href="#%24File%3a%3aFetch%3a%3aDEBUG">$File::Fetch::DEBUG</a><li><a href="#%24File%3a%3aFetch%3a%3aBLACKLIST">$File::Fetch::BLACKLIST</a><li><a href="#%24File%3a%3aFetch%3a%3aMETHOD_FAIL">$File::Fetch::METHOD_FAIL</a></ul><li><a href="#MAPPING">MAPPING</a><li><a href="#FREQUENTLY-ASKED-QUESTIONS">FREQUENTLY ASKED QUESTIONS</a><ul><li><a href="#So-how-do-I-use-a-proxy-with-File%3a%3aFetch%3f">So how do I use a proxy with File::Fetch?</a><li><a href="#I-used-'lynx'-to-fetch-a-file%2c-but-its-contents-is-all-wrong!">I used 'lynx' to fetch a file, but its contents is all wrong!</a><li><a href="#Files-I'm-trying-to-fetch-have-reserved-characters-or-non-ASCII-characters-in-them.-What-do-I-do%3f">Files I'm trying to fetch have reserved characters or non-ASCII characters in them. What do I do?</a></ul><li><a href="#TODO">TODO</a><li><a href="#BUG-REPORTS">BUG REPORTS</a><li><a href="#AUTHOR">AUTHOR</a><li><a href="#COPYRIGHT">COPYRIGHT</a></ul><a name="NAME"></a><h1>NAME</h1>

<p>File::Fetch - A generic file fetching mechanism</p>

<a name="SYNOPSIS"></a><h1>SYNOPSIS</h1>

<pre class="verbatim"> <a class="l_k" href="../functions/use.html">use</a> <span class="w">File::Fetch</span><span class="sc">;</span></pre>

<pre class="verbatim"> <span class="c">### build a File::Fetch object ###</span>

<a class="l_k" href="../functions/my.html">my</a> <span class="i">$ff</span> = <span class="w">File::Fetch</span><span class="w">->new</span><span class="s">(</span><span class="w">uri</span> <span class="cm">=></span> <span class="q">'<a href="http://some.where.com/dir/a.txt">http://some.where.com/dir/a.txt</a>'</span><span class="s">)</span><span class="sc">;</span></pre>

<pre class="verbatim"> <span class="c">### fetch the uri to cwd() ###</span>

<a class="l_k" href="../functions/my.html">my</a> <span class="i">$where</span> = <span class="i">$ff</span><span class="i">->fetch</span><span class="s">(</span><span class="s">)</span> or <a class="l_k" href="../functions/die.html">die</a> <span class="i">$ff</span><span class="i">->error</span><span class="sc">;</span></pre>

<pre class="verbatim"> <span class="c">### fetch the uri to /tmp ###</span>

<a class="l_k" href="../functions/my.html">my</a> <span class="i">$where</span> = <span class="i">$ff</span><span class="i">->fetch</span><span class="s">(</span> <span class="w">to</span> <span class="cm">=></span> <span class="q">'/tmp'</span> <span class="s">)</span><span class="sc">;</span></pre>

<pre class="verbatim"> <span class="c">### parsed bits from the uri ###</span>

<span class="i">$ff</span><span class="i">->scheme</span><span class="sc">;</span>

100

101

102

103

<a name="DESCRIPTION"></a><h1>DESCRIPTION</h1>

104

<p>File::Fetch is a generic file fetching mechanism.</p>

105

<p>It allows you to fetch any file pointed to by a <code class="inline"><span class="w">ftp</span></code>

106

, <code class="inline"><span class="w">http</span></code>

107

108

109

, or <code class="inline"><span class="w">rsync</span></code>

110

uri by a number of different means.</p>

111

<p>See the <code class="inline"><span class="w">HOW</span> <span class="w">IT</span> <span class="w">WORKS</span></code>

112

section further down for details.</p>

113

<a name="ACCESSORS"></a><h1>ACCESSORS</h1>

114

<p>A <code class="inline"><span class="w">File::Fetch</span></code>

115

object has the following accessors</p>

116

<ul>

117

118

<p>The uri you passed to the constructor</p>

119

</li>

120

<li><a name="%24ff-%3escheme"></a><b>$ff->scheme</b>

121

<p>The scheme from the uri (like 'file', 'http', etc)</p>

122

</li>

123

124

<p>The hostname in the uri. Will be empty if host was originally

125

'localhost' for a '<a href="file://">file://</a>' url.</p>

126

</li>

127

128

<p>On operating systems with the concept of a volume the second element

129

of a <a href="file://">file://</a> is considered to the be volume specification for the file.

130

Thus on Win32 this routine returns the volume, on other operating

131

systems this returns nothing.</p>

132

<p>On Windows this value may be empty if the uri is to a network share, in

133

which case the 'share' property will be defined. Additionally, volume

134

specifications that use '|' as ':' will be converted on read to use ':'.</p>

135

<p>On VMS, which has a volume concept, this field will be empty because VMS

136

file specifications are converted to absolute UNIX format and the volume

137

information is transparently included.</p>

138

</li>

139

<li><a name="%24ff-%3eshare"></a><b>$ff->share</b>

140

<p>On systems with the concept of a network share (currently only Windows) returns

141

the sharename from a <a href="file:////">file:////</a> url. On other operating systems returns empty.</p>

142

</li>

143

144

<p>The path from the uri, will be at least a single '/'.</p>

145

</li>

146

147

<p>The name of the remote file. For the local file name, the

148

result of $ff->output_file will be used.</p>

149

</li>

150

<li><a name="%24ff-%3eoutput_file"></a><b>$ff->output_file</b>

151

<p>The name of the output file. This is the same as $ff->file,

152

but any query parameters are stripped off. For example:</p>

153

<pre class="verbatim"> http://example.com/index.html?x=y</pre><p>would make the output file be <code class="inline"><a class="l_k" href="../functions/index.html">index.html</a></code> rather than

154

<code class="inline"><a class="l_k" href="../functions/index.html">index.html?x=y</a></code>.</p>

155

</li>

156

</ul>

157

<a name="METHODS"></a><h1>METHODS</h1>

158

<a name="%24ff-%3d-File%3a%3aFetch-%3enew(-uri-%3d%3e-'http%3a%2f%2fsome.where.com%2fdir%2ffile.txt'-)%3b"></a><h2>$ff = File::Fetch->new( uri => '<a href="http://some.where.com/dir/file.txt">http://some.where.com/dir/file.txt</a>' );</h2>

159

<p>Parses the uri and creates a corresponding File::Fetch::Item object,

160

that is ready to be <code class="inline"><span class="w">fetch</span></code>

161

ed and returns it.</p>

162

<p>Returns false on failure.</p>

163

<a name="%24ff-%3efetch(-%5bto-%3d%3e-%2fmy%2foutput%2fdir%2f%5d-)"></a><h2>$ff->fetch( [to => /my/output/dir/] )</h2>

164

<p>Fetches the file you requested. By default it writes to <code class="inline"><span class="i">cwd</span><span class="s">(</span><span class="s">)</span></code>

165

166

but you can override that by specifying the <code class="inline"><span class="w">to</span></code>

167

argument.</p>

168

<p>Returns the full path to the downloaded file on success, and false

169

on failure.</p>

170

<a name="%24ff-%3eerror(%5bBOOL%5d)"></a><h2>$ff->error([BOOL])</h2>

171

<p>Returns the last encountered error as string.

172

Pass it a true value to get the <code class="inline"><span class="i">Carp::longmess</span><span class="s">(</span><span class="s">)</span></code>

173

output instead.</p>

174

<a name="HOW-IT-WORKS"></a><h1>HOW IT WORKS</h1>

175

<p>File::Fetch is able to fetch a variety of uris, by using several

176

external programs and modules.</p>

177

<p>Below is a mapping of what utilities will be used in what order

178

for what schemes, if available:</p>

179

180

181

<span class="w">ftp</span> <span class="cm">=></span> <span class="w">LWP</span><span class="cm">,</span> <span class="w">Net::FTP</span><span class="cm">,</span> <span class="w">wget</span><span class="cm">,</span> <span class="w">curl</span><span class="cm">,</span> <span class="w">ncftp</span><span class="cm">,</span> <span class="w">ftp</span>

182

<span class="w">rsync</span> <span class="cm">=></span> <span class="w">rsync</span></pre>

183

<p>If you'd like to disable the use of one or more of these utilities

184

and/or modules, see the <code class="inline"><span class="i">$BLACKLIST</span></code>

185

variable further down.</p>

186

<p>If a utility or module isn't available, it will be marked in a cache

187

(see the <code class="inline"><span class="i">$METHOD_FAIL</span></code>

188

variable further down), so it will not be

189

tried again. The <code class="inline"><span class="w">fetch</span></code>

190

method will only fail when all options are

191

exhausted, and it was not able to retrieve the file.</p>

192

<p>A special note about fetching files from an ftp uri:</p>

193

<p>By default, all ftp connections are done in passive mode. To change

194

that, see the <code class="inline"><span class="i">$FTP_PASSIVE</span></code>

195

variable further down.</p>

196

<p>Furthermore, ftp uris only support anonymous connections, so no

197

named user/password pair can be passed along.</p>

198

<p><code class="inline">/bin/ftp</code> is blacklisted by default; see the <code class="inline"><span class="i">$BLACKLIST</span></code>

199

variable

200

further down.</p>

201

<a name="GLOBAL-VARIABLES"></a><h1>GLOBAL VARIABLES</h1>

202

<p>The behaviour of File::Fetch can be altered by changing the following

203

global variables:</p>

204

<a name="%24File%3a%3aFetch%3a%3aFROM_EMAIL"></a><h2>$File::Fetch::FROM_EMAIL</h2>

205

<p>This is the email address that will be sent as your anonymous ftp

206

password.</p>

207

<p>Default is <code class="inline"><span class="w">File</span>-<span class="w">Fetch</span><span class="i">@example</span>.<span class="w">com</span></code>

208

.</p>

209

<a name="%24File%3a%3aFetch%3a%3aUSER_AGENT"></a><h2>$File::Fetch::USER_AGENT</h2>

210

<p>This is the useragent as <code class="inline"><span class="w">LWP</span></code>

211

will report it.</p>

212

<p>Default is <code class="inline"><span class="w">File::Fetch</span>/<span class="i">$VERSION</span></code>

213

.</p>

214

<a name="%24File%3a%3aFetch%3a%3aFTP_PASSIVE"></a><h2>$File::Fetch::FTP_PASSIVE</h2>

215

<p>This variable controls whether the environment variable <code class="inline"><span class="w">FTP_PASSIVE</span></code>

216

217

and any passive switches to commandline tools will be set to true.</p>

218

<p>Default value is 1.</p>

219

<p>Note: When $FTP_PASSIVE is true, <code class="inline"><span class="w">ncftp</span></code>

220

will not be used to fetch

221

files, since passive mode can only be set interactively for this binary</p>

222

<a name="%24File%3a%3aFetch%3a%3aTIMEOUT"></a><h2>$File::Fetch::TIMEOUT</h2>

223

<p>When set, controls the network timeout (counted in seconds).</p>

224

<p>Default value is 0.</p>

225

<a name="%24File%3a%3aFetch%3a%3aWARN"></a><h2>$File::Fetch::WARN</h2>

226

<p>This variable controls whether errors encountered internally by

227

<code class="inline"><span class="w">File::Fetch</span></code>

228

should be <code class="inline"><span class="w">carp</span></code>

229

'd or not.</p>

230

<p>Set to false to silence warnings. Inspect the output of the <code class="inline"><span class="i">error</span><span class="s">(</span><span class="s">)</span></code>

231

232

method manually to see what went wrong.</p>

233

<p>Defaults to <code class="inline"><span class="w">true</span></code>

234

.</p>

235

<a name="%24File%3a%3aFetch%3a%3aDEBUG"></a><h2>$File::Fetch::DEBUG</h2>

236

<p>This enables debugging output when calling commandline utilities to

237

fetch files.

238

This also enables <code class="inline"><span class="w">Carp::longmess</span></code>

239

errors, instead of the regular

240

241

errors.</p>

242

<p>Good for tracking down why things don't work with your particular

243

setup.</p>

244

<p>Default is 0.</p>

245

<a name="%24File%3a%3aFetch%3a%3aBLACKLIST"></a><h2>$File::Fetch::BLACKLIST</h2>

246

<p>This is an array ref holding blacklisted modules/utilities for fetching

247

files with.</p>

248

<p>To disallow the use of, for example, <code class="inline"><span class="w">LWP</span></code>

249

and <code class="inline"><span class="w">Net::FTP</span></code>

250

, you could

251

set $File::Fetch::BLACKLIST to:</p>

252

<pre class="verbatim"> <span class="i">$File::Fetch::BLACKLIST</span> = <span class="s">[</span><span class="q">qw|lwp netftp|</span><span class="s">]</span></pre>

253

<p>The default blacklist is [qw|ftp|], as <code class="inline">/bin/ftp</code> is rather unreliable.</p>

254

<p>See the note on <code class="inline"><span class="w">MAPPING</span></code>

255

below.</p>

256

<a name="%24File%3a%3aFetch%3a%3aMETHOD_FAIL"></a><h2>$File::Fetch::METHOD_FAIL</h2>

257

<p>This is a hashref registering what modules/utilities were known to fail

258

for fetching files (mostly because they weren't installed).</p>

259

<p>You can reset this cache by assigning an empty hashref to it, or

260

individually remove keys.</p>

261

<p>See the note on <code class="inline"><span class="w">MAPPING</span></code>

262

below.</p>

263

<a name="MAPPING"></a><h1>MAPPING</h1>

264

<p>Here's a quick mapping for the utilities/modules, and their names for

265

the $BLACKLIST, $METHOD_FAIL and other internal functions.</p>

266

267

<span class="w">Net::FTP</span> <span class="cm">=></span> <span class="w">netftp</span>

268

269

270

<span class="w">ncftp</span> <span class="cm">=></span> <span class="w">ncftp</span>

271

272

273

<span class="w">rsync</span> <span class="cm">=></span> <span class="w">rsync</span></pre>

274

<a name="FREQUENTLY-ASKED-QUESTIONS"></a><h1>FREQUENTLY ASKED QUESTIONS</h1>

275

<a name="So-how-do-I-use-a-proxy-with-File%3a%3aFetch%3f"></a><h2>So how do I use a proxy with File::Fetch?</h2>

276

<p><code class="inline"><span class="w">File::Fetch</span></code>

277

currently only supports proxies with LWP::UserAgent.

278

You will need to set your environment variables accordingly. For

279

example, to use an ftp proxy:</p>

280

<pre class="verbatim"> <span class="i">$ENV</span>{<span class="w">ftp_proxy</span>} = <span class="q">'foo.com'</span><span class="sc">;</span></pre>

281

<p>Refer to the LWP::UserAgent manpage for more details.</p>

282

<a name="I-used-'lynx'-to-fetch-a-file%2c-but-its-contents-is-all-wrong!"></a><h2>I used 'lynx' to fetch a file, but its contents is all wrong!</h2>

283

284

can only fetch remote files by dumping its contents to <code class="inline"><span class="w">STDOUT</span></code>

285

286

which we in turn capture. If that content is a 'custom' error file

287

(like, say, a <code class="inline">404 handler</code>), you will get that contents instead.</p>

288

<p>Sadly, <code class="inline"><span class="w">lynx</span></code>

289

doesn't support any options to return a different exit

290

code on non-<code class="inline">200 OK</code> status, giving us no way to tell the difference

291

between a 'successfull' fetch and a custom error page.</p>

292

<p>Therefor, we recommend to only use <code class="inline"><span class="w">lynx</span></code>

293

as a last resort. This is

294

why it is at the back of our list of methods to try as well.</p>

295

<a name="Files-I'm-trying-to-fetch-have-reserved-characters-or-non-ASCII-characters-in-them.-What-do-I-do%3f"></a><h2>Files I'm trying to fetch have reserved characters or non-ASCII characters in them. What do I do?</h2>

296

<p><code class="inline"><span class="w">File::Fetch</span></code>

297

is relatively smart about things. When trying to write

298

a file to disk, it removes the <code class="inline"><span class="w">query</span> <span class="w">parameters</span></code>

299

(see the

300

<code class="inline"><span class="w">output_file</span></code>

301

method for details) from the file name before creating

302

it. In most cases this suffices.</p>

303

<p>If you have any other characters you need to escape, please install

304

the <code class="inline"><span class="w">URI::Escape</span></code>

305

module from CPAN, and pre-encode your URI before

306

passing it to <code class="inline"><span class="w">File::Fetch</span></code>

307

. You can read about the details of URIs

308

and URI encoding here:</p>

309

310

<ul>

311

<li><a name="Implement-%24PREFER_BIN"></a><b>Implement $PREFER_BIN</b>

312

<p>To indicate to rather use commandline tools than modules</p>

313

</li>

314

</ul>

315

<a name="BUG-REPORTS"></a><h1>BUG REPORTS</h1>

316

<p>Please report bugs or other issues to <bug-file-fetch@rt.cpan.org<gt>.</p>

317

<a name="AUTHOR"></a><h1>AUTHOR</h1>

318

<p>This module by Jos Boumans <kane@cpan.org>.</p>

319

<a name="COPYRIGHT"></a><h1>COPYRIGHT</h1>

320

<p>This library is free software; you may redistribute and/or modify it

321

under the same terms as Perl itself.</p>

322

</div>

323

324

</div>

325

</div>

326

327

328

329

330

331

</div>

332

<h1>Search:</h1>

333

<p>

334

335

336

337

</form>

338

</p>

339

340

<h2>Labels:</h2>

341

<p>

342

343

</p>

344

345

</div>

346

</div>

347

</div>

348

349

</div>

350

351

</body>

352

</html>

Older »