l i n u x - u s e r s - g r o u p - o f - d a v i s
L U G O D
 
Next Meeting:
January 6: Social gathering
Next Installfest:
TBD
Latest News:
Nov. 18: Club officer elections
Page last updated:
2005 Mar 02 14:28

The following is an archive of a post made to our 'vox-tech mailing list' by one of its subscribers.

Report this post as spam:

(Enter your email address)
Re: [vox-tech] Perl question - How do I use an HTTP header to downloada file [SOLVED]?
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [vox-tech] Perl question - How do I use an HTTP header to downloada file [SOLVED]?



Jay Strauss wrote:
Hi,

Using LWP::UserAgent and its pals, I get back the response (below via Data::Dumper). If I use the same $request object and feed it to www::mechanize it does the right thing and the data file is in the ->content. I can't see www::mech does it, or how to do it properly with LWP::UserAgent

Basically the response is saying, go to this location with this cookie and download the file. How am I supposed to use this header? Am I suppose to feed this header into some other type of object and do a request on it?

Any suggestions?

Thanks
Jay

$VAR1 = bless( {
'_protocol' => 'HTTP/1.1',
'_content' => '<html><head><title>Object moved</title></head><body>^M
<h2>Object moved to <a href=\'/DelayedQuote/QuoteData.dat\'>here</a>.</h2>^M
</body></html>^M
',
'_rc' => '302',
'_headers' => bless( {
'client-response-num' => 1,
'cache-control' => 'private',
'set-cookie' => [

'CBOESiteTrackingID=d58134de-4d88-4d13-9d5e-c831599506a6; expires=Tue, 17-Feb-2105 12:17:19 GMT; path=/',

'WEBTRENDS_ID=; expires=Wed, 16-Feb-2005 12:17:19 GMT; path=/',

'DownLoadError=; expires=Wed, 16-Feb-2005 12:17:19 GMT; path=/',

'QueryData=79D55F9BFFCA04C80D64EFC35FDECCEC487BB6D44EF9493E2214453402DC21A172A00E496D6348D6C2B9CD6E60C0BC172ACE78326718524614957081569C6A566053D27798472A2A5F90F1CBC849A20ED2FDB94956EFC33E; path=/'
],
'location' => '/DelayedQuote/QuoteData.dat',
'date' => 'Thu, 17 Feb 2005 12:17:19 GMT',
'client-peer' => '198.160.148.116:80',
'content-length' => '144',
'x-aspnet-version' => '1.1.4322',
'client-date' => 'Thu, 17 Feb 2005 12:14:20 GMT',
'content-type' => 'text/html; charset=iso-8859-1',
'title' => 'Object moved',
'server' => 'Microsoft-IIS/6.0'
}, 'HTTP::Headers' ),
'_msg' => 'Found',
'_request' => bless( {
'_content' => '__EVENTTARGET=&__EVENTARGUMENT=&NOVIEWSTATE=dDwtODc5MTM4MTUwOzs%2B6wDeQv0gEC342sUPKG%2B3n6ZStJ8%3D&ucHeader%3AucCBOEHeaderLinks%3AucCBOEHeaderSearch%3Asearchtext=&txtTicker=QQQQ&cmdSubmit=Download',
'_uri' => bless( do{\(my $o = 'http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx')}, 'URI::http' ),
'_headers' => bless( {

'user-agent' => 'Windows IE 6',

'content-type' => 'application/x-www-form-urlencoded',

'content-length' => 196
}, 'HTTP::Headers' ),
'_method' => 'POST'
}, 'HTTP::Request' )
}, 'HTTP::Response' );
_______________________________________________
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech


By adding the "referer" things worked out:

#!/usr/bin/perl

use WWW::Mechanize;
use LWP::UserAgent;
use HTML::Form;
use Data::Dumper;

$url    = "http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx";;
$agent  = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)';
$symbol = 'QQQQ';
$ua     = LWP::UserAgent->new(cookie_jar=>{}, agent=>$agent);

push @{ $ua->requests_redirectable }, 'POST';
$resp = $ua->get($url);

$form = HTML::Form->parse($resp->content,$resp->base);
$form->find_input('__VIEWSTATE')->name('NOVIEWSTATE');
$form->value(txtTicker => $symbol);

my $req = $form->click;
$req->header('referer' => $url);

my $resp = $ua->request($req);
print $resp->content;



Jay
_______________________________________________
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech



LinkedIn
LUGOD Group on LinkedIn
Sign up for LUGOD event announcements
Your email address:
facebook
LUGOD Group on Facebook
'Like' LUGOD on Facebook:

Hosting provided by:
Sunset Systems
Sunset Systems offers preconfigured Linux systems, remote system administration and custom software development.

LUGOD: Linux Users' Group of Davis
PO Box 2082, Davis, CA 95617
Contact Us

LUGOD is a 501(c)7 non-profit organization
based in Davis, California
and serving the Sacramento area.
"Linux" is a trademark of Linus Torvalds.

Sponsored in part by:
Appahost Applications
For a significant contribution towards our projector, and a generous donation to allow us to continue meeting at the Davis Library.