Gossamer Forum
Home : General : Perl Programming :

retrieve multiple files - PayPal $20 for solution

Quote Reply
retrieve multiple files - PayPal $20 for solution
Hi,

I need to retrieve multiple files (about 408 to be exact) and wanted to automate the process.

I've used something like this to retrieve pages before:

`wget -Oxml1.xml 'http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=1' `;
`wget -Oxml2.xml 'http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=2' `;
`wget -Oxml3.xml 'http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=3' `;

But I need to go up to

`wget -Oxml408.xml 'http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=408' `;

Since I'm basically a perl virgin, I wasn't sure how to code this, although I believe it should be relatively simple.

The urls above are just examples, the data I want isn't actually there.

Once I get all of these files, I'd like to combine them into one XML file. The first and last files will be different, but I can manually adjust those. So, basically all I'd need to do for docs xml2.xml through xml407.xml is strip everything before the first instance of <Item> and at the bottom </Items></ItemSearchResponse> needs to be stripped off.

So each file would now consist of something like:

<Item>data here</Item>
<Item>data here</Item>
<Item>data here</Item>

Then xml2.xml through xml407.xml needs to be combined to merge all of these items.

I'd gladly paypal $20 to whomever can help me with a solution that works.

Thanks!

Robert
http://www.pcprofiles.com
PC Profiles and hardware reviews

Last edited by:

Robert_B: Aug 27, 2005, 12:03 PM
Quote Reply
Re: [Robert_B] retrieve multiple files - PayPal $20 for solution In reply to
Never mind; crafted up a PHP solution from bits here and there.

Robert
http://www.pcprofiles.com
PC Profiles and hardware reviews
Quote Reply
Re: [Robert_B] retrieve multiple files - PayPal $20 for solution In reply to
How bout shell:

Code:
gossamer-threads@gossamer gossamer-threads $ for i in `seq 1 5`; do echo wget -Oxml$i.xml "http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=$i"; done
wget -Oxml1.xml http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=1
wget -Oxml2.xml http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=2
wget -Oxml3.xml http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=3
wget -Oxml4.xml http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=4
wget -Oxml5.xml http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&ItemPage=5
gossamer-threads@gossamer gossamer-threads $


Change the 5 to a 408, and then if you are happy with how things look, remove the 'echo', and run it.

Cheers,

Alex
--
Gossamer Threads Inc.