Gossamer Forum
Quote Reply
Spider Job
Dear sir:

I am looking for a simple script that can spider a specific list of links (FFAP), get the description, title, the last modified date -will be set as add_date- and webmaster e-mail off the each link, then write the information into a file in an SQL format (validate mod) so I can import it into my database.

(The category will be a 0 since it will be assigned manually upon visiting the site).

Regards

Christa

Quote Reply
Re: Spider Job In reply to
Hi Christa,

This shouldn't be too difficult.

However, would you mind sending me more details on the script?

Specifically:
1. How are the links stored? (In a single file or in database?)
2. Do you have an idea how you'd like us to get the add_date/modified_date and webmaster email?
3. Are the spidered links going to be entered into Links for later validation?

Then we can get a better idea of how you would like us to put it all together and we can provide you with an estimate on how much and how long it will take.

You can post here or email me directly at aki@gossamer-threads.com.

Cheers,
Aki

Quote Reply
Re: Spider Job In reply to
Hello Back

1. How are the links stored? (In a single file or in database?)

they will be stored in s ingle file to be imported through SQL into validate table latter.
2. Do you have an idea how you'd like us to get the add_date/modified_date and webmaster email?
I was thinking about assigning the last modified date on the page as the default date.
Or if that is too hard we can give the option to enter a default date before running the script.
(you just made me think about giving the option to enter a default script before running te script, that will be great if you are spidering a page with links that will fit in one category).
3. Are the spidered links going to be entered into Links for later validation?

Yes they will

regards
christa

Quote Reply
Re: Spider Job In reply to
Hi Christa,

The job would cost approximately 250$US.

If you are interested, send me an email to discuss specifics and we will start the project ASAP!

Cheers,
Aki (aki@gossamer-threads.com)