it's dmoz.cgi that doesn't seem to work right when it comes to spidering URL's with it.

I will wait for your e-mail, haven't got anything as of yet :)

Thanks btw!

dmoz.cgi doesn't spider anything. it only indexes categories. does it do it after you add a category and its on the success page? I resent the email.

Lavon Russell
LookHard Mods
lavon@lh.links247.net

*I suddenly feel very stupid!*



Erm, I thought it did spider URL's?

So. I can use DMOZ.CGI to get the DMOZ categories but it doesn't actually then spider the sites in those categories?

no, dmoz.cgi is the file that handles which templates show up.
It browses the dmoz category structures
it will put the category you type in in a database.
And deletes records from that database.
gofetch.cgi is the one that does the spidering for both the dmoz cats/and the urls.
Anyway, i may need server ftp access, admin & pass b/c this problem is hard to find, and i saw that there's a bug in your delete function. I haven't seen it anywhere else.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Okay.... have you sent me that e-mail yet as I haven't got anything...

I sent it twice to the email you replied to me with. If i can post attachments in pm, i'll just private message them to you.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Didn't get the e-mails :(

but I got the Private Message with the Attachments - thank you!

(wow, cool forum software!)


----- ooooppppps! ---------

Apologies: my server e-mail was incorrectly set. It's now fixed. They were waiting for me on the server....... Sorry about that!

Hi,

I installed your script earlier on without any real problems.

This time I used your automatic installation feature and I get the curl prompt etc.

Is it possible for you to include a site_html_templates.pl and links.cfg so as to make things easier?

I do appreciate the time and effort that you have put into this script.

IceWater

so you installed the script before so you had everything in links.cfg and site_html_templates taken care of so you didn't need those redone. And now you get a curl unknown tag. That could be with the code i have. Let me check.
oh, btw... when do you see this? only when you try to add categories to the que right?
Lavon Russell
LookHard Mods
lavon@lh.links247.net

ok, i found it. In addurl.html i use <%curl%> but forgot to put in the source for the install under sub ddurl to put curl => $curl,
Just add that under turl => $turl,
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Hi Bmxer,

I do not know what I have done but the site_html_templates.pl is all messed up. Is it possbile for you to send me the complete files which are necessary to run the script. The script is great and I cannot wait to use it but at the moment I have all this 'things' happening.

For instance, when I spider dmoz then I get the following urls: http://www.dmoz.org/Regions and so on. I do not get the actual url for the link in question.

I know that I ask for a lot but I would really appreciate if you could send me the files.

I also understand if you do not wish to send the files.

Many Thanks,

IceWater

can you send me an url to where this happened?
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Find this line
print SPIDER "$ID|$mytitle|http://$url|$lastupd|$FORM{'Category'}|$mydescrip||$email|0|No|No|0|0|Yes\n";

The first occurence of this line. Change the http://$url to http://$myurl . Only make this change for the dmoz part.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Thanks for your reply. It works perfect now.

Many thanks for all your support.

IceWater

no problem
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Hi Bmxer,

Love gofetch2.2.

One question. Using the mod will mean you are bringing in a lot of links from DMOZ. Is there a way to auto validate.

I seefollowing sub in gofetch.cgi

# Print links to spider database

# ------------------------------------

# Open db to add spidered links in

open (SPIDER, ">>$db_valid_name") or &cgierr ("Can't open for output counter file. Reason: $!");

if ($db_use_flock) { flock (SPIDER, 2) or &cgierr ("Can't get file lock. Reason: $!"); }

print SPIDER "$ID|$mytitle|http://$myurl|$lastupd|$FORM{'Category'}|$mydescrip||$email|0|No|No|0|0|No\n";

close SPIDER; # Closing above Opening

}



To make it auto validate, is all I need to change is $db_valid_name to $db_links_name in order to self validate or can you think of some other changes?

Thanks.




Webmaster
http://www.e-bannerx.com

Hi there,

I am having a problem spidering

The error I get is:

<<File Not Found

The requested URL was not found on this server.>>


The location bar says: http://www.kemetonline.com/cgi-bin/links2/admin/gofetch.cgi?categoryinput=arts%2Fliterature%2Fmyths_and_folktales%2Fmyths%2Fegyptian%2Fgods_and_goddesses&Category=AE_Netjeru_Deities&email=webmaster%40kemetonline.com&=Spider

So I have dmoz.cgi here:
http://kemetonline.com/cgi-bin/links2/dmoz.cgi
I scanned in some categories

Then I go to gofetch.cgi:

http://kemetonline.com/...n/links2/gofetch.cgi

There is now a pull down list of spidered categories and another to choose categories in my database to load them into

Thanks, Again. In advance

Thanks, Yes that would be all that's necessary.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

I don't understand. When i went to your urls everything seemed fine.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Yes, I thought all was fine also.
However it doesn't seem to actually spider the urls and add them to the database

example
I go to my gofetch.cgi
http://kemetonline.com/...n/links2/gofetch.cgi
I select the first category
(which is arts/literature/mths_and-folktales/myths/egyptian/gods-and-goddesess)

I choose category for import: AE_Netjeru_Deities

put in an email address..renee@kemetonline.com

and press spider

Nothing happens except that I get brought to an url not found page...and on return to the gofetch cgi page I see:
Error no category selected. You must have fetched without
choosing a category

Renee

Well, I'll say you should update to 2.2 because you're using 2.0 and 2.2. And I saw all the cgi-bin/ urls setup to spider search and you should know gofetch wasn't setup to do that. It will only spider the first page of results and thats it. I could add something like this in 3.0 since it's pretty much done but I just haven't released it yet.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Bmxer,

Have you thought of adapting your gofetch scripts to be able to go through the clickbank database (adding your seller code) and then adding to your links database?

I have no idea what is possible and what is not (zero idea on programming over here) but just a thought......

By the way I love gofetch.....but I will now have to upgrade to sql. (dammm)

Thanks

Fulcan




Webmaster
http://www.e-bannerx.com

Never thought about clickbank. Don't really know what it is, but I can take a look.

I would have rather made gofetch for sql but I don't know it yet, and don't have Links SQL , but Imagine you can still have it work. Just keep an installation of Links 2 to get the links in a text file, and then I'm sure there a scripts to put the text data into an sql table.
Lavon Russell
LookHard Mods
lavon@lh.links247.net

Hi Bmxer,

If click bank could be done it would br great. I have tested click bank by adding a number of links to my directory already and what is good about it is that you earn a commission each time some clicks the link and then purchases. So far (over four months ) I have received two commissions this way (not a lot) but now that I can add more links in my directory and hopefully therefore get more traffic from google that will change.

Clickbank can be found here. Just go to Affiliate programs and then go to clickbank hoplinks.

It may be a small way for us all to make a few bucks on the side from our directories...

Regards

Fulcan




Webmaster
http://www.e-bannerx.com

not quite sure whats wrong with the setup but I'm getting this error...Error including libraries: syntax error at site_html_templates.pl line 39, near "db_dmoz_url"Compilation failed in require at dmoz.cgi line 35.Make sure they exist, permissions are set properly, and paths are set correctly.