Gossamer Forum: General: Perl Programming: Regex Problem...

Jan 30, 2003, 3:36 AM

Andy

Veteran / Moderator (18441 posts)

Jan 30, 2003, 3:36 AM

Post #1 of 13

Shortcut

Regex Problem...

Can anyone suggest some regex to replace this? The aim of it is to grab all URL's starting with http:// in a HTML page.

Code:
  my @html = get($page); 

foreach (@html) { 

     # now scan through to find URL's... 
      m,http://(.+?),g and print $1; 

}

It just doesn't seem to be matching Unsure

Cheers

Andy (mod)
andy@ultranerds.co.uk

Jan 30, 2003, 3:53 AM

Paul

Veteran (19537 posts)

Jan 30, 2003, 3:53 AM

Post #2 of 13

Shortcut

Re: [Andy] Regex Problem... In reply to

Your regex will match any element of @html matching http:// but will match until the very end of the string.

I was playing with URL regexs for one of my own scripts and came up with this...not sure how good it is but it works for my purposes.

Code:
my $html = get($url); 

while ($html =~ m|http://((?:[a-z0-9-]+\.)?[a-z0-9-]+\.[a-z0-9]+(?:\.[a-z0-9\.]*)?)|sg) { 
    print $1; 
}

Last edited by:

Paul: Jan 30, 2003, 3:55 AM

Jan 30, 2003, 4:00 AM

Andy

Veteran / Moderator (18441 posts)

Jan 30, 2003, 4:00 AM

Post #3 of 13

Shortcut

Re: [Paul] Regex Problem... In reply to

Doesn't match anything (i.e no page output Unsure

)

Thanks for the try though :)

Andy (mod)
andy@ultranerds.co.uk

Jan 30, 2003, 4:04 AM

Paul

Veteran (19537 posts)

Jan 30, 2003, 4:04 AM

Post #4 of 13

Shortcut

Re: [Andy] Regex Problem... In reply to

Seems ok for me.

Code:

Code:
    my $url = 'http://www.adsql.com/'; 
    print "Content-type: text/html\n\n"; 
    print $url =~ m|http://((?:[a-z0-9-]+\.)?[a-z0-9-]+\.[a-z0-9]+(?:\.[a-z0-9\.]*)?)|;

Output:

www.adsql.com

Jan 30, 2003, 4:10 AM

Andy

Veteran / Moderator (18441 posts)

Jan 30, 2003, 4:10 AM

Post #5 of 13

Shortcut

Re: [Paul] Regex Problem... In reply to

Web Images Groups Directory News

• Advanced Search
• Preferences
• Language Tools
New! Take your search further. Take a Google Tour.

Advertise with Us - Search Solutions - Services & Tools - Jobs, Press, & Help

©2003 Google - Searching 3,083,324,652 web pages

Spider

Please tick which sites you want to spider, and get the details for....

SELECT ALL?

Thats pretty much what I see (a few extra buttons, but the forum wont let me show them). The code I was using, is;

Code:
  my @html = get($page); 

foreach (@html) { 

  m|http://((?:[a-z0-9-]+\.)?[a-z0-9-]+\.[a-z0-9]+(?:\.[a-z0-9\.]*)?)|sg and print "$1<BR>"; 
  print $_; 
}

As you can see, its a basic adaption of yours, to accommidate the array. However, nothing is printed out URL wise Unsure

Cheers

Andy (mod)
andy@ultranerds.co.uk