Gossamer Forum
Home : Products : Gossamer Links : Development, Plugins and Globals :

New plugin: thumbnail preview of web pages

Quote Reply
New plugin: thumbnail preview of web pages
Hello,
I've just finished "Thumbnail" plugin. It show thumbnail preview of web pages. Images are from Thumbnail.cz. You can use preview of links in DMOZ for free.
More information: http://www.thumbnail.cz/glinks.html
Quote Reply
Re: [mirek] New plugin: thumbnail preview of web pages In reply to
Nice plugin Cool


I checked your site earlier, and I always wanted to know how do you take automatic website snapshots (website rendering)? Angelic


Ok-ok, I know, it should be some patented technology, as there are not too much services with such feature, and there are a lot, who would like to have such service... Wink

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [webmaster33] New plugin: thumbnail preview of web pages In reply to
Quote:
I checked your site earlier, and I always wanted to know how do you take automatic website snapshots (website rendering)? Angelic

We use Win32::OLE, and then a Screenshot capturing module <G> (has to be on an NT machine though).

Obviously, you could use an X-Server on a UNIX machine, but with the M$ setup, you make sure that it all comes out exactly how it was intereded to by the owner Smile

Cheers

Andy (mod)
andy@ultranerds.co.uk
Want to give me something back for my help? Please see my Amazon Wish List
GLinks ULTRA Package | GLinks ULTRA Package PRO
Links SQL Plugins | Website Design and SEO | UltraNerds | ULTRAGLobals Plugin | Pre-Made Template Sets | FREE GLinks Plugins!
Quote Reply
Re: [Andy] New plugin: thumbnail preview of web pages In reply to
Basically you start an IE, then capture the browser content?

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [webmaster33] New plugin: thumbnail preview of web pages In reply to
We use Linux machines for rendering (=> no IE). But we also use browser, which render page. And then the page is saved to jpg. Result is same as screenshot, but technically it is not screenshot. This approach is also better, because we know when page is loaded. This is much faster.
Quote Reply
Re: [mirek] New plugin: thumbnail preview of web pages In reply to
Ah, so you are using the Xvfb or xwd way?

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [mirek] New plugin: thumbnail preview of web pages In reply to
Just found an activex plugin, which can does the same under Windows, too.

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [webmaster33] New plugin: thumbnail preview of web pages In reply to
Most of these "capture" programs use a screen capture after the browser has rendered the image to the screen.

There are a number of perl modules that can do this now, some (most) run under ActiveState Perl (windows).

We've been using one on XP (home) for awhile now, but windows has a tendency to hang. There are some conditions that cause unknown windows errors, and all sorts of other issues.

It would be great to be able to render the image to a buffer, not a screen, and then capture that buffer. By bypassing the video driver completely you'd increase stability significantly.

I've been hoping that maybe the mozilla project or one of the other open source browser products would spin off a rendering engine interface just for this purpose.

Right now, most of the tools use the MSIE OLE(?) module to do it. (in otherwords, most run on Windows).

We've talked with alex about setting up a machine to do this on Linux with Xwindows(?) but security and costs have been an issue. It *almost* happened a couple of months ago. Had a client that needed a huge number of thumbnails, and a regularly updated system, but they fizzled out.


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: [pugdog] New plugin: thumbnail preview of web pages In reply to
Thanks for the detailed info.

Anyway, doing HTML to IMAGE conversion is interesting subject with high demand for such feature, so I believe, there will be sometime a cross-os solution. Yes, likely within the Mozilla project.

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [webmaster33] New plugin: thumbnail preview of web pages In reply to
In Reply To:
Ah, so you are using the Xvfb or xwd way?
Neither one. We are using konqueror, which is slightly modified. It is benefit of open source :). You can not really do it with IE until you are MS. So when we can use konqueror core, there is no need for xwd or similar screenshot utility. We render page directly from buffer. However we still use Xserver (but not Xvfb) for some weird pages.
Quote Reply
Re: [pugdog] New plugin: thumbnail preview of web pages In reply to
In Reply To:
We've talked with alex about setting up a machine to do this on Linux with Xwindows(?) but security and costs have been an issue. It *almost* happened a couple of months ago. Had a client that needed a huge number of thumbnails, and a regularly updated system, but they fizzled out.
Yes, it is not problem to make screenshots yourself. But every utility for making thumbnail preview of web pages is *SLOW*. Even if you build dedicated server, it take whole month to generate thumbnails for small catalogue with 1000 links. And you spend nearly same amount of money on electricity as on subscription on Thumbnail.cz. If you will count price of hardware and your time, you get much higher cost.
And if you want to make faster system, you must spend a lot of time (and money) in development. And the results are still unsure.
Quote Reply
Re: [mirek] New plugin: thumbnail preview of web pages In reply to
I was calculating, if I'd do link directory, how much income would be possible, and how much would cost to get thumbnails for the website. Unfortunately the result was, that I could not afford so high prices you use. It's not your fault, just the internet business nowadays doesn't pays so well to afford such high, monthly recurring costs.

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...
Quote Reply
Re: [mirek] New plugin: thumbnail preview of web pages In reply to
Quote:
Yes, it is not problem to make screenshots yourself. But every utility for making thumbnail preview of web pages is *SLOW*. Even if you build dedicated server, it take whole month to generate thumbnails for small catalogue with 1000 links. And you spend nearly same amount of money on electricity as on subscription on Thumbnail.cz. If you will count price of hardware and your time, you get much higher cost.


I don't know where you get those numbers, about a year ago, when I started this, I posted that even with the beta program I had, and the lousy "hack" I made, I could do over 300 screen shots an hour, _more_ if the system behaved. The only problem was I had to have someone sitting at the console, since every so often a website would cause the system to "explode" and it would not reset.

I can do about 500/hour now, but still, it requires a human to watch it.

It can be done on a *very* slow/old machine, even on windows 98.

If I had the resources, I could set up 5 or 10 old machines in a "console" like I used to have years ago (before the Internet ,when every machine had to have it's own dial-up, and it was cheaper and faster to use more machines than one fast one), and have a kid sit in the middle of them playing video games, or doing homework, resetting them as they hung.

Believe me, it's not generation of the images that is the big problem, it's the _AUTOMATED_ process that is not good.

In doing a "free" service, to serve the images for other sites, then it gets into bandwidth costs, etc.

I've never looked at it that way. I've always wanted to create my "NexusLinks" directory, populate it with the images, maintain themm, allow people to add/recieve links to populate their niche directories, etc.

I've gotten one step closer, with my keyword search/dump.


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: [webmaster33] New plugin: thumbnail preview of web pages In reply to
Quote:
I was calculating, if I'd do link directory, how much income would be possible, and how much would cost to get thumbnails for the website. Unfortunately the result was, that I could not afford so high prices you use. It's not your fault, just the internet business nowadays doesn't pays so well to afford such high, monthly recurring costs.


We posted prices, I think they are still posted on http://UltraNerds.com for us to do it for you. We did restrict, at that time, to directories under 10,000 links, for the problems mentioned above.

But, as time has moved on, we are willing to expand the services, *and* we can create the thumbnails for you in any size you want, and with or without a black border. (I think http://origami.net or http://pugs.net have them mixed up in the directory).

We can also restrict it to only links that do not show an image from the OTP/Thumbshots project.

We need access to your site, though, since we run some scripts that check images, *AND* we store _all_ the images on your site, in a cache, so you need the disk space.

At the expense of diskspace, we've found performance is _significantly_ better than relying on outside servers.

We also can tune a set up to your needs.

For a *lot* of reasons, we like to consider the image cache to be a real part of your directory, not an add on. You have links that are not in the DMOZ database, and those images will not be served by the OTP server.

Also, and this is where the NexusLinks comes in, we will add your links to our database, and update the images regularly. You'll be able to "fetch" updated copies of the images using a plugin. This has not been developed yet, but it's been planned, and has two different development concepts. Each has benefits, and both may be implemented.

The reason for this is two-fold. Image generation software can kick the #*%Q out of a server, and it consumes huge amounts of memory, and resources, that have nothing to do with web serving. Moving the image generation programs off, to other servers makes a huge amount of sense. The other reason is that most directories are most concerned with acquiring new links, content, and data, and the screen shots are not really something most webmasters want to have to worry about. If you add a link, and the thumbnail is missing, the program should automatically check our database, if it's there, it fetches it, if not, it queues the request, and the image is generated. The program will cache a blank image, that expires in 24-hours, and will attempt a re-fetch. This keeps requests to our servers down, and hopefully within 24-48 hours the new thumbnail will be ready for you.

Additionally, check the next post for more information on the Nexus Links project.


PUGDOG� Enterprises, Inc.

The best way to contact me is to NOT use Email.
Please leave a PM here.
Quote Reply
Re: [mirek] New plugin: thumbnail preview of web pages In reply to
In Reply To:
Hello,
I've just finished "Thumbnail" plugin. It show thumbnail preview of web pages. Images are from Thumbnail.cz. You can use preview of links in DMOZ for free.
More information: http://www.thumbnail.cz/glinks.html
Very nice.

One question - how can I make the preview image a link to the site?
Blacknight Solutions - Hosting
Search.ie
Armchair.ie
Quote Reply
Re: [blacknight] New plugin: thumbnail preview of web pages In reply to
Another thing I've just spotted is that I'm getting "Preview by Thumbnail.cz" output when no thumbnail is present instead of a transparent image....
Blacknight Solutions - Hosting
Search.ie
Armchair.ie
Quote Reply
Re: [blacknight] New plugin: thumbnail preview of web pages In reply to
In Reply To:
One question - how can I make the preview image a link to the site?
I'm using this code:

Code:
<a href="<%URL%>" OnClick="location.href='<%db_cgi_url%>/jump.cgi?ID=<%ID%>'; return false;"><%Plugins::Thumbnail::display($URL)%></a>
This code counts popularity of links in JavaScript enabled browser, but people can see target URL in status line and is also nice to old browser.
I can put it next release of Thumbnail plugin.
Quote Reply
Re: [blacknight] New plugin: thumbnail preview of web pages In reply to
In Reply To:
Another thing I've just spotted is that I'm getting "Preview by Thumbnail.cz" output when no thumbnail is present instead of a transparent image....
Where? I do not see this behavior.
Quote Reply
Re: [mirek] New plugin: thumbnail preview of web pages In reply to
Have a look at:
http://www.browse.ie/Computers_and_Internet/index.html
Blacknight Solutions - Hosting
Search.ie
Armchair.ie
Quote Reply
Re: [blacknight] New plugin: thumbnail preview of web pages In reply to
In Reply To:
Have a look at:
http://www.browse.ie/Computers_and_Internet/index.html
Oops, I see.
I repaired the bug.
Quote Reply
Re: [mirek] New plugin: thumbnail preview of web pages In reply to
Do I need to do anything on my end?
Blacknight Solutions - Hosting
Search.ie
Armchair.ie
Quote Reply
Re: [blacknight] New plugin: thumbnail preview of web pages In reply to
In Reply To:
Do I need to do anything on my end?
No. As far as I can see. On your pages is displayed 1*1 transparent pixel when there is no thumbnail.
Quote Reply
Re: [webmaster33] New plugin: thumbnail preview of web pages In reply to
I thought you might be interested in this:


http://andy.5263.org/screengrab/


Best regards,

Erich
Quote Reply
Re: [erichcyber] New plugin: thumbnail preview of web pages In reply to
Thanks Erich! Nice hit! Cool

Best regards,
Webmaster33


Paid Support
from Webmaster33. Expert in Perl programming & Gossamer Threads applications. (click here for prices)
Webmaster33's products (upd.2004.09.26) | Private message | Contact me | Was my post helpful? Donate my help...