Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ModPerl: ModPerl

Framework for handling Query tasks that take a LONG time

 

 

ModPerl modperl RSS feed   Index | Next | Previous | View Threaded


ichudov at gmail

Jul 22, 2012, 8:20 PM

Post #1 of 5 (1001 views)
Permalink
Framework for handling Query tasks that take a LONG time

I am working on my internal IT system for my business. It is, of course,
mod_perl based, as is the rest of what I do (algebra.com etc).

As a part of it, I have an intranet web form that starts a task, which
happens to take a long time, such as minutes.

I would like to handle this correctly.

While I read some helpful suggestions, my first instinct would be to look
for some existing framework based on mod_perl, instead of trying to
reinvent the wheel.

Would anyone happen to have a suggestion of such a perl module or framework
for executing long queries in a nice and graceful manner?

Specifically:

1) Start a long query
2) Display an intermediate page that self refreshes
3) Display progress of a task
4) Displaying correct results when task completes.

Thanks!

i


torsten.foertsch at gmx

Jul 23, 2012, 3:12 AM

Post #2 of 5 (954 views)
Permalink
Re: Framework for handling Query tasks that take a LONG time [In reply to]

On 07/23/2012 05:20 AM, Igor Chudov wrote:
> I am working on my internal IT system for my business. It is, of course,
> mod_perl based, as is the rest of what I do (algebra.com etc).
>
> As a part of it, I have an intranet web form that starts a task, which
> happens to take a long time, such as minutes.
>
> I would like to handle this correctly.
>
> While I read some helpful suggestions, my first instinct would be to look
> for some existing framework based on mod_perl, instead of trying to
> reinvent the wheel.
>
> Would anyone happen to have a suggestion of such a perl module or framework
> for executing long queries in a nice and graceful manner?
>
> Specifically:
>
> 1) Start a long query
> 2) Display an intermediate page that self refreshes
> 3) Display progress of a task
> 4) Displaying correct results when task completes.

I have one I have written for a client. I'll ask if I can publish it. If
they approve it it will need a bit of renaming. Also, currently the
documentation is in German.

Torsten


Alexander.Elgert at external

Jul 23, 2012, 5:39 AM

Post #3 of 5 (962 views)
Permalink
AW: Framework for handling Query tasks that take a LONG time [In reply to]

In my opinion, this is the correct way.

There are many web applications, whichs polls every few seconds to give the server a hint, that the client is interested in the answer.
(This way you are able to abort a long_running_whatever.)
On newer applications, JavaScript is used, of course - so you can do what you like in the meanwhile, without any need to update the page itself.
As soon as the call comes back, you can call an update function to handle the output.

There are progress bars available too:
http://docs.jquery.com/UI/Progressbar

Greetings,
Alexander

--
Deutsche Telekom AG
Seamless ICT Security Infrastructure & Management
im Auftrag T-Systems International GmbH
Dipl. Inf Alexander Elgert
Langwadener Strasse 17
64625 Bensheim
+49 176 22 717 661 (Mobil)
+49 671 83419-12 (Tel)
+49 671 83419-30 (Fax)
E-Mail: alexander.elgert [at] gmx

________________________________________
Von: Igor Chudov [ichudov [at] gmail]
Gesendet: Montag, 23. Juli 2012 05:20
An: Mod_Perl
Betreff: Framework for handling Query tasks that take a LONG time

I am working on my internal IT system for my business. It is, of course, mod_perl based, as is the rest of what I do (algebra.com etc).

As a part of it, I have an intranet web form that starts a task, which happens to take a long time, such as minutes.

I would like to handle this correctly.

While I read some helpful suggestions, my first instinct would be to look for some existing framework based on mod_perl, instead of trying to reinvent the wheel.

Would anyone happen to have a suggestion of such a perl module or framework for executing long queries in a nice and graceful manner?

Specifically:

1) Start a long query
2) Display an intermediate page that self refreshes
3) Display progress of a task
4) Displaying correct results when task completes.

Thanks!

i


Alexander.Elgert at external

Jul 23, 2012, 12:10 PM

Post #4 of 5 (953 views)
Permalink
AW: Framework for handling Query tasks that take a LONG time [In reply to]

In my opinion, this is the correct way.

There are many web applications, whichs polls every few seconds to give the server a hint, that the client is interested in the answer.
(This way you are able to abort a long_running_whatever.)
On newer applications, JavaScript is used, of course - so you can do what you like in the meanwhile, without any need to update the page itself.
As soon as the call comes back, you can call an update function to handle the output.

There are progress bars available too:
http://docs.jquery.com/UI/Progressbar

I know, this is not a complete solution to your question, but just a little hint.

Greetings,
Alexander


--
Deutsche Telekom AG
Seamless ICT Security Infrastructure & Management
im Auftrag T-Systems International GmbH
Dipl. Inf Alexander Elgert
Langwadener Strasse 17
64625 Bensheim
+49 176 22 717 661 (Mobil)
+49 671 83419-12 (Tel)
+49 671 83419-30 (Fax)
E-Mail: alexander.elgert [at] gmx

________________________________________
Von: Igor Chudov [ichudov [at] gmail]
Gesendet: Montag, 23. Juli 2012 05:20
An: Mod_Perl
Betreff: Framework for handling Query tasks that take a LONG time

I am working on my internal IT system for my business. It is, of course, mod_perl based, as is the rest of what I do (algebra.com etc).

As a part of it, I have an intranet web form that starts a task, which happens to take a long time, such as minutes.

I would like to handle this correctly.

While I read some helpful suggestions, my first instinct would be to look for some existing framework based on mod_perl, instead of trying to reinvent the wheel.

Would anyone happen to have a suggestion of such a perl module or framework for executing long queries in a nice and graceful manner?

Specifically:

1) Start a long query
2) Display an intermediate page that self refreshes
3) Display progress of a task
4) Displaying correct results when task completes.

Thanks!

i


ichudov at gmail

Jul 26, 2012, 5:43 AM

Post #5 of 5 (945 views)
Permalink
Re: Framework for handling Query tasks that take a LONG time [In reply to]

Hi Guys,

I wanted to say thank you for your support and ideas.

I did not find something that I liked as much as to adopt it right away.

So, I wrote my own module for this, which will, no duobt, evolve some more.

I may open source it.

The way it works is as follows:

1) The initiating web page starts a long task by executing a call to
start_long_task, passing it a ref to a subroutine to be executed that could
take a long time.

A few things happen:
* Process forks
* Output to STDOUT is redirected to a variable
* An IFRAME tag is output, pointing to the monitoring script (see below).
* The subref is executed
* When it finishes, it writes the output from the variable to
file /tmp/longtask.<taskid>

2) The monitoring script is a two liner:

#!/usr/bin/perl
use LongWebTask; serve_long_web_task;

serve_long_web_task is the subroutine that handles everything.

* The monitoring script is watching the output in /tmp/longtask.<taskid>
file. It keeps refreshing the iframe content until it finds said file, at
which point it outputs its content and deletes temporary files.

I am including the source here, but it is a work in progress. I would like
to hear comments as to whether this would be appropriate for CPAN. I have
another module there called Net::eBay.

#!/usr/bin/perl

use strict;
use warnings;

package LongWebTask;

use Time::HiRes qw( gettimeofday );
use Digest::MD5 qw( md5_hex );
use CGI;
use File::Slurp;

use strict;

require Exporter;
use vars qw($VERSION @ISA @EXPORT @EXPORT_OK %EXPORT_TAGS);
@ISA = qw(Exporter);
$VERSION = 2000.0821;

@EXPORT= qw(
start_long_web_task
serve_long_web_task
);

sub start_long_web_task {
my ($cgi, $options, $subref, @args) = @_;

my $progress_url = $options->{progress_url}
|| die "progress_url parameter not defined, cannot create iframe";
my $refresh_frequency = $options->{refresh_frequency} || 10;

my ($seconds, $microseconds) = gettimeofday;
my $randstr = $$ . $seconds . $microseconds . rand( 1E9 );

my $jobid = md5_hex( $randstr );

my $iframe_width = $options->{iframe_width} || "90%";
my $iframe_height = $options->{iframe_height} || "1000";

print "<IFRAME SRC=\"$progress_url?jobid=$jobid\&action=show\"
WIDTH=$iframe_width HEIGHT=$iframe_height></IFRAME>";

if ( fork ) { # Parent

} else { # Child

my $output = "";
close STDOUT;
open STDOUT, ">", \$output;

my $rc = &$subref( @args );

close STDOUT;

my $outfn = outputfn( $jobid );
write_file( "$outfn.new", $output );
rename( "$outfn.new", $outfn );

exit( $rc );
}
}

sub outputfn {
my ($jobid) = @_;
return "/tmp/longtask.$jobid.output";
}

sub progressfn {
my ($jobid) = @_;
return "/tmp/longtask.$jobid.progress";
}

sub serve_long_web_task {

my $cgi = new CGI;

my $jobid = $cgi->param( 'jobid' )
|| die "$0: Parameter jobid is not defined in serve_long_web_task";
my $action = $cgi->param( 'action' )
|| die "$0: Parameter action is not defined in serve_long_web_task";

$jobid =~ /^\w+$/
or die "Job id is illegal: '$jobid'";

if ( $action eq 'show' ) {

print $cgi->header;

my $outfn = outputfn( $jobid );

if ( -e $outfn ) {

print read_file( $outfn );
unlink( outputfn ( $jobid ) );
unlink( progressfn( $jobid ) );

} else {
print $cgi->start_html(
'-head' => $cgi->meta(
{
-http_equiv =>
'Refresh',
-content => 3,
}
),
);
print "Job is in progress. <BR>\n" . `date `;
}


}
}

1;

ModPerl modperl RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.