Mailing List Archive: ModPerl: ModPerl

Adding Post Data to a SubRequest

Index | Next | Previous | View Threaded

jordan at viviotech

Dec 14, 2011, 4:03 PM

Post #1 of 7 (629 views)
Permalink

Adding Post Data to a SubRequest

I've been working to find a solution to adding Post data to a sub
request for a few days now (since last week when I posted about
retrieving the POST data in the first place).

Adding the POST data to the $subr object using write doesn't work,
because it tries to add it to the output response and not the input request.

I then found a message on google that indicated that you could modify
the incoming request POST data via a filter, so I've written something
similar to the following:

------------------------------------------
use Apache2::SubRequest ();
use Apache2::ServerUtil ();
use Apache2::RequestUtil ();
use Apache2::RequestRec ();
use Apache2::RequestIO ();
use Apache2::Filter ();
use Apache2::Log ();
use APR::Table ();
use APR::Brigade ();
use APR::Bucket ();

sub handler {
my $r = shift;
our @ProxyInputContent = ();

sub proxy_input_filter {
my ( $f, $bb ) = @_;
my $ba = $bb->bucket_alloc();
my $newBucket = '';
# remove the existing buckets
$bb->cleanup;
# create buckets using the orig request POST data
foreach ( @ProxyInputContent ) {
# the current array element will be represented as $_
# create a new bucket
$newBucket = APR::Bucket->new($ba, $_);
# put the new bucket in the subr brigade
$bb->insert_head($newBucket);
}
# the subrequest should now contain the post data
return Apache2::Const::OK;
}

# handle POST requests
if ( $r->method() eq "POST" ) {

# set the subrequest method to also be a post
$subr->method('POST');

# create a post data buffer
my $PostBuffer = '';

# loop over each line of data
while($r->read($PostBuffer, 1024)) {

# add the content to the ProxyInputContent Array
push(@ProxyInputContent, $PostBuffer);
}

# make the subrequest go through the input filter so we can add the
post data to it.
$subr->add_input_filter( \&proxy_input_filter );

}

# perform the proxy request
$subr->run;
------------------------------------------

However, this doesn't work. I still don't see the POST data on the
Tomcat side.

And, being the noob I am, I haven't yet figured out how to log the data
from the filter to I can make sure I'm doing it right.

So, my questions are simple:

1) Is there an ideal way to add POST data to a subrequest that I'm
missing here?

2) Is it possible to log data from a filter so I can debug it?

Thank you *so much* for your help and patience with this. I truly
appreciate your time.

--
Warm Regards,
Jordan Michaels

torsten.foertsch at gmx

Dec 15, 2011, 1:23 AM

Post #2 of 7 (594 views)
Permalink

Re: Adding Post Data to a SubRequest [In reply to]

On Wednesday, 14 December 2011 16:03:18 Jordan Michaels wrote:
> 1) Is there an ideal way to add POST data to a subrequest that I'm
> missing here?

No, subrequests don't have a request body by definition. It's hard coded in
httpd 2.2.x.

Enclosed you find a patch for httpd 2.2.21 that makes is possible to attach
the request body of the main request to a subreq made by mod_includes . The intent at the time of writing of the patch was
to send a POST request via mod_proxy to a backend server passing the original
req body where the subreq was initiated by the mod_include response handler.
It turned out to be quite tricky.

Maybe it helps.

Torsten Förtsch

--
Need professional modperl support? Hire me! (http://foertsch.name)

Like fantasy? http://kabatinte.net

Attachments:

virtual_include_method-2.2.21.patch (12.7 KB)

jordan at viviotech

Dec 15, 2011, 12:09 PM

Post #3 of 7 (598 views)
Permalink

Re: Adding Post Data to a SubRequest [In reply to]

Aw, that's terribly sad news. No wonder I couldn't get it to work! lol

That's awesome about the patch, but the module I'm working is intended
to be distributed, and expecting end users to compile in a patch would
probably too much to ask of them.

The module I'm working on updates the headers and proxies the requests
for specific file types off to Tomcat HTTP and AJP ports. I suppose I
can look into using LWP::UserAgent for this kind of work instead of a
subrequest, I just don't want to have to "re-invent the wheel" of trying
to do things mod_proxy_http and mod_proxy_ajp are already doing.

I wonder if I can set the hander for the LWP::UserAgent request to be
the proxy_handler like we're doing with the subrequests now... hmmm...

Thank you very much for your help Torsten. I really appreciate your
wisdom and experience in this area.

Warm Regards,
Jordan Michaels

On 12/15/2011 01:23 AM, Torsten Förtsch wrote:
> On Wednesday, 14 December 2011 16:03:18 Jordan Michaels wrote:
>> 1) Is there an ideal way to add POST data to a subrequest that I'm
>> missing here?
>
> No, subrequests don't have a request body by definition. It's hard coded in
> httpd 2.2.x.
>
> Enclosed you find a patch for httpd 2.2.21 that makes is possible to attach
> the request body of the main request to a subreq made by mod_includes. The intent at the time of writing of the patch was
> to send a POST request via mod_proxy to a backend server passing the original
> req body where the subreq was initiated by the mod_include response handler.
> It turned out to be quite tricky.
>
> Maybe it helps.
>
> Torsten Förtsch
>

torsten.foertsch at gmx

Dec 15, 2011, 12:17 PM

Post #4 of 7 (594 views)
Permalink

Re: Adding Post Data to a SubRequest [In reply to]

On Thursday, 15 December 2011 12:09:05 Jordan Michaels wrote:
> The module I'm working on updates the headers and proxies the requests
> for specific file types off to Tomcat HTTP and AJP ports.

Then maybe you don't need subrequests at all. A translation handler may be
sufficient. Even an internal redirect will do if invoked before the response
phase. (The core response handler calls ap_discard_request_body.)

Torsten Förtsch

--
Need professional modperl support? Hire me! (http://foertsch.name)

Like fantasy? http://kabatinte.net

jordan at viviotech

Dec 15, 2011, 12:40 PM

Post #5 of 7 (593 views)
Permalink

Re: Adding Post Data to a SubRequest [In reply to]

Well, I do need mod_rewrite to be run. A lot of our users use SES url's
and then use rewrite rules to parse them out. I also have specific
requests to leave .htaccess functionality in there (for secured
directories and so forth).

So yeah, the more typical Apache functionality we can leave in the
better. It's a pretty tall order.

I've configured my perl module using AddHandler in Apache, like so:

AddHandler perl-script [my file extensions]
PerlRequire /path/to/mymod.pm
PerlHandler mymod

and that seems to allow me to do everything I'm needing to do so far,
but I admit I haven't tested it thoroughly so far. I'm still trying to
get POST data passed.

Thank you again, very much, for your help!

Warm Regards,
Jordan Michaels

On 12/15/2011 12:17 PM, Torsten Förtsch wrote:
> On Thursday, 15 December 2011 12:09:05 Jordan Michaels wrote:
>> The module I'm working on updates the headers and proxies the requests
>> for specific file types off to Tomcat HTTP and AJP ports.
>
> Then maybe you don't need subrequests at all. A translation handler may be
> sufficient. Even an internal redirect will do if invoked before the response
> phase. (The core response handler calls ap_discard_request_body.)
>
> Torsten Förtsch
>

aw at ice-sa

Dec 15, 2011, 11:40 PM

Post #6 of 7 (592 views)
Permalink

Re: Adding Post Data to a SubRequest [In reply to]

Hi.
If it may contribute something to the question :
I have a case where, within a response handler, I need to make a call to a back-end Tomcat
through mod_jk (a simple GET, not a POST). I initially tried to do this through a
sub-request, and was getting segfaults in Apache for my trouble.
Rather than investigating this, I switched to making the call through LWP, and it has been
working fine for a couple of years, on a variety of systems and Apache versions.

Jordan Michaels wrote:
> Well, I do need mod_rewrite to be run. A lot of our users use SES url's
> and then use rewrite rules to parse them out. I also have specific
> requests to leave .htaccess functionality in there (for secured
> directories and so forth).
>
> So yeah, the more typical Apache functionality we can leave in the
> better. It's a pretty tall order.
>
> I've configured my perl module using AddHandler in Apache, like so:
>
> AddHandler perl-script [my file extensions]
> PerlRequire /path/to/mymod.pm
> PerlHandler mymod
>
> and that seems to allow me to do everything I'm needing to do so far,
> but I admit I haven't tested it thoroughly so far. I'm still trying to
> get POST data passed.
>
> Thank you again, very much, for your help!
>
> Warm Regards,
> Jordan Michaels
>
> On 12/15/2011 12:17 PM, Torsten Förtsch wrote:
>> On Thursday, 15 December 2011 12:09:05 Jordan Michaels wrote:
>>> The module I'm working on updates the headers and proxies the requests
>>> for specific file types off to Tomcat HTTP and AJP ports.
>>
>> Then maybe you don't need subrequests at all. A translation handler
>> may be
>> sufficient. Even an internal redirect will do if invoked before the
>> response
>> phase. (The core response handler calls ap_discard_request_body.)
>>
>> Torsten Förtsch
>>
>

jordan at viviotech

Dec 16, 2011, 12:28 PM

Post #7 of 7 (589 views)
Permalink

Re: Adding Post Data to a SubRequest [In reply to]

Interesting. I appreciate the feedback! So far I have basic proxying
working nicely in LWP, so I'm only delayed about a week.

I ran into Apache segfaults too when using subrequests. I debugged it by
adding logging points throughout my code, and if a segfault was hit
after a specific log point, I knew the segfault was caused by the code
that came after the one particular log point.

In my case, it turned out to be how I was passing subrequest headers off
to a filter. I was able to replace the filter with a while loop, and
that got rid of the segfaults for me. I realize not everyone can replace
their filters with while loops, but that's what worked for me!

Thanks again, Torsten and Andre, for your wisdom and experience. I
deeply appreciate it! =)

Warm Regards,
Jordan Michaels

On 12/15/2011 11:40 PM, André Warnier wrote:
> Hi.
> If it may contribute something to the question :
> I have a case where, within a response handler, I need to make a call to
> a back-end Tomcat through mod_jk (a simple GET, not a POST). I initially
> tried to do this through a sub-request, and was getting segfaults in
> Apache for my trouble.
> Rather than investigating this, I switched to making the call through
> LWP, and it has been working fine for a couple of years, on a variety of
> systems and Apache versions.
>
>
> Jordan Michaels wrote:
>> Well, I do need mod_rewrite to be run. A lot of our users use SES
>> url's and then use rewrite rules to parse them out. I also have
>> specific requests to leave .htaccess functionality in there (for
>> secured directories and so forth).
>>
>> So yeah, the more typical Apache functionality we can leave in the
>> better. It's a pretty tall order.
>>
>> I've configured my perl module using AddHandler in Apache, like so:
>>
>> AddHandler perl-script [my file extensions]
>> PerlRequire /path/to/mymod.pm
>> PerlHandler mymod
>>
>> and that seems to allow me to do everything I'm needing to do so far,
>> but I admit I haven't tested it thoroughly so far. I'm still trying to
>> get POST data passed.
>>
>> Thank you again, very much, for your help!
>>
>> Warm Regards,
>> Jordan Michaels
>>
>> On 12/15/2011 12:17 PM, Torsten Förtsch wrote:
>>> On Thursday, 15 December 2011 12:09:05 Jordan Michaels wrote:
>>>> The module I'm working on updates the headers and proxies the requests
>>>> for specific file types off to Tomcat HTTP and AJP ports.
>>>
>>> Then maybe you don't need subrequests at all. A translation handler
>>> may be
>>> sufficient. Even an internal redirect will do if invoked before the
>>> response
>>> phase. (The core response handler calls ap_discard_request_body.)
>>>
>>> Torsten Förtsch
>>>
>>
>

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads