Mailing List Archive: Catalyst: Users

Dispatching with Chained vs HTTP method

Index | Next | Previous | View Threaded

tjc at wintrmute

Apr 30, 2008, 1:26 AM

Post #1 of 18 (399 views)
Permalink

Dispatching with Chained vs HTTP method

just looking for some advice on the best way to do something..

So I wrote a controller class using Chained that basically auto-converts any
DBIx::Schema (which includes a tiny extra base class itself) into a REST API..
Well, a simple one anyway - it supports find and search so far, and foreign
keys in the objects get serialised into hashes of the URIs to fetch them.

All working nicely so far, but this is all for GET queries.

But the behaviour should be different depending upon whether you
GET /item/1234
or
DELETE /item/1234
etc.

I was thinking something like this: (Abbreviated code below)

------------
sub item_by_id : Chained CaptureArgs(2) {
my ($self, $c, $type, $id) = @_;
$c->stash->{item} = $c->model("DB:$type")->find($id);
}

sub delete : Chained('item_by_id') Args ActionClass('MethodDELETE') {
my ($self, $c) = @_;
$c->stash->{item}->delete;
}

sub modify : Chained('item_by_id') Args ActionClass('MethodPUT') {
my ($self, $c) = @_;
$c->stash->{item}->update($c->request->params);
}
------------
Then the appropriate ActionClasses would check if the $c->req->method eq 'GET'
or 'DELETE' or whatever.

But I was just wondering if you had other ideas, and if using ActionClasses
with Chained actions is crackfuelled and going to lead to a world of misery and
pain.

Cheers!
Toby

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

pagaltzis at gmx

Apr 30, 2008, 2:07 AM

Post #2 of 18 (390 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

* Toby Corkindale <tjc[at]wintrmute.net> [2008-04-30 10:40]:
> I was thinking something like this: (Abbreviated code below)

http://search.cpan.org/dist/Catalyst-Action-REST/
http://search.cpan.org/dist/Catalyst-Request-REST-ForBrowsers/

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

zzbbyy at gmail

Apr 30, 2008, 2:24 AM

Post #3 of 18 (387 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

Hi Toby,

I don't know if you are aware - but building a REST-like CRUD
interface to DBIx::Schema is my long term goal (started with
Catalyst::Example::InstantCRUD). Do you think we could collaborate?

Cheers,
Zbyszek

On Wed, Apr 30, 2008 at 10:26 AM, Toby Corkindale <tjc[at]wintrmute.net> wrote:
> just looking for some advice on the best way to do something..
>
> So I wrote a controller class using Chained that basically auto-converts any
> DBIx::Schema (which includes a tiny extra base class itself) into a REST API..
> Well, a simple one anyway - it supports find and search so far, and foreign
> keys in the objects get serialised into hashes of the URIs to fetch them.
>
> All working nicely so far, but this is all for GET queries.
>
> But the behaviour should be different depending upon whether you
> GET /item/1234
> or
> DELETE /item/1234
> etc.
>
> I was thinking something like this: (Abbreviated code below)
>
> ------------
> sub item_by_id : Chained CaptureArgs(2) {
> my ($self, $c, $type, $id) = @_;
> $c->stash->{item} = $c->model("DB:$type")->find($id);
> }
>
> sub delete : Chained('item_by_id') Args ActionClass('MethodDELETE') {
> my ($self, $c) = @_;
> $c->stash->{item}->delete;
> }
>
> sub modify : Chained('item_by_id') Args ActionClass('MethodPUT') {
> my ($self, $c) = @_;
> $c->stash->{item}->update($c->request->params);
> }
> ------------
> Then the appropriate ActionClasses would check if the $c->req->method eq 'GET'
> or 'DELETE' or whatever.
>
> But I was just wondering if you had other ideas, and if using ActionClasses
> with Chained actions is crackfuelled and going to lead to a world of misery and
> pain.
>
> Cheers!
> Toby
>
> _______________________________________________
> List: Catalyst[at]lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>

--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

onken at houseofdesign

Apr 30, 2008, 2:46 AM

Post #4 of 18 (386 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

Hi Zbyszek,

do you plan FormFu support? In case the submitted values are not valid
the response is a xml/json/whatever which includes the error messages.
That would be really great :)

cheers,

moritz

Am 30.04.2008 um 11:24 schrieb Zbigniew Lukasiak:

> Hi Toby,
>
> I don't know if you are aware - but building a REST-like CRUD
> interface to DBIx::Schema is my long term goal (started with
> Catalyst::Example::InstantCRUD). Do you think we could collaborate?
>
> Cheers,
> Zbyszek
>
> On Wed, Apr 30, 2008 at 10:26 AM, Toby Corkindale
> <tjc[at]wintrmute.net> wrote:
>> just looking for some advice on the best way to do something..
>>
>> So I wrote a controller class using Chained that basically auto-
>> converts any
>> DBIx::Schema (which includes a tiny extra base class itself) into a
>> REST API..
>> Well, a simple one anyway - it supports find and search so far, and
>> foreign
>> keys in the objects get serialised into hashes of the URIs to fetch
>> them.
>>
>> All working nicely so far, but this is all for GET queries.
>>
>> But the behaviour should be different depending upon whether you
>> GET /item/1234
>> or
>> DELETE /item/1234
>> etc.
>>
>> I was thinking something like this: (Abbreviated code below)
>>
>> ------------
>> sub item_by_id : Chained CaptureArgs(2) {
>> my ($self, $c, $type, $id) = @_;
>> $c->stash->{item} = $c->model("DB:$type")->find($id);
>> }
>>
>> sub delete : Chained('item_by_id') Args ActionClass('MethodDELETE') {
>> my ($self, $c) = @_;
>> $c->stash->{item}->delete;
>> }
>>
>> sub modify : Chained('item_by_id') Args ActionClass('MethodPUT') {
>> my ($self, $c) = @_;
>> $c->stash->{item}->update($c->request->params);
>> }
>> ------------
>> Then the appropriate ActionClasses would check if the $c->req-
>> >method eq 'GET'
>> or 'DELETE' or whatever.
>>
>> But I was just wondering if you had other ideas, and if using
>> ActionClasses
>> with Chained actions is crackfuelled and going to lead to a world
>> of misery and
>> pain.
>>
>> Cheers!
>> Toby
>>
>> _______________________________________________
>> List: Catalyst[at]lists.scsys.co.uk
>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>> Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
>> Dev site: http://dev.catalyst.perl.org/
>>
>
>
>
> --
> Zbigniew Lukasiak
> http://brudnopis.blogspot.com/
>
> _______________________________________________
> List: Catalyst[at]lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

zzbbyy at gmail

Apr 30, 2008, 3:03 AM

Post #5 of 18 (385 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Wed, Apr 30, 2008 at 11:46 AM, Moritz Onken <onken[at]houseofdesign.de> wrote:
> Hi Zbyszek,
>
> do you plan FormFu support? In case the submitted values are not valid the
> response is a xml/json/whatever which includes the error messages.
> That would be really great :)

I have not yet settled down with the choice of the form processor.
I've started with FormFu - but now I am playing with Rose::HTML::Form
- and I found it much more flexible.

--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

onken at houseofdesign

Apr 30, 2008, 3:19 AM

Post #6 of 18 (387 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

Maybe you could make it modular so we can easily add another processor
like formfu

Am 30.04.2008 um 12:03 schrieb Zbigniew Lukasiak:

> On Wed, Apr 30, 2008 at 11:46 AM, Moritz Onken
> <onken[at]houseofdesign.de> wrote:
>> Hi Zbyszek,
>>
>> do you plan FormFu support? In case the submitted values are not
>> valid the
>> response is a xml/json/whatever which includes the error messages.
>> That would be really great :)
>
> I have not yet settled down with the choice of the form processor.
> I've started with FormFu - but now I am playing with Rose::HTML::Form
> - and I found it much more flexible.
>
> --
> Zbigniew Lukasiak
> http://brudnopis.blogspot.com/
>
> _______________________________________________
> List: Catalyst[at]lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

tjc at wintrmute

Apr 30, 2008, 9:23 PM

Post #7 of 18 (381 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Wed, Apr 30, 2008 at 11:24:36AM +0200, Zbigniew Lukasiak wrote:
> Hi Toby,
>
> I don't know if you are aware - but building a REST-like CRUD
> interface to DBIx::Schema is my long term goal (started with
> Catalyst::Example::InstantCRUD). Do you think we could collaborate?

Sure, if you liked.

I'm not sure I want to aim for total DBIx::Class:Schema emulation, as I think
there are many corner cases which could start getting hard to implement /well/.

I'd rather approach creating something which does a sub-set of the features,
but does them well. For instance, how would you handle transactions? Or
specifying complex searches with aggregate clauses and self-joins? Those would
seem to be more suited to an RPC protocol rather than an object-based REST
situation. I think?

The system I have so far works quite well for find and search, and I have a
client library which re-vivifies stuff back into Perl DBIC-like objects..
So in theory you can have an application which can operate upon a local
database, OR a remote database, transparently. (As long as it only uses the
simpler DBIC methods)

I'm learning more about the innards of Catalyst and DBIx::Class in the process
too :)

Toby

> On Wed, Apr 30, 2008 at 10:26 AM, Toby Corkindale <tjc[at]wintrmute.net> wrote:
> > just looking for some advice on the best way to do something..
> >
> > So I wrote a controller class using Chained that basically auto-converts any
> > DBIx::Schema (which includes a tiny extra base class itself) into a REST API..
> > Well, a simple one anyway - it supports find and search so far, and foreign
> > keys in the objects get serialised into hashes of the URIs to fetch them.
> >
> > All working nicely so far, but this is all for GET queries.
> >
> > But the behaviour should be different depending upon whether you
> > GET /item/1234
> > or
> > DELETE /item/1234
> > etc.
> >
> > I was thinking something like this: (Abbreviated code below)
> >
> > ------------
> > sub item_by_id : Chained CaptureArgs(2) {
> > my ($self, $c, $type, $id) = @_;
> > $c->stash->{item} = $c->model("DB:$type")->find($id);
> > }
> >
> > sub delete : Chained('item_by_id') Args ActionClass('MethodDELETE') {
> > my ($self, $c) = @_;
> > $c->stash->{item}->delete;
> > }
> >
> > sub modify : Chained('item_by_id') Args ActionClass('MethodPUT') {
> > my ($self, $c) = @_;
> > $c->stash->{item}->update($c->request->params);
> > }
> > ------------
> > Then the appropriate ActionClasses would check if the $c->req->method eq 'GET'
> > or 'DELETE' or whatever.
> >
> > But I was just wondering if you had other ideas, and if using ActionClasses
> > with Chained actions is crackfuelled and going to lead to a world of misery and
> > pain.
> >
> > Cheers!
> > Toby

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

zzbbyy at gmail

Apr 30, 2008, 11:41 PM

Post #8 of 18 (377 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Thu, May 1, 2008 at 6:23 AM, Toby Corkindale <tjc[at]wintrmute.net> wrote:
> On Wed, Apr 30, 2008 at 11:24:36AM +0200, Zbigniew Lukasiak wrote:
> > Hi Toby,
> >
> > I don't know if you are aware - but building a REST-like CRUD
> > interface to DBIx::Schema is my long term goal (started with
> > Catalyst::Example::InstantCRUD). Do you think we could collaborate?
>
> Sure, if you liked.
>
> I'm not sure I want to aim for total DBIx::Class:Schema emulation, as I think
> there are many corner cases which could start getting hard to implement /well/.
>
> I'd rather approach creating something which does a sub-set of the features,
> but does them well. For instance, how would you handle transactions? Or

My understanding is that you can nest transactions in DBIC - so my
plan is just add a transaction wherever I think it is needed.

> specifying complex searches with aggregate clauses and self-joins? Those would
> seem to be more suited to an RPC protocol rather than an object-based REST
> situation. I think?
>

I admit that my main goal is mostly a kind of browser-REST (but with
the option for 'real' REST) - so I start with HTML forms (for now I
discovered Rose::HTML::Form - and I like it) that would define the
available searches. I guess this might look restricting for you. By
the way have you seen my Advent article:
http://catalyst.perl.org/calendar/2007/16 ?

> The system I have so far works quite well for find and search, and I have a
> client library which re-vivifies stuff back into Perl DBIC-like objects..
> So in theory you can have an application which can operate upon a local
> database, OR a remote database, transparently. (As long as it only uses the
> simpler DBIC methods)
>

Great! This looks like rather outside of the scope of my work - but
if we manage to do it in some unified fashion so that people could mix
those things - this would be great.

Zbyszek
http://brudnopis.blogspot.com/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

tjc at wintrmute

May 6, 2008, 6:05 PM

Post #9 of 18 (322 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Thu, May 01, 2008 at 08:41:26AM +0200, Zbigniew Lukasiak wrote:
> On Thu, May 1, 2008 at 6:23 AM, Toby Corkindale <tjc[at]wintrmute.net> wrote:
> > On Wed, Apr 30, 2008 at 11:24:36AM +0200, Zbigniew Lukasiak wrote:
> > > Hi Toby,
> > >
> > > I don't know if you are aware - but building a REST-like CRUD
> > > interface to DBIx::Schema is my long term goal (started with
> > > Catalyst::Example::InstantCRUD). Do you think we could collaborate?
> >
> > Sure, if you liked.
> >
> > I'm not sure I want to aim for total DBIx::Class:Schema emulation, as I
> > think there are many corner cases which could start getting hard to
> > implement /well/.
> >
> > I'd rather approach creating something which does a sub-set of the
> > features, but does them well. For instance, how would you handle
> > transactions? Or
>
> My understanding is that you can nest transactions in DBIC - so my
> plan is just add a transaction wherever I think it is needed.

Ah, I was thinking of transactions vs a REST API, eg:
PUT /user/1234/account_balance?subtract=1
POST /user/4567/account_balance?add=1
Since those are two separate HTTP requests, and REST specifically states you
cannot maintain state on the server, how would you perform those two
operations inside a transaction?

(My "solution" is to implement it in one request, like:
PUT /user/1234/money_transfer?user=4567;amount=1
However that is not CRUD-like, nor a direct mapping of DBIC functionality to
REST)

> > specifying complex searches with aggregate clauses and self-joins? Those
> > would seem to be more suited to an RPC protocol rather than an
> > object-based REST situation. I think?
>
> I admit that my main goal is mostly a kind of browser-REST (but with
> the option for 'real' REST) - so I start with HTML forms (for now I
> discovered Rose::HTML::Form - and I like it) that would define the
> available searches. I guess this might look restricting for you. By
> the way have you seen my Advent article:
> http://catalyst.perl.org/calendar/2007/16 ?

Oh, right.
I'm definitely making a 'real' REST interface; I'm basing it around a DBIC
Schema for the moment, but I'm not attempting to provide 100% of the DBIC
features via REST; I figure if you want that, you can just use a real DB
connection.
Instead I'm trying to provide a simple set of basic REST calls, and a framework
for dealing with foreign keys, and then allow extensions on top of that for
things such as the example above where you can't use simple calls.

I think it sounds like we're really working towards fairly different goals at
the moment.
Whereas you're looking at using forms and validation there, I'm more interested
in having a DB which provides the validation via constraints and stored
procedures, and using Catalyst as a fairly thin layer to access it and provide
extensions via the DBIC-SChema class.

The REST API has proven quite cute when used against a quickly hacked-up pure
javascript web UI though; it's blindingly fast compared to having pages built
and served and rendered!

Toby

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

adam.clarke at strategicdata

May 6, 2008, 10:30 PM

Post #10 of 18 (323 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On 07/05/2008, at 11:05 AM, Toby Corkindale wrote:

> Ah, I was thinking of transactions vs a REST API, eg:
> PUT /user/1234/account_balance?subtract=1
> POST /user/4567/account_balance?add=1
> Since those are two separate HTTP requests, and REST specifically
> states you
> cannot maintain state on the server, how would you perform those two
> operations inside a transaction?
>
> (My "solution" is to implement it in one request, like:
> PUT /user/1234/money_transfer?user=4567;amount=1
> However that is not CRUD-like, nor a direct mapping of DBIC
> functionality to
> REST)

The solution suggested in "Restful Web Services" is to POST to a
"factory" resource which creates you with a transaction resource. e.g.
"POST /transactions/account-transfer" returns "Location: /
transactions/account-transfer/11a5", where the 11a5 is a unique
transaction identifier.

Then "PUT /transactions/account-transfer/11a5/accounts/checking/11",
where 11 is the account identifier. The body carries the transaction
details, in the example the balances are adjusted absolutely, i.e.
"balance=150". A similar PUT is sent for the other account.

Once the required components of the transaction have been PUT it is
possible to rollback by DELETEing the transaction resource or commit
it by putting "committed=true" to the resource.

While seeming a bit fiddly, it does keep the state on the client and
allows the client to make (at least some of) the commit / rollback
decision rather than (only) the server.

--
Adam

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

tjc at wintrmute

May 6, 2008, 10:57 PM

Post #11 of 18 (324 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

Hi Adam,

On Wed, May 07, 2008 at 03:30:12PM +1000, Adam Clarke wrote:
> On 07/05/2008, at 11:05 AM, Toby Corkindale wrote:
>
>> Ah, I was thinking of transactions vs a REST API, eg:
>> PUT /user/1234/account_balance?subtract=1
>> POST /user/4567/account_balance?add=1
>> Since those are two separate HTTP requests, and REST specifically states
>> you
>> cannot maintain state on the server, how would you perform those two
>> operations inside a transaction?
>>
>> (My "solution" is to implement it in one request, like:
>> PUT /user/1234/money_transfer?user=4567;amount=1
>> However that is not CRUD-like, nor a direct mapping of DBIC functionality
>> to
>> REST)
>
> The solution suggested in "Restful Web Services" is to POST to a "factory"
> resource which creates you with a transaction resource. e.g. "POST
> /transactions/account-transfer" returns "Location:
> /transactions/account-transfer/11a5", where the 11a5 is a unique
> transaction identifier.
>
> Then "PUT /transactions/account-transfer/11a5/accounts/checking/11", where
> 11 is the account identifier. The body carries the transaction details, in
> the example the balances are adjusted absolutely, i.e. "balance=150". A
> similar PUT is sent for the other account.
>
> Once the required components of the transaction have been PUT it is
> possible to rollback by DELETEing the transaction resource or commit it by
> putting "committed=true" to the resource.
>
> While seeming a bit fiddly, it does keep the state on the client and allows
> the client to make (at least some of) the commit / rollback decision rather
> than (only) the server.

I've read parts of RESTful Web Services, but not that bit.. I'll have to go
back and look.

I wonder how one goes about implementing such a transaction on the server
side.. One would not want to lock DB rows indefinitely, waiting for the client
to finally complete the transaction. But if one just recorded the queries and
then executed them all (internally) at the end, then other risks exist, eg:

$id = POST transaction
$amount = GET /user/1/account_balance
$amount2 = GET /user/2/account_balance
PUT /user/1/account_balance/$amount-1
PUT /user/2/account_balance/$amount+1
PUT transaction/$id?completed

In that example, user one might receive their paycheque after you've queried
their account balance, but before you've written the new amount.

Hmm. I suppose one could try for the method of actually locking the DB rows
until the REST completed the transaction, and putting some kind of timeout on
it; but anything that does long DB locks scares me with the potential of
deadlocks.

Have you implemented anything like this? I'm curious to know how well it might
work (or not work) in practice.

Toby
(Who likes REST but still isn't sure about doing double-entry bookkeeping with
it)

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

tjc at wintrmute

May 6, 2008, 11:09 PM

Post #12 of 18 (323 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Wed, May 07, 2008 at 03:57:07PM +1000, Toby Corkindale wrote:
[snip]
> $id = POST transaction
> $amount = GET /user/1/account_balance
> $amount2 = GET /user/2/account_balance
> PUT /user/1/account_balance/$amount-1
> PUT /user/2/account_balance/$amount+1
Whoops, that should read:
> PUT /user/2/account_balance/$amount2+1

Go me and my badly named example variables :/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

zzbbyy at gmail

May 7, 2008, 12:55 AM

Post #13 of 18 (320 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Wed, May 7, 2008 at 7:57 AM, Toby Corkindale <tjc[at]wintrmute.net> wrote:
> Hi Adam,
>
>
>
> On Wed, May 07, 2008 at 03:30:12PM +1000, Adam Clarke wrote:
> > On 07/05/2008, at 11:05 AM, Toby Corkindale wrote:
> >
> >> Ah, I was thinking of transactions vs a REST API, eg:
> >> PUT /user/1234/account_balance?subtract=1
> >> POST /user/4567/account_balance?add=1
> >> Since those are two separate HTTP requests, and REST specifically states
> >> you
> >> cannot maintain state on the server, how would you perform those two
> >> operations inside a transaction?
> >>
> >> (My "solution" is to implement it in one request, like:
> >> PUT /user/1234/money_transfer?user=4567;amount=1
> >> However that is not CRUD-like, nor a direct mapping of DBIC functionality
> >> to
> >> REST)
> >
> > The solution suggested in "Restful Web Services" is to POST to a "factory"
> > resource which creates you with a transaction resource. e.g. "POST
> > /transactions/account-transfer" returns "Location:
> > /transactions/account-transfer/11a5", where the 11a5 is a unique
> > transaction identifier.
> >
> > Then "PUT /transactions/account-transfer/11a5/accounts/checking/11", where
> > 11 is the account identifier. The body carries the transaction details, in
> > the example the balances are adjusted absolutely, i.e. "balance=150". A
> > similar PUT is sent for the other account.
> >
> > Once the required components of the transaction have been PUT it is
> > possible to rollback by DELETEing the transaction resource or commit it by
> > putting "committed=true" to the resource.
> >
> > While seeming a bit fiddly, it does keep the state on the client and allows
> > the client to make (at least some of) the commit / rollback decision rather
> > than (only) the server.
>
> I've read parts of RESTful Web Services, but not that bit.. I'll have to go
> back and look.
>
> I wonder how one goes about implementing such a transaction on the server
> side.. One would not want to lock DB rows indefinitely, waiting for the client
> to finally complete the transaction. But if one just recorded the queries and
> then executed them all (internally) at the end, then other risks exist, eg:
>
> $id = POST transaction
> $amount = GET /user/1/account_balance
> $amount2 = GET /user/2/account_balance
> PUT /user/1/account_balance/$amount-1
> PUT /user/2/account_balance/$amount+1
> PUT transaction/$id?completed

How about:

$id = POST transaction
PUT /transaction/$id/payer/1
PUT /transaction/$id/receiver/2
PUT /transaction/$id/amount/1
PUT /transaction/$id?completed

or even

$id = POST transaction
PUT /transaction/$id?payer=1;receiver=2;amount=1
PUT /transaction/$id?completed

And - yeah - looks like we have differnet goals. But I'll watch your
project proceeding :)

--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

adam.clarke at strategicdata

May 7, 2008, 1:02 AM

Post #14 of 18 (321 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On 07/05/2008, at 3:57 PM, Toby Corkindale wrote:

> On Wed, May 07, 2008 at 03:30:12PM +1000, Adam Clarke wrote:
>>
>> The solution suggested in "Restful Web Services" is to POST to a
>> "factory"
>> resource which creates you with a transaction resource. e.g. "POST
>> /transactions/account-transfer" returns "Location:
>> /transactions/account-transfer/11a5", where the 11a5 is a unique
>> transaction identifier.
>>
>> Then "PUT /transactions/account-transfer/11a5/accounts/checking/
>> 11", where
>> 11 is the account identifier. The body carries the transaction
>> details, in
>> the example the balances are adjusted absolutely, i.e.
>> "balance=150". A
>> similar PUT is sent for the other account.
>>
>> Once the required components of the transaction have been PUT it is
>> possible to rollback by DELETEing the transaction resource or
>> commit it by
>> putting "committed=true" to the resource.
>>
>> While seeming a bit fiddly, it does keep the state on the client
>> and allows
>> the client to make (at least some of) the commit / rollback
>> decision rather
>> than (only) the server.
>
> I've read parts of RESTful Web Services, but not that bit.. I'll
> have to go
> back and look.
>
> I wonder how one goes about implementing such a transaction on the
> server
> side.. One would not want to lock DB rows indefinitely, waiting for
> the client
> to finally complete the transaction. But if one just recorded the
> queries and
> then executed them all (internally) at the end, then other risks
> exist, eg:

I haven't done this before, but I have thought about it a bit. I think
I would handle this as a two-phase commit. PostgreSQL has "PREPARE
TRANSACTION" which allows you to start a transaction and assign it a
"transaction_id" for use with a subsequent "COMMIT TRANSACTION". I
would also use Multi-Version Concurrency Control (MVCC) rather than
any kind of blocking locks to minimise the impact of the longer
transaction lifetime.

This would at least keep a good deal of the hard work in the DB.

Cheers
--
Adam

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

tjc at wintrmute

May 7, 2008, 1:13 AM

Post #15 of 18 (323 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Wed, May 07, 2008 at 09:55:10AM +0200, Zbigniew Lukasiak wrote:
> On Wed, May 7, 2008 at 7:57 AM, Toby Corkindale <tjc[at]wintrmute.net> wrote:
[snip]
> > I wonder how one goes about implementing such a transaction on the server
> > side.. One would not want to lock DB rows indefinitely, waiting for the
> > client to finally complete the transaction. But if one just recorded the
> > queries and then executed them all (internally) at the end, then other
> > risks exist, eg:
> >
> > $id = POST transaction
> > $amount = GET /user/1/account_balance
> > $amount2 = GET /user/2/account_balance
> > PUT /user/1/account_balance/$amount-1
> > PUT /user/2/account_balance/$amount+1
> > PUT transaction/$id?completed
>
> How about:
>
> $id = POST transaction
> PUT /transaction/$id/payer/1
> PUT /transaction/$id/receiver/2
> PUT /transaction/$id/amount/1
> PUT /transaction/$id?completed
>
> or even
>
> $id = POST transaction
> PUT /transaction/$id?payer=1;receiver=2;amount=1
> PUT /transaction/$id?completed

There are other ways of doing it that would avoid the problem, but I'm just
trying to demonstrate potential flaws in any transaction that does not do
locking to prevent other things accessing it at the same time.
Perhaps I should try another example if the other wasn't clear.

Given this series of calls:
<transaction>
1) get value of item
2) get users bank balance
if balance > item-value, then:
3) subtract value from bank balance
4) assign item to user
</transaction>

What if the internet connection died for 30 seconds between step (2) and (4),
and in that 30 seconds, the user bought something elsewhere, thus reducing his
or her balance to below the value?

That's the sort of race condition that real databases can avoid, because
they'll basically stop anyone else modifying the balance after you've checked
it, until you commit or rollback the transaction.
We could do that too..
But imagine if we locked the user's DB row at step (2).. but then imagine if
the internet connection to the person who started the transaction died.. for
hours.. and during that time no other transactions could be made!
That's not so good either.

Toby

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

tjc at wintrmute

May 7, 2008, 2:10 AM

Post #16 of 18 (320 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Wed, May 07, 2008 at 06:02:46PM +1000, Adam Clarke wrote:
> On 07/05/2008, at 3:57 PM, Toby Corkindale wrote:
>> On Wed, May 07, 2008 at 03:30:12PM +1000, Adam Clarke wrote:
>>>
>>> The solution suggested in "Restful Web Services" is to POST to a
>>> "factory"
>>> resource which creates you with a transaction resource. e.g. "POST
>>> /transactions/account-transfer" returns "Location:
>>> /transactions/account-transfer/11a5", where the 11a5 is a unique
>>> transaction identifier.
>>>
>>> Then "PUT /transactions/account-transfer/11a5/accounts/checking/11",
>>> where
>>> 11 is the account identifier. The body carries the transaction details,
>>> in
>>> the example the balances are adjusted absolutely, i.e. "balance=150". A
>>> similar PUT is sent for the other account.
>>>
>>> Once the required components of the transaction have been PUT it is
>>> possible to rollback by DELETEing the transaction resource or commit it
>>> by
>>> putting "committed=true" to the resource.
>>>
>>> While seeming a bit fiddly, it does keep the state on the client and
>>> allows
>>> the client to make (at least some of) the commit / rollback decision
>>> rather
>>> than (only) the server.
>>
>> I've read parts of RESTful Web Services, but not that bit.. I'll have to go
>> back and look.
>>
>> I wonder how one goes about implementing such a transaction on the server
>> side.. One would not want to lock DB rows indefinitely, waiting for the
>> client to finally complete the transaction. But if one just recorded the
>> queries and then executed them all (internally) at the end, then other risks
>> exist, eg:
>
> I haven't done this before, but I have thought about it a bit. I think I
> would handle this as a two-phase commit. PostgreSQL has "PREPARE
> TRANSACTION" which allows you to start a transaction and assign it a
> "transaction_id" for use with a subsequent "COMMIT TRANSACTION". I would
> also use Multi-Version Concurrency Control (MVCC) rather than any kind of
> blocking locks to minimise the impact of the longer transaction lifetime.

I'm not sure the former command does what we'd like it to - after running it, I
don't think you can add any further commands to it; it's merely held in stasis
until you commit/rollback it, and you can start another transaction meanwhile.
I *think* but I haven't used it either.

Regarding the MVCC; that's a rather good idea, although my understanding of
postgres is that it will start blocking in conditions where you're updating the
same thing. (Edit: Just tested - it seems to)
That said, I don't know how other systems handle it, or if in fact the
SET SERIALISATION parameter to psql can alter this behaviour..
Are there other MVCC implementations which manage it?
You seem to have a good idea about using that rather than locking.

Something else occured to me - Have you had any experience at trying to get DB
transactions to span connections? Since the HTTP requests could hit different
processes for each request (or possibly even different servers in a farm)..
I suppose one could push the DB requests to a back-end processing daemon that
could ensure a consistent connection was used, but again seems to be tieing up
resources if there's a network drop-out. I haven't looked into that at all
really though.
I was just assuming one might have to use locks as they would span, but that
would be ugly, and breaks the concept of not storing state on the server for
REST.

> This would at least keep a good deal of the hard work in the DB.

*nods* I agree that's the best option.
I'm mainly familiar with Postgres (and a little of MySQL) so I don't know if
the commercial DBs have added features to help with these issues already?

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

ggoebel at goebel

May 7, 2008, 5:38 AM

Post #17 of 18 (313 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

Zbigniew Lukasiak wrote:
> On Wed, May 7, 2008 at 7:57 AM, Toby Corkindale <tjc[at]wintrmute.net> wrote:
>> $id = POST transaction
>> $amount = GET /user/1/account_balance
>> $amount2 = GET /user/2/account_balance
>> PUT /user/1/account_balance/$amount-1
>> PUT /user/2/account_balance/$amount+1
>> PUT transaction/$id?completed
>>
>
> How about:
>
> $id = POST transaction
> PUT /transaction/$id/payer/1
> PUT /transaction/$id/receiver/2
> PUT /transaction/$id/amount/1
> PUT /transaction/$id?completed
>
> or even
>
> $id = POST transaction
> PUT /transaction/$id?payer=1;receiver=2;amount=1
> PUT /transaction/$id?completed
>
A couple years back when I hacked together REST(like) CRUD with
Catalyst... I opted for the latter. As it seemed at the time to be a
better way to represent the possible keys to tuples.

I haven't been able to pay much attention to this thread. Not enough
tuits. Sorry if I'm going over ground which has already been covered...

But I hope the people who are implementing it spend some time thinking
about making the interface introspective.

Also important is how to allow people to limit which sets of tuples and
relationships are publically accessible. For production work the default
should probably require the REST interfaces to be explicitly published.
Otherwise, with any set of tables with more than a handful of records,
it will be fairly simple to bring the database to its knees with a URL
that performs multiple joins on a large set of records. As a compromise,
you might allow primary key candidates (keys which match exactly one
record) and "have one" relationships to be public by default, but not
"have many" or "many to many" relationships.

I think thinking in terms of using the underlying databases'
transactions across multiple GET | PUT | POST | DELETE's is a mistake.
For the reasons others have already pointed out. Except where you
collect the data for your complex set of interrelated tuples and commit
it in one go. Which of course means you need to be able to catch and
handle the conflicts which can occur between starting to create your set
of tuples and finally committing it.

cheers and good luck...

Garrett

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

paddy at panici

May 7, 2008, 5:51 AM

Post #18 of 18 (313 views)
Permalink

Re: Dispatching with Chained vs HTTP method [In reply to]

On Wed, May 07, 2008 at 08:38:18AM -0400, Garrett Goebel wrote:
<snip>
>
> Also important is how to allow people to limit which sets of tuples and
> relationships are publically accessible. For production work the default
> should probably require the REST interfaces to be explicitly published.
> Otherwise, with any set of tables with more than a handful of records,
> it will be fairly simple to bring the database to its knees with a URL
> that performs multiple joins on a large set of records. As a compromise,
> you might allow primary key candidates (keys which match exactly one
> record) and "have one" relationships to be public by default, but not
> "have many" or "many to many" relationships.

or ask the database how long the query will take and then limit on that?

Regards,
Paddy

_______________________________________________
List: Catalyst[at]lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst[at]lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com