Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Zope: Users

problem with commits in SQLAlchemyDA

 

 

Zope users RSS feed   Index | Next | Previous | View Threaded


maric at aristote

Aug 17, 2009, 6:54 AM

Post #1 of 5 (1017 views)
Permalink
problem with commits in SQLAlchemyDA

Hi all,

I wanted to use SQALchemyDA with a standalone Zope 2.11 (we only depends
on CMFCore).

i tried to get it up using easy_install, which is pretty
straightforward, once removed the incompatible zope.component package
needed in the dependencies.

It ends up with the following installed packages (skipping zope 3
components) :
zope.sqlalchemy-0.4-py2.4.egg
z3c.sqlalchemy-1.3.10.1-py2.4.egg
SQLAlchemy-0.5.5-py2.4.egg
Products.SQLAlchemyDA-0.4.1-py2.4.egg

At first glance, this works well, but the DA actually can't manage to
commit any sql requests (I try only for insert in Mysql and postgres but
I'm sure it's always true).

This is quite an unexpectable behavior for a DA.

I think the problem is related to this thread I found on tg-trunk
newsgroup :

http://www.mail-archive.com/turbogears-trunk[at]googlegroups.com/msg07302.html

There is a misleading comment in the docstring of
zope.sqlalchemy.datamanager.join_transaction speaking of a
DirtyAfterFlush SessionExtension, which I never seen except in plone code.

In the last I finally manage to make it work by monkey patching
SQLAlchemyDA itself this way :


try :
from Products.SQLAlchemyDA import da
from z3c.sqlalchemy import getSAWrapper, createSAWrapper
if '0.4.1' in da.__file__ and not hasattr(da.SAWrapper,
'_patched__wrapper_property') :
da.SAWrapper._patched__wrapper_property = da.SAWrapper._wrapper

def _always_invalidated_wrapper(self):
"""The property '_wrapper' patched by CFENet to correct what
seems to be a bug in SQLAlchemyDA which prevents any
commit."""
from zope.sqlalchemy.datamanager import STATUS_INVALIDATED
if self.dsn:
try:
return getSAWrapper(self.util_id)
except ValueError:
return createSAWrapper(
self.dsn, forZope=True,
transactional=self.transactional,
engine_options={'convert_unicode' :
self.convert_unicode,
'encoding' : self.encoding},
extension_options={'initial_state': # the whole
# point is here
STATUS_INVALIDATED},
name=self.util_id)
return None

da.SAWrapper._wrapper = property(_always_invalidated_wrapper)

except ImportError :
pass


Anyone has any insights about this problem, any comments on my solution,
did it have been reported and corrected, or am I simply missing something ?


--
_____________

Maric Michaud

_______________________________________________
Zope maillist - Zope[at]zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )


robert at redcor

Aug 17, 2009, 7:04 AM

Post #2 of 5 (928 views)
Permalink
Re: problem with commits in SQLAlchemyDA [In reply to]

I do not see any reference to mark_changed
you have to call it before any transaction.commit() to tell the zope transaction
machinery that it has to commit you changes also.
robert

Maric Michaud schrieb:
> Hi all,
>
> I wanted to use SQALchemyDA with a standalone Zope 2.11 (we only depends
> on CMFCore).
>
> i tried to get it up using easy_install, which is pretty
> straightforward, once removed the incompatible zope.component package
> needed in the dependencies.
>
> It ends up with the following installed packages (skipping zope 3
> components) :
> zope.sqlalchemy-0.4-py2.4.egg
> z3c.sqlalchemy-1.3.10.1-py2.4.egg
> SQLAlchemy-0.5.5-py2.4.egg
> Products.SQLAlchemyDA-0.4.1-py2.4.egg
>
> At first glance, this works well, but the DA actually can't manage to
> commit any sql requests (I try only for insert in Mysql and postgres but
> I'm sure it's always true).
>
> This is quite an unexpectable behavior for a DA.
>
> I think the problem is related to this thread I found on tg-trunk
> newsgroup :
>
> http://www.mail-archive.com/turbogears-trunk[at]googlegroups.com/msg07302.html
>
> There is a misleading comment in the docstring of
> zope.sqlalchemy.datamanager.join_transaction speaking of a
> DirtyAfterFlush SessionExtension, which I never seen except in plone code.
>
> In the last I finally manage to make it work by monkey patching
> SQLAlchemyDA itself this way :
>
>
> try :
> from Products.SQLAlchemyDA import da
> from z3c.sqlalchemy import getSAWrapper, createSAWrapper
> if '0.4.1' in da.__file__ and not hasattr(da.SAWrapper,
> '_patched__wrapper_property') :
> da.SAWrapper._patched__wrapper_property = da.SAWrapper._wrapper
>
> def _always_invalidated_wrapper(self):
> """The property '_wrapper' patched by CFENet to correct what
> seems to be a bug in SQLAlchemyDA which prevents any
> commit."""
> from zope.sqlalchemy.datamanager import STATUS_INVALIDATED
> if self.dsn:
> try:
> return getSAWrapper(self.util_id)
> except ValueError:
> return createSAWrapper(
> self.dsn, forZope=True,
> transactional=self.transactional,
> engine_options={'convert_unicode' :
> self.convert_unicode,
> 'encoding' : self.encoding},
> extension_options={'initial_state': # the whole
> # point is here
> STATUS_INVALIDATED},
> name=self.util_id)
> return None
>
> da.SAWrapper._wrapper = property(_always_invalidated_wrapper)
>
> except ImportError :
> pass
>
>
> Anyone has any insights about this problem, any comments on my solution,
> did it have been reported and corrected, or am I simply missing something ?
>
>

_______________________________________________
Zope maillist - Zope[at]zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )


maric at aristote

Aug 17, 2009, 7:24 AM

Post #3 of 5 (933 views)
Permalink
Re: problem with commits in SQLAlchemyDA [In reply to]

robert rottermann a écrit :
> I do not see any reference to mark_changed
> you have to call it before any transaction.commit() to tell the zope
transaction
> machinery that it has to commit you changes also.
> robert
>

In fact, what I understood is that zope.sqlalchemy, by default, bypasses
the commit if status is not manually set as 'changed' (see the docstring
I quoted in my previous mail). The alternative is to build the
SessionExtension with initial_state == STATUS_INVALIDATED.

My point is that it should be the default for a DA, as it is intended to
be used mainly by zsql methods which doesn't do nothing to the
transaction state.

Maybe I wasn't clear but the patch works well with my existing zsql code.

--
_____________

Maric Michaud
_______________________________________________
Zope maillist - Zope[at]zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )


robert at redcor

Aug 17, 2009, 7:58 AM

Post #4 of 5 (930 views)
Permalink
Re: problem with commits in SQLAlchemyDA [In reply to]

Maric Michaud schrieb:
> robert rottermann a écrit :
> > I do not see any reference to mark_changed
> > you have to call it before any transaction.commit() to tell the zope
> transaction
> > machinery that it has to commit you changes also.
> > robert
> >
>
> In fact, what I understood is that zope.sqlalchemy, by default, bypasses
> the commit if status is not manually set as 'changed' (see the docstring
> I quoted in my previous mail). The alternative is to build the
> SessionExtension with initial_state == STATUS_INVALIDATED.
>
> My point is that it should be the default for a DA, as it is intended to
> be used mainly by zsql methods which doesn't do nothing to the
> transaction state.
>
> Maybe I wasn't clear but the patch works well with my existing zsql code.
>
it is a longtime that i worked with zsql methods.
however I think the difference is, that zsql handles transactions itself. it
wraps every zqls method in a start/end transaction. pair.

this is similarly done by zope.sqlalchemy. but it will only commit, when you
tell it that the session is dirty by calling mark_changed

import transaction
from zope.sqlalchemy import mark_changed.
here is how I use it
def addKtyp(self, form, commit=1):
"""
create a new ktyp
return id of that ktyp
"""
session = self.getSession()
...

if values:
...
if not ktyp:
...
session.add(ktyp)
# only mark the session, do not yet commit
self.mark_changed()
if commit:
transaction.commit()
# we only know the id after a commit !!!
ktyp_id = ktyp.ktypid
return ktyp_id

robert
_______________________________________________
Zope maillist - Zope[at]zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )


lists at zopyx

Aug 30, 2009, 7:49 AM

Post #5 of 5 (795 views)
Permalink
Re: problem with commits in SQLAlchemyDA [In reply to]

On 17.08.09 16:24, Maric Michaud wrote:
> robert rottermann a écrit :
> > I do not see any reference to mark_changed
> > you have to call it before any transaction.commit() to tell the zope
> transaction
> > machinery that it has to commit you changes also.
> > robert
> >
>
> In fact, what I understood is that zope.sqlalchemy, by default, bypasses
> the commit if status is not manually set as 'changed' (see the docstring
> I quoted in my previous mail). The alternative is to build the
> SessionExtension with initial_state == STATUS_INVALIDATED.
>
> My point is that it should be the default for a DA, as it is intended to
> be used mainly by zsql methods which doesn't do nothing to the
> transaction state.
>
As author of SQLAlchemyDA I would assume that the transaction integration
works out-of-the-box with Zope - however I have tested SQLAlchemyDA with
the latest zope.sqlalchemy/SQLAlchemy versions. SQLAlchemy together
with the underlying z3c.sqlalchemy/zope.sqlalchemy works with the latest
Zope versions in various projects out-of-the-box using Zope 2.11.

There is one project that required a special

extension_options={'initial_state':STATUS_CHANGED},

parameter (used for the
scoped_session(extension=ZopeTransactionExtension(**extension_options))
call).

Not sure if this is related to your issue.

Andreas


--
ZOPYX Ltd. & Co KG \ ZOPYX & Friends
Charlottenstr. 37/1 \ The experts for your Python, Zope and
D-72070 Tübingen \ Plone projects
www.zopyx.com, info[at]zopyx.com \ www.zopyx.de/friends, friends[at]zopyx.de
------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting
Attachments: lists.vcf (0.31 KB)

Zope users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.