Mailing List Archive: Zope: XML

ANNOUNCE: PT_XPath

Index | Next | Previous | View Threaded

delza at alliances

Oct 18, 2001, 10:13 PM

Post #1 of 10 (1877 views)
Permalink

ANNOUNCE: PT_XPath

Thanks to Martijn F. and Evan I have posted my first Zope product:

http://www.zope.org/Members/DaddyGravity/PT_XPath/index_html

It adds an xpath: expression type to TALES expressions in Zope Page
Templates. If you don't know what XPath is, it might not help you much %-)

Currently supports ParsedXML and well-formed XML in a string. Namespaces

should be fully supported. I'm still working on Zope Hierarchy-as-XML
and other document types.

Feedback encouraged and welcome.

Thanks guys!

--Dethe

--

Dethe Elza (delza [at] burningtiger)
Chief Mad Scientist
Burning Tiger Technologies (http://burningtiger.com)
Living Code Weblog (http://livingcode.ca)

faassen at vet

Oct 19, 2001, 3:26 AM

Post #2 of 10 (1804 views)
Permalink

Re: ANNOUNCE: PT_XPath [In reply to]

Dethe Elza wrote:
> Thanks to Martijn F. and Evan I have posted my first Zope product:
>
> http://www.zope.org/Members/DaddyGravity/PT_XPath/index_html
>
> It adds an xpath: expression type to TALES expressions in Zope Page
> Templates. If you don't know what XPath is, it might not help you much %-)
>
> Currently supports ParsedXML and well-formed XML in a string. Namespaces
> should be fully supported.

Cool! I'll take a peek. :)

> I'm still working on Zope Hierarchy-as-XML
> and other document types.

I'm also working on ZDOM, i.e. Zope Hierarchy-as-XML. Right now I'm focusing
on making it work read-only, and I do have a list of what methods need
to be implemented for a read-only DOM. Care to work together?

See also here:

http://dev.zope.org/Wikis/DevSite/Proposals/ZDOMXPath

Jay Dylan is also on this list, I believe.

What we'd need is somekind of registry where for an arbitrary Zope object a
DOM object will be returned, if possible. If not possible, some kind of
DOM object should still be returned, I'm thinking about something like
an <unknown /> element node.

Element name would often be determined by meta_type, perhaps with spaces
replaced by underscores, as I'm not sure on spaces in element names.

Most of the work will go into designing the wrappers. I'd like all of this
to be New Component Religion (like the piece of code that looks up the
right DOM wrapper), but since that's not done let's just implement
something that works, and care about new religion specifics later..

Regards,

Martijn

delza at landry

Oct 19, 2001, 10:26 AM

Post #3 of 10 (1812 views)
Permalink

Re: ANNOUNCE: PT_XPath [In reply to]

Martijn Faassen wrote:
> > I'm still working on Zope Hierarchy-as-XML
> > and other document types.
>
> I'm also working on ZDOM, i.e. Zope Hierarchy-as-XML. Right now I'm focusing
> on making it work read-only, and I do have a list of what methods need
> to be implemented for a read-only DOM. Care to work together?

I'd love to help. I'll try to get up to speed with what's been done so far
first, then see what I can add. I just finished up a big contract, so
hopefully I will have more time to put into this. Where can I find the
list of methods that need to be implemented.

> See also here:
>
> http://dev.zope.org/Wikis/DevSite/Proposals/ZDOMXPath

Wow. Very ambitious. This would turn Zope into a very full-featured
XML respository. I wonder if it would be make sense to push some of the
DOM support down into ZODB? I can't claim to understand all the issues
there, though.

> What we'd need is somekind of registry where for an arbitrary Zope object a
> DOM object will be returned, if possible. If not possible, some kind of
> DOM object should still be returned, I'm thinking about something like
> an <unknown /> element node.

I'm afraid I don't understand the need for a registry. Shouldn't we be
patching low-level Zope objects with DOM methods so all objects inherit
the DOM API? I've just started reading the ZDOMIssues document, so
undoubtably I'm missing something.

> Element name would often be determined by meta_type, perhaps with spaces
> replaced by underscores, as I'm not sure on spaces in element names.

That's correct, no XML whitespace (space, tab, various line breaks, etc.)
is allowed in element names.

> Most of the work will go into designing the wrappers. I'd like all of this
> to be New Component Religion (like the piece of code that looks up the
> right DOM wrapper), but since that's not done let's just implement
> something that works, and care about new religion specifics later..

I've read the manifesto for the new religion, but I don't really have a clue
about the actual implementation of it, I'm afraid.

Looking forward to working together.

--Dethe

==========================================================================
| dethe [at] burningtiger | http://livingcode.manilasites.com |
==========================================================================
"Well I've wrestled with reality for thirty-five years now, doctor, and I'm
happy to state I've finally won out over it." -- Elwood P. Dowd, Harvey

delza at landry

Oct 19, 2001, 11:15 AM

Post #4 of 10 (1823 views)
Permalink

Re: ANNOUNCE: PT_XPath [In reply to]

> > Wow. Very ambitious. This would turn Zope into a very full-featured
> > XML respository. I wonder if it would be make sense to push some of the
> > DOM support down into ZODB? I can't claim to understand all the issues
> > there, though.
>
> How do you see pushing that into the ZODB? The ZODB just stores objects,
> right, DOM or not?

Well, I'm thinking in terms of a generic XML repository, like Tamino or
XML-DB. If we're using XPath as a general query mechanism over a database, it
makes sense to put it *in* the database. On the other hand, this wouldn't
necessarily have to be baked into the ZODB, but could be an API which sits
between the ZODB and Zope. Maybe I'm confusing the goals with those of
other projects and this doesn't make sense at all.

One of the things I like about ParsedXML, btw, is its balance of effeciency
with persistence: Store one object in the ZODB, but make it look like an
entire DOM tree to the user (at least, that's how I think it works!). Part of
the reason I start thinking about the ZODB when we talk about querying for
objects as if they were all XML is that it would be handy to do even outside
of Zope, in other projects which use the ZODB. I would like to be able to use
ParsedXML for general XML persistence from Python, but it seems to be closely
tied to Zope.

> > I'm afraid I don't understand the need for a registry. Shouldn't we be
> > patching low-level Zope objects with DOM methods so all objects inherit
> > the DOM API? I've just started reading the ZDOMIssues document, so
> > undoubtably I'm missing something.
>
> If you mean by 'low level Zope objects' things like Folder and DTML Method,
> I'm much in favor in making new objects that layer over these
> Zope objects. That'd be a much cleaner design. The *old* ZDOM works
> with mixin classes, but it never got very far and I think it's
> just too complicated.
>
> So I see somekind of registry that knows how to map any Zope object it
> encounters to a DOM object (usually an element with some attributes
> and probably some other nodes inside). This way you could even have
> different registries that do different mappings, some more detailed
> than the other.

I can see how a registry could be useful, but I think it should be used to
*override* some reasonable default behavior. That is, any Zopish object
should be accessible via ZDOM, and objects which have special needs should be
able to register those to override the defaults.

I'll read up on your thoughts on the matter.

--Dethe

==========================================================================
| dethe [at] burningtiger | http://livingcode.manilasites.com |
==========================================================================
"Well I've wrestled with reality for thirty-five years now, doctor, and I'm
happy to state I've finally won out over it." -- Elwood P. Dowd, Harvey

faassen at vet

Oct 19, 2001, 11:21 AM

Post #5 of 10 (1800 views)
Permalink

Re: ANNOUNCE: PT_XPath [In reply to]

delza [at] landry wrote:
> > http://dev.zope.org/Wikis/DevSite/Proposals/ZDOMXPath
>
> Wow. Very ambitious. This would turn Zope into a very full-featured
> XML respository. I wonder if it would be make sense to push some of the
> DOM support down into ZODB? I can't claim to understand all the issues
> there, though.

How do you see pushing that into the ZODB? The ZODB just stores objects,
right, DOM or not?

> > What we'd need is somekind of registry where for an arbitrary Zope object a
> > DOM object will be returned, if possible. If not possible, some kind of
> > DOM object should still be returned, I'm thinking about something like
> > an <unknown /> element node.
>
> I'm afraid I don't understand the need for a registry. Shouldn't we be
> patching low-level Zope objects with DOM methods so all objects inherit
> the DOM API? I've just started reading the ZDOMIssues document, so
> undoubtably I'm missing something.

If you mean by 'low level Zope objects' things like Folder and DTML Method,
I'm much in favor in making new objects that layer over these
Zope objects. That'd be a much cleaner design. The *old* ZDOM works
with mixin classes, but it never got very far and I think it's
just too complicated.

So I see somekind of registry that knows how to map any Zope object it
encounters to a DOM object (usually an element with some attributes
and probably some other nodes inside). This way you could even have
different registries that do different mappings, some more detailed
than the other.

Some vague pondering of mine about this in the context of new religion
can be found here:

http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/ExplicitVersusImplicitFeatureWrapping

and also here:

http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/FeatureFinding

[snip]
> > Most of the work will go into designing the wrappers. I'd like all of this
> > to be New Component Religion (like the piece of code that looks up the
> > right DOM wrapper), but since that's not done let's just implement
> > something that works, and care about new religion specifics later..
>
> I've read the manifesto for the new religion, but I don't really have a clue
> about the actual implementation of it, I'm afraid.

There doesn't seem to be any actual implementation yet, so your lack of
clue is understandable. ;)

I do have a bit of non functional documentation sitting on my computer
here that I hope to whip into shape enough for demo purposes soon.

Regards,

Martijn

faassen at vet

Oct 19, 2001, 3:54 PM

Post #6 of 10 (1815 views)
Permalink

Re: ANNOUNCE: PT_XPath [In reply to]

delza [at] landry wrote:
> > > Wow. Very ambitious. This would turn Zope into a very full-featured
> > > XML respository. I wonder if it would be make sense to push some of the
> > > DOM support down into ZODB? I can't claim to understand all the issues
> > > there, though.
> >
> > How do you see pushing that into the ZODB? The ZODB just stores objects,
> > right, DOM or not?
>
> Well, I'm thinking in terms of a generic XML repository, like Tamino or
> XML-DB. If we're using XPath as a general query mechanism over a database, it
> makes sense to put it *in* the database. On the other hand, this wouldn't
> necessarily have to be baked into the ZODB, but could be an API which sits
> between the ZODB and Zope. Maybe I'm confusing the goals with those of
> other projects and this doesn't make sense at all.

You could use the plain ZODB as a DOM repository, which is of course nice
and would be of use in Zope. What would also be nice however is to enable
the use of XPath (and perhaps later XSLT) in other contexts, such as
existing Zope databases. For that you either need to equip current
object with DOM powers (using mixin classes), or you create a new set
of objects that knows how to give DOM interfaces to the existing data.

> One of the things I like about ParsedXML, btw, is its balance of effeciency
> with persistence: Store one object in the ZODB, but make it look like an
> entire DOM tree to the user (at least, that's how I think it works!).

Right; the entire tree is currently persistent only as a whole, not as
the individual objects. This has advantages, mostly I think storage
efficiency and probably also some use efficiency. The disadvantage is
that it doesn't scale all the way up to huge documents and its'
probably relatively inefficient for DOM manipulations, as changes to
the DOM cause the entire tree to be stored again in a new transaction..

Eventually a hybrid segmented DOM approach may be the best solution.

> Part of
> the reason I start thinking about the ZODB when we talk about querying for
> objects as if they were all XML is that it would be handy to do even outside
> of Zope, in other projects which use the ZODB. I would like to be able to use
> ParsedXML for general XML persistence from Python, but it seems to be closely
> tied to Zope.

So you're thinking about a generic way to represent arbitrary Python
objects as XML. It's an interesting idea that I need to think about a bit
more. I'm talking about a higher level representation of particular
existing Zope objects. The advantage I see in the latter is that it
may make more sense to a human than the direct representation of object
internals. An advantage of a more generic representation would be
that you represent absolutely everything, but of course that can
also be a weakness -- it is what makes XML .zexps rather hard to
read.

> > > I'm afraid I don't understand the need for a registry. Shouldn't we be
> > > patching low-level Zope objects with DOM methods so all objects inherit
> > > the DOM API? I've just started reading the ZDOMIssues document, so
> > > undoubtably I'm missing something.
> >
> > If you mean by 'low level Zope objects' things like Folder and DTML Method,
> > I'm much in favor in making new objects that layer over these
> > Zope objects. That'd be a much cleaner design. The *old* ZDOM works
> > with mixin classes, but it never got very far and I think it's
> > just too complicated.
> >
> > So I see somekind of registry that knows how to map any Zope object it
> > encounters to a DOM object (usually an element with some attributes
> > and probably some other nodes inside). This way you could even have
> > different registries that do different mappings, some more detailed
> > than the other.
>
> I can see how a registry could be useful, but I think it should be used to
> *override* some reasonable default behavior. That is, any Zopish object
> should be accessible via ZDOM, and objects which have special needs should be
> able to register those to override the defaults.

That would be an interesting compromise, though I'm still not convinced
you could actually expose the implementation details... hm, what about security
issues, for instance?

Regards,

Martijn

djay at avaya

Oct 21, 2001, 4:31 PM

Post #7 of 10 (1820 views)
Permalink

RE: ANNOUNCE: PT_XPath [In reply to]

> -----Original Message-----
> From: Martijn Faassen [mailto:faassen [at] vet]
> Sent: Saturday, 20 October 2001 8:55 AM
> To: delza [at] landry
> Cc: Dethe Elza; zope-xml [at] zope
> Subject: Re: [Zope-xml] ANNOUNCE: PT_XPath
>
>
> > One of the things I like about ParsedXML, btw, is its
> balance of effeciency
> > with persistence: Store one object in the ZODB, but make it
> look like an
> > entire DOM tree to the user (at least, that's how I think
> it works!).
>
> Right; the entire tree is currently persistent only as a whole, not as
> the individual objects. This has advantages, mostly I think storage
> efficiency and probably also some use efficiency. The disadvantage is
> that it doesn't scale all the way up to huge documents and its'
> probably relatively inefficient for DOM manipulations, as changes to
> the DOM cause the entire tree to be stored again in a new
> transaction..
>
> Eventually a hybrid segmented DOM approach may be the best solution.

Which is a much better idea if your treating a document as a database. Much
more efficient for heavy querying. The only problem is you need a flexible
way to represent the policy of what becomes objects and what elements are
stored as one object. Perhaps some extension DTD could mark where the
chunking occurs.

> > Part of
> > the reason I start thinking about the ZODB when we talk
> about querying for
> > objects as if they were all XML is that it would be handy
> to do even outside
> > of Zope, in other projects which use the ZODB. I would
> like to be able to use
> > ParsedXML for general XML persistence from Python, but it
> seems to be closely
> > tied to Zope.
>
> So you're thinking about a generic way to represent arbitrary Python
> objects as XML. It's an interesting idea that I need to think
> about a bit
> more. I'm talking about a higher level representation of particular
> existing Zope objects. The advantage I see in the latter is that it
> may make more sense to a human than the direct representation
> of object
> internals. An advantage of a more generic representation would be
> that you represent absolutely everything, but of course that can
> also be a weakness -- it is what makes XML .zexps rather hard to
> read.

I think this problem will be solved to some extent if the component
architecture is going to be a level between zope and the ZODB. I'm not sure
if it is or not but this would mean you could request a given ZODB object as
XML and it would happen for you. As to what the default view for unknown
object types is... it's kind of abitrary. You can't make much guarentees
about what the XML would look like for some new object type so why put
anything in at all. I prefere the <unknown> object representation.

> > I can see how a registry could be useful, but I think it
> should be used to
> > *override* some reasonable default behavior. That is, any
> Zopish object
> > should be accessible via ZDOM, and objects which have
> special needs should be
> > able to register those to override the defaults.
>
> That would be an interesting compromise, though I'm still not
> convinced
> you could actually expose the implementation details... hm,
> what about security
> issues, for instance?

A registry seems to be how the component archetecture is going to work so
I'd say that would be the way to go. Less change when the component
archecture does appear. Plus it was the advantage that no internal zope code
needs to be modified which makes this much better for distribution as a
product.

mj at zope

Oct 22, 2001, 8:40 AM

Post #8 of 10 (1808 views)
Permalink

Re: ANNOUNCE: PT_XPath [In reply to]

On Mon, Oct 22, 2001 at 09:31:15AM +1000, Jay, Dylan wrote:
> > Eventually a hybrid segmented DOM approach may be the best solution.
>
> Which is a much better idea if your treating a document as a database. Much
> more efficient for heavy querying. The only problem is you need a flexible
> way to represent the policy of what becomes objects and what elements are
> stored as one object. Perhaps some extension DTD could mark where the
> chunking occurs.

I think that XPath expressions would be a better siolution for specifying
the chunking borders.

> > > Part of the reason I start thinking about the ZODB when we talk about
> > > querying for objects as if they were all XML is that it would be handy
> > > to do even outside of Zope, in other projects which use the ZODB. I
> > > would like to be able to use ParsedXML for general XML persistence
> > > from Python, but it seems to be closely tied to Zope.

Note that in my opinion, XML is not really suitable as a storage format. It
is great for interchange between different entities because of the efficient
and standard choices for parsing XML, but as soon as you rely on XML for
storage, things become tricky and inefficient.

You would be much better off converting from and to XML, which is what most
storage solutions do.

I really h�ve yet to come across convincing reasons for having a DOM on the
server side, which is not a good situation to be in, seeing I am the owner
of ParsedXML at Zope Corp... :| XMLPath expressions in TAL could just maybe
be a reason..

> > So you're thinking about a generic way to represent arbitrary Python
> > objects as XML. It's an interesting idea that I need to think about a
> > bit more. I'm talking about a higher level representation of particular
> > existing Zope objects. The advantage I see in the latter is that it may
> > make more sense to a human than the direct representation of object
> > internals. An advantage of a more generic representation would be that
> > you represent absolutely everything, but of course that can also be a
> > weakness -- it is what makes XML .zexps rather hard to read.

To me the advantage would be for you to be able to use a standard,
widespread API for object tree manipulation, the W3C DOM. Except that the
DOM was designed for a particalr type of object tree, with a well-defined
and finite set of object types and manipulations. A ZODB-stored object tree
will almost always *not* fit that model, and making it fit will always
necessitate cutting out functionality and power.

--
Martijn Pieters
| Software Engineer mailto:mj [at] zope
| Zope Corporation http://www.zope.com/
| Creators of Zope http://www.zope.org/
---------------------------------------------

sean.upton at uniontrib

Oct 22, 2001, 8:48 AM

Post #9 of 10 (1795 views)
Permalink

RE: ANNOUNCE: PT_XPath [In reply to]

What would be nice, perhaps is a new product: 'bigXML' is a folderish object
containing 1..n ParsedXML objects, and acts as a proxy component that uses
each ParsedXML instance to store DocumentFragments, as delineated by rules
written as a bunch of properties in that folder that constitute rules for
what gets stored where (perhaps expressed using XPath as a notation).

This would allow for use of the existing ParsedXML framework, and allow
improvements made to it to trickle into any product that proxied it. I know
there is only so many degrees of 'proxying' that we should do, but this
seems like a simple enough approach to get the job done? It would allow any
document adminstrator that could write XPath to manage how branches are
stored, which means they can control how the XML is optimized for their
particular use case.

Sean

-----Original Message-----
From: Jay, Dylan [mailto:djay [at] avaya]
Sent: Sunday, October 21, 2001 4:31 PM
To: zope-xml [at] zope
Subject: RE: [Zope-xml] ANNOUNCE: PT_XPath

> -----Original Message-----
> From: Martijn Faassen [mailto:faassen [at] vet]
> Sent: Saturday, 20 October 2001 8:55 AM
> To: delza [at] landry
> Cc: Dethe Elza; zope-xml [at] zope
> Subject: Re: [Zope-xml] ANNOUNCE: PT_XPath
>
>
> > One of the things I like about ParsedXML, btw, is its
> balance of effeciency
> > with persistence: Store one object in the ZODB, but make it
> look like an
> > entire DOM tree to the user (at least, that's how I think
> it works!).
>
> Right; the entire tree is currently persistent only as a whole, not as
> the individual objects. This has advantages, mostly I think storage
> efficiency and probably also some use efficiency. The disadvantage is
> that it doesn't scale all the way up to huge documents and its'
> probably relatively inefficient for DOM manipulations, as changes to
> the DOM cause the entire tree to be stored again in a new
> transaction..
>
> Eventually a hybrid segmented DOM approach may be the best solution.

Which is a much better idea if your treating a document as a database. Much
more efficient for heavy querying. The only problem is you need a flexible
way to represent the policy of what becomes objects and what elements are
stored as one object. Perhaps some extension DTD could mark where the
chunking occurs.

> > Part of
> > the reason I start thinking about the ZODB when we talk
> about querying for
> > objects as if they were all XML is that it would be handy
> to do even outside
> > of Zope, in other projects which use the ZODB. I would
> like to be able to use
> > ParsedXML for general XML persistence from Python, but it
> seems to be closely
> > tied to Zope.
>
> So you're thinking about a generic way to represent arbitrary Python
> objects as XML. It's an interesting idea that I need to think
> about a bit
> more. I'm talking about a higher level representation of particular
> existing Zope objects. The advantage I see in the latter is that it
> may make more sense to a human than the direct representation
> of object
> internals. An advantage of a more generic representation would be
> that you represent absolutely everything, but of course that can
> also be a weakness -- it is what makes XML .zexps rather hard to
> read.

I think this problem will be solved to some extent if the component
architecture is going to be a level between zope and the ZODB. I'm not sure
if it is or not but this would mean you could request a given ZODB object as
XML and it would happen for you. As to what the default view for unknown
object types is... it's kind of abitrary. You can't make much guarentees
about what the XML would look like for some new object type so why put
anything in at all. I prefere the <unknown> object representation.

> > I can see how a registry could be useful, but I think it
> should be used to
> > *override* some reasonable default behavior. That is, any
> Zopish object
> > should be accessible via ZDOM, and objects which have
> special needs should be
> > able to register those to override the defaults.
>
> That would be an interesting compromise, though I'm still not
> convinced
> you could actually expose the implementation details... hm,
> what about security
> issues, for instance?

A registry seems to be how the component archetecture is going to work so
I'd say that would be the way to go. Less change when the component
archecture does appear. Plus it was the advantage that no internal zope code
needs to be modified which makes this much better for distribution as a
product.

_______________________________________________
Zope-xml mailing list
Zope-xml [at] zope
http://lists.zope.org/mailman/listinfo/zope-xml

kra at monkey

Oct 24, 2001, 2:26 PM

Post #10 of 10 (1810 views)
Permalink

Re: ANNOUNCE: PT_XPath [In reply to]

Martijn Pieters <mj [at] zope> writes:

> I really h�ve yet to come across convincing reasons for having a DOM on the
> server side [...]

> To me the advantage would be for you to be able to use a standard,
> widespread API for object tree manipulation, the W3C DOM.

That's not a convincing reason? :)

I personally am only excited about the DOM because it is a standard -
a DOM interface allows tools that consume a DOM tree, such as XPath
and XSLT engines, to work with your objects.

> Except that the
> DOM was designed for a particalr type of object tree, with a well-defined
> and finite set of object types and manipulations. A ZODB-stored object tree
> will almost always *not* fit that model, and making it fit will always
> necessitate cutting out functionality and power.

Really? I'm not coming up with a good example of that.

The DOM fulfills the XML Infoset, the Infoset represents tree-based
data - an XML document also represents an Infoset instance, modulo
syntactic noise such as whitespace and entity references.

Any Python object can be represented by XML, and unless your XML
representation relies on that syntactic noise (and it shouldn't),
it can also be represented by a DOM tree.

Of course, any one-size-fits-all XML object representation will be
unwieldy for many objects, but I don't see why that matters
(currently). If you want XPath, don't look at the ugly DOM tree, let
the XPath processor do that for you. If you do need a cleaner DOM
tree - for my favorite example, if you want a StructuredText
representation of a Zope object that reads and writes via its DOM, and
you want a cleaner representation presented before the StructuredText
layer - then clean up what your processor sees with DOM Traversal or
XSLT.

Or, if you have a specialized XML or DOM usage, or need more
efficiency, use a specialized DOM representation for your class. That
will be an issue anyway, I think, since the DOM is document-centric
and people won't want to use the entire ZODB as a single document.
There will be times when the expected usage will influence the
representation.

But these are all efficiency issues, not functionality issues.

--
Karl Anderson kra [at] monkey http://www.monkey.org/~kra/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads