Mailing List Archive: Bricolage: devel

[Bricolage-Devel] A few questions [LONG]

Index | Next | Previous | View Threaded

darren at iterx

Feb 9, 2002, 3:43 PM

Post #1 of 22 (5505 views)
Permalink

[Bricolage-Devel] A few questions [LONG]

Firstly a big thanks for putting together Bricolage, I had just about
given up looking for a usable Perl based CMS.

Anyway, I've knocked together a prototype a system with Bricolage
providing the CMS functionality. During this process I have run into a
few questions + bugs that hopefully you can resolve (apologies if any of
these are documented on the todo list or as known bugs - I've only skip
read through these documents). In no particular order:

* The media publishing mechanism seems to break if you have a media item
with an unknown content type (I tried to publish a .gz file). It works
fine if you add the appropriate information to the media_type and
media_type_ext tables. Also, is there any reason why .gz files are
missing from the default entries (given that we have things such as
.tgz).

* There a a few typos in Bric::Biz::Asset::Business. Briefly, put
get_all_keywords() only returned the assets keywords and it's
get_source__id() no get_source_id(). I've attached a patch fixing this.
Also, the patch contains a modification to enable a slug to be specified
for a "fixed" page (I can't see any good reason as to why you can't
specify one - though I might be missing something).

* Why can't a subelement of an element be the element again. For
instance I want to define nested section containers recursively i.e

section
section
section

I've attached a patch that enables this.

* Why are elements tied to a specific burner? I can't quite see the
logic behind this. Surely, you should just assign a burner to an output
channel, thus enabling element reuse if you ever decide to use a
different burner (or want to mix and match).

* I'm not sure that the user/group authorisation mechanism is 100%
correct. At present, it isn't possible to give a user membership to the
user admin or group admin groups without also giving them the ability to
upgrade themselves to become global administrators.

I've also been playing around retrofitting an XML/XSLT burner into
Bricolage. At present, there are two parts to the burner. There is a
generator, that automatically converts an asset object into XML and
then, if any XSLT templates have been specified (via Bricolage's
templating mechanism), to apply them before outputing the results. At
present, it's build around LibXML/LibXSLT, but there is no reason why
this dependancy can't be abstracted out, so that other XSLT processors
can pluged in.

The generator simply iterates through the tree, creating XML nodes for
each element within the asset in turn. The node is generated using the
following rules:

* The node name is set to the element or field names converted to lower
case, spaces removed and anything in brackets stripped.

* Any element or field names beginning with @ are represented by
attributes.

* The story metadata is automatically inserted under the root node, with
the name info appended to the root name e.g:

<article>
<articleinfo>

</articleinfo>
</article>

* Related stories and media items are represented by xlinks

This is flexible enough for you to be able to model things such as
editing docbook documents within Bricolage (though, things such as
editing tables is a bit cumbersome).

Anyway, I've included a patch containing my progress so far. The main
thing that is currently missing is documentation and examples, which I
can easily knock together if requested. Debugging XSLT templates is/can
be painful under the current system (in part due to LibXML/LibXSLT being
particularly bad in this respect) and there is no provision for things
such as namespacing.

Also, I've had some thoughts as to how to retrofit namespacing and
greater flexibility in the auto generation of XML. Firstly, we can add
namespacing by just adding a new hidden custom field type to Bricolage.
In this we can squirrel away the namespace decalarations and any other
static data. Secondly, if we add a label property to an element (in a
similar way to a field), then we can use the name as a pseudo XPath
mapping to where the value should be inserted into the document tree,
e.g

@foo => insert as attribute of the current node
data/foo => insert <data><foo>[VALUE]</foo></data> under the
current node

Hopefully, the above makes some kind of sense... it doesn't help that
there is a real terminology problem with the use of the word element ;-)

Thanks...

Darren

Attachments:

bricolage-business.patch (1.57 KB)

bricolage-recursive-elements.patch (0.87 KB)

bricolage-xml.patch (18.8 KB)

david at kineticode

Feb 9, 2002, 6:44 PM

Post #2 of 22 (5182 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On Sat, 2002-02-09 at 15:43, Darren Graves wrote:
> Firstly a big thanks for putting together Bricolage, I had just about
> given up looking for a usable Perl based CMS.

Cool -- glad to have you joining us!

> * The media publishing mechanism seems to break if you have a media item
> with an unknown content type (I tried to publish a .gz file).

<snip />

I believe I fixed this problem with the 1.2.0 release. See the
DEF_MEDIA_TYPE directive as documented in Bric::Admin. I'm aware that
the media type management could use an interface for adding media types,
however. I'll add that to the To Do list.

> * There a a few typos in Bric::Biz::Asset::Business. Briefly, put
> get_all_keywords() only returned the assets keywords and it's
> get_source__id() no get_source_id(). I've attached a patch fixing this.

Thanks -- good catch! I've added the patch.

> Also, the patch contains a modification to enable a slug to be specified
> for a "fixed" page (I can't see any good reason as to why you can't
> specify one - though I might be missing something).

Well, the original idea was that Fixed stories wouldn't have slugs. But
with the proposed addition of a preference specifying how story URIs are
built, it will become something we'll want. Right now, slugs aren't used
for anything for fixed stories, but in anticipation of the preference,
I'll go ahead and apply your patch so that they're at least stored in
the database for future use.

> * Why can't a subelement of an element be the element again. For
> instance I want to define nested section containers recursively i.e

<snip />

Because one should not be able to make top-level elements (i.e., story
and media types) be subelements. Actually, I think it'd be cool to allow
this eventually, but I think it'd cause more problems than it's worth at
this point.

But I don't see why non-top-level elements can't contain themselves. In
that case, the change would be

$excl_sub = sub { $_->[0]->get_top_level || ! chk_authz($_[0], READ, 1) };

What does everybody think?

> * Why are elements tied to a specific burner? I can't quite see the
> logic behind this. Surely, you should just assign a burner to an output
> channel, thus enabling element reuse if you ever decide to use a
> different burner (or want to mix and match).

I'm not sure about this. Sam would know better, I think.

> * I'm not sure that the user/group authorisation mechanism is 100%
> correct. At present, it isn't possible to give a user membership to the
> user admin or group admin groups without also giving them the ability to
> upgrade themselves to become global administrators.

Well, one could call this a feature. In most systems that I'm aware of,
if you make someone an administrator, that means they can administer
themselves. Um, but you're right, they can't *increase* their own
permissions. Would you file a bug report on that? I'll have to think
about how to work around that issue.

http://bricolage-bugzilla.thepirtgroup.com/

> I've also been playing around retrofitting an XML/XSLT burner into
> Bricolage. At present, there are two parts to the burner. There is a
> generator, that automatically converts an asset object into XML and
> then, if any XSLT templates have been specified (via Bricolage's
> templating mechanism), to apply them before outputing the results. At
> present, it's build around LibXML/LibXSLT, but there is no reason why
> this dependancy can't be abstracted out, so that other XSLT processors
> can pluged in.

Excellent! You might want to look at the SOAP stuff that Sam is writing
for the 1.4.0 release. It's in CVS right now.

> The generator simply iterates through the tree, creating XML nodes for
> each element within the asset in turn. The node is generated using the
> following rules:
>
> * The node name is set to the element or field names converted to lower
> case, spaces removed and anything in brackets stripped.
>
> * Any element or field names beginning with @ are represented by
> attributes.
>
> * The story metadata is automatically inserted under the root node, with
> the name info appended to the root name e.g:
>
> <article>
> <articleinfo>
> 
> </articleinfo>
> </article>
> * Related stories and media items are represented by xlinks

Hmmm...I think I'd rather see each of these in an "Element" tag that
then has a "name" attribute. But again, you should get with Sam on what
he's doing with SOAP. I think he might be creating a DTD for Bricolage
stories.

> This is flexible enough for you to be able to model things such as
> editing docbook documents within Bricolage (though, things such as
> editing tables is a bit cumbersome).

<snip />

This looks like a good contribution. Hopefully I can look at it in more
detail so I can respond to your message some more later this week.

Regards,

David
--
David Wheeler AIM: dwTheory
david [at] kineticode ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory [at] jabber
Kineticode. Setting knowledge in motion.(sm)

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

sam at tregar

Feb 9, 2002, 7:11 PM

Post #3 of 22 (5188 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On 9 Feb 2002, Darren Graves wrote:

> * Why are elements tied to a specific burner? I can't quite see the
> logic behind this. Surely, you should just assign a burner to an output
> channel, thus enabling element reuse if you ever decide to use a
> different burner (or want to mix and match).

The idea was that you might want to use different burners for different
story types within a single output channel. Which is a pretty weird idea,
isn't it? I think I'd be in favor of a patch that moved the choice to the
output channel.

> I've also been playing around retrofitting an XML/XSLT burner into
> Bricolage.

Neat! Have you had a look at my Bric::SOAP development project? You can
look through the docs at:

http://bricolage.thepirtgroup.com/docs/Bric/SOAP.html

> There is a generator, that automatically converts an asset object into
> XML

Check out Bric::SOAP::Story->export() for my implementation of this
functionality. I went with a more rigid structure for asset data, partly
due to the fact that I was building an XML Schema for it. Here's a sample
story with some elements:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assets xmlns="http://bricolage.sourceforge.net/assets.xsd">
<story id="1024" element="Story">
<name>Story One</name>
<description>a description of story one</description>
<slug>one</slug>
<primary_uri>/2002/01/29/one</primary_uri>
<priority>3</priority>
<publish_status>0</publish_status>
<active>1</active>
<source>Internal</source>
<cover_date>2002-01-29T20:02:00Z</cover_date>
<categories>
<category primary="1">/</category>
</categories>
<keywords>
<keyword>key one</keyword>
<keyword>key two</keyword>
</keywords>
<contributors></contributors>
<elements>
<data order="1" element="Deck">deck one</data>
<container order="1" element="Page">
<data order="1" element="Paragraph">para one</data>
<data order="2" element="Paragraph">para two</data>
<container order="1" element="Inset">
<data order="1" element="Copy">inset copy</data>
</container>
</container>
</elements>
</story>
</assets>

You can find the XML Schema for this document in the Bric::SOAP docs I
pointed at above.

-sam

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

darren at iterx

Feb 11, 2002, 4:07 AM

Post #4 of 22 (5180 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

> I believe I fixed this problem with the 1.2.0 release. See the
> DEF_MEDIA_TYPE directive as documented in Bric::Admin. I'm aware that
> the media type management could use an interface for adding media types,
> however. I'll add that to the To Do list.

I was using a pretty recent CVS version (it was certainly later than
1.2.0). If I get some time, I'll have a deeper look at what was going
wrong and this time acutally record the error message ;-)

<snip>
> Because one should not be able to make top-level elements (i.e., story
> and media types) be subelements. Actually, I think it'd be cool to allow
> this eventually, but I think it'd cause more problems than it's worth at
> this point.
>
> But I don't see why non-top-level elements can't contain themselves. In
> that case, the change would be
>
> $excl_sub = sub { $_->[0]->get_top_level || ! chk_authz($_[0], READ, 1) };
>
> What does everybody think?

Ahhh.... I thought there was a catch. However, the above makes sense
(personally, I only need it for non-top-level elements).

<snip>
> > I've also been playing around retrofitting an XML/XSLT burner into
> > Bricolage. At present, there are two parts to the burner. There is a
> > generator, that automatically converts an asset object into XML and
> > then, if any XSLT templates have been specified (via Bricolage's
> > templating mechanism), to apply them before outputing the results. At
> > present, it's build around LibXML/LibXSLT, but there is no reason why
> > this dependancy can't be abstracted out, so that other XSLT processors
> > can pluged in.
>
> Excellent! You might want to look at the SOAP stuff that Sam is writing
> for the 1.4.0 release. It's in CVS right now.

I've had a good look now.

> > The generator simply iterates through the tree, creating XML nodes for
> > each element within the asset in turn. The node is generated using the
> > following rules:
> >
> > * The node name is set to the element or field names converted to lower
> > case, spaces removed and anything in brackets stripped.
> >
> > * Any element or field names beginning with @ are represented by
> > attributes.
> >
> > * The story metadata is automatically inserted under the root node, with
> > the name info appended to the root name e.g:
> >
> > <article>
> > <articleinfo>
> > 
> > </articleinfo>
> > </article>
> > * Related stories and media items are represented by xlinks
>
> Hmmm...I think I'd rather see each of these in an "Element" tag that
> then has a "name" attribute. But again, you should get with Sam on what
> he's doing with SOAP. I think he might be creating a DTD for Bricolage
> stories.

My main problems with outputting XML along the lines of (which I take is
the structure your alluding to):

<element name="article>
<element name="articleinfo">
....
</element>
</element>

is that firstly it starts to become really cumbersome when you start to
apply XSLT transforms,etc to them. e.g

<xsl:template
match="element[@name='article']/element[@name='articleinfo']">
vs
<xsl:template match="article/articleinfo">

and it only gets worse the complexity of your template increases.
Secondly, from an XML centric point of view, IMHO it's not a very
natural way to model the data (for a dead start I think that the
"element" element is redundent). One of the reasons I choose to output
the data in the way that I did (though currently, its far from perfect),
was because I could use the Bricolage elements to model a story, that
would output in most cases (without need to apply XSLT transforms, etc)
something that conformed to the XML output DTD I required.

Darren

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

darren at iterx

Feb 11, 2002, 5:11 AM

Post #5 of 22 (5183 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

<snip>
> > I've also been playing around retrofitting an XML/XSLT burner into
> > Bricolage.
>
> Neat! Have you had a look at my Bric::SOAP development project? You can
> look through the docs at:
>
> http://bricolage.thepirtgroup.com/docs/Bric/SOAP.html

I've now had a chance to have a better look at the SOAP stuff.
Certainly, my initial reaction is that if possible, things like the XML
representation of an XML object should be the same. Anyway, I thought
I'd note down some of my initial feedback/thoughts on the SOAP docs and
how XML support can be better integrated within Bricolage (however, this
comes with the warning that certain aspects haven't been fully thought
through).

Firstly, for the asset data, I can understand why you went for the XML
structure you did, but where there any reasons/problems apart from
general complexity for not to do a more natural mapping. Personally, I
can see quite a few advantages for going structure like:

<asset>
<story>

<deck>deck one</deck>
<paragraph>...</paragraph>
<inset>

</inset>
</story>
</asset>

Firstly, I feel this is a more direct representation of the data
represented within the asset object (rather than a representation of how
Bricolage currently stores the data). One of the major downsides to
this, is that we would need to dynamically generate the XML schemas for
the asset.

This raises also an interesting point with the current implementation.
With the current schema, we can only easily validate that the the story
XML is a vaild story interchange document - not that the underlying
story data is valid (I assume that we would need to go through the
document element by element, grabbing the details for the element
definitions and validating against that).

Personally (from an XML point of view), I like to view the element
definitions within Bricolage as a XML Schema/DTD definition and stories,
etc instances of these. If you take it to the extreme, I'd think it
might be cool to take your "industry standard" schema and just be able
to import it straight into Bricolage, which would automatically create
the element definations for you (maybe with some prompting for you to
decide what widgets should be used in a particular instance). This also
would make things such as importing XML documents directly into
Bricolage simpler, etc (I'm not saying this can't be done with the
current scheme, it's just going to be from a users prespective
harder/require more preprocessing). Also, once you can export the
Bricolage element definitions as a say an XML schema, you open the
possibility of integrating in 3rd party XML editors,etc for editing the
stories.

Finally, a couple of random points regarding the current Bricolage SOAP
specification:

* With the namespace for the asset document, it's probably best to point
to some netural URI rather than the XML schema. e.g
xmlns="http://bricolage.sourceforge.net/assets" rather than
xmlns="http://bricolage.sourceforge.net/assets.xsd"
That way you can easily set up a convention to point to other types of
schemas (e.g just append the file extension). Also, have you considered
using something like purl (www.purl.org) to have a less sourceforge
dependant name (just in case you choose to host else where at a latter
date and given the relative pain of changing namespaces).

* I think that the order attribute within containers elements for a
document is really overkill. Within XML documents, the order of the
elements is usually taken to be significant and you ignore at your own
peril.

Anyway, that's my random thoughts/feelings for now.

Darren

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

david at kineticode

Feb 11, 2002, 7:50 AM

Post #6 of 22 (5182 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On Mon, 2002-02-11 at 04:07, Darren Graves wrote:

> I was using a pretty recent CVS version (it was certainly later than
> 1.2.0). If I get some time, I'll have a deeper look at what was going
> wrong and this time acutally record the error message ;-)

If you find it's not working for some reason, please file a bug.

> > $excl_sub = sub { $_->[0]->get_top_level || ! chk_authz($_[0], READ, 1) };
> >
> > What does everybody think?

Actually, now that I think about it, this is wrong. It would have to be

$excl_sub = sub { ($->[0]->get_id == $id && $_->[0]->get_top_level)
|| ! chk_authz($_[0], READ, 1) ];

However, that's overkill, since, IIRC, none of those elements will ever
be top-level, because the call to list() has top_level => 0. So
actually, your original patch should work fine.

$excl_sub = sub { ! chk_authz($_[0], READ, 1) };

Anyway, anybody got any objections to allowing subelements be
subelements of themselves?

> > Excellent! You might want to look at the SOAP stuff that Sam is writing
> > for the 1.4.0 release. It's in CVS right now.
>
> I've had a good look now.

Cool -- I saw your message to Sam. Thanks for the feedback! More
eyeballs will definitely make Bricolage better.

<snip />

> and it only gets worse the complexity of your template increases.
> Secondly, from an XML centric point of view, IMHO it's not a very
> natural way to model the data (for a dead start I think that the
> "element" element is redundent). One of the reasons I choose to output
> the data in the way that I did (though currently, its far from perfect),
> was because I could use the Bricolage elements to model a story, that
> would output in most cases (without need to apply XSLT transforms, etc)
> something that conformed to the XML output DTD I required.

But isn't the problem with that that the DTD would have to be different
for every story type?

Thanks for all your work and thoughts -- big help!

Regards,

David
--
David Wheeler AIM: dwTheory
david [at] kineticode ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory [at] jabber
Kineticode. Setting knowledge in motion.(sm)

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

david at kineticode

Feb 11, 2002, 7:53 AM

Post #7 of 22 (5181 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On Mon, 2002-02-11 at 05:11, Darren Graves wrote:

<snip />

> Also, have you considered
> using something like purl (www.purl.org) to have a less sourceforge
> dependant name (just in case you choose to host else where at a latter
> date and given the relative pain of changing namespaces).

Actually, I was thinking of getting a permanent URL home for Bricolage.
bricolagehq.* is available. What does everybody think? Other name
suggestions are welcome. I figure we can just point such a domain name
at bricolage.thepirtgroup.com.

I'm going to leave the rest of this discussion for you and Sam to hammer
out for now. I'm moving today!

Regards,

David
--
David Wheeler AIM: dwTheory
david [at] kineticode ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory [at] jabber
Kineticode. Setting knowledge in motion.(sm)

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

sam at tregar

Feb 11, 2002, 8:04 AM

Post #8 of 22 (5181 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On 11 Feb 2002, Darren Graves wrote:

> Firstly, for the asset data, I can understand why you went for the XML
> structure you did, but where there any reasons/problems apart from
> general complexity for not to do a more natural mapping. Personally, I
> can see quite a few advantages for going structure like:

The main reason was validation. There's no easy way to validate a
structure with variable element names. You can't publish an XML schema
and describing it takes long-winded paragraphs.

Also, ease of implementation. I needed to strike a balance between
finding a nice format and being close enough to Bricolage's internal
data-structure to make writing the interface layer possible. Every so
often I need to step back and remind myself that I'm not going to solve
all of Bricolage's problems in the SOAP interface - some problems will
need deeper solutions.

> This raises also an interesting point with the current implementation.
> With the current schema, we can only easily validate that the the story
> XML is a vaild story interchange document - not that the underlying
> story data is valid (I assume that we would need to go through the
> document element by element, grabbing the details for the element
> definitions and validating against that).

Correct. This is a general problem with XML Schemas - they're only
capable of validating two properties: data structure and data types.
Obviously any reasonably complicated application is going to have
constraints that XML Schema can't cover.

> Personally (from an XML point of view), I like to view the element
> definitions within Bricolage as a XML Schema/DTD definition and stories,
> etc instances of these.

This, I think, is the way of the future. I've had thoughts along these
lines many times during the SOAP project. The problem is, it's not
feasible right now. It would take a total revision of the Asset
management systems to even approach this kind of a design.

> * With the namespace for the asset document, it's probably best to point
> to some netural URI rather than the XML schema. e.g
> xmlns="http://bricolage.sourceforge.net/assets" rather than
> xmlns="http://bricolage.sourceforge.net/assets.xsd"
> That way you can easily set up a convention to point to other types of
> schemas (e.g just append the file extension).

My thinking in putting .xsd there was to draw attention to the fact that
there's an XML Schema for the document. It's just a hint, but it might
keep people from asking "where's the DTD" too often!

> * I think that the order attribute within containers elements for a
> document is really overkill. Within XML documents, the order of the
> elements is usually taken to be significant and you ignore at your own
> peril.

Not true. Consider that Bricolage allows you to intersperse Data and
Container objects inside a Container. Since the Schema requires all Data
before Containers the only way to communicate this is with an order
attribute.

> Anyway, that's my random thoughts/feelings for now.

Keep 'em coming! I'm happy to have someone looking over my work.

-sam

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

drew at drewtaylor

Feb 11, 2002, 8:24 AM

Post #9 of 22 (5183 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

At 07:53 AM 2/11/2002 -0800, David Wheeler wrote:

>Actually, I was thinking of getting a permanent URL home for Bricolage.
>bricolagehq.* is available. What does everybody think? Other name
>suggestions are welcome. I figure we can just point such a domain name
>at bricolage.thepirtgroup.com.

That style domain works for Mason, so I say go for it.

Drew

Drew Taylor JA[P|m_p|SQL]H
http://www.drewtaylor.com/ Just Another Perl|mod_perl|SQL Hacker
mailto:drew [at] drewtaylor *** God bless America! ***

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

darren at iterx

Feb 12, 2002, 8:32 AM

Post #10 of 22 (5182 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

Grrr... you've gone and done it now - I've now forced myself to properly
read the XML Schema specification ;-) Personally, I'm not a big fan of
XML Schema, much prefering something like RELAX NG or Schematron.
Anyway, based on my still limited understanding of XML Schemas...

> > Firstly, for the asset data, I can understand why you went for the XML
> > structure you did, but where there any reasons/problems apart from
> > general complexity for not to do a more natural mapping. Personally, I
> > can see quite a few advantages for going structure like:
>
> The main reason was validation. There's no easy way to validate a
> structure with variable element names. You can't publish an XML schema
> and describing it takes long-winded paragraphs.

Are we talking at cross purposes here - I proposing that we dynamically
generate an XML Schema covering the structure of the data, based upon
the users content model they have set up (I'll ignore the point of how
we come up with that schema for the moment). We can then validate the
schema against that and nor does it matter how verbose that schema is
(since it's dynamically generated).

In fact using XML Schema, I can see a way to break down the problem into
2 parts. We can pretty much use the current asset XML description/XML
Schema as a wrapper for the data. The asset XML schema can then just
have

<xsd:any namespace="http://..." processContent="strict" />

to covering the main data. When it comes to validation, the processor
should then pull in the data specific schema to validate it.

As for generating the XML Schema, well the current Bricolage element and
field definitions can be interogated. Then it's relatively straight
forward to express these as an XML Schema and assigned into the owners
namespace.

> Also, ease of implementation. I needed to strike a balance between
> finding a nice format and being close enough to Bricolage's internal
> data-structure to make writing the interface layer possible. Every so
> often I need to step back and remind myself that I'm not going to solve
> all of Bricolage's problems in the SOAP interface - some problems will
> need deeper solutions.

I agree whole heartedly (quite a bit of what I'm suggesting is probably
not practicle in the short term) - part of what I trying to do is get a
better feel for how far we can push the integration of XML into
Bricolage (from my point of view, I need to get working a fairly XML
centric CMS).

<snip>
> > Personally (from an XML point of view), I like to view the element
> > definitions within Bricolage as a XML Schema/DTD definition and stories,
> > etc instances of these.
>
> This, I think, is the way of the future. I've had thoughts along these
> lines many times during the SOAP project. The problem is, it's not
> feasible right now. It would take a total revision of the Asset
> management systems to even approach this kind of a design.

Personally, I don't think that the current implementation is too far
away from doing this (ignoring things such as importing general schema
and creating element definitions), though certain aspects wouldn't be
pretty. However, there is quite a hefty chunk of work associated with
this and there are still important issues as to things such as what do
you use to do XML Schema validation under Perl.

Anyway, if I get some spare time, I might have a go at prototyping
dynamically generating the XML Schemas, etc (though no promises).

On a different level, this also raises the underlying question of how
far do people want XML support within Bricolage to go in the future. Is
making the internal element definitions effectively an XML schema taking
things a little too far?

<snip>
> > * I think that the order attribute within containers elements for a
> > document is really overkill. Within XML documents, the order of the
> > elements is usually taken to be significant and you ignore at your own
> > peril.
>
> Not true. Consider that Bricolage allows you to intersperse Data and
> Container objects inside a Container. Since the Schema requires all Data
> before Containers the only way to communicate this is with an order
> attribute.

True, I missed that. Thinking about it, it's a bit poor that you can't
easily express interspersed Data and Container objects within XML
Schemas. However, can't you do something like (if I'm reading the
specifcation correctly):

<xsd:group name="bric-objects">
<xsd:choice>
<xsd:element name="data">....</xsd:element>
<xsd:element name="container">....</xsd:element>
</xsd:choice>
<xsd:group>

and then in the asset schema definition

....
<xsd:element name="elements">
<xsd:complexType>
<xsd:sequence>
<xsd:element ref="bric-objects" minOccurs="0" maxOccurs="unbounded"/>
</xsd:sequence>
....

Darren

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

david at kineticode

Feb 12, 2002, 9:14 AM

Post #11 of 22 (5184 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On Tue, 2002-02-12 at 08:32, Darren Graves wrote:

<snip />

> On a different level, this also raises the underlying question of how
> far do people want XML support within Bricolage to go in the future. Is
> making the internal element definitions effectively an XML schema taking
> things a little too far?

Yes, it is. But if it *does* make sense to do a kind of dynamic schema
definition like you're talking about, then it makes sense to me that the
SOAP interface for elements would take Schemas for import and export.
That'd be kinda cool, actually. What do you two think?

Regards,

David
--
David Wheeler AIM: dwTheory
david [at] kineticode ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory [at] jabber
Kineticode. Setting knowledge in motion.(sm)

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

sam at tregar

Feb 12, 2002, 10:32 AM

Post #12 of 22 (5187 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On 12 Feb 2002, Darren Graves wrote:

> Grrr... you've gone and done it now - I've now forced myself to properly
> read the XML Schema specification ;-) Personally, I'm not a big fan of
> XML Schema, much prefering something like RELAX NG or Schematron.

Yeah, they're nice enough but they're not W3C and as a result they're not
being used nearly as much as XML Schema. Just look at the SOAP spec for a
good example - it has XML Schema built-in with some vague hand-waving
about allowing other schema systems.

> > The main reason was validation. There's no easy way to validate a
> > structure with variable element names. You can't publish an XML schema
> > and describing it takes long-winded paragraphs.
>
> Are we talking at cross purposes here - I proposing that we dynamically
> generate an XML Schema covering the structure of the data, based upon
> the users content model they have set up (I'll ignore the point of how
> we come up with that schema for the moment). We can then validate the
> schema against that and nor does it matter how verbose that schema is
> (since it's dynamically generated).

Right, I think I see where you're going. The problem is, in my opinion,
that XML Schemas have two uses - run-time validation and programmer
documentation. A dynamic schema might be great for run-time validation
but it stinks as documentation. It would take numerous examples to convey
the range of potential schemas.

> As for generating the XML Schema, well the current Bricolage element and
> field definitions can be interogated.

This is *far* more difficult than you might imagine! There's some deep
cruft in the element and field definition system. I have grand schemes to
clean it out but Bric::SOAP has to come first.

> I agree whole heartedly (quite a bit of what I'm suggesting is probably
> not practicle in the short term) - part of what I trying to do is get a
> better feel for how far we can push the integration of XML into
> Bricolage (from my point of view, I need to get working a fairly XML
> centric CMS).

Ok, that's cool. Note, however, that Bricolage isn't even close to being
XML-centric. On the contrary, it's highly RDBMS-centric with a dash of
Java-esque OO for flavor. Changing that would be roughly equivalent to a
full rewrite. We can certainly build XML services onto the Bricolage core
- Bric::SOAP, Bric::Util::Burner::XSLT, etc - but that's not going to make
Bricolage XML-centric!

> However, there is quite a hefty chunk of work associated with
> this and there are still important issues as to things such as what do
> you use to do XML Schema validation under Perl.

I have two options at present, neither of which are ready for prime-time

- The xerces-c++ project has XML Schema support in their CVS version.
This is what I use to run regression tests at present - see
lib/Bric/SOAP/Story.pl for an example. If/when they release 1.7 then
I might be able to use their XML::Xerces perl library to do run-time
validation.

- Andy Wardley wrote a Perl module called XML::Schema and then
disappeared. I have a copy of version 0.05 and it looks pretty sweet.
If someone, maybe me, releases it with some improvements then this
might be a viable alternative.

> On a different level, this also raises the underlying question of how
> far do people want XML support within Bricolage to go in the future. Is
> making the internal element definitions effectively an XML schema taking
> things a little too far?

XML for XML's sake? That would be going too far. On the other hand, if
we can find uses for XML technology that let us do new things then I'm all
for it.

> True, I missed that. Thinking about it, it's a bit poor that you can't
> easily express interspersed Data and Container objects within XML
> Schemas. However, can't you do something like (if I'm reading the
> specifcation correctly):

That looks like it would work. Let me put it into XML Spy and Xerces and
see if they both like it. Thanks!

-sam

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

sam at tregar

Feb 12, 2002, 10:38 AM

Post #13 of 22 (5181 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On 12 Feb 2002, David Wheeler wrote:

> Yes, it is. But if it *does* make sense to do a kind of dynamic schema
> definition like you're talking about, then it makes sense to me that the
> SOAP interface for elements would take Schemas for import and export.
> That'd be kinda cool, actually. What do you two think?

It's a great idea whose time has not yet come. The XML Schema parsing and
generation technology simply isn't available.

Also, I think we need to do some cleanup of the underlying element
definition system before doing this will be feasible. I've got some ideas
about that - look for an RFC in a couple weeks after I finish a first
draft of Bric::SOAP::Element.

-sam

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

darren at iterx

Feb 12, 2002, 11:14 AM

Post #14 of 22 (5184 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

> > I was using a pretty recent CVS version (it was certainly later than
> > 1.2.0). If I get some time, I'll have a deeper look at what was going
> > wrong and this time acutally record the error message ;-)
>
> If you find it's not working for some reason, please file a bug.

Actually, I found out what was wrong. In the bricolage.conf file, the
default for DEF_MEDIA_TYPE is:

DEF_MEDIA_TYPE = 'text/html'

Of course, when this is used internally the quotes remain. This screws
up the query to find the media type id. Changing it to remove the
quotes, i.e

DEF_MEDIA_TYPE = text/html

fixes everything.

Darren

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

david at kineticode

Feb 12, 2002, 4:00 PM

Post #15 of 22 (5183 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On Tue, 2002-02-12 at 11:14, Darren Graves wrote:

<snip />

> DEF_MEDIA_TYPE = text/html
>
> fixes everything.

Duh. Thanks for spotting that, Darren! I've fixed it in the Release_1_2
branch of CVS.

Regards,

David

--
David Wheeler AIM: dwTheory
david [at] kineticode ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory [at] jabber
Kineticode. Setting knowledge in motion.(sm)

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

david at kineticode

Feb 16, 2002, 6:07 PM

Post #16 of 22 (5181 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On Tue, 2002-02-12 at 10:38, Sam Tregar wrote:

> It's a great idea whose time has not yet come. The XML Schema parsing and
> generation technology simply isn't available.

Um, maybe I'm showing my ignorance here, but isn't an XML Schema
document an XML document? You could create it with XML::Writer or some
such, and then use a validator to validate it, no?

> Also, I think we need to do some cleanup of the underlying element
> definition system before doing this will be feasible. I've got some ideas
> about that - look for an RFC in a couple weeks after I finish a first
> draft of Bric::SOAP::Element.

I heartily agree with this. I hope I can get some time to work on it,
too, in the next couple of months...

Regards,

David

--
David Wheeler AIM: dwTheory
david [at] kineticode ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory [at] jabber
Kineticode. Setting knowledge in motion.(sm)

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

david at kineticode

Feb 18, 2002, 12:55 PM

Post #17 of 22 (5182 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

sam at tregar

Feb 18, 2002, 3:17 PM

Post #18 of 22 (5185 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On 16 Feb 2002, David Wheeler wrote:

> On Tue, 2002-02-12 at 10:38, Sam Tregar wrote:
>
> > It's a great idea whose time has not yet come. The XML Schema parsing and
> > generation technology simply isn't available.
>
> Um, maybe I'm showing my ignorance here, but isn't an XML Schema
> document an XML document?

Yes. Of course, that's just the tip of the iceberg. It's also a data
description language with a some rather complicated processing rules.

> You could create it with XML::Writer or some such,

Sure, you can write an XML Schema with XML::Writer but I'd hardly call it
a good solution. You might as well say "isn't a C program written in
machine code? So why would I need a compiler? I could just use my trusty
assembler!" To make efficient use of the format you have to be able to
operate at a higher level of abstraction.

> and then use a validator to validate it, no?

Perl does not have a working XML Schema validator at present. This is the
really critical missing piece for any design relying on dynamic usage of
XML Schema. I think this should be rectified soon - either by
the creation of XML::Schema or by the Xerces project releasing their
working validator that's now in CVS.

-sam

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

david at kineticode

Feb 18, 2002, 6:23 PM

Post #19 of 22 (5182 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On Mon, 2002-02-18 at 15:17, Sam Tregar wrote:

<snip />

> Perl does not have a working XML Schema validator at present. This is the
> really critical missing piece for any design relying on dynamic usage of
> XML Schema. I think this should be rectified soon - either by
> the creation of XML::Schema or by the Xerces project releasing their
> working validator that's now in CVS.

Well then, the complexities of the Schema syntax aside, would that make
such a dynamic schema creation and validation scheme for elements
possible?

D

--
David Wheeler AIM: dwTheory
david [at] kineticode ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory [at] jabber
Kineticode. Setting knowledge in motion.(sm)

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

sam at tregar

Feb 18, 2002, 7:49 PM

Post #20 of 22 (5179 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On 18 Feb 2002, David Wheeler wrote:

> On Mon, 2002-02-18 at 15:17, Sam Tregar wrote:
>
> > Perl does not have a working XML Schema validator at present. This is the
> > really critical missing piece for any design relying on dynamic usage of
> > XML Schema. I think this should be rectified soon - either by
> > the creation of XML::Schema or by the Xerces project releasing their
> > working validator that's now in CVS.
>
> Well then, the complexities of the Schema syntax aside, would that make
> such a dynamic schema creation and validation scheme for elements
> possible?

Most likely, yes. I've been focused solely on the validation side of
things but it seems to me that any good XML Schema solution should have
facilities for easy generation too.

-sam

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

david at kineticode

Feb 18, 2002, 10:14 PM

Post #21 of 22 (5182 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On Mon, 2002-02-18 at 19:49, Sam Tregar wrote:

> Most likely, yes. I've been focused solely on the validation side of
> things but it seems to me that any good XML Schema solution should have
> facilities for easy generation too.

Well, if it seems like this is immanent, should we work on adding the
capability now? Does it make sense to have a rigid schema now if we're
just going to make it dynamic later?

David

--
David Wheeler AIM: dwTheory
david [at] kineticode ICQ: 15726394
Yahoo!: dew7e
Jabber: Theory [at] jabber
Kineticode. Setting knowledge in motion.(sm)

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

sam at tregar

Feb 18, 2002, 11:16 PM

Post #22 of 22 (5185 views)
Permalink

Re: [Bricolage-Devel] A few questions [LONG] [In reply to]

On 18 Feb 2002, David Wheeler wrote:

> On Mon, 2002-02-18 at 19:49, Sam Tregar wrote:
> >
> > Most likely, yes. I've been focused solely on the validation side of
> > things but it seems to me that any good XML Schema solution should have
> > facilities for easy generation too.
>
> Well, if it seems like this is immanent, should we work on adding the
> capability now?

I guess that depends on how much you like writing pseudo-code! I'm all
for writing specs but I like to have some idea how things will work while
I'm writing them. With this much up in the air I'm not ready to commit to
any details. Which doesn't mean no one can work on it; it just means it
won't be me.

> Does it make sense to have a rigid schema now if we're just going to
> make it dynamic later?

Yes, because it works now. 1.3.0 contains a majority of the planned SOAP
functionality working and tested. It is ready to be released and has been
for more than a week now. Going back to the design stages would be a
needless delay in an almost complete project.

I think it makes more sense to consider a project like this for the 1.5
series. That might even give the technology a chance to materialize in
time to be used!

-sam

_______________________________________________
Bricolage-Devel mailing list
Bricolage-Devel [at] lists
https://lists.sourceforge.net/lists/listinfo/bricolage-devel

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads