Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

ANN: a mini-language for encapsulating deep-copy operations on Python data structures

 

 

Python python RSS feed   Index | Next | Previous | View Threaded


showell30 at yahoo

Nov 18, 2009, 12:49 AM

Post #1 of 2 (133 views)
Permalink
ANN: a mini-language for encapsulating deep-copy operations on Python data structures

During the last few days I have written code in support of a small DDL
language that encapsulates a concise representation of the
manipulations needed to make a deep subcopy of a Python-like data
structure. It is inspired by syntax from mainstream modern languages,
including, of course, Python. The DDL can be converted to an AST. That
AST can be walked to either generate Python code that expresses the
mapping or to generate Python objects that can execute the mapping.
Either of the prior outputs can then subsequently be applied to real
world inputs to create new Python data structures from old ones, using
the mechanisms specified in the original DDL for attribute access,
dictionary lookup, iteration, method invocation, etc.

Here is an example of the DDL (and I hate the terminology "DDL," just
cannot think of anything better):


{
'show_table_of_contents',
'author' {
.person 'name',
.person 'location' as city,
.favorite_books()[
.title,
.cost() as expense
] as books}
}


There are more details here:

http://showellonprogramming.blogspot.com/2009/11/more-on-python-deep-copy-schema.html

Apart from shamelessly plugging my blog, I am hoping to generate
ideas, constructive criticism, etc.

Feel free to comment here or on the blog. So far there are three
entries on the blog, all pertaining to Python.

The implementation of the idea is quite new, but it is a topic that I
have been pondering for a long time--no matter how expressive your
programming language of choice might be, there are certain programming
tasks that just seem needlessly tedious. The mini-language above
proposes to solve one problem but solve it well.

I am particularly interested to find out whether somebody has tried
something like this before.





--
http://mail.python.org/mailman/listinfo/python-list


showell30 at yahoo

Nov 18, 2009, 9:09 AM

Post #2 of 2 (120 views)
Permalink
Re: ANN: a mini-language for encapsulating deep-copy operations on Python data structures [In reply to]

On Nov 18, 4:34 am, "M.-A. Lemburg" <m...@egenix.com> wrote:
> Steve Howell wrote:
> > [...]
> > Here is an example of the DDL (and I hate the terminology "DDL," just
> > cannot think of anything better):
>
> >             {
> >                 'show_table_of_contents',
> >                 'author' {
> >                     .person 'name',
> >                     .person 'location' as city,
> >                     .favorite_books()[.
> >                         .title,
> >                         .cost() as expense
> >                         ] as books}
> >             }
>
> > There are more details here:
>
> >http://showellonprogramming.blogspot.com/2009/11/more-on-python-deep-...

Thanks for your feedback, Marc-Andre! Comments inline...

>
> Personally, I find the explicit approach more intuitive, since
> you immediately see what you are going to get:
>
> show_table_of_contents = context['show_table_of_contents']
> author = context['author']
> d = {
>     'show_table_of_contents': show_table_of_contents,
>     'author': {
>         'name': author.person['name'],
>         'city': author.person['location'],
>         'books': [.{
>             'title': item.title,
>             'expense': item.cost(),
>             }
>             for item in author.favorite_books()],
>         },
>     }
>

I understand what you mean about the explicit approach. When you go
for brevity, you sacrifice some explicitness. One of the things I'm
actually looking to do next is to be able to generate code like what
you've written above. You could make the generated code even more
procedural in terms of how it walks the structure--for example,
replace the list comprehension with a for loop. This could be useful
for inserting tracing code, for example.

> I particularly find this part non-intuitive:
>
> >                     .favorite_books()[.
> >                         .title,
> >                         .cost() as expense
> >                         ] as books}
>

Yep, I think what I really want is this:

.favorite_books()[{
.title,
.cost() as expense
}] as books}

I am not sure the curlies make the transformation completely
intuitive, but they are more suggestive that you are generating a list
of dictionaries.

> and would leave in the square brackets for __getitem__
> lookups on these:
>
> >                     .person 'name',
> >                     .person 'location' as city,
>

Yep, I agree. The square brackets were not a deliberate omission.
They were just slightly more tricky to implement because of '[' having
double syntactical meaning.

> For additional inspiration, you might want to look at XSLT
> which provides similar transformations on XML data structures.
>
> There are also a number of other transformation languages:
>
> http://en.wikipedia.org/wiki/Model_Transformation_Languagehttp://en.wikipedia.org/wiki/Data_transformation
>

Thanks for the links!

> Regarding the term "DDL": that's normally used for "data definition
> language" and doesn't really have all that much to do with
> transforming data. You normally define data structures using
> DDL - without actually putting data into those structures.
>
> Why not "PyDTL".... Python data transformation language ?!
>

Yep, I like that suggestion. The language does indeed describe a
transformation. In some ways I was wanting to think of it more
broadly, in terms that the transformation also acts as a schema for
any input objects. For example, you could use '{.title, .publisher,
{.name} }' as a schema to validate that an object behaves like a
book. I suppose that's still a transformation in an abstract sense,
where the output is just True, False, or maybe even the slice of the
transformation that can/cannot be executed.

--
http://mail.python.org/mailman/listinfo/python-list

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.