Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Bugs

[issue14385] Support other types than dict for __builtins__

 

 

Python bugs RSS feed   Index | Next | Previous | View Threaded


report at bugs

Mar 21, 2012, 6:35 PM

Post #1 of 14 (161 views)
Permalink
[issue14385] Support other types than dict for __builtins__

New submission from STINNER Victor <victor.stinner [at] gmail>:

CPython expects __builtins__ to be a dict, but it is interesting to be able to use another type. For example, my pysandbox project (sandbox to secure Python) requires a read-only mapping for __builtins__.

The PEP 416 was rejected, so there is no builtin frozendict type, but it looks like the dictproxy type will be exposed as a public type.

Attached patch uses PyDict_CheckExact() to check if __builtins__ is a dict and add a "slow-path" for other types. The overhead on runtime performance should be very low (near zero), PyDict_CheckExact() just dereference a pointer (to read the object type) and compare two pointers.

The patch depends on issue #14383 patch (identifier.patch) for the __build_class__ identifier. I can write a new patch without this dependency if needed.

----------
components: Interpreter Core
files: builtins.patch
keywords: patch
messages: 156534
nosy: haypo
priority: normal
severity: normal
status: open
title: Support other types than dict for __builtins__
type: enhancement
versions: Python 3.3
Added file: http://bugs.python.org/file24989/builtins.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 21, 2012, 6:50 PM

Post #2 of 14 (156 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

See the issue #14386 which exposes dictproxy as a public type.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 21, 2012, 7:03 PM

Post #3 of 14 (159 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

Example combining patches of #14385 and #14386 to run code with read-only __builtins__:
----------- test.py -------------
ns={'__builtins__': __builtins__.__dict__}
exec(compile("__builtins__['superglobal']=1; print(superglobal)", "test", "exec"), ns)
ns={'__builtins__': dictproxy(__builtins__.__dict__)}
exec(compile("__builtins__['superglobal']=2; print(superglobal)", "test", "exec"), ns)
----------- end of test.py -----

Output:
--------
$ ./python test.py
1
Traceback (most recent call last):
File "x.py", line 4, in <module>
exec(compile("__builtins__['superglobal']=1; print(superglobal)", "test", "exec"), ns)
File "test", line 1, in <module>
TypeError: 'dictproxy' object does not support item assignment
--------

Note: this protection is not enough to secure Python, but it is an important part of a Python sandbox.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 22, 2012, 5:38 AM

Post #4 of 14 (154 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

> Note: this protection is not enough to secure Python,
> but it is an important part of a Python sandbox.

Oh, and by the way, I workaround the lack of read-only mapping in pysandbox by removing dict methods: dict.__init__(), dict.clear(), dict.update(), etc. This is a problem because these methods are useful in Python.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Mar 25, 2012, 4:59 PM

Post #5 of 14 (153 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

With my patch, Python doesn't check __builtins__ type whereas ceval.c replaces any lookup error by a NameError. Example:

$ ./python
Python 3.3.0a1+ (default:f8d01c8baf6a+, Mar 26 2012, 01:44:48)
>>> code=compile("print('Hello World!')", "", "exec")
>>> exec(code,{'__builtins__': {'print': print}})
Hello World!
>>> exec(code,{'__builtins__': {}})
NameError: name 'print' is not defined
>>> exec(code,{'__builtins__': 1})
NameError: name 'print' is not defined

It should only replace the current exception by NameError if the current exception is a LookupError.

And my patch on LOAD_GLOBAL is not correct, it does still call PyDict_GetItem. I'm waiting until #14383 is done before writing a new patch.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 5, 2012, 5:09 AM

Post #6 of 14 (142 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

New version:
- if __build_class__ is missing, raise a NameError instead of surprising ImportError
- add tests
- if PyObject_GetItem on __builtins__ or globals fail, only raise NameError if the exception is a KeyError

Before my patch, a new dict was created for builtins if __builtins__ exists in global but is not a dict. With my patch, the __builtins__ is kept and the type is checked at runtime. If __builtins__ is not a mapping, an exception is raised on lookup in ceval.c.

We may check __builtins__ type in PyFrame_New() using:

PyDict_Check(builtins) || (PyMapping_Check(mapping) && !PyList_Check(mapping) && !PyTuple_Check(mapping))

(PyDict_Check(builtins) is checked first for performance)

----------
Added file: http://bugs.python.org/file25132/builtins-2.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 15, 2012, 3:31 PM

Post #7 of 14 (138 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

Changes by STINNER Victor <victor.stinner [at] gmail>:


Removed file: http://bugs.python.org/file24989/builtins.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 15, 2012, 4:49 PM

Post #8 of 14 (140 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

Oops, patch version 2 was not correct: I forgot a { ... } in ceval.c.

New patch fixing this issue but leaves also the LOAD_GLOBAL code unchanged : keep the goto and don't try to factorize the 3 last instructions. LOAD_GLOBAL is really critical in performance.

With patch version 3, the overall overhead is +0.4% according to pybench.

----------
Added file: http://bugs.python.org/file25230/builtins-3.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 15, 2012, 4:49 PM

Post #9 of 14 (138 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

Changes by STINNER Victor <victor.stinner [at] gmail>:


Removed file: http://bugs.python.org/file25132/builtins-2.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 15, 2012, 4:53 PM

Post #10 of 14 (139 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

Changes by STINNER Victor <victor.stinner [at] gmail>:


----------
nosy: +pitrou

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 18, 2012, 4:54 AM

Post #11 of 14 (141 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

STINNER Victor <victor.stinner [at] gmail> added the comment:

- assert(!builtins || PyDict_Check(builtins));
+ assert(!builtins);

Oops, the assert must be replaced with assert(builtins != NULL) -> fixed in patch version 4.

----------
Added file: http://bugs.python.org/file25257/builtins-4.patch

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 18, 2012, 3:26 PM

Post #12 of 14 (140 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

Antoine Pitrou <pitrou [at] free> added the comment:

This looks fine.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 18, 2012, 4:00 PM

Post #13 of 14 (140 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

Roundup Robot <devnull [at] psf> added the comment:

New changeset e3ab8aa0216c by Victor Stinner in branch 'default':
Issue #14385: Support other types than dict for __builtins__
http://hg.python.org/cpython/rev/e3ab8aa0216c

----------
nosy: +python-dev

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com


report at bugs

Apr 18, 2012, 4:02 PM

Post #14 of 14 (140 views)
Permalink
[issue14385] Support other types than dict for __builtins__ [In reply to]

Changes by STINNER Victor <victor.stinner [at] gmail>:


----------
resolution: -> fixed
status: open -> closed

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue14385>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Python bugs RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.