Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

Retrieving result from embedded execution

 

 

Python python RSS feed   Index | Next | Previous | View Threaded


mephisto_9000 at hotmail

May 8, 2012, 12:07 PM

Post #1 of 6 (365 views)
Permalink
Retrieving result from embedded execution

Hello everyone,
We are trying to implement our own interactive interpreter in our applicationusing an embedded Python interpreter.
I was wondering what would be the best way to retreive as text the result of executing Python code. The text must be exactly the same as it would be in thestandalone interpreter.
We used to do this by changing the embedded interpreter's sys.stdout and sys.stderrwith a FILE created in C++. We could then execute code using the PyRun_InteractiveOnefunction and easily retrieve the result from the FILE. The problem is, our applicationshouldn't create any files if possible.
We also tried replacing sys.stdout and sys.stderr with a custom made Python objectwhich could be more easily read from C++. We then used the function PyRun_Stringto execute the code. The problem here is that using the Py_file_input start tokendoesn't write everything we need to sys.stdout. (i.e. executing 2+2 does not write 4).It also doesn't print a traceback to sys.stderr when there is an exception. Using theother two start tokens is impossible since we need to be able to execute more than one instruction at once. We also tried using the function PyRun_SimpleString but weencounter the same problems.
We are using python 2.7.
Any suggestions?
Thank you for your time.
Also, I'm new to mailling lists, what is the proper way to reply to someone? Should Ireply directly to someone's email adress or is there a way to answer trough the server.
F. Lab


wrw at mac

May 8, 2012, 12:44 PM

Post #2 of 6 (350 views)
Permalink
Re: Retrieving result from embedded execution [In reply to]

On May 8, 2012, at 3:07 PM, F L wrote:

> Hello everyone,
>
> We are trying to implement our own interactive interpreter in our application
> using an embedded Python interpreter.
>
> I was wondering what would be the best way to retreive as text the result of
> executing Python code. The text must be exactly the same as it would be in the
> standalone interpreter.
>
> We used to do this by changing the embedded interpreter's sys.stdout and sys.stderr
> with a FILE created in C++. We could then execute code using the PyRun_InteractiveOne
> function and easily retrieve the result from the FILE. The problem is, our application
> shouldn't create any files if possible.
>
> We also tried replacing sys.stdout and sys.stderr with a custom made Python object
> which could be more easily read from C++. We then used the function PyRun_String
> to execute the code. The problem here is that using the Py_file_input start token
> doesn't write everything we need to sys.stdout. (i.e. executing 2+2 does not write 4).
> It also doesn't print a traceback to sys.stderr when there is an exception. Using the
> other two start tokens is impossible since we need to be able to execute more than
> one instruction at once. We also tried using the function PyRun_SimpleString but we
> encounter the same problems.
>
> We are using python 2.7.
>
> Any suggestions?
>
> Thank you for your time.

[byte]

I'm pretty new to Python myself, and I may not understand what you are trying to do, but have you looked at the subprocess module?

Documented here: http://docs.python.org/library/subprocess.html

Using subprocess.Popen would let you hook stdin and staout directly to pipes that will pass data back to the calling program with no intermediate steps.

-Bill


rosuav at gmail

May 10, 2012, 7:48 PM

Post #3 of 6 (341 views)
Permalink
Re: Retrieving result from embedded execution [In reply to]

On Wed, May 9, 2012 at 5:07 AM, F L <mephisto_9000 [at] hotmail> wrote:
> Hello everyone,
>
> We are trying to implement our own interactive interpreter in our
> application
> using an embedded Python interpreter.
>
> I was wondering what would be the best way to retreive as text the result of
> executing Python code. The text must be exactly the same as it would be in
> the
> standalone interpreter.

Greetings!

The standalone interpreter - I assume you mean the interactive prompt?
It has slightly different handling of things (for instance, naked
expressions being written to standard output) from the more "usual"
invocation of the interpreter.

For your embedded Python, probably the easiest thing to do is to not
try to use interactive mode, but script mode:

PyObject* PyRun_String(const char *str, int start, PyObject *globals,
PyObject *locals)

Pass it a string of Python code and a dictionary to use for globals
and locals (they can be the same dict). Then when that finishes,
retrieve from that dictionary some predetermined name (eg "Output").
The Python code then needs simply to assign to Output and all will
work.

Another way to do it is to call a function inside the Python code, and
retrieve its return value.

> We are using python 2.7.
>
> Any suggestions?

Unless you have particular need to stick to the 2.x branch, I would
suggest moving to 3.3; many things are easier (especially Unicode).
But both work.

> Also, I'm new to mailling lists, what is the proper way to reply to someone?
> Should I
> reply directly to someone's email adress or is there a way to answer trough
> the server.

The usual convention is to reply on-list, unless it's particularly
private. Many mail clients will handle this conveniently; otherwise,
just manually replace the To address with the list address. As long as
you use reply (rather than starting a fresh email), all posts will be
correctly threaded both on the list and on the synchronized newsgroup
(comp.lang.python).

Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list


stefan_ml at behnel

May 10, 2012, 9:45 PM

Post #4 of 6 (340 views)
Permalink
Re: Retrieving result from embedded execution [In reply to]

F L, 08.05.2012 21:07:
> We are trying to implement our own interactive interpreter in our
> applicationusing an embedded Python interpreter. I was wondering what
> would be the best way to retreive as text the result of executing
> Python code. The text must be exactly the same as it would be in
> thestandalone interpreter. We used to do this by changing the embedded
> interpreter's sys.stdout and sys.stderrwith a FILE created in C++. We
> could then execute code using the PyRun_InteractiveOnefunction and
> easily retrieve the result from the FILE. The problem is, our
> applicationshouldn't create any files if possible. We also tried
> replacing sys.stdout and sys.stderr with a custom made Python
> objectwhich could be more easily read from C++. We then used the
> function PyRun_Stringto execute the code. The problem here is that using
> the Py_file_input start tokendoesn't write everything we need to
> sys.stdout. (i.e. executing 2+2 does not write 4).It also doesn't print
> a traceback to sys.stderr when there is an exception. Using theother two
> start tokens is impossible since we need to be able to execute more than
> one instruction at once. We also tried using the function
> PyRun_SimpleString but weencounter the same problems.

As was already suggested, the PyRun_*() functions are a better choice for
executing code from C code, but they will not automatically divert
sys.stdout and sys.stderr for you, which are global settings of the
interpreter, not local to an execution. So you will get the result of the
evaluation back, but not any output that may have been printed during the
execution. You have to change those yourself, as you apparently already did.

However, have you tried using a pipe for them instead of a real file? That
would allow you to retrieve the output without needing to pass through a
file in the file system. You can also replace sys.stdout/err with arbitrary
objects in Python space, which could then forward the output in any way you
want.

But, if you control the code that is being executed, it would really be
best to *not* print anything out directly, but to use the logging module
for output, where you can control much better what exactly gets propagated
and to what target.

And when the code fails and raises an exception, you can get at the stack
trace (that the normal interpreter would just print out) using the
traceback module.


> Also, I'm new to mailling lists, what is the proper way to reply to
> someone? Should Ireply directly to someone's email adress or is there a
> way to answer trough the server.

A couple of more hints in addition to what Chris Angelico said: reply below
the text your are responding to (i.e. not above the text) or split it into
sections and reply inline (as I did). Cut down the original text to what is
needed for readers to understand the context of your answer. Write plain
text mails instead of HTML mails.

Your problem description was very good, BTW. It included the perfect amount
of information about what you tried and what didn't work for you about it.

Stefan

--
http://mail.python.org/mailman/listinfo/python-list


jeanpierreda at gmail

May 11, 2012, 10:36 AM

Post #5 of 6 (342 views)
Permalink
Re: Retrieving result from embedded execution [In reply to]

On Fri, May 11, 2012 at 12:45 AM, Stefan Behnel <stefan_ml [at] behnel> wrote:
> However, have you tried using a pipe for them instead of a real file? That
> would allow you to retrieve the output without needing to pass through a
> file in the file system. You can also replace sys.stdout/err with arbitrary
> objects in Python space, which could then forward the output in any way you
> want.

Surely a pipe would cause a deadlock if the pipe fills up if Python
and the C program are running in the same process/thread?

I'm not too familiar with these sorts of things.

-- Devin
--
http://mail.python.org/mailman/listinfo/python-list


rosuav at gmail

May 11, 2012, 11:05 AM

Post #6 of 6 (339 views)
Permalink
Re: Retrieving result from embedded execution [In reply to]

On Sat, May 12, 2012 at 3:36 AM, Devin Jeanpierre
<jeanpierreda [at] gmail> wrote:
> On Fri, May 11, 2012 at 12:45 AM, Stefan Behnel <stefan_ml [at] behnel> wrote:
>> However, have you tried using a pipe for them instead of a real file? That
>> would allow you to retrieve the output without needing to pass through a
>> file in the file system. You can also replace sys.stdout/err with arbitrary
>> objects in Python space, which could then forward the output in any way you
>> want.
>
> Surely a pipe would cause a deadlock if the pipe fills up if Python
> and the C program are running in the same process/thread?

Yes, it would; how much data are you looking at retrieving, and is it
all at script-end or do you need to process it concurrently? If the
latter, then your two best options are a pipe (and running the Python
script in another thread, possibly a separate process) or a callback
of some sort (in which case the script hands you a line or block of
output and says "Deal with this and get back to me") - eg by replacing
sys.stdout, or defining a function that the Python script calls
explicitly.

This is starting to sound similar to something I did at work. A C++
program uses a block of Python code as a sort of uber-config-file. It
would initialize the Python environment, import some modules, etc,
once, and keep the globals around (this allows Python globals to be
retained). Every time script-configurable code is to be run:

1) Prepare a dictionary called Input with a whole lot of
parameters/information/etc
2) Create an empty dictionary and call it Output
3) Execute a block of code (retrieved from the database)
4) Inspect the Output dictionary and use that to control subsequent behaviour.

As a concept, it worked quite well. I do not, however, advise doing
this if your Python scripts come from untrusted sources, as it is
impossible to guarantee safety. (For that reason we actually shifted
to Javascript for that project.) But if you're using this as a
powerful and simple config-file and you know you can trust everything
that will be run, then it's a pretty awesome way of doing things. It
takes deliberately malicious code to break the system.

ChrisA
--
http://mail.python.org/mailman/listinfo/python-list

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.