Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Python: Python

RotatingFileHandler Fails

 

 

Python python RSS feed   Index | Next | Previous | View Threaded


cookfitz at gmail

Mar 3, 2012, 3:42 AM

Post #1 of 5 (493 views)
Permalink
RotatingFileHandler Fails

The RotatingFileHandler running on win 7 64-bit; py 2.7 is failing
when the script launches a process using subprocess.Popen. Works fine
if the subprocess is not launched

The exception thrown
Traceback (most recent call last):
File "C:\Python27\lib\logging\handlers.py", line 78, in emit
self.doRollover()
File "C:\Python27\lib\logging\handlers.py", line 141, in doRollover
os.rename(self.baseFilename, dfn)
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process

Anyone have an idea how to fix this?


import os, sys
import logging
import logging.handlers
import subprocess

def chomp(s):
"remove trailing carriage return"
if s[-1:]=='\n': return s[:-1]
else: return s

logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %
(name)-2s %(levelname)-8s %(threadName)-12s %(message)s')
q5Logger = logging.getLogger('Q5')
logFileHandler = logging.handlers.RotatingFileHandler(filename='c:\
\logger\\q5.log', mode= 'a', maxBytes=100, backupCount=5)
logFileHandler.setFormatter(logging.Formatter('%(asctime)s %(name)-2s %
(levelname)-8s %(threadName)-12s %(message)s'))
logFileHandler.setLevel(logging.DEBUG)
q5Logger.addHandler(logFileHandler)

progOutput = subprocess.Popen(r'dir *.*', shell=True, bufsize=1000,
stdout=subprocess.PIPE).stdout
line = progOutput.readline()
while (line) != "":
q5Logger.info( chomp(line))
line = progOutput.readline()
rc = progOutput.close()
--
http://mail.python.org/mailman/listinfo/python-list


jeanmichel at sequans

Mar 5, 2012, 2:27 AM

Post #2 of 5 (478 views)
Permalink
Re: RotatingFileHandler Fails [In reply to]

nac wrote:
> The RotatingFileHandler running on win 7 64-bit; py 2.7 is failing
> when the script launches a process using subprocess.Popen. Works fine
> if the subprocess is not launched
>
> The exception thrown
> Traceback (most recent call last):
> File "C:\Python27\lib\logging\handlers.py", line 78, in emit
> self.doRollover()
> File "C:\Python27\lib\logging\handlers.py", line 141, in doRollover
> os.rename(self.baseFilename, dfn)
> WindowsError: [Error 32] The process cannot access the file because it
> is being used by another process
>
> Anyone have an idea how to fix this?
>
>
> import os, sys
> import logging
> import logging.handlers
> import subprocess
>
> def chomp(s):
> "remove trailing carriage return"
> if s[-1:]=='\n': return s[:-1]
> else: return s
>
> logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %
> (name)-2s %(levelname)-8s %(threadName)-12s %(message)s')
> q5Logger = logging.getLogger('Q5')
> logFileHandler = logging.handlers.RotatingFileHandler(filename='c:\
> \logger\\q5.log', mode= 'a', maxBytes=100, backupCount=5)
> logFileHandler.setFormatter(logging.Formatter('%(asctime)s %(name)-2s %
> (levelname)-8s %(threadName)-12s %(message)s'))
> logFileHandler.setLevel(logging.DEBUG)
> q5Logger.addHandler(logFileHandler)
>
> progOutput = subprocess.Popen(r'dir *.*', shell=True, bufsize=1000,
> stdout=subprocess.PIPE).stdout
> line = progOutput.readline()
> while (line) != "":
> q5Logger.info( chomp(line))
> line = progOutput.readline()
> rc = progOutput.close()
>
I don't think you can open a file from 2 different processes, hence the
error message. The only exception would be that one of them opens it in
read only mode which won't happen for log file.

You cannot fix it, it is the operand system that blocks the operation.
Using thread may allow you to log into the same file, but could bring a
bunch of other problems.

I know some ppl use a log server, your processes using a socket (client)
logger. You could do that on your local machine. There's a tutorial on
that in the logging documentation.

JM
--
http://mail.python.org/mailman/listinfo/python-list


cookfitz at gmail

Mar 5, 2012, 6:17 PM

Post #3 of 5 (478 views)
Permalink
Re: RotatingFileHandler Fails [In reply to]

On Mar 5, 4:27 am, Jean-Michel Pichavant <jeanmic...@sequans.com>
wrote:
> nac wrote:
> > The RotatingFileHandler running on win 7 64-bit; py 2.7 is failing
> > when the script launches a process using subprocess.Popen. Works fine
> > if the subprocess is not launched
>
> > The exception thrown
> > Traceback (most recent call last):
> >   File "C:\Python27\lib\logging\handlers.py", line 78, in emit
> >     self.doRollover()
> >   File "C:\Python27\lib\logging\handlers.py", line 141, in doRollover
> >     os.rename(self.baseFilename, dfn)
> > WindowsError: [Error 32] The process cannot access the file because it
> > is being used by another process
>
> > Anyone have an idea how to fix this?
>
> > import os, sys
> > import logging
> > import logging.handlers
> > import subprocess
>
> > def chomp(s):
> >       "remove trailing carriage return"
> >       if s[-1:]=='\n': return s[:-1]
> >       else: return s
>
> > logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %
> > (name)-2s %(levelname)-8s %(threadName)-12s %(message)s')
> > q5Logger = logging.getLogger('Q5')
> > logFileHandler = logging.handlers.RotatingFileHandler(filename='c:\
> > \logger\\q5.log', mode= 'a', maxBytes=100, backupCount=5)
> > logFileHandler.setFormatter(logging.Formatter('%(asctime)s %(name)-2s %
> > (levelname)-8s %(threadName)-12s %(message)s'))
> > logFileHandler.setLevel(logging.DEBUG)
> > q5Logger.addHandler(logFileHandler)
>
> > progOutput = subprocess.Popen(r'dir *.*', shell=True, bufsize=1000,
> > stdout=subprocess.PIPE).stdout
> > line = progOutput.readline()
> > while (line) != "":
> >       q5Logger.info( chomp(line))
> >       line = progOutput.readline()
> > rc = progOutput.close()
>
> I don't think you can open a file from 2 different processes, hence the
> error message. The only exception would be that one of them opens it in
> read only mode which won't happen for log file.
>
> You cannot fix it, it is the operand system that blocks the operation.
> Using thread may allow you to log into the same file, but could bring a
> bunch of other problems.
>
> I know some ppl use a log server, your processes using a socket (client)
> logger. You could do that on your local machine. There's a tutorial on
> that in the logging documentation.
>
> JM

Thanks for taking the time to look at the problem. In case you are
interested I was able to get this to work with the code at URL below.
Seemed to work because the subprocess I am running is not actually
using the log file but the OS make it inherit a handle to the log
file. The code below disables that inheritance. Do you see any
problems with the approach?

http://bugs.python.org/file14420/NTSafeLogging.py
--
http://mail.python.org/mailman/listinfo/python-list


arun.sathyan at ust-global

Mar 7, 2012, 5:20 AM

Post #4 of 5 (481 views)
Permalink
Re: RotatingFileHandler Fails [In reply to]

Hi nac,

NTSafeLogging.py is working fine without any errors, but its not rotating
the log files as rotatingfilehandler does.
Do you have any working sample with NTSafeLogging which rotates the log
file.

logging issue with subprocess.Popen can be solved using this code

import threading
lock = threading.RLock()
def acquire_lock():
lock.acquire()

def release_lock():
lock.release()

call the acquire_lock() at the begining of method and release_lock() at
the end





--
View this message in context: http://python.6.n6.nabble.com/RotatingFileHandler-Fails-tp4542769p4554381.html
Sent from the Python - python-list mailing list archive at Nabble.com.
--
http://mail.python.org/mailman/listinfo/python-list


arun.sathyan at ust-global

Mar 7, 2012, 7:33 AM

Post #5 of 5 (480 views)
Permalink
Re: RotatingFileHandler Fails [In reply to]

Hi,

Actually NonInheritedRotatingFileHandler is rotating the log files but some
times it falis and showing I/O errors while the log file limit reaches the
given size.

Thanks

Arun


--
View this message in context: http://python.6.n6.nabble.com/RotatingFileHandler-Fails-tp4542769p4554781.html
Sent from the Python - python-list mailing list archive at Nabble.com.
--
http://mail.python.org/mailman/listinfo/python-list

Python python RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.