Mailing List Archive: Python: Bugs

[issue7337] Add lossy queue to queue library module

Index | Next | Previous | View Threaded

report at bugs

Nov 17, 2009, 2:40 AM

Post #1 of 3 (160 views)
Permalink

[issue7337] Add lossy queue to queue library module

New submission from Ben Bass <benpaulbass [at] googlemail>:

Many applications would benefit from 'connectionless' queues, i.e. they
don't want to care whether anything is reading from the other end.
Using current queue module classes this is not practical, because there
is a choice between unbounded memory consumption or blocking. I propose
adding a 'LossyQueue' class in the queue module which would allow
bounded memory consumption without blocking on put. (i.e. items are
dropped in fifo manner beyond a certain limit). In my view this is at
least as natural as the PriorityQueue and LifoQueue extensions in that
module.

Outline as follows:

class LossyQueue(Queue):
"Queue subclass which drops items on overflow"
def _init(self, maxsize):
if maxsize > 0:
# build the deque with maxsize limit
self.queue = deque(maxlen=maxsize)
else:
# same as normal Queue instance
self.queue = collections.deque()
# deque alone handles maxsize,
# so we pretend we have none
self.maxsize = 0

if there is interest in this I will offer a proper patch with docs and
tests.

----------
components: Library (Lib)
messages: 95374
nosy: bpb
severity: normal
status: open
title: Add lossy queue to queue library module
type: feature request
versions: Python 2.7, Python 3.2

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue7337>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Nov 17, 2009, 11:39 AM

Post #2 of 3 (143 views)
Permalink

[issue7337] Add lossy queue to queue library module [In reply to]

Raymond Hettinger <rhettinger [at] users> added the comment:

I'm curious about your "many applications would benefit from
'connectionless' queues". What do you have in mind? Is there any
reason those apps cannot use collections.deque() directly?

----------
assignee: -> rhettinger
nosy: +rhettinger

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue7337>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

report at bugs

Nov 18, 2009, 1:44 AM

Post #3 of 3 (131 views)
Permalink

[issue7337] Add lossy queue to queue library module [In reply to]

Ben Bass <benpaulbass [at] googlemail> added the comment:

'connectionless' is from how I see it as an analogy with UDP (vs TCP);
why not just use a deque is primarily about having the same API - a
client (getter) of the queue shouldn't know or care whether it is a
'lossy' queue or a normal queue. I guess most uses of a normal queue
(excepting the 'task' functions) could just use a deque, but it wouldn't
feel natural.

Use cases: non-critical event/status reporting is my canonical example.
Specific examples:
- a program which executes a long running process in a thread. It wants
to update a GUI progress bar or similar, which must occur in a different
thread because of the GUI model. By using a LossyQueue, the server
thread is simplified; it doesn't have to care whether anything is
listening on the other end, allowing greater decoupling (e.g. no changes
required if there isn't a GUI). LossyQueues become part of the interface
which can be used or not as required.
- emulating/providing wrapper around UDP sockets
- many application protocols support a get/set/report type interface
with the addition of asynchronous events (e.g. SNMP, Netconf, SCPI). In
these type of applications a suitable abstraction might be a normal
Queue(s) for the standard commands and a LossyQueue for the events
(which some applications might not care about). The point is that to
the user of this abstraction, these two interfaces look the same.

The 'server' doesn't care if a client is listening or not (it won't
block and it won't use unlimited memory)
The 'client' (if it wants to use it) doesn't know that it isn't a normal
queue (same API).
-> decouples server and client tasks.

----------

_______________________________________
Python tracker <report [at] bugs>
<http://bugs.python.org/issue7337>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads