internal.crashQueue

A multiprocessing queue class that clears out the queue if an exception is raised.

class bpreveal.internal.crashQueue.CrashQueue(maxsize=-1, timeout=240)

A queue that closes itself if a timeout expires during a put.

Parameters:

• maxsize (int) – The maximum size of the queue. Default is unlimited.

• timeout (float) – The time, in seconds, that blocking operations should wait before erroring out.

This class fixes a problem where a producer is feeding objects into a queue and then the consumer crashes. In this instance, the producer thread will raise a queue.Full exception, but the process can’t close because there’s still data to write to the queue. But since the consumer has crashed, this creates deadlock. If a write to a queue in this class times out, it breaks that deadlock by allowing the process to exit before the queue is flushed. This class is, more or less, a subclass of multiprocessing.Queue.

empty()

Is the queue currently empty?

Return type:

bool

full()

Is the queue currently full?

Return type:

bool

put(obj, timeout=None)

Put the given object on the queue. If timeout expires, clear the queue.

Parameters:

• obj (Any) – Any (picklable) object.

• timeout (float | None) – The maximum time to wait, in seconds. If this expires, then the queue gets cancel_join_thread called on it and the queue.Full exception is raised.

Return type:

None

get(timeout=None)

Get an object from the queue.

Parameters:

timeout (float | None) – The number of seconds to wait before raising an exception. If omitted, the timeout given in the initializer.

Returns:

The object from the queue.

Return type:

Any

close()

Indicate that no more items will be added to the queue.

Automatically called when the queue is garbage collected.

Return type:

None