rocksq.nonblocking

class rocksq.nonblocking.MpmcQueue(path, ttl, max_inflight_ops=1000)

A persistent queue with a ttl that supports multiple consumers marked with labels. This is a non-blocking implementation. All methods return the future-like object MpmcResponse which must be used to get the actual response.

Parameters:

• path (str) – The path to the queue.

• ttl (int) – The amount of seconds after which the element in the queue will be removed. TTL is not strict. It means that the element will remain in the queue for TTL seconds after insertion and the queue will make efforts to remove the element after TTL seconds but it is not guaranteed to be done immediately. Thus, consumers can retrieve expired but not removed elements.

• max_inflight_ops (int) – The maximum number of inflight operations. If the number of inflight operations reached its limit, further ops are blocked until the capacity is available. Default to 1_000.

Raises:

PyRuntimeError – If the queue could not be created.

add(items, no_gil=True)

Adds items to the queue.

GIL: the method can optionally be called without the GIL.

Parameters:

• items (list of bytes) – The items to add to the queue.

• no_gil (bool) – If True, the method will be called without the GIL. Default is True.

Raises:

PyRuntimeError – If the method fails.

Returns:

The future-like object which must be used to get the actual response. For the add operation, the response object is only useful to call for is_ready().

Return type:

MpmcResponse

disk_size

Returns the disk size of the queue.

Raises:

PyRuntimeError –

Returns:

The future-like object which must be used to get the actual response. For the size operation, the response object is useful to call for is_ready(), try_get() and get().

Return type:

MpmcResponse

inflight_ops

labels

Returns the consumer labels.

Returns:

labels – The consumer labels.

Return type:

list of str

len

Returns the length of the queue.

Raises:

PyRuntimeError – If the method fails.

Returns:

The future-like object which must be used to get the actual response. For the length operation, the response object is useful to call for is_ready(), try_get() and get().

Return type:

MpmcResponse

next(label, start_position, max_elements=1, no_gil=True)

Retrieves items from the queue.

GIL: the method can optionally be called without the GIL.

Parameters:

• label (str) – The consumer label that determines the start position in the queue to retrieve elements. If the label does not exist the start position is determined by ``option` parameter. If the label exists the start position is the next element after the last call of this method. If some elements are expired between the last and this call next non-expired elements will be retrieved.

• start_position (StartPosition) – The option that determines the start position in the queue to retrieve elements if the consumer label does not exist.

• max_elements (int) – The maximum number of elements to retrieve. Default is 1.

• no_gil (bool) – If True, the method will be called without the GIL. Default is True.

Raises:

PyRuntimeError – If the method fails.

Returns:

The future-like object which must be used to get the actual response. For the add operation, the response object is useful to call for is_ready(), try_get() and get().

Return type:

MpmcResponse

remove_label(label, no_gil=True)

Remove the consumer label from the queue.

GIL: the method can optionally be called without the GIL.

Parameters:

• label (str) – The consumer label to remove.

• no_gil (bool) – If True, the method will be called without the GIL. Default is True.

Raises:

PyRuntimeError – If the method fails.

Returns:

True if the consumer label existed, False otherwise.

Return type:

bool

class rocksq.nonblocking.MpmcResponse

get()

Returns the response in a blocking way.

GIL: the method releases the GIL

Raises:

PyRuntimeError – If the method fails.

Returns:

The response when it is ready.

Return type:

MpmcResponseVariant

is_ready

Checks if the response is ready.

Returns:

True if the response is ready, False otherwise.

Return type:

bool

try_get()

Returns the response if it is ready.

Raises:

PyRuntimeError – If the method fails.

Returns:

• MpmcResponseVariant – The response if it is ready,

• None – otherwise.

class rocksq.nonblocking.MpmcResponseVariant

A response variant containing the actual data for add, next, size and length operations of MpmcQueue. The object is created only by the library, there is no public constructor.

data

Returns the data for the next() operation.

Raises:

PyRuntimeError – If the method fails.

Returns:

• (list of bytes, bool) – The data for the next() operation if the operation was successful,

• None – if the future doesn’t represent the next() operation.

labels

Returns the data for the get_labels() operation.

Raises:

PyRuntimeError – If the method fails.

Returns:

• list of str – The data for the get_labels() operation if the operation was successful,

• None – if the future doesn’t represent the get_labels() operation.

len

Returns the length of the queue.

Raises:

PyRuntimeError – If the method fails.

Returns:

• int – The length of the queue if the operation was successful,

• None – if the future doesn’t represent the length() operation.

removed_label

Returns the result of remove_label() operation.

Raises:

PyRuntimeError – If the method fails.

Returns:

• bool – True if the consumer label existed, False otherwise.

• None – if the future doesn’t represent the size() operation.

size

Returns the size of the queue.

Raises:

PyRuntimeError – If the method fails.

Returns:

• int – The size of the queue if the operation was successful,

• None – if the future doesn’t represent the size() operation.

class rocksq.nonblocking.PersistentQueueWithCapacity(path, max_elements=1000000000, max_inflight_ops=1000)

A persistent queue with a fixed capacity. This is a non-blocking implementation. All methods return the future-like object Response which must be used to get the actual response.

Parameters:

• path (str) – The path to the queue.

• max_elements (int) – The maximum number of elements the queue can hold. Default to 1_000_000_000.

• max_inflight_ops (int) – The maximum number of inflight operations. If the number of inflight operations reached its limit, further ops are blocked until the capacity is available. Default to 1_000.

Raises:

PyRuntimeError – If the queue could not be created.

disk_size

Returns the disk size of the queue.

Raises:

PyRuntimeError –

Returns:

The future-like object which must be used to get the actual response. For the size operation, the response object is useful to call for is_ready(), try_get() and get().

Return type:

Response

inflight_ops

len

Returns the length of the queue.

Raises:

PyRuntimeError – If the method fails.

Returns:

The future-like object which must be used to get the actual response. For the length operation, the response object is useful to call for is_ready(), try_get() and get().

Return type:

Response

payload_size

Returns the payload size of the queue.

Raises:

PyRuntimeError –

Returns:

The future-like object which must be used to get the actual response. For the size operation, the response object is useful to call for is_ready(), try_get() and get().

Return type:

Response

pop(max_elements=1, no_gil=True)

Retrieves items from the queue.

GIL: the method can optionally be called without the GIL.

Parameters:

• max_elements (int) – The maximum number of elements to retrieve. Default to 1.

• no_gil (bool) – If True, the method will be called without the GIL. Default is True.

Raises:

PyRuntimeError – If the method fails.

Returns:

The future-like object which must be used to get the actual response. For the pop operation, the response object is useful to call for is_ready(), try_get() and get().

Return type:

Response

push(items, no_gil=True)

Adds items to the queue.

GIL: the method can optionally be called without the GIL.

Parameters:

• items (list of bytes) – The items to add to the queue.

• no_gil (bool) – If True, the method will be called without the GIL. Default is True.

Raises:

PyRuntimeError – If the method fails.

Returns:

The future-like object which must be used to get the actual response. For the push operation, the response object is only useful to call for is_ready().

Return type:

Response

class rocksq.nonblocking.Response

get()

Returns the response in a blocking way.

GIL: the method releases the GIL

Raises:

PyRuntimeError – If the method fails.

Returns:

The response when it is ready.

Return type:

ResponseVariant

is_ready

Checks if the response is ready.

Returns:

True if the response is ready, False otherwise.

Return type:

bool

try_get()

Returns the response if it is ready.

Raises:

PyRuntimeError – If the method fails.

Returns:

• ResponseVariant – The response if it is ready,

• None – otherwise.

class rocksq.nonblocking.ResponseVariant

A response variant containing the actual data for push, pop, size and length operations of PersistentQueueWithCapacity. The object is created only by the library, there is no public constructor.

data

Returns the data for the pop() operation.

Raises:

PyRuntimeError – If the method fails.

Returns:

• list of bytes – The data for the pop() operation if the operation was successful,

• None – if the future doesn’t represent the pop() operation.

len

Returns the length of the queue.

Raises:

PyRuntimeError – If the method fails.

Returns:

• int – The length of the queue if the operation was successful,

• None – if the future doesn’t represent the length() operation.

size

Returns the size of the queue.

Raises:

PyRuntimeError – If the method fails.

Returns:

• int – The size of the queue if the operation was successful,

• None – if the future doesn’t represent the size() operation.