API

Dogpile

class dogpile.dogpile.Dogpile(expiretime, init=False, lock=None)

Dogpile lock class.

Provides an interface around an arbitrary mutex that allows one thread/process to be elected as the creator of a new value, while other threads/processes continue to return the previous version of that value.

Parameters:

• expiretime – Expiration time in seconds. Set to None for never expires.
• init – if True, set the ‘createdtime’ to the current time.
• lock – a mutex object that provides acquire() and release() methods.

acquire(creator, value_fn=None, value_and_created_fn=None)

Acquire the lock, returning a context manager.

Parameters:

• creator – Creation function, used if this thread is chosen to create a new value.
• value_fn – Optional function that returns the value from some datasource. Will be returned if regeneration is not needed.
• value_and_created_fn – Like value_fn, but returns a tuple of (value, createdtime). The returned createdtime will replace the “createdtime” value on this dogpile lock. This option removes the need for the dogpile lock itself to remain persistent across usages; another dogpile can come along later and pick up where the previous one left off.

createdtime = -1

The last known ‘creation time’ of the value, stored as an epoch (i.e. from time.time()).

If the value here is -1, it is assumed the value should recreate immediately.

has_value

Return true if the creation function has proceeded at least once.

is_expired

Return true if the expiration time is reached, or no value is available.

exception dogpile.dogpile.NeedRegenerationException

An exception that when raised in the ‘with’ block, forces the ‘has_value’ flag to False and incurs a regeneration of the value.

class dogpile.dogpile.SyncReaderDogpile(*args, **kw)

Provide a read-write lock function on top of the Dogpile class.

acquire_write_lock()

Return the “write” lock context manager.

This will provide a section that is mutexed against all readers/writers for the dogpile-maintained value.

NameRegistry

class dogpile.nameregistry.NameRegistry(creator)

Generates and return an object, keeping it as a singleton for a certain identifier for as long as its strongly referenced.

e.g.:

class MyFoo(object):
    "some important object."
    def __init__(self, identifier):
        self.identifier = identifier

registry = NameRegistry(MyFoo)

# thread 1:
my_foo = registry.get("foo1")

# thread 2
my_foo = registry.get("foo1")

Above, my_foo in both thread #1 and #2 will be the same object. The constructor for MyFoo will be called once, passing the identifier foo1 as the argument.

When thread 1 and thread 2 both complete or otherwise delete references to my_foo, the object is removed from the NameRegistry as a result of Python garbage collection.

NameRegistry is a utility object that is used to maintain new Dogpile objects against a certain key, for as long as that particular key is referenced within the application. An application can deal with an arbitrary number of keys, ensuring that all threads requesting a certain key use the same Dogpile object, without the need to maintain each Dogpile object persistently in memory.

Parameters:creator – A function that will create a new value, given the identifier passed to the NameRegistry.get() method.

get(identifier, *args, **kw)

Get and possibly create the value.

Parameters:

• identifier – Hash key for the value. If the creation function is called, this identifier will also be passed to the creation function.
• **kw (*args,) – Additional arguments which will also be passed to the creation function if it is called.

Utilities

class dogpile.readwrite_lock.ReadWriteMutex

A mutex which allows multiple readers, single writer.

ReadWriteMutex uses a Python threading.Condition to provide this functionality across threads within a process.

The Beaker package also contained a file-lock based version of this concept, so that readers/writers could be synchronized across processes with a common filesystem. A future Dogpile release may include this additional class at some point.

acquire_read_lock(wait=True)

Acquire the ‘read’ lock.

acquire_write_lock(wait=True)

Acquire the ‘write’ lock.

release_read_lock()

Release the ‘read’ lock.

release_write_lock()

Release the ‘write’ lock.