TorchCraftAI
A bot for machine learning research on StarCraft: Brood War
|
A buffered consumer that sends data via ZeroMQ. More...
#include <zmqbufferedconsumer.h>
Public Member Functions | |
ZeroMQBufferedConsumer (uint8_t nthreads, size_t maxQueueSize, std::vector< std::string > endpoints, std::shared_ptr< zmq::context_t > context=nullptr) | |
~ZeroMQBufferedConsumer () | |
void | enqueue (T arg) |
void | updateEndpoints (std::vector< std::string > endpoints) |
A buffered consumer that sends data via ZeroMQ.
The intended use-case is for this class to be used together with ZeroMQBufferedConsumer to implement distributed producer-consumer setups. Suppose you have an existing setup that looks like this, with sections of your code producing items of type T and other sections consuming them:
[Producer] -> [Consumer]
Then, assuming that items can be serialized with cereal, the ZeroMQBufferedConsumer/Producer classes enable the following design:
[Producer] -> [ZeroMQBufferedConsumer] | TCP | [ZeroMQBufferedProducer] -> [Consumer]
As in common::BufferedConsumer, you specify the number of threads and a queue size. In addition, you supply a list of end points that ZeroMQBufferedProducer instances have been bound to. Data will be send to end-points in a round-robin fashion. If producer endpoints don't accept new data (because their queue is full and items are not consumed fast enough), enqueue()
will eventually block and perform retries.
cpid::ZeroMQBufferedConsumer< T >::ZeroMQBufferedConsumer | ( | uint8_t | nthreads, |
size_t | maxQueueSize, | ||
std::vector< std::string > | endpoints, | ||
std::shared_ptr< zmq::context_t > | context = nullptr |
||
) |
cpid::ZeroMQBufferedConsumer< T >::~ZeroMQBufferedConsumer | ( | ) |
void cpid::ZeroMQBufferedConsumer< T >::enqueue | ( | T | arg | ) |
void cpid::ZeroMQBufferedConsumer< T >::updateEndpoints | ( | std::vector< std::string > | endpoints | ) |