As one possible alternative for handling the imaging data which is going to be produced at some of the beam lines of the European Spallation Source, we have developed an areaDetector to Apache Kafka plugin. The plugin guarantees (within reasonable limitations) that data produced by an areaDetector driver is delivered to an Apache Kafka data broker. The data can then be retrieved from the data broker for as long as it is cached. An areaDetector driver which retrieves data from an Apache Kafka broker and makes it available to the IOC has also been developed. 

The code for both these modules is available in the following repository: https://github.com/ess-dmsc/ad-kafka-interface

The documentation of the modules should get most people somewhat familiar with EPICS up and running relatively quickly. Most, if not all, of the functions and classes used in the modules are documented which should make it relatively easy for a developer to get an understanding of how the plugin and driver are implemented. Unit tests exists for large parts of the code though coverage can (and probably should) be improved. Some performance and integration tests have been done but the methods used for doing these can also be improved.

If you are interested in the project, feel free to suggest improvements. The project will likely see some features and other improvements being implemented in the coming months regardless.