Quick essay on building a replicated in-memory queue.
Just an ESSAY, not a working solution.
This is just a quick micro-service skeleton.
No use-case in mind, so I won't add distributed consistency and coordination management.
Depending on the given use-case we may go for a CRDT based solution to avoid coordination.
Nevertheless in-memory queue is modeled to handle operation replication.
A node is run on 2 TCP ports:
queue operationsportcluster operationsport
Cluster operations are interfaced under cluster/ folder.
Queue operations are almost implemented on queue/ folder.
I used plain TCP for simplicity, but HTTPS, gRPC, whatever could be used for both communications.
Also Go-Kit could be used as a general good practice micro-service design.
I avoided cluster management is bounded to the cluster package.
Cluster models cluster membership management via:
- 2 request methods
- 2 receiver channels.
Publisher is supposed to manage synchronization in background mode.
It also allows domain code to publish queue changes to the cluster.
Node daemon logic is under node/.
Several connection handlers are used to concurrent update the queue.
Queue is managed via command-handler pattern to easily support replication operations.
make run
This essay is not aimed to be compiled.
To-do: add tracing capabilities go get github.com/jaegertracing/jaeger
To profile CPU usage: make profile-cpu
Memory profile could also be added