Elasticsearch backend configuration options

[iddan]

Originally from @olivere
I'm the author of github.com/olivere/elastic that you're using as a storage backend for Elasticsearch. I've seen issues popping up in Elastic that describe connection problems when using ES as a backend with Cayley.

These connection problems typically stem from the fact that Elastic (the Go client) is using a process called sniffing to do client-side load-balancing between the nodes of an Elasticsearch cluster (I didn't invent this: All the official Elasticsearch clients have this.) However, sniffing may fail due to the nodes not being routable (e.g. running ES in a Docker container will return private IPs by default).

Long story short: If you're using Elastic, it might be necessary to pass additional options into the elastic.NewClient(...) function, e.g. for disabling sniffing or disabling health checks. However, Cayley is currently not passing any additional parameters, and I don't see how I might be able to configure this in Cayley.

But maybe I have a solution.

Elastic has a subpackage github.com/olivere/elastic/config that allows to users to pass a URL with query parameters. These parameters will be parsed and a *config.Config structure is returned. You can pass that *config.Config into elastic.NewClientFromConfig(*config.Config) to get a client.

That'd allow to keep your configuration simple (it's just a URL) and make users capable of configuring their Elasticsearch connection however they need.

I'm not sure if you're interested in this feature, but if you are and I can help, just let me know. :-)