Would you accept a PR for a new JDBC integration plugin?

[rocco8620]

Hello,
I have been using logstash for some time now (2 years) and used extensively the jdbc_static plugin. I noticed that it works well to enrich events with the result of complex queries but performs poorly in the combo:

• Simple query lookup by key, like SELECT col1, col2 FROM local_table WHERE id = :my_id
• High throughput pipeline (>10k events/s) where the lookup is executed for every event

I am not the only one experiencing this problem and the ending up using the translate filter instead. Some examples:

logstash-plugins/logstash-filter-jdbc_static#27

https://discuss.elastic.co/t/logstash-jdbc-static-filter-plugin-performing-very-poorly-nearly-not-at-all/214292/25

https://discuss.elastic.co/t/jdbc-static-filter/266162/12

So i though about creating a new JDBC integration plugin to fill that gap. I am thinking about using the same logic as jdbc_static to query a remote database, an hashmap to keep the data in ram, and an interface similar to the one of translate to do lookups and enrich the events in the pipeline.

This benefits from both the periodic data update of jdbc_static (no intermediate dictionary files, no cron jobs to keep them up to date, etc for the translate plugin) and the fast lookup of translate (no complex query parsing is executed and similar when I only need a primary key lookup).

I am willing to implement this new plugin and submit a PR if you like the idea and would merge it. I can send more details if needed. Let me know your thoughts or if you have any questions.

Edit:

I just noticed the same idea surfaced on the translate plugin side. logstash-plugins/logstash-filter-translate#68