how to implement a two-step data load

[freesinger]

Discussed in #6624

^{Originally posted by freesinger December 27, 2022}
Hi all, I am using fluent-bit to collect log and import an OLAP DB. Unfortunatly, this DB supports only a two-step data load, which upload to s3 first and pull from s3 bucket via HTTP.
After reading fluent-bit document, it seems no pre-built output plufin can realize my purpose, here are some methods may be word:

• use Lua Filter and perform upload and pull in it, if it is legal to have no Output stage? Or I just put a dummy output to fulfill?
• upload via s3 output and pull via http output, if it is legal to have 2 Ouptut stages?
  Or any other workround? Any help will be appreciated.

[freesinger]

https://github.com/fluent/fluent-bit-go seems to be a workaround