Skip to content

Commit

Permalink
fix: Improve ZstdCompressor implementation
Browse files Browse the repository at this point in the history
Signed-off-by: ddukbg <[email protected]>
  • Loading branch information
ddukbg committed Nov 2, 2024
1 parent 7e28a32 commit 4d2dbff
Showing 1 changed file with 11 additions and 3 deletions.
14 changes: 11 additions & 3 deletions lib/fluent/plugin/out_s3.rb
Original file line number Diff line number Diff line change
Expand Up @@ -632,16 +632,24 @@ def compress(chunk, tmp)
end

class ZstdCompressor < Compressor
def initialize(level = 3)
@level = level
end

def ext
'zst'.freeze
end

def content_type
'application/x-zst'.freeze
end

def compress(chunk, tmp)
compressed_data = Zstd.compress(chunk.read, level: @level)
uncompressed_data = ''
chunk.open do |io|
uncompressed_data = io.read
end
compressed_data = Zstd.compress(uncompressed_data, level: @level)
tmp.write(compressed_data)
rescue => e
log.warn "zstd compression failed: #{e.message}"
Expand Down

0 comments on commit 4d2dbff

Please sign in to comment.