Thumbnail improvements

[naanlizard]

Is your feature request related to a problem? Please describe.
This is partly a request for clarification in the docs as well as features.

For our purposes, input streams may be arbitrary aspect ratios. This means that for creating thumbnails, we need to not adjust the aspect ratio at all. This can be done either by taking a thumbnail of the stream and using black bars to fill in empty space (rather than stretch), taking the thumbnail as the maximum dimension that fits in the target thumbnail size, taking a full size screenshot, or some other method.

This second mode is used already in OME for video transcoding - see here and perhaps can be easily added to the thumbnailer, or may already work that way - if so, the documentation should be updated to describe that

Additionally, again for our purposes we do not need thumbnails more frequently than 5 seconds at most. Perhaps this can already be done, but it is not clear in the documentation here whether you can input fractional and/or decimal values.

Lastly, it would be useful to have multiple thumbnail providers for multiple sizes we may need - full size is one of the more common sizes we need, but also smaller thumbnails and sometimes medium size. I think this is already possible, and if it is, I will make a PR to update the gitbook

[bchah]

In FFMPEG you could use this filter to pad it to an aspect ratio of your choosing (just set the resolution)

-vf scale=256:144:force_original_aspect_ratio=decrease,pad=256:144:-1:-1:color=black

I agree it would be cool if OME just gave us thumbnails in their native aspect ratio, where the size dimensions provided are treated as a maximum size rather than an absolute size.

[naanlizard]

Yes, we do something similar with ffmpeg and a cronjob, but this isn't a clean solution and it would be nice if we could just pull a thumbnail from OME (whether by http or not)

[Keukhan]

@naanlizard

Thanks for the great report.

Like the video scaling option, the thumbnail image can be rescaling while keeping the aspect ratio. see the example below.
However, it does not support the option to keep the aspect ratio along with the margins for certain resolutions.

<Image> <Codec>jpeg</Codec> <Width>320</Width> <Framerate>0.2</Framerate> </Image>

It's not clearly written in the manual, but if you want to generate a thumbnail image once every 5 seconds. try setting 0.2 as in the example above. It is an option to generate 0.2 images per second.

I would be very grateful if you could PR about the insufficient contents of the gitbook.

Thanks.

[naanlizard]

Iirc performance even with a very low bitrate was horrible compared to ffmpeg. I'll see if we can test again soon

[naanlizard]

Just to confirm, there haven't been updates to improve the performance of thumbnail generation, correct? If thumb generation is now closer to ffmpeg efficiency, we'll more aggressively test this :)

[Keukhan]

@naanlizard

FFmpeg performs decoding only in the section necessary for thumbnail generation, so CPU usage is low. on the other hand, OME decodes all video frames regardless of the thumbnail framerate, so CPU usage is relatively high. this is because the decoded frame is designed as an efficient structure for ABR encoding, not just a thumbnail.
If only thumbnail image generation is used without an encoding such as H264 or VP8, it can be improved to reduce CPU occupancy by performing decoding only in an appropriate section. I'll put that in the plan for performance improvement. :)

In addition, Output size is fixed, and a method of filling the padding according to the ratio of the original image will be added soon.
I'll let you know when a feature is added. 👍

[naanlizard]

Once there are performance improvements we will try and benchmark it. We're very happy with ffmpeg thumbnail generation performance so if it is similar, we'd definitely switch for convenience