Adding resamplers kernels and named resizers and kernel control to Convert()

[DTL2020]

As noted in https://forum.doom9.org/showthread.php?p=1983080#post1983080 post by FranceBB it is good to add more resamplers kernels and named resizers to AVS core. It is SinPowResize and possibly UserDefined2 (where 2 is only number of kernel sum coefficients used, may be more for better precision but harder for user to enter and adjust, also typical software solutions using controlled kernels nowdays typically limited with 2 max only user-defined control params, like Bicubic). Also in compare with Spline and many other kernels with lots of 'magic numbers' the UserDefined* kernel is based on 'natural enough for DSP' sinc and do not have any built-in magic numbers at all.

Implementation compatible with AVS resampler is in the jpsdr's plugin: https://github.com/jpsdr/ResampleMT/blob/master/ResampleMT/resample_functions.cpp
Simple C-only, no asm optimization required as it startup-time-only resampler program init.

It will be addition of 2 more resampling kernels with 'conditioning' at dowsampling activity to currently only GaussResize having same feature at some range of control param values.

And second as different resampling kernels may be used in Convert() filter for scaling UV channels - it is good to add kernel control params (param1 and param2) to Convert() so user can adjust kernels by the needs of current use case.

[pinterf]

Thanks, I can see that syncfilter taps is increased from 20 to 150 as well.

[DTL2020]

As about SincResize and its current issues with current resampling engine (#182) - in jpsdr's plugin also implemented some 'workaround' version of Sinc kernel (SincLin2) having at least taps/2 full-blood sinc kernel and some linear fading weightining to zero at the end.
https://github.com/jpsdr/ResampleMT/blob/32ba5c731aefcb64959e09480421d195da6979b4/ResampleMT/resample_functions.cpp#L318
So it works without that issues with resampler at least for 8bit precision and may be 16. The 32bit float may still have some residuals (with low enough taps numbers).

So instead of SincResize(taps=N) with edge of kernel distortions user can use SincLin2Resize(taps=Nx2) and having at least same number N of full-blood sinc lobes not degraded with any weighting used.

In theory it may be merged to SincResize() adding param like 'fix_end= true/false'. If fix_end=true - use internal taps = taps_x_2 and apply some fading to zero weighting to the last half of taps.

[pinterf]

I think I ported them all, I'm gonna test them a bit.

Three new functions:

SinPowerResize "cii[src_left]f[src_top]f[src_width]f[src_height]f[p]f"
parameters like GaussResize: optional "p"
Default: p=2.5

SincLin2Resize "cii[src_left]f[src_top]f[src_width]f[src_height]f[taps]i"
parameters like SincFilter or LanczosFilter: optional "taps"
Default taps=15

UserDefined2Resize "cii[b]f[c]f[src_left]f[src_top]f[src_width]f[src_height]f"
parameters like BicubicResize: Optional "b" and "c"
Default b=121.0, c=19.0

and their equivalent for the ConvertToXXXX family "chromaresample" parameters:
"sinpow", "sinclin2" and "userdefined2"

EDIT:
parameters are not yet implemented
'param1' would hold for 'taps'/'p'/'b', 'param2' for 'c'

[DTL2020]

I can see that syncfilter taps is increased from 20 to 150 as well.

It was done to check the computing issues with 'too short' truncated sinc kernel. That edges-computing bugs disappear (at least with 8 bit precition) without sinc kernel changing at taps about 70+. Mostly probably because sinc is slowly enough self-fading to zero function and only at taps about 70 and more it self-fades to zero good enough so existing resampler engine provide finally good (expected) output. But in everyday life close to no one uses such long sinc kenels simply to fix end of computing issues and have too low computing performance.
In theory the LanczosResize with taps about Nx3 or more make close result to SincResize of taps N. So SincResize is only shortest and fastest to compute (lowest 'support/window' for resampler) with highest 'sharpnes' at given number of 'taps'. But with its pure form and very short truncated kernel it also produces that end of kernel computing issues. Adding some more taps to the end with weighting to 0 fix issue but make some performance drop because resampler need to process more data. So the research task if someone have time may be in selecting shortest possible addition to standard non-changed sinc kernel and its weighting shape to fix end-compute issues (for each processing precision - 8, 16, 32foat (if possible)). The currently used linear 45-degree fading to zero as ending weighting starting from 0.5 length of kernel may be not perfect solution in terms of performance. It is close to fisrt solution that fix issue and short enough to compute. At least it shoud provide a bit better performance in compare with Lanczos weighting of larger taps number used and with comparable 'sharpness' (defined by number of non or as low as possible weighted number of sinc lobes starting from zero). In everyday life usage it is typically enough to have about up to 5...7 non-weighted sinc taps from zero and some computing/convoluting engine without additional bugs. So SincLin2 with default taps=15 provide at least 7 taps with full-sinc and some workaround to fix computing issues. But it not guaranteed from existance even shorter and faster to compute version of ending weighting having also required minimum full-strike sinc taps from zero and making result free from ending issues. So SincLin2 is not really separate resize kernel and only current programming workaround for SincResize and placed inbetween LanczosResize (or many other more or less smoothly weighted sincs) (slower) and SincResize (fastest but providing some issues at low taps number).

Most of 'resizers' based on weighted sinc were loaded with 'science of spectrums'. So we have lots of different weightings like Lanczos/Blackman/Hanning/Hamming/.. and many more. And with computing issue of too truncated sinc in DSP/image processing we have non-spectrum-science but simple math+programming computing issue so its additional sinc-shaping is need to be designed simply fast enough to compute without visible errors (at least when residual computing error fall below 1 LSB in current used precision it is enough fixed state). Or may be other solutions may exist (may be some resampler fix or redesign ?) so we can continue use SincResize unchanged - but I still do not know. If we where at education company this simple enough math task may be given to some student to think some hours/days/months and present some research results at this problem with may be better solution in compare with existing SincLin2 for each output bitdepth.

"'param1' would hold for 'taps'/'p'/'b', 'param2' for 'c'"

Different kernels may have different param types (integer and float). Will this add additional issues or we can use float everywhere ?

[Reel-Deal]

@DTL2020

For documentation purposes, can you provided a brief user-friendly description of each filter? I was going thru the "Sin squared kernel for resize (Gaussresize mod)" thread to see if I can find more info but I got overwhelmed. I also looked in the ResampleMT docs but only the b and c parameters of UserDefined2Resize are described.

Friendly descriptions:

UserDefined2Resize is a pure sinc-based resizer, its kernel is just a sum of weighted sincs:
return c*sinc(x+2) + b*sinc(x+1) + a*sinc(x) + b*sinc(x-1) + c*sinc(x-2)
Where a=1 and b and c are user-defined weights.

Source: https://forum.doom9.org/showthread.php?p=1939704#post1939704

[DTL2020]

Both SinPowResize and UserDefined2 mostly designed as downsampling resizers. Also can be used as convolution filter only with very small but non-zero src_left param for example. Their params allow to control the 'sharpness' of output downsampled frame adjusting to desired 'looking' or 'makeup' (as soft film-look without over/under shoots at all or more sharper video-look with more or less over/undershooting at levels transients). Also may be used for high quality antialiasing work. Example:

Subtitle(size = 10x of required)
SinPowResize or UserDefined2Resize (width/10, height/10)

SinPow is easier to adjust with single control param and UserDefined2 is harder to adjust because of 2 control params typically required to change at once (to keep result from ringing as much as possible) but may provide better quality.

For recommended (but not limited to) b and c control values for UserDefined2Resize only possible to add to documentation the (left part for b and c only) of table from https://forum.doom9.org/showthread.php?p=1951250#post1951250 post. The idea of b and c values range - it can be directly set as samples values in 8bit limited range and to see its spatial responce in AVS (applying Sinc/SincLin2 resize with enlarging several times). So zero value of b and c is 16 (16.0). When both b and c set to 16 - the impulse kernel is equal to SincResize with taps about 3. There also exist some web-based tool for manual adjusting of b and c values in their possible range +250.. -50 to get non-ringing kernel of the possible 'sharpness' (it is expected the resize result inherit the properties of resizer's kernel, but it may be not completely correct with small enough shrinking ratios and move close to complete inheritance at shrinking ratios about 10 and more). It may be possible to add it to some web hosting and to offline documentation (it is HTML and javascript drawing tool). I will try to search it at home.

With low shrinking ratios (like about 2 for example downsizing from HD to SD) the kernel actively interacts with spectrum of source and best b and c may be not as table shows. But for high shrinking ratios (like in initial data production like text antialiasing or 3D graphics rasterizing for moving pictures environment with about 5..10x downsize ratios) table values may be used as good enough reference.

The SinPow kernel have not very nice range of single float control param p in typical useful range of 2.5f to 3.0 (or may be to 3.5f). Where 2.5 is 'very sharp' and 3.0+ may be closer to Gauss in softness.

One of the main purpose of these resizers is attempt to create content 'conditioned to channel' where 'channel' assumed to be Nyquist frequency limited and also as digital it also may have 'Gibbs phenomenon' causing 'ringing' after upsizing or converted to analog form (via DAC). So the 'conditioning' is both fighting aliasing and ringing. Both resizers can produce single lobe 'peaking' (single overshoots and undershoots arount level transient) close to BicubicResize and it may enhance visual sharpness (acutance - https://en.wikipedia.org/wiki/Acutance ).

Yes - for UserDefined2 it is good to add its kernel equation. May be we will get some users comments about making it better some day. The math idea of this kernel: the linear combination of band-limited functions is also band-limited. So as sinc(x) is band-limited base of this kernel, so any linear combination defined by b and c control params will form band-limited kernel too.

As for SincLin2 it is simply workaround supplement to SincResize and for being 'not-worse' in full-strike sinc taps count recommended to set SincLin2(taps) = 2 times larger of 'classic' SincResize(taps).

[DTL2020]

@Reel-Deal

Attached is web-browser based util for selecting pairs of b and c control values for UserDefined2 kernel giving possibly lowest possible (with 2 members only after zero) ringing while having different shape of transient (with or without over/under shoots). It tested in Google Chrome but some fields may not work in other browsers (may be need fix by better html/script developer). Also it may help to understand how samples values create real interpolated data for displaying and where is the Gibbs-ringing come from. It may be included into HTML documentation or embedded into website help/wiki.

ud2_util.zip

[DTL2020]

After more testing and experiments:
It was found the performance of UserDefined2 in residual ringing significantly depends on 'support' size used. Most changes occur in range of 'support' between current 2.0 and 3.0.
Increasing 'support' to 4 change very few (at least with tested 'sharp' b/c of about 80/-20). So current hard truncation of 'support' to 2.0 makes its performance in residual ringing close to SinPowResize.
Hard truncating of kernel with support of 2.0 create still enough 'non-linear' processing (close to that with SinPow kernel) and it cause a bit more sharp result but with more residual ringing and artifacts.

So recommended changes:

1. Add 3rd control param to UserDefined2Resize named 'support' or may be 's' to make naming shorter and faster to type at scriptwriting. Default of 2.2 float and valid range of 2.0 to 3.0 (may be to 4.0 if someone need, also it good to check if resampler actually requests x>3 with support>3 at line

   AviSynthPlus/avs_core/filters/resample_functions.cpp

   Line 308 in e9924f8

   if (x <= 3) // ?

   . If not - may be if() condition may be removed at all and kernel size will be limited to filter's support). Default 2.2 float will more differentiate UserDefined2Resize from result of SinPowResize.

2. Adding 3rd control param to resizer again changes number of control params for ConvertXXX filters but if possible it is good to add param3 to ConvertXXX. If not added - lets make default support param to UserDefined2 kernel of 2.2.

Addition to documentation: Adjusting support/s param of UserDefined2Resize from 2.0 to higher values allow better control over residual ringing but makes result a bit softer.

Also the current defaults of b and c 121/19 for UserDefined2Resize create very soft (film-look makeup) result at downsizing. May be change to more sharper like 90/-12.

Current test of resizers for 2:1 downsize and checking for displaying with taps=8 sincresize:
https://drive.google.com/file/d/1ds4x-63WnpT905nZVcbnELMfHKj3Cr_w/view?usp=sharing

Script is
LoadPlugin("ffms2.dll")
LoadPlugin("ResampleMT.dll")

FFmpegSource2("src.avi")
Crop(800,500, 440, 220)
bicub_0d5=BicubicResize(width/2, height/2, b=-0.5, c=0.25).SincLin2ResizeMT(width2, height2, taps=8).Subtitle("Bicubic b=0.5, c=-0.25")
lz4=Lanczos4Resize(width/2, height/2).SincLin2ResizeMT(width2, height2, taps=8).Subtitle("Lanczos4")
sin_pow=SinPowResizeMT(width/2, height/2, p=2.7).SincLin2ResizeMT(width2, height2, taps=8).Subtitle("SinPow p=2.7")
ud2_75_s22=UserDefined2Resize(width/2, height/2,b=75,c=-25).SincLin2ResizeMT(width2, height2, taps=8).Subtitle("UD2 b=75 c=-25 s2.2")

v1=StackVertical(bicub_0d5, ud2_75_s22)
v2=StackVertical(sin_pow, lz4)
StackHorizontal(v1, v2)

2.2 support for UserDefined2Resize was tested with custom debug build of current avisynth+ sources.

[DTL2020]

Created pull-request to jpsdr's plugin with updates to UserDefined2Resize - jpsdr/ResampleMT#4 . It may be used here too.

[DTL2020]

Some more idea - the adjustable (float) s-param of support may be added to many other AVS resizers. At least at most of sinc-based (Lanczos) and Gauss (may be also Bicubic ?). It is nice that AVS resampler accept support as float so if someone interested - may play with float-support param for resizing. May be Dogway will make tests and additions to his resizers pack.

Currently high-blurring Gauss (with some p-param values) may be limited with too small support internal default value. Also playing with support around 2 may give visibly different results with ringing-kernels like Sinc/Lanczos and others.

Unfortunately for more hacky kernel of SinPow-resize it looks not possible to adjust support without significant increasing of issues (or may be only very small play around default 2 possible).