OSS-Fuzz build does not find codec dependencies

[farindk]

OSS-Fuzz builds libaom, x265, and libde265, but does not find them when building libheif, resulting in only enabling the 'uncompressed' codec (mentioned in #1345 (comment)). Link to Build Logs.

Additionally to fixing this, we should also enable the other codecs in the build.

[bobfriesenhahn]

The GraphicsMagick oss-fuzz build is doing much better regarding libheif. I think that currently only one useful dependency is excluded (because a pkg-config file is now missing).

Feel free to borrow ideas from GraphicsMagick's oss-fuzz support at https://foss.heptapod.net/graphicsmagick/graphicsmagick/-/tree/branch/default/fuzzing?ref_type=heads, and particularly the "oss-fuzz-build.sh" script at https://foss.heptapod.net/graphicsmagick/graphicsmagick/-/blob/branch/default/fuzzing/oss-fuzz-build.sh?ref_type=heads

If you see something that I should be doing better, please make me aware of it.

[bobfriesenhahn]

The GraphicsMagick oss-fuzz build does not actually need x265 yet, but I tried to re-add it to the oss-fuzz build while using https://bitbucket.org/multicoreware/x265_git/src/stable/ as the source repository path (as libheif's oss-fuzz build does).

The build failed while building libheif with this error:

[ 81%] Building CXX object libheif/CMakeFiles/heif.dir/plugins/encoder_mask.cc.o
cd /src/heif_build/libheif && /usr/local/bin/clang++ -DENABLE_MULTITHREADING_SUPPORT=1 -DENABLE_PARALLEL_TILE_DECODING=1 -DHAVE_AOM_DECODER=1 -DHAVE_AOM_ENCODER=1 -DHAVE_LIBDE265=1 -DHAVE_LIBSHARPYUV=1 -DHAVE_UNISTD_H -DHAVE_VISIBILITY -DHAVE_X265=1 -DIS_BIG_ENDIAN=0 -DLIBHEIF_EXPORTS -I/src/heif_build -I/src/libheif/libheif -I/src/libheif/libheif/api -I/src/libheif/include/libheif -I/src/libheif/include -I/work/include -I/work/include/webp -O1   -fno-omit-frame-pointer   -gline-tables-only   -Wno-error=enum-constexpr-conversion   -Wno-error=incompatible-function-pointer-types   -Wno-error=int-conversion   -Wno-error=deprecated-declarations   -Wno-error=implicit-function-declaration   -Wno-error=implicit-int   -Wno-error=vla-cxx-extension   -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -stdlib=libc++ -g -std=c++20 -fPIC -fvisibility=hidden -fvisibility-inlines-hidden   -Werror -Wall -Wsign-compare -Wconversion -Wno-sign-conversion -Wno-error=conversion -Wno-error=unused-parameter -Wno-error=deprecated-declarations -Wno-error=tautological-compare -Wno-error=tautological-constant-out-of-range-compare -Wno-error=potentially-evaluated-expression -Wno-tautological-constant-out-of-range-compare -Werror -MD -MT libheif/CMakeFiles/heif.dir/plugins/encoder_mask.cc.o -MF CMakeFiles/heif.dir/plugins/encoder_mask.cc.o.d -o CMakeFiles/heif.dir/plugins/encoder_mask.cc.o -c /src/libheif/libheif/plugins/encoder_mask.cc
/src/libheif/libheif/plugins/encoder_x265.cc:918:23: error: cannot initialize a parameter of type 'x265_picture *' with an rvalue of type 'x265_picture **'
  918 |                       &out_pic);
      |                       ^~~~~~~~
/src/libheif/libheif/plugins/encoder_x265.cc:993:38: error: cannot initialize a parameter of type 'x265_picture *' with an rvalue of type 'x265_picture **'
  993 |                                      &out_pic);
      |                                      ^~~~~~~~
2 errors generated.
make[2]: *** [libheif/CMakeFiles/heif.dir/build.make:849: libheif/CMakeFiles/heif.dir/plugins/encoder_x265.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/src/heif_build'
make[1]: *** [CMakeFiles/Makefile2:198: libheif/CMakeFiles/heif.dir/all] Error 2
make[1]: Leaving directory '/src/heif_build'
make: *** [Makefile:159: all] Error 2

[farindk]

The build failed while building libheif with this error:

x265 had changed the signature of the encoder_encode() function in version 212. See #1314 and https://bitbucket.org/multicoreware/x265_git/issues/952/crash-in-libheif-tests.

It seems that they changed it back in version 213. I have adapted the #if in commit 9c39813
Now, it should compile again.

[bobfriesenhahn]

I have verified that GraphicsMagick oss-fuzz builds with everything (including x265) now, using "https://bitbucket.org/multicoreware/x265_git/src/stable/". I am not sure if GraphicsMagick should switch to using "https://bitbucket.org/multicoreware/x265_git/src/master/" since I have little knowledge of the x265 project and how stable it is on a day-to-day basis.

[farindk]

I'm also not sure whether there is much benefit in using x265 master instead of stable. I just know that we constantly had fuzzing alarms because of issues inside x265. Often false positives because of read access to uninitialized data (which can be happen, for example, in the border padding region of images without any noticeable effect).