Skip to content

Commit

Permalink
Merge pull request #90 from kamsar/feature/avif
Browse files Browse the repository at this point in the history
Avif support. Only works on async mode at the moment, some issue with…
  • Loading branch information
markgibbons25 authored Jan 10, 2022
2 parents 1c50f07 + c66a946 commit 9776f9a
Show file tree
Hide file tree
Showing 44 changed files with 1,968 additions and 183 deletions.
66 changes: 56 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,11 @@ Dianoga supports:
* PNGs (via PNGOptimizer - lossless / pngquant - lossy)
* SVGs (via SVGO - lossless, and automatic gzipping of SVG media responses)
* WebP (via cwebp - lossless or lossy)
* Avif (via avifenc - lossless or lossy)
* JPEG XL (via cjxl - lossless or lossy)
* Auto convert JPEG/PNG/GIF to WebP based on browser support
* Auto convert JPEG/PNG to Avif based on browser support
* Auto convert JPEG/PNG/GIF to JPEG XL (jxl) based on browser support

Additional format support is possible to add via new processors in the `dianogaOptimize` pipeline.

Expand Down Expand Up @@ -63,30 +67,72 @@ To perform a manual installation:
If you are enabling the SVGO optimiser, you'll also need the [Dianoga.svgtools](https://www.nuget.org/packages/Dianoga.svgtools) NuGet package.
This is simply a prepackaged compiled version of SVGO called SVGOP from [here](https://github.com/twardoch/svgop).

## WebP feature
## Next-gen Formats Support

WebP is is an image format employing both lossy and lossless compression. It is currently developed by Google, based on technology acquired with the purchase of On2 Technologies. WebP file size is [25%-34% smaller compared to JPEG file size](https://developers.google.com/speed/webp/docs/webp_study) and [26% smaller for PNG](https://developers.google.com/speed/webp/docs/webp_lossless_alpha_study). [All evergreen browsers except Safari currently support WebP](https://caniuse.com/#feat=webp). By default WebP optimization is disabled since you need to do some due diligence around reviewing configs, how it works, and importantly if you are using a CDN you need to do some extra steps.
Next-gen images use formats with superior compression and quality characteristics compared to their GIF, JPEG, and PNG ancestors. These image formats support advanced features designed to take up less data while maintaining a high quality level, making them perfect for web use.

### How WebP optimization works:
Browser sends request to server to get image. If browser supports WebP image format then it sends "image/webp" value in Accept header. It is possible to detect this header on server and return WebP image to browser instead of JPEG or PNG. If browser doesn't support WebP then other image optimizers are executed if they are enabled.
### How Next-gen Formats Optimization Works:

### How to enable WebP support:
1. Enable `Dianoga.WebP.config.disabled` config and adjust any parameters if you require lossless or higher quality than the default
2. Open web.config and change line
Browser sends request to server to get image. It sends list of accepted image formats in the `Accept` header, e.g. it can be `image/avif,image/webp,image/apng,image/*,*/*;q=0.8`. Presence of `image/webp` means that this browser supports `WebP` image format. Absense of `image/jxl` means that this browser does not support `JPEG XL` format. It is possible to check this header on server side and return `WebP` format image to browser instead of `JPEG`, `PNG` or `GIF`. If browser doesn't support any next-gen formats then other image optimizers are executed if they are enabled.

### How to Enable Next-gen Formats Support:

1. Open web.config and change line

`<add verb="*" path="sitecore_media.ashx" type="Sitecore.Resources.Media.MediaRequestHandler, Sitecore.Kernel" name="Sitecore.MediaRequestHandler" />`

to

`<add verb="*" path="sitecore_media.ashx" type="Dianoga.MediaRequestHandler, Dianoga" name="Sitecore.MediaRequestHandler" />`
`<add verb="*" path="sitecore_media.ashx" type="Dianoga.NextGenFormats.MediaRequestHandler, Dianoga" name="Sitecore.MediaRequestHandler" />`

OR if you use SXA

`<add verb="*" path="sitecore_media.ashx" type="Dianoga.MediaRequestHandlerXA, Dianoga" name="Sitecore.MediaRequestHandler" />`
`<add verb="*" path="sitecore_media.ashx" type="Dianoga.NextGenFormats.MediaRequestHandlerXA, Dianoga" name="Sitecore.MediaRequestHandler" />`

OR if you have a custom `MediaRequestHandler` then you need to make some changes - see `MediaRequestHandler.cs`

3. If you run Sitecore under CDN: carefully review and enable `Dianoga.WebP.CDN.config.disabled`, and disable `Dianoga.Strategy.GetMediaStreamSync.config`.
2. If you run Sitecore under CDN: review and enable `Dianoga.NextGenFormats.CDN.config.disabled`. It will add `?extension=<list of supported extensions>` query parameter to all images present on your pages.

3. Enable any next-gen formats configuration files that you want. e.g.: `z.01.Dianoga.NextGenFormats.WebP.config.disabled`, `z.02.Dianoga.NextGenFormats.Avif.config.disabled`, `z.03.Dianoga.NextGenFormats.Jxl.config.disabled`

4. Review files that you have enabled and adjust any parameters if you require lossless or higher quality than the default

5. Adjust order of next-gen formats configuration files.

WebP, Avif and JPEG XL formats are not convertable to each other. Dianoga works with file stream and сonsistently converts file stream using optimizers.
e.g. JPEG > mozjpeg > Avif. Once stream is converted to one next-gen format it could not be easily reconverted to another format, e.g. Avif <=> WebP, because current encoders doesn't support it.

Configuration files should be applied in **reversed** priority order. The first format should be applied in last confiration file.

E.g. We want to support all JPEG XL, Avif and WebP in next priority 1. JPEG XL 2. Avif 3. WebP. If browser supports JPEG XL then try to use it. If browser doesn't support JPEG XL then check if browser support Avif and try to use it. If browser doesn't support both JPEG XL and Avif then check if browser supports WebP and try to use it.

Then next-gen configuration file names should have prefixes to put files in the proper order: **z.01**.Dianoga.NextGenFormats.WebP.config.disabled, **z.02**.Dianoga.NextGenFormats.Avif.config.disabled, **z.03**.Dianoga.NextGenFormats.Jxl.config.disabled

### Next-gen formats list

#### WebP

WebP is is an image format employing both lossy and lossless compression. It is currently developed by Google, based on technology acquired with the purchase of On2 Technologies. WebP file size is [25%-34% smaller compared to JPEG file size](https://developers.google.com/speed/webp/docs/webp_study) and [26% smaller for PNG](https://developers.google.com/speed/webp/docs/webp_lossless_alpha_study). [All evergreen browsers except Safari currently support WebP](https://caniuse.com/#feat=webp).

By default WebP optimization is disabled since you need to do some due diligence around reviewing configs, how it works, and importantly if you are using a CDN you need to do some extra steps.

If you want to enable usage of `WebP` format, please rename `z.01.Dianoga.NextGenFormats.WebP.config.disabled` to `z.01.Dianoga.NextGenFormats.WebP.config`

#### Avif

Avif is a modern image format based on the AV1 video format. AVIF generally has better compression than WebP, JPEG, PNG and GIF and is designed to supersede them. AVIF competes with JPEG XL which has worse support, similar compression quality and is generally seen as more feature-rich than AVIF. [Avif is supported by Chrome and Firefox browsers](https://caniuse.com/#feat=avif).

By default Avif optimization is disabled since you need to do some due diligence around reviewing configs, how it works, and importantly if you are using a CDN you need to do some extra steps.

If you want to enable usage of `Avif` format, please rename `z.02.Dianoga.NextGenFormats.Avif.config.disabled` to `z.02.Dianoga.NextGenFormats.Avif.config`

#### JPEG XL

JPEG XL is a modern image format optimized for web environments. JPEG XL generally has better compression than WebP, JPEG, PNG and GIF and is designed to supersede them. JPEG XL competes with AVIF which has better support, similar compression quality but fewer features overall. JPEG XL is not supported yet by modern browsers. [But support could be enabled via flags in Chrome, Firefox, Opera and Edge](https://caniuse.com/?search=jpeg%20xl).

By default JPEG XL optimization is disabled since you need to do some due diligence around reviewing configs, how it works, and importantly if you are using a CDN you need to do some extra steps.

If you want to enable usage of `JPEG XL` format, please rename `z.03.Dianoga.NextGenFormats.Jxl.config.disabled` to `z.02.Dianoga.NextGenFormats.Jxl.config`


## Upgrade
Expand Down
104 changes: 104 additions & 0 deletions src/Dianoga.Tests/NextGenFormats/HelpersTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Web;
using Dianoga.NextGenFormats;
using FluentAssertions;
using Moq;
using Xunit;

namespace Dianoga.Tests.NextGenFormats
{
public class HelpersTests
{
[Fact]
public void GetSupportedFormats_ShouldCallAndReturnValueFromdianogaGetSupportedFormatsPipeline()
{
//Arrange
var context = new Mock<HttpContextBase>();
var request = new Mock<HttpRequestBase>();
var acceptTypes = new[] {"image/webp"};
request.SetupGet(r => r.AcceptTypes).Returns(acceptTypes);
context.Setup(ctx => ctx.Request).Returns(request.Object);
var helpers = new Helpers();
var pipelineHelpers = new Mock<PipelineHelpers>();
pipelineHelpers.Setup(h => h.RunDianogaGetSupportedFormatsPipeline(It.IsAny<string[]>())).Returns("webp").Verifiable();
helpers.PipelineHelpers = pipelineHelpers.Object;

//Act
var result = helpers.GetSupportedFormats(context.Object);

//Assert
result.Should().Be("webp", "");
pipelineHelpers.Verify(m=>m.RunDianogaGetSupportedFormatsPipeline(acceptTypes), Times.Once());
}

[Fact]
public void GetSupportedFormats_ShouldNotCallDianogaGetSupportedFormatsPipeline_WhenNoAcceptTypes()
{
//Arrange
var context = new Mock<HttpContextBase>();
var request = new Mock<HttpRequestBase>();
context.Setup(ctx => ctx.Request).Returns(request.Object);
var helpers = new Helpers();
var pipelineHelpers = new Mock<PipelineHelpers>();
helpers.PipelineHelpers = pipelineHelpers.Object;

//Act
var result = helpers.GetSupportedFormats(context.Object);

//Assert
result.Should().Be(String.Empty, "");
pipelineHelpers.Verify(m => m.RunDianogaGetSupportedFormatsPipeline(It.IsAny<string[]>()), Times.Never);
}

[Fact]
public void GetCustomOptions_ShouldTakeValueFromQueryString_WhenExtensionQueryStringIsPresent()
{
//Arrange
var context = new Mock<HttpContextBase>();
var request = new Mock<HttpRequestBase>();
context.Setup(ctx => ctx.Request).Returns(request.Object);
var acceptTypes = new[] { "image/webp" };
request.SetupGet(r => r.AcceptTypes).Returns(acceptTypes);
var queryString = new NameValueCollection();
queryString.Add("extension", "webp,avif");
request.SetupGet(r => r.QueryString).Returns(queryString);
var helpers = new Helpers();
var pipelineHelpers = new Mock<PipelineHelpers>();
helpers.PipelineHelpers = pipelineHelpers.Object;

//Act
var result = helpers.GetCustomOptions(context.Object);

//Assert
result.Should().Be("webp,avif", "");
pipelineHelpers.Verify(m => m.RunDianogaGetSupportedFormatsPipeline(It.IsAny<string[]>()), Times.Never);
}

[Fact]
public void GetCustomOptions_ShouldTakeValueFromFromAcceptTypes_WhenExtensionQueryStringIsEmpty()
{
//Arrange
var context = new Mock<HttpContextBase>();
var request = new Mock<HttpRequestBase>();
context.Setup(ctx => ctx.Request).Returns(request.Object);
var acceptTypes = new[] { "image/webp", "image/avif" };
request.SetupGet(r => r.AcceptTypes).Returns(acceptTypes);
var queryString = new NameValueCollection();
queryString.Add("extension", "");
request.SetupGet(r => r.QueryString).Returns(queryString);
var helpers = new Helpers();
var pipelineHelpers = new Mock<PipelineHelpers>();
pipelineHelpers.Setup(h => h.RunDianogaGetSupportedFormatsPipeline(It.IsAny<string[]>())).Returns("webp,avif").Verifiable();
helpers.PipelineHelpers = pipelineHelpers.Object;

//Act
var result = helpers.GetCustomOptions(context.Object);

//Assert
result.Should().Be("webp,avif", "");
pipelineHelpers.Verify(m => m.RunDianogaGetSupportedFormatsPipeline(It.IsAny<string[]>()), Times.Once);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
using System;
using System.Collections.Specialized;
using System.Web;
using Dianoga.NextGenFormats;
using Dianoga.NextGenFormats.Pipelines.DianogaGetSupportedFormats;
using Moq;
using FluentAssertions;
using Xunit;

namespace Dianoga.Tests.NextGenFormats
{
public class CheckSupportTests
{
[Fact]
public void ShouldFindFormatSupportInAccepts_WhenItIsPresent()
{
//Arrange
var args = new SupportedFormatsArgs()
{
Input = "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
Prefix = "image/"
};

var CheckSupport = new CheckSupport()
{
Extension = "webp"
};

//Act
CheckSupport.Process(args);

//Assert
args.Extensions.Should().HaveCount(1);
args.Extensions.Should().Contain("webp");
}

[Fact]
public void ShouldNotFindFormatSupportInAccepts_WhenItIsAbsent()
{
//Arrange
var args = new SupportedFormatsArgs()
{
Input = "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
Prefix = "image/"
};

var CheckSupport = new CheckSupport()
{
Extension = "jxl"
};

//Act
CheckSupport.Process(args);

//Assert
args.Extensions.Should().HaveCount(0);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
using System.Diagnostics;
using System.IO;
using Dianoga.Optimizers;
using Dianoga.Optimizers.Pipelines.DianogaAvif;
using FluentAssertions;
using FluentAssertions.Common;
using Xunit;
using Xunit.Abstractions;

namespace Dianoga.Tests.Optimizers.Pipelines.DianogaAvif
{
public class AvifOptimizerTests
{
ITestOutputHelper output;
public AvifOptimizerTests(ITestOutputHelper output)
{
this.output = output;
}

[Fact]
public void ShouldReturnOriginalStreamWhenOptimizedImageSizeIsGreater()
{
Test(@"TestImages\small.jpg",
@"..\..\..\..\Dianoga\Dianoga Tools\avif\avifenc.exe",
"-s 10 -l", out var args, out var startingSize);
args.Stream.Length.Should().Be(startingSize).And.BeGreaterThan(0);
args.IsOptimized.Should().BeFalse();
}

[Fact]
public void ShouldSquishLosslessSmallPng()
{
Test(@"TestImages\small.png",
@"..\..\..\..\Dianoga\Dianoga Tools\avif\avifenc.exe",
"-s 10 -l", out var args, out var startingSize);
args.Stream.Length.Should().BeLessThan(startingSize).And.BeGreaterThan(0);
args.IsOptimized.Should().BeTrue();
}

[Fact]
public void ShouldSquishLosslessLargePngButBeTooBig()
{
Test(@"TestImages\large.png",
@"..\..\..\..\Dianoga\Dianoga Tools\avif\avifenc.exe",
"-s 10 -l", out var args, out var startingSize);
args.Stream.Length.Should().Be(startingSize).And.BeGreaterThan(0);
args.IsOptimized.Should().BeFalse();
}

[Fact]
public void ShouldSquishLossySmallJpegDefaults()
{
Test(@"TestImages\small.jpg",
@"..\..\..\..\Dianoga\Dianoga Tools\avif\avifenc.exe",
"-s 10", out var args, out var startingSize);
args.Stream.Length.Should().BeLessThan(startingSize).And.BeGreaterThan(0);
args.IsOptimized.Should().BeTrue();
}

[Fact]
public void ShouldSquishLossyLargeJpegDefaults()
{
Test(@"TestImages\large.jpg",
@"..\..\..\..\Dianoga\Dianoga Tools\avif\avifenc.exe",
"-s 10", out var args, out var startingSize);
args.Stream.Length.Should().BeLessThan(startingSize).And.BeGreaterThan(0);
args.IsOptimized.Should().BeTrue();
}

[Fact]
public void ShouldNotSquishCorruptedJpegLossy()
{
Test(@"TestImages\corrupted.jpg",
@"..\..\..\..\Dianoga\Dianoga Tools\avif\avifenc.exe",
"-s 10", out var args, out var startingSize);
args.Stream.Length.Should().IsSameOrEqualTo(startingSize);
args.IsOptimized.Should().BeFalse();
}

[Fact]
public void ShouldSquishLossySmallPngDefaults()
{
Test(@"TestImages\small.png",
@"..\..\..\..\Dianoga\Dianoga Tools\avif\avifenc.exe",
"-s 10", out var args, out var startingSize);
args.Stream.Length.Should().BeLessThan(startingSize).And.BeGreaterThan(0);
args.IsOptimized.Should().BeTrue();
}

[Fact]
public void ShouldSquishLossyLargePngDefaults()
{
Test(@"TestImages\large.png",
@"..\..\..\..\Dianoga\Dianoga Tools\avif\avifenc.exe",
"-s 10", out var args, out var startingSize);
args.Stream.Length.Should().BeLessThan(startingSize).And.BeGreaterThan(0);
args.IsOptimized.Should().BeTrue();
}

private void Test(string imagePath, string exePath, string exeArgs, out OptimizerArgs argsOut, out long startingSize)
{
var inputStream = new MemoryStream();

using (var testJpeg = File.OpenRead(imagePath))
{
testJpeg.CopyTo(inputStream);
}

var sut = new AvifOptimizer();
sut.ExePath = exePath;
sut.AdditionalToolArguments = exeArgs;

var opts = new Sitecore.Resources.Media.MediaOptions();
opts.CustomOptions["extension"] = "avif";
var args = new OptimizerArgs(inputStream, opts, imagePath);

startingSize = args.Stream.Length;

var stopwatch = new Stopwatch();
stopwatch.Start();
sut.Process(args);
stopwatch.Stop();
output.WriteLine($"Time: {stopwatch.ElapsedMilliseconds}ms");

argsOut = args;
}

}
}
Loading

0 comments on commit 9776f9a

Please sign in to comment.