[2.7] The journey to fix a circular reference caused memory leak

[bohanyang]

The Problem

After upgrading my project from 2.6 to 2.7, PHP begins crashing with Fatal error: Allowed memory size of 134217728 bytes exhausted.
On the console, which don't have a memory limit, the PHP process ate all of the memory and 100% the CPU core.

Finding the cause

I turned on Xdebug and made a breakpoint in the Symfony\Component\HttpKernel\Kernel::handle() method, which should be called upon request. I've also set it to break on every PHP errors and exceptions in VS Code.

I issued a request with my browser and the process paused by an warning from a xdebug_get_function_stack() call:

Warning: Function must be enabled in php.ini by setting 'xdebug.mode' to 'develop'

I have no idea what this mean but when looking at the long long looping call stack, the cause seems to be found.

The circular reference: Data transformers are instantiated during AbstractItemNormalizer instantiation

I have a data transformer that requires NormalizerInterface and DenormalizerInterface in the constructor, that are autowired with Symfony\Component\Serializer\Serializer.

To instantiate a Serializer, which contains a chain of normalizers, ApiPlatform\Serializer\ItemNormalizer will be instantiated, which requires a chain of data transformers.

In 2.7, the following code is added to trigger deprecation for custom data transformers.

core/src/Serializer/AbstractItemNormalizer.php

Lines 119 to 126 in 693b03a

	// Just skip our data transformer to trigger a proper deprecation
	$customDataTransformers = array_filter(\is_array($dataTransformers) ? $dataTransformers : iterator_to_array($dataTransformers), function ($dataTransformer) {
	return !$dataTransformer instanceof MessengerDataTransformer;
	});
	if (\count($customDataTransformers)) {
	trigger_deprecation('api-platform/core', '2.7', 'The DataTransformer pattern is deprecated, use a Provider or a Processor and either use your input or return a new output there.');
	}

$dataTransformers is an iterator of data transformers, injected by Symfony DI's tagged iterator.

core/src/Symfony/Bundle/Resources/config/api.xml

Line 70 in 693b03a

<argument type="tagged" tag="api_platform.data_transformer" on-invalid="ignore" />

Previously the data transformers are instantiated only when the chain is being iterated (e.g. when actually performing (de)normalization), which happens after the root Serializer's instantiation.

However in 2.7, all data transformers will be instantiated upon the root Serializer's instantiation, causes infinite loop and memory leak.

Fix

I found that my custom data transformer only needs my custom (de)normalizer, rather than the whole normalizer chain.
After specify that in the service config (I'm using PHP format: config/services.php), it fixed!

// Hydrator is my custom (de)normalizer.

$services->set(ServiceInputTransformer::class)
        ->arg(NormalizerInterface::class, service(Hydrator::class))
        ->arg(DenormalizerInterface::class, service(Hydrator::class));

Hope my journey can help you if you met the same issue!

[RobinHoutevelts]

Thanks for the writeup!

[mikhail-marynich-omnilog]

Thanks !