Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add System.Text.Json serialization/deserialization supports #10217

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

filzrev
Copy link
Contributor

@filzrev filzrev commented Sep 20, 2024

This draft PR intended to add functionality for gradual migration from Newtonsoft.Json to System.Text.Json. (Related issue #9727)

1. chore: refactoring JsonUtility code to support newtonsoftjson/systemtxtjson switching based on type (fda9353)

  • Refactor JsonUtility related code to support switching NewtonsoftJson/SystemTextJson by type.

2. chore: add json/yaml serialization test infrastructure (92a47a6)

  • Add test infrastructure code to validate serialization/deserialization results.
  • Validate JSON serialization results has no diffs between Newtonsoft.Json/SystemTextJson.
  • Validate JSON serialize/deserialize roundtrip result object comparison between Newtonsoft.Json/SystemTextJson.
  • Validate YAML related roundtrip results. (Object->YAML->Object and Object->JSON->Object)

3. chore: switch XRefMap serialization logics to use System.Text.Json (643c89b)

  • Switch XRefMap serialization/deserialization to use JsonUtility and System.Text.Json.
  • Add XRefMap serialization/deserialization tests.

4. chore: add json roundtrip test with JsonUtility (2fa823a)

  • Add JSON serialization/deserialization round trip tests with JsonUtility

5. chore: Add MarkdigExtensionSetting json roundtrip tests (7b3dd78)

  • Add System.Text.Json based serialization/deserialization for MarkdigExtensionSetting.

6. chore: add ManagedReference YAML roundtrip tests (9492ef8)

  • Add System.Text.Json based serialization/deserialization for ManagedReference.PageViewModel
    • Modify CompositeDictionary related property to add private init and JsonInclude attribute
      (It's required because System.Text.Json can't deserialize getter-only property)

7. chore: add UniversalReference YAML roundtrip tests (12c7394)

  • Add System.Text.Json based serialization/deserialization for UniversalReference.PageViewModel

8. chore: add ApiPage YAML roundtrip tests (448c933)

  • Add Yaml serialization/deserialization tests for ApiPage

9. chore: add FilterConfig YAML roundtrip tests (a9fde45)

  • Add Yaml serialization/deserialization tests for FilterConfig
  • Add ‘[JsonIgnore]‘ attribute to Rule getter-only property.

10. chore: Add Manifest model json roundtrip tests (9c13117)

  • Add JSON serialization/deserialization tests for Manifest
  • Add empty default constructor and init properties to support System.Text.Json deserialization.

11. chore: change JavascriptEncoder settings and add related tests (9466b91)

  • Enable UnsafeRelaxedJsonEscaping JSON Serialization Encoder. (By default. all non-ACSII chars are escaped)
  • Add JSON encoder related tests.

12. chore: add TOC model YAML roundtrip tests (da94915)

  • Add JSON serialization/deserialization tests for TOC
  • Add CustomEqualityEquivalencyStep that is used by BeEquivalent comparison that comparing between deserialized models(Newtonsoft/SysteTextJson/YamlDotNet)
  • Modify YamlSerializationTest.cs code that test s CustomEqualityEquivalencyStep behaviors

13. chore: add dummy property name for deserialize metadata key data (d101fdc)

  • Add dummy property name (__metadata__) to Metadata property that using JsonIgnore attribute.
    Without this setting, SystemTextJson skip deserialization of items that have metadata key.
    (See: https://github.com/dotnet/runtime/issues/68895)

14. chore: add FileMapping/FileItems converters and JSON roundtrip tests (7b06344)

  • Add JSON serialization/deserialization tests for FileMapping/FileItems

15. chore: disable MarkdigExtensionSettingConverter JSON formatting (8eaed9f)

  • Disable JSON formatting for MarkdigExtensionSettingConverter serialization

16. chore: add ListWithStringFallback converters and JSON roundtrip test (948052a)

  • Add JSON serialization/deserialization tests for ListWithStringFallback

17. chore: add FileMetadataPairs converters and JSON roundtrip test (6eb819c)

  • Add JSON serialization/deserialization tests for FileMetadataPairs
  • Modify test assertion logics to use FluentAssertion's EquivalentTo
    (NewtonsoftJson deserialize integer to long. but SystemTextJson implementation try to deserialize as int)

18. chore: add MergeJsonConfig converters and JSON roundtrip test (b8e4d85)

19. chore: add BuildJsonConfig JSON roundtrip test (80d10a7)

20. chore: add MetadataJsonConfig JSON roundtrip test (c62a025)

21. chore: add DocfxConfig JSON roundtrip test(28c55aa)

22. chore: Switch to use SystemTextJson by default and fix related tests (850625c)

23. chore: modify converter to throw JsonException if unexpected token found (f89752a)

  • Modify JsonConverter to throw JsonException.

Future tasks (that is not handled in this PR)

  • Switch RestApiRootItemViewModel serialization/deserialization to use System.Text.Json
    • RestApi related code using NewtonsoftJson heavily. So currently it's leaves as is.
  • Modify remaining code that use JToken/JObject
    • HtmlCodeSnippetRenderer using JSON Path. System.Text.Json is not support this feature.
    • Remove NewtonsoftJson library dependencies.

Known compatibility issues

  1. string to bool/numeric conversion is not supported by default (e.g. "prop": "true")
  2. JsonUtility's behavior differences.
    2.1. numeric value is deserialized as appropriate types (byte/short/long/ulong/float/double)
    (NewtonsoftJson deserialize numeric value to fixed type (long or double) )
    2.2. Dictionary<string, object> data is deserialized to object dictionary. And child array element is deserialized to List.
    (NewtonsoftJson deserialize these data to JToken)

@filzrev filzrev force-pushed the feature/add-system-text-json-supports branch from 165bb8a to cf7bf6c Compare September 20, 2024 08:22
@filzrev filzrev force-pushed the feature/add-system-text-json-supports branch from 46ab297 to cab3a99 Compare September 26, 2024 22:45
@filzrev filzrev force-pushed the feature/add-system-text-json-supports branch from cab3a99 to e137e2d Compare September 26, 2024 22:48
@filzrev filzrev force-pushed the feature/add-system-text-json-supports branch from e137e2d to fed5d64 Compare September 26, 2024 22:48
@filzrev filzrev force-pushed the feature/add-system-text-json-supports branch from fed5d64 to e7c8a77 Compare September 27, 2024 12:29
@filzrev filzrev force-pushed the feature/add-system-text-json-supports branch from e7c8a77 to 750bb7d Compare September 27, 2024 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant