This draft PR intended to add functionality for gradual migration from Newtonsoft.Json to System.Text.Json. (Related issue #9727)

1. chore: refactoring JsonUtility code to support newtonsoftjson/systemtxtjson switching based on type (fda9353)

• Refactor JsonUtility related code to support switching NewtonsoftJson/SystemTextJson by type.

2. chore: add json/yaml serialization test infrastructure (92a47a6)

• Add test infrastructure code to validate serialization/deserialization results.
• Validate JSON serialization results has no diffs between Newtonsoft.Json/SystemTextJson.
• Validate JSON serialize/deserialize roundtrip result object comparison between Newtonsoft.Json/SystemTextJson.
• Validate YAML related roundtrip results. (Object->YAML->Object and Object->JSON->Object)

3. chore: switch XRefMap serialization logics to use System.Text.Json (643c89b)

• Switch XRefMap serialization/deserialization to use JsonUtility and System.Text.Json.
• Add XRefMap serialization/deserialization tests.

4. chore: add json roundtrip test with JsonUtility (2fa823a)

• Add JSON serialization/deserialization round trip tests with JsonUtility

5. chore: Add MarkdigExtensionSetting json roundtrip tests (7b3dd78)

• Add System.Text.Json based serialization/deserialization for MarkdigExtensionSetting.

6. chore: add ManagedReference YAML roundtrip tests (9492ef8)

• Add System.Text.Json based serialization/deserialization for ManagedReference.PageViewModel
  • Modify CompositeDictionary related property to add private init and JsonInclude attribute
    (It's required because System.Text.Json can't deserialize getter-only property)

7. chore: add UniversalReference YAML roundtrip tests (12c7394)

• Add System.Text.Json based serialization/deserialization for UniversalReference.PageViewModel

8. chore: add ApiPage YAML roundtrip tests (448c933)

• Add Yaml serialization/deserialization tests for ApiPage

9. chore: add FilterConfig YAML roundtrip tests (a9fde45)

• Add Yaml serialization/deserialization tests for FilterConfig
• Add ‘[JsonIgnore]‘ attribute to Rule getter-only property.

10. chore: Add Manifest model json roundtrip tests (9c13117)

• Add JSON serialization/deserialization tests for Manifest
• Add empty default constructor and init properties to support System.Text.Json deserialization.

11. chore: change JavascriptEncoder settings and add related tests (9466b91)

• Enable UnsafeRelaxedJsonEscaping JSON Serialization Encoder. (By default. all non-ACSII chars are escaped)
• Add JSON encoder related tests.

12. chore: add TOC model YAML roundtrip tests (da94915)

• Add JSON serialization/deserialization tests for TOC
• Add CustomEqualityEquivalencyStep that is used by BeEquivalent comparison that comparing between deserialized models(Newtonsoft/SysteTextJson/YamlDotNet)
• Modify YamlSerializationTest.cs code that test s CustomEqualityEquivalencyStep behaviors

13. chore: add dummy property name for deserialize metadata key data (d101fdc)

• Add dummy property name (__metadata__) to Metadata property that using JsonIgnore attribute.
  Without this setting, SystemTextJson skip deserialization of items that have metadata key.
  (See: https://github.com/dotnet/runtime/issues/68895)

14. chore: add FileMapping/FileItems converters and JSON roundtrip tests (7b06344)

• Add JSON serialization/deserialization tests for FileMapping/FileItems

15. chore: disable MarkdigExtensionSettingConverter JSON formatting (8eaed9f)

• Disable JSON formatting for MarkdigExtensionSettingConverter serialization

16. chore: add ListWithStringFallback converters and JSON roundtrip test (948052a)

• Add JSON serialization/deserialization tests for ListWithStringFallback

17. chore: add FileMetadataPairs converters and JSON roundtrip test (6eb819c)

• Add JSON serialization/deserialization tests for FileMetadataPairs
• Modify test assertion logics to use FluentAssertion's EquivalentTo
  (NewtonsoftJson deserialize integer to long. but SystemTextJson implementation try to deserialize as int)

18. chore: add MergeJsonConfig converters and JSON roundtrip test (b8e4d85)

19. chore: add BuildJsonConfig JSON roundtrip test (80d10a7)

20. chore: add MetadataJsonConfig JSON roundtrip test (c62a025)

21. chore: add DocfxConfig JSON roundtrip test(28c55aa)

22. chore: Switch to use SystemTextJson by default and fix related tests (850625c)

23. chore: modify converter to throw JsonException if unexpected token found (f89752a)

• Modify JsonConverter to throw JsonException.

Future tasks (that is not handled in this PR)

• Switch RestApiRootItemViewModel serialization/deserialization to use System.Text.Json
  • RestApi related code using NewtonsoftJson heavily. So currently it's leaves as is.
• Modify remaining code that use JToken/JObject
  • HtmlCodeSnippetRenderer using JSON Path. System.Text.Json is not support this feature.
  • Remove NewtonsoftJson library dependencies.

Known compatibility issues

1. string to bool/numeric conversion is not supported by default (e.g. "prop": "true")
2. JsonUtility's behavior differences.
   2.1. numeric value is deserialized as appropriate types (byte/short/long/ulong/float/double)
   (NewtonsoftJson deserialize numeric value to fixed type (long or double) )
   2.2. Dictionary<string, object> data is deserialized to object dictionary. And child array element is deserialized to List.
   (NewtonsoftJson deserialize these data to JToken)