Newtonsoft.Json → System.Text.Json Migration

[NikolaMilosavljevic]

Summary of changes

1. Core Shared Infrastructure (1 file)

• JExtensions.cs (src/Shared/JExtensions.cs) — Most significant change. This shared file (linked into multiple projects) was the central Newtonsoft.Json helper. Completely rewritten to use System.Text.Json.Nodes (JsonNode, JsonObject, JsonArray, JsonValue) instead of JObject/JToken/JArray. Key changes:
  • All extension methods now operate on JsonObject/JsonNode/JsonArray instead of Newtonsoft types
  • ParseJsonNode() / ParseJsonObject() helpers with JsonDocumentOptions configured for comment handling and trailing commas
  • ToJsonString() for serialization using JsonSerializer
  • String/int/bool/enum/array/object accessor methods (ToString(), ToInt32(), Get<T>(), etc.) rewritten for STJ APIs
  • JsonNodeComparer and JsonNodeEqualityComparer utility classes added

2. Package References (5 files)

• Directory.Packages.props — Removed Newtonsoft.Json package reference; added System.Text.Json and JsonSchema.Net (for JSON schema validation in tests).
• 4 csproj files — Replaced Newtonsoft.Json with System.Text.Json, conditioned with Condition="'$(TargetFrameworkIdentifier)' != '.NETCoreApp'" to avoid NU1510 package pruning warnings (System.Text.Json is inbox for .NET 10+, but still needed for net472/netstandard2.0):
  • src/Microsoft.TemplateEngine.Edge/Microsoft.TemplateEngine.Edge.csproj
  • src/Microsoft.TemplateEngine.Orchestrator.RunnableProjects/Microsoft.TemplateEngine.Orchestrator.RunnableProjects.csproj
  • src/Microsoft.TemplateEngine.Utils/Microsoft.TemplateEngine.Utils.csproj
  • src/Microsoft.TemplateSearch.Common/Microsoft.TemplateSearch.Common.csproj
• Microsoft.TemplateSearch.TemplateDiscovery.csproj — Removed Newtonsoft.Json reference.
• RunnableProjects.UnitTests.csproj — Replaced Newtonsoft.Json with JsonSchema.Net for schema validation tests.

3. Production Source Files (62 files)

Microsoft.TemplateEngine.Edge (6 files)

• All converted from JObject/JToken to JsonObject/JsonNode

Microsoft.TemplateEngine.Orchestrator.RunnableProjects (32 files)

Microsoft.TemplateSearch.Common (11 files)

Tools/TemplateDiscovery (12 files)

JsonEncodeValueFormFactory.cs

Added JavaScriptEncoder.UnsafeRelaxedJsonEscaping so " encodes as \" (matching Newtonsoft behavior) instead of STJ's default \u0022.

4. Test Files (24 files)

All test files updated to use System.Text.Json APIs instead of Newtonsoft:

5. Test Data Fixes (4 files)

Removed duplicate $schema properties from template.json files that Newtonsoft silently tolerated but STJ/JsonSchema.Net rejects:

• test_templates/TemplateConditionalProcessing/.template.config/template.json
• test_templates/Invalid/InvalidHostData/.template.config/template.json
• test_templates/SourceWithExcludeAndWithout/With/.template.config/template.json
• test_templates/SourceWithExcludeAndWithout/Without/.template.config/template.json

6. Behavioral Differences Addressed

Issue	Newtonsoft Behavior	STJ Behavior	Fix Applied
Single-quoted JSON	Allowed	Rejected	Fixed test data to use double quotes
Unquoted property names	Allowed	Rejected	Fixed test data to use quoted names
Boolean serialization	"True" / "False"	"true" / "false"	Updated test assertions
" encoding in JSON strings	\"	\u0022	Added UnsafeRelaxedJsonEscaping
Duplicate JSON keys	Last value wins	Throws ArgumentException	Removed duplicates from test data
JsonNode.ToString() format	Indented	Indented (but ToJsonString() is compact)	Used ToJsonString() in assertions

7. Validation

VMR build

[NikolaMilosavljevic]

2 integration tests are failing - working on fixes.

[NikolaMilosavljevic]

Fixed integration tests with cad05c0

Summary of fixes

1. JExtensions.ToInt32() — string-typed integer parsing

File: src/Shared/JExtensions.cs

ToInt32 wasn't handling string-typed values like "precedence": "100". Newtonsoft's JToken.Value<int>() auto-converted strings to ints; STJ's ToJsonString() returns "\"100\"" (with quotes), failing int.TryParse. Fixed by reusing the already-correct TryParseInt() helper.

This fixed the ValidateCommand_BasicTest snapshot mismatch (MV009 was falsely triggering for all templates).

2. ExportCommandTests.cs — trailing commas

File: test/Microsoft.TemplateEngine.Authoring.CLI.IntegrationTests/ExportCommandTests.cs

Added JsonDocumentOptions { AllowTrailingCommas = true, CommentHandling = JsonCommentHandling.Skip } to all JsonNode.Parse calls. Template.json files contain trailing commas that Newtonsoft tolerated but STJ rejects by default.

3. BlobStorageTemplateInfo.cs & LegacyBlobTemplateInfo.cs — NotImplementedException in properties

Files:

• src/Microsoft.TemplateSearch.Common/TemplateDiscoveryMetadata/BlobStorageTemplateInfo.cs
• tools/Microsoft.TemplateSearch.TemplateDiscovery/Results/LegacyBlobTemplateInfo.cs

STJ serializes through interface types (ICacheTag, ICacheParameter, ITemplateParameter), ignoring [JsonIgnore] on the concrete class. Newtonsoft used runtime types and respected [JsonIgnore]. Changed ~15 throwing properties to return safe defaults.

4. TemplateDiscoveryTests.CanReadAuthor — array vs scalar

File: test/Microsoft.TemplateSearch.TemplateDiscovery.IntegrationTests/TemplateDiscoveryTests.cs

V1 format serializes Owners (an IReadOnlyList<string>) as a JSON array. The migration incorrectly used .GetValue<string>() (scalar only); fixed to .AsArray().Select(...).Single() matching the original .Values().Single().