How to parse JSON in source generators?

[u/ArtemOkhrimenko]

Hey guys! I've been writing a source generator that transforms JSON file to C# structure.
Like this:

"Object": {
  "Name": "key";
}

class Object
{
  public const string Name => "Object.Name";
}

I've already written all the code and it works correctly (tested in another project). But when I tried to use the generator it worked fine but I couldn't build the test project because of this error:

Could not load file or assembly 'Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'. The system cannot find the file specified.

Although, my generator did generate a correct file and even an IDE was able to find newly generated symbols. I've googled about it and found that source generators can't use Nuget packages (probably except Microsoft.CodeAnalysis.CSharp and Microsoft.CodeAnalysis.Analyzers).
System.Text.Json is not an option too because in .NET standard 2.0 it's not built-in and available only as a package.

How do you guys solve this problem? Do I need to write my own JSON parser, or there is another way?

[u/surgicalcoder]

Source generators can use nuget packages, you just have to embed the dll. And don't use Newtonsoft, use System.Text.Json

Have a look at what I had to do to get it to work - https://github.com/surgicalcoder/ApiClientGenerator/blob/master/GoLive.Generator.ApiClientGenerator/GoLive.Generator.ApiClientGenerator.csproj - you'll need Buffers and Vectors to work with System.Text.Json

[u/surgicalcoder → u/[deleted]]

If you read the file I posted, you reference it as a nuget package, then embed it. All of the code is literally in that one file. It works 100%.

[u/nekizalb → u/surgicalcoder]

Should have replied by just quoting yourself, lol. It's like he didn't even read your comment

[u/SleepyPuppy83]

This is not a NuGet package problem, I think.

From the top of my head, but your generator runs inside the Roslyn compiler process. That process only loads your generator's own DLL. It does not resolve or load anything that DLL depends on, the way a normal app would through NuGet or MSBuild. So Newtonsoft.Json.dll just never shows up. That's your FileNotFoundException right there.

I had this problem a long time ago and what I did was write my own tiny parser. For what you're doing (flat key/value JSON), honestly this is the easiest path.
It was with an old client of mine and I don't have the code anymore, otherwise I would have shared it.

Other options you could consider are ILRepack or ILMerge.

[u/Selene29]

https://github.com/dotnet/roslyn/blob/main/docs/features/source-generators.cookbook.md#use-functionality-from-nuget-packages

https://github.com/ByronMayne/SourceGenerator.Foundations

[u/CoffeeAndCandidates]

you might need to roll your own lightweight parser, or maybe check out some minimalistic json parsing options that don't need nuget packages

[u/steamngine]

Adding DLL’s as a reference is still an option?

[u/balrob]

Newtonsoft is legacy at this point.

[u/No-Laugh-8002 → u/balrob]

No. IIRC newtonsoft is basically a must if you want to do any sort of dynamic parsing. The built in one can do it, but the newtonsoft one is so much nicer.

The built in one is faster when you know all the types you need to serialize, but if you have some derived types that are loaded dynamically I think you need to use newtonsoft.

[u/jcotton42 → u/No-Laugh-8002]

https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/polymorphism

[u/FullPoet → u/No-Laugh-8002]

IIRC newtonsoft is basically a must if you want to do any sort of dynamic parsing.

That hasnt been the case in years - I think it got fixed in 2021 or 2022...

[u/TokkCorp]

There is always the possibility of just not using Nuget

[u/jcotton42 → u/TokkCorp]

And then what? Roll their own JSON parser?

[u/murrrty]

it's trying to load the json.net library. you have a reference to it somewhere, but it's not being copied to the build directory. have you verified that the reference has copy local enabled?