.NET 11 中 JSON Lines 支持
Intro
JSON Lines（JSONL）是一种基于文本的数据格式，其中每一行代表一个有效的 JSON 对象。这种结构允许逐行轻松解析和处理数据，使其适合高效处理大型数据集和流式内容。elasticsearch 批量上传数据时也在使用 JSON lines，除此之外现在很多 AI 工具也在使用 JSON Lines 来记录操作历史等包括 ClaudeCode, Codex 等， A2UI (Agent to UI) 就采用了 JSONL，这是一个用于代理驱动接口的声明式 UI 协议。
在 .NET 11 Preview 5 中实现了 JSON lines 的序列化支持
New API
新增的 API 如下：
namespace System.Text.Json;
public static partial class JsonSerializer
{
// Stream overloads
public static Task SerializeAsyncEnumerable<TValue>(
Stream utf8Json,
IAsyncEnumerable<TValue> value,
bool topLevelValues = false,
JsonSerializerOptions? options = null,
CancellationToken cancellationToken = default);
public static Task SerializeAsyncEnumerable<TValue>(
Stream utf8Json,
IAsyncEnumerable<TValue> value,
JsonTypeInfo<TValue> jsonTypeInfo,
bool topLevelValues = false,
CancellationToken cancellationToken = default);
// PipeWriter overloads
public static Task SerializeAsyncEnumerable<TValue>(
PipeWriter utf8Json,
IAsyncEnumerable<TValue> value,
bool topLevelValues = false,
JsonSerializerOptions? options = null,
CancellationToken cancellationToken = default);
public static Task SerializeAsyncEnumerable<TValue>(
PipeWriter utf8Json,
IAsyncEnumerable<TValue> value,
JsonTypeInfo<TValue> jsonTypeInfo,
bool topLevelValues = false,
CancellationToken cancellationToken = default);
}对应的反序列化 API 已经在 .NET 9 里已经支持基于 Stream 的反序列化
namespace System.Text.Json
{
public partial static class JsonSerializer
{
public static IAsyncEnumerable<T> DeserializeAsyncEnumerable<T>(Stream utf8Json, bool topLevelValues, JsonSerializerOptions options = null, CancellationToken cancellationToken = default);
public static IAsyncEnumerable<T> DeserializeAsyncEnumerable<T>(Stream utf8Json, JsonTypeInfo<T> jsonTypeInfo, bool topLevelValues, CancellationToken cancellationToken = default);
}
}并在 .NET 10 里支持了基于 Pipe 的反序列化
namespace System.Text.Json;
public partial static class JsonSerializer
{
+ public static IAsyncEnumerable<TValue?> DeserializeAsyncEnumerable<TValue>(
+ PipeReader utf8Json,
+ JsonTypeInfo<TValue> jsonTypeInfo,
+ bool topLevelValues,
+ CancellationToken cancellationToken = default);
+ [RequiresUnreferencedCode(SerializationUnreferencedCodeMessage)]
+ [RequiresDynamicCode(SerializationRequiresDynamicCodeMessage)]
+ public static IAsyncEnumerable<TValue?> DeserializeAsyncEnumerable<TValue>(
+ PipeReader utf8Json,
+ bool topLevelValues,
+ JsonSerializerOptions? options = null,
+ CancellationToken cancellationToken = default);
}Sample
首先我们准备一个方法返回一个 IAsyncEnumerable 对象
private static async IAsyncEnumerable<LogEntry> GetLogsAsync()
{
yield return new("xxx begin");
await Task.Delay(2000);
yield return new("xxx processing");
await Task.Delay(2000);
yield return new("xxx end");
}
internal record LogEntry(string Message)
{
public DateTimeOffset Time { get; init; } = DateTimeOffset.Now;
}使用示例如下：
// Serialize an IAsyncEnumerable<T> to a stream as JSON lines
var logs = GetLogsAsync();
using var ms = new MemoryStream();
await JsonSerializer.SerializeAsyncEnumerable(ms, logs, true);
ms.Seek(0, SeekOrigin.Begin);
using var reader = new StreamReader(ms);
var output = await reader.ReadToEndAsync();
Console.WriteLine($"[{DateTimeOffset.Now}]: {output}");输出结果如下：
{"Message":"xxx begin","Time":"2026-06-14T20:19:42.2798459+08:00"}
{"Message":"xxx processing","Time":"2026-06-14T20:19:44.2942854+08:00"}
{"Message":"xxx end","Time":"2026-06-14T20:19:46.2952025+08:00"}上面时使用 Stream 的一个示例，下面是一个基于 Pipe 的示例
// Serialize an IAsyncEnumerable<T> to a PipeWriter as JSON lines
var logs = GetLogsAsync();
var pipe = new Pipe();
var writerTask = JsonSerializer.SerializeAsyncEnumerable(pipe.Writer, logs, true)
.ContinueWith(_ => pipe.Writer.Complete());
var readerTask = ReadFromPipeAsync(pipe.Reader);
await Task.WhenAll(writerTask, readerTask);
async Task ReadFromPipeAsync(PipeReader reader)
{
while (true)
{
// Wait for data from the writer
var result = await reader.ReadAsync();
var buffer = result.Buffer;
// Process whatever data is available
if (!buffer.IsEmpty)
{
// Note: In real networking code, you must handle incomplete messages here.
// For this example, we just read everything available.
var text = Encoding.UTF8.GetString(buffer);
Console.Write($"[{DateTimeOffset.Now}]: {text}");
}
// Tell the reader how much data we consumed so it can free the memory
reader.AdvanceTo(buffer.End);
// Stop if the writer called Complete()
if (result.IsCompleted)
{
await reader.CompleteAsync();
break;
}
}
}输出结果如下：
目前 ASP.NET Core 还没有较好的 JSON lines 支持，我们可以基于新的 API 来实现一个新的 JsonLinesResult
internal sealed class JsonLinesResult<TValue>(IAsyncEnumerable<TValue> result) : IResult
{
private const string ContentType = "application/jsonl;charset=utf-8";
private readonly IAsyncEnumerable<TValue> _result = result;
public async Task ExecuteAsync(HttpContext httpContext)
{
httpContext.Response.Headers.ContentType = ContentType;
var jsonOptions = httpContext.RequestServices
.GetRequiredService<IOptions<JsonOptions>>().Value;
await JsonSerializer.SerializeAsyncEnumerable(httpContext.Response.BodyWriter, _result,
true, jsonOptions.SerializerOptions, httpContext.RequestAborted);
}
}可以再添加一个 Results 的扩展方法来和其他的 Result 保持一致的使用方式如 Results.Ok 可以使用 C# 14 的 extension 实现
public static class JsonLinesExtension
{
extension(Results)
{
public static IResult JsonLines<TValue>(IAsyncEnumerable<TValue> result)
{
return new JsonLinesResult<TValue>(result);
}
}
}使用示例如下：
var builder = WebApplication.CreateSlimBuilder();
var app = builder.Build();
app.MapGet("/logs", (HttpContext context) =>
{
var logs = GetLogsAsync();
return Results.JsonLines(logs);
});
await app.RunAsync("http://localhost:5100");调用 API 输出如下：
using var httpClient = new HttpClient();
using var response = await httpClient.GetAsync(
"http://localhost:5100/logs",
HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();
var responseStream = await response.Content.ReadAsStreamAsync();
var logEntries = JsonSerializer.DeserializeAsyncEnumerable<LogEntry>(
responseStream, true, JsonSerializerOptions.Web);
await foreach (var logEntry in logEntries)
{
Console.WriteLine($"[{DateTimeOffset.Now}]: {JsonSerializer.Serialize(logEntry)}");
}
More
JSON lines 支持主要是针对于流式输出的，目前 ASP.NET Core 的支持还不完善，希望后续能够能够完善，感觉 JSON lines 比之前的实现返回 json array 的方式更好，没有 [] 每一行都是一个单独的数据对象，格式更加清晰统一，如果有大量的数据需要流式处理不妨试一试看看。
