Applications that exchange structured binary data frequently need to read files, interpret network communications, or interact with external devices. Incremental reading capability is introduced in System by NET 11 RC 2. When the input arrives in several pieces, putting the complete payload into a single buffer before parsing it might add needless memory strain and delay processing.Formats.Cbor. A CBOR document that comes in chunks can be processed by the modified CborReader without the need to first copy the complete document into contiguous memory.
The new functionality is useful for developers building network protocols, data ingestion services, and applications that handle structured binary messages. It also introduces an important responsibility: the application must recognize when more input is needed and provide the next chunk without losing unread data.
This article explains how incremental CBOR reading works, how to use the new APIs, and what to consider when implementing a reliable parser.
What Is CBOR?
CBOR stands for Concise Binary Object Representation. It is a binary data format designed to represent structured values such as integers, strings, arrays, maps, Boolean values, and other data types.
Unlike JSON, which represents values using readable text, CBOR uses a compact binary encoding. This can make it suitable for protocols where message size and efficient data exchange matter.
For example, a device might send a measurement containing a sensor identifier and a numeric reading. A CBOR array could represent those values as two elements:
using System.Formats.Cbor;
var writer = new CborWriter();
writer.WriteStartArray(2);
writer.WriteTextString("sensor-17");
writer.WriteInt32(28);
writer.WriteEndArray();
byte[] payload = writer.Encode();The example creates a two-element CBOR array containing a sensor identifier and a reading. The resulting byte array can be transmitted or stored.
Before .NET 11 RC 2, developers using CborReader generally needed to provide the encoded data through a complete input buffer. Incremental reading changes how the reader can consume input when a document arrives in separate chunks.
What Is New in .NET 11 RC 2?
The updated CborReader supports input that is delivered incrementally.
Three elements are particularly important:
isFinalBlock: Indicates whether the supplied input represents the final block of the CBOR document.NeedsMoreData: Indicates that the reader cannot continue because additional input is required.SlideData: Supplies another input block while allowing the reader to continue processing the document.
Incremental reading requires CborConformanceMode.Lax. Other conformance modes reject a non-final block because they do not support this incremental input behavior.
The main advantage is that the parser no longer needs to wait for the entire encoded document before it can begin reading its contents.
However, incremental reading does not automatically make every application memory-efficient. The application must still manage incoming buffers, preserve unread bytes, and avoid accumulating the complete payload unnecessarily.
How Incremental CBOR Reading Works
Consider a CBOR array containing two integers: 10 and 20.
Its encoded representation can be split across two input chunks. The first chunk contains the array header and the first integer, while the second chunk contains the remaining integer.
The reader can process the first value immediately and request more input when it reaches the incomplete portion of the document.
Step 1: Create a Reader With an Incomplete Input Block
The first chunk is supplied with isFinalBlock set to false.
This tells the reader that the current buffer may not contain the entire document.
Step 2: Read the Available Values
The reader processes the array header and reads the first integer.
When it reaches the second integer, the reader determines whether sufficient input is available to continue.
Step 3: Check Whether More Data Is Required
Use PeekState() to inspect the next reader state without consuming the next value.
If the state is CborReaderState.NeedsMoreData, the application must provide another chunk before continuing.
Step 4: Supply the Next Chunk
Call SlideData with the additional bytes. Set isFinalBlock to true when the supplied block completes the document.
The reader can then continue parsing from its current position.
Practical C# Example
The following example demonstrates the incremental reading pattern introduced in .NET 11 RC 2.
using System;
using System.Formats.Cbor;
ReadOnlyMemory<byte> firstChunk =
new byte[] { 0x82, 0x0A };
ReadOnlyMemory<byte> nextChunk =
new byte[] { 0x14 };
var reader = new CborReader(
firstChunk,
new CborReaderOptions
{
ConformanceMode = CborConformanceMode.Lax
},
isFinalBlock: false);
reader.ReadStartArray();
int first = reader.ReadInt32();
if (reader.PeekState() == CborReaderState.NeedsMoreData)
{
if (reader.BytesRemaining == 0)
{
reader.SlideData(
nextChunk,
isFinalBlock: true);
}
else
{
throw new InvalidOperationException(
"Additional buffer management is required.");
}
}
int second = reader.ReadInt32();
reader.ReadEndArray();
Console.WriteLine(first);
Console.WriteLine(second);The expected output is:
10
20The first byte, 0x82, represents an array containing two elements. The following byte, 0x0A, represents the integer 10. The final byte, 0x14, represents the integer 20.
The reader processes the first integer before the second chunk is supplied.
The example also checks BytesRemaining. This matters because a production implementation cannot assume that every incomplete read occurs exactly at the end of its current buffer. If unread bytes remain, the application must preserve them when managing the next input block.
This sample illustrates the API pattern with a deliberately simple message. A real network parser needs a more complete buffer-management strategy.
Handling Network Streams in Production
Incremental CBOR reading is especially relevant when a message arrives through a network connection.
A network read does not necessarily correspond to one complete CBOR value. A single message may arrive in several reads, and one read may contain bytes belonging to more than one logical value.
For that reason, a production parser should separate transport handling from CBOR decoding.
Manage Incomplete Input
Keep track of the bytes that have not yet been consumed. When additional input arrives, combine or slide the remaining data into the next reader buffer according to the API’s buffer-management requirements.
Do not discard unread bytes simply because another network read has completed.
Handle End-of-Stream Correctly
If the transport ends before the CBOR document is complete, the application should report an incomplete message rather than treating it as a valid document.
Set isFinalBlock to true only when the application knows that no more bytes belong to the current document.
Validate the Parsed Data
Successful CBOR decoding does not mean that the input satisfies your application’s business rules.
For example, a decoded sensor reading might still be outside the permitted range. Validate required fields, numeric limits, message structure, and any protocol-specific constraints before acting on the data.
Apply Input Limits
Incremental processing does not eliminate the possibility of oversized messages or deeply nested data.
Set appropriate limits on message size, nesting depth, and processing time. Reject malformed input early and avoid allocating large buffers based solely on untrusted length information.
Incremental Reading Compared With Complete-Buffer Reading
| Consideration | Complete-buffer reading | Incremental reading |
|---|---|---|
| Input requirement | Complete encoded document | Input can arrive in chunks |
| Initial parsing | Starts after the complete buffer is available | Can start with an incomplete block |
| Buffer management | Generally simpler | Requires careful handling of unread data |
| Memory usage | May require the entire document in memory | Can reduce the need for a complete contiguous copy |
| Implementation complexity | Lower | Higher |
| Suitable scenarios | Small messages and files | Streamed or chunked input |
Incremental reading is not automatically faster for every workload. For small messages, a complete-buffer approach may remain simpler and perfectly adequate.
The new APIs are most valuable when input naturally arrives in chunks or when avoiding an additional contiguous copy matters to the application.
Advantages and Disadvantages
Advantages
Earlier processing: Applications can begin decoding values before the entire document arrives.
Reduced buffering requirements: The parser does not need to receive the complete document in one contiguous memory block.
Better support for streamed data: The API fits workflows involving network messages and other chunked input sources.
Explicit incomplete-input handling: NeedsMoreData provides a clear indication that the reader requires additional bytes.
Disadvantages
More complicated buffer management: Applications must preserve unread bytes and provide subsequent blocks correctly.
Conformance restrictions: Incremental reading requires lax CBOR conformance mode.
No automatic transport integration: Developers still need to read from streams or network connections and manage the incoming data.
Validation remains necessary: The API decodes CBOR but does not replace application-level security and data validation.
Common Mistakes and Troubleshooting
The Reader Reports NeedsMoreData
This means the current input is insufficient to continue decoding. Obtain the next chunk and supply it through SlideData.
Do not attempt to read the next value until the required input is available.
The Parser Loses Data Between Chunks
Review the buffer-management logic. Any unread bytes must remain available when the next chunk is supplied.
Test with documents split at different byte positions, including boundaries inside strings, integers, and nested structures.
Incremental Reading Fails With a Different Conformance Mode
Use CborConformanceMode.Lax for incremental input. Other conformance modes reject non-final blocks.
If your application requires stricter conformance, evaluate whether you can buffer the complete document and validate it using the required mode.
The Application Still Uses Too Much Memory
Check whether the application accumulates every received chunk in a MemoryStream or another growing buffer before parsing.
Incremental decoding only reduces unnecessary buffering when the surrounding application also manages memory appropriately.
Summary
.NET 11 RC 2 adds incremental reading support to CborReader, allowing developers to process CBOR documents delivered in chunks without first copying the entire document into contiguous memory.
The key elements are the isFinalBlock constructor argument, CborReaderState.NeedsMoreData, and SlideData. Together, they allow a parser to read available values, request additional input, and continue decoding as more bytes arrive.
For production applications, the most important considerations are correct buffer management, proper end-of-stream handling, input limits, and application-level validation. Incremental reading is a useful option for streamed binary data, but complete-buffer parsing may remain the better choice for small and self-contained messages.
Best and Most Recommended ASP.NET Core 11.0 Hosting
Fortunately, there are a number of dependable and recommended web hosts available that can help you gain control of your website’s performance and improve your ASP.NET Core 11 web ranking. HostForLIFE.eu is highly recommended. In Europe, HostForLIFE.eu is the most popular option for first-time web hosts searching for an affordable plan. Their standard price begins at only €3.49 per month. Customers are permitted to choose quarterly and annual plans based on their preferences. HostForLIFE.eu guarantees “No Hidden Fees” and an industry-leading ’30 Days Cash Back’ policy. Customers who terminate their service within the first thirty days are eligible for a full refund.
By providing reseller hosting accounts, HostForLIFE.eu also gives its consumers the chance to generate income. You can purchase their reseller hosting account, host an unlimited number of websites on it, and even sell some of your hosting space to others. This is one of the most effective methods for making money online. They will take care of all your customers’ hosting needs, so you do not need to fret about hosting-related matters.
