Improved Support for ref and unsafe in Async Methods and Iterators in C#

Back to: C# New Features Tutorials

Improved Support for ref and unsafe in C#

In this article, I will discuss Improved Support for ref and unsafe in Async Methods and Iterators in C# with Examples. Please read our previous articles discussing Partial Properties and Indexers in C# with Examples. In C# 13, the language has improved the ability to use ref local variables and unsafe code within async methods and iterators. This opens up new possibilities for developers working with memory-efficient data structures like System.Span<T> and System.ReadOnlySpan<T>, and enables more powerful, efficient, and flexible data processing techniques.

What are Async Methods?

An async method allows you to perform asynchronous operations while the rest of your program continues running. Async methods enable non-blocking execution, meaning they do not block the main thread while waiting for long-running operations (e.g., database queries, file I/O, web requests) to complete. The following are the Key Points to remember:

• Non-blocking: Async methods allow other tasks to run while waiting for long-running operations.
• Concurrency: They are commonly used in I/O-bound tasks, where performing multiple tasks concurrently improves performance.

Example Syntax of an Async Method:

public async Task<int> FetchDataAsync()
{
    await Task.Delay(1000);  // Simulate an asynchronous operation (e.g., web request)
    return 42;  // Return the result
}

What are Iterators?

An iterator method uses the yield return keyword to generate a sequence of values lazily. Iterators allow you to enumerate over a collection without loading it fully into memory at once. This is especially useful when working with large datasets or when you need to process items individually. The following are the Key Points to remember:

• Lazy Evaluation: Iterators create lazy collections where values are generated on demand.
• Memory-Efficient: They allow processing large datasets without simultaneously loading everything into memory.

Example Syntax of an Iterator:

public IEnumerable<int> GetNumbers()
{
    for (int i = 0; i < 10; i++)
    {
        yield return i;  // Lazily return numbers one by one
    }
}

When to Use Async Methods:

• I/O-Bound Operations: Async methods are useful for tasks like reading files, querying a database, or making HTTP requests.
• UI Applications: Async methods allow the UI to remain responsive by executing long-running operations in the background.
• Parallel Processing: Async methods efficiently manage multiple parallel tasks without blocking the main thread.

When to Use Iterators:

• Lazy Data Generation: When you want to generate or process large sets of data lazily.
• Memory-Efficient Iteration: For large collections, like streaming data or logs, without loading everything into memory.
• Custom Collection Types: When building collections with custom iteration logic (e.g., filtering or transforming data during iteration).

For a better understanding, please check the image below:

The ref and unsafe Keywords in Async Methods and Iterators Before C# 13

Before C# 13, there were the following limitations:

• Ref parameters could not be used in async methods due to the way the state machine works for async methods.
• Unsafe code (e.g., direct memory manipulation) was not allowed in async methods or iterators because it interfered with the state machine generated by them.

Performing low-level memory operations in async code or lazy evaluations using iterators was challenging.

C# 13 Improvements: ref and unsafe in Async Methods and Iterators

With C# 13, the language has relaxed these restrictions, allowing:

• Ref parameters to be used in async methods.
• Unsafe code to be used in iterators.
• Improved functionality of ref struct types like System.Span<T> and System.ReadOnlySpan<T> in both async methods and iterators.

Improvements in C# 13 for ref Parameters in Async Methods

In C# 13, ref parameters are now allowed in async methods, enabling you to modify local variables by reference asynchronously, which wasn’t possible before. Key points to note:

• Ref local variables can now span across await boundaries.
• Ref variables cannot be used across yield return boundaries (i.e., after a yield return statement, ref variables cannot be accessed).

Example: Using ref in Async Methods in C# 13

Using ref in async methods—Bank Account Withdrawal: In a banking application, a customer makes a withdrawal request. We need to modify the balance by reference, as the balance is stored in memory and needs to be updated asynchronously. In the past, this was impossible in async methods because ref parameters were not allowed.

With C# 13, we can use ref in async methods to modify the balance directly without returning a new value. This ensures we don’t need to create additional copies of the account balance, making the system more memory-efficient.

namespace BankingAppExample
{
    public class Program
    {
        // Simulated account balance
        static double accountBalance = 1000.00;

        static async Task Main(string[] args)
        {
            Console.WriteLine($"Initial Account Balance: ${accountBalance}");

            // Perform an async withdrawal operation
            await WithdrawMoneyAsync(200);

            Console.WriteLine($"Final Account Balance: ${accountBalance}");
        }

        // Async method to withdraw money and modify balance by reference
        static async Task WithdrawMoneyAsync(double amount)
        {
            await SimulateAsyncProcessing();  // Simulate some async operation (e.g., database call)

            // Modify the account balance by reference using ref
            ref double balance = ref GetAccountBalance();
            ProcessWithdrawal(ref balance, amount);  // Perform withdrawal
        }

        // Simulate async processing (e.g., database or network request)
        static async Task SimulateAsyncProcessing()
        {
            await Task.Delay(1000);  // Simulate async work (e.g., contacting a database)
            Console.WriteLine("Processing withdrawal...");
        }

        // Get reference to account balance
        static ref double GetAccountBalance()
        {
            return ref accountBalance;  // Return the reference to the account balance
        }

        // Process the withdrawal and modify the balance
        static void ProcessWithdrawal(ref double balance, double amount)
        {
            if (balance >= amount)
            {
                balance -= amount;  // Decrease balance by the withdrawal amount
                Console.WriteLine($"Withdrawal of ${amount} successful.");
            }
            else
            {
                Console.WriteLine("Insufficient funds.");
            }
        }
    }
}

Explanation of the Code:

• Initial Balance: The accountBalance is simulated as a variable in memory.
• Async Withdrawal: The WithdrawMoneyAsync method is async and allows for non-blocking operations (simulating a time-consuming withdrawal process, like querying the bank).
• ref Parameter: The accountBalance is passed as a ref parameter to the ProcessWithdrawal method. The ref keyword ensures that we modify the balance directly without making a copy.
• Async Operations: The SimulateAsyncProcessing() method simulates async operations (like checking the balance from a database or remote server).
• Withdrawal: If the balance is sufficient, it is decreased by the withdrawal amount, and the updated balance is printed.

Output:

Improvements in C# 13 for unsafe Code in Iterators

In C# 13, you can now use unsafe code inside iterators. Unsafe code refers to operations that involve low-level memory manipulation, such as pointer arithmetic or manual memory allocation. You can use unsafe code in iterators, but the yield return statement and yield break must occur within safe contexts.

Example: Using unsafe in Iterators in C# 13

Using unsafe in Iterators—Processing Large Log Files: In a log file processing system, we need to process large log files and extract specific entries efficiently. Since log files can be huge (gigabytes), we want to process the data lazily without loading the entire file into memory. Unsafe code allows for direct memory manipulation, enabling fast, low-level processing of the log file.

In C# 13, we can now use unsafe code inside iterators. This allows us to manipulate data directly in memory and yield results lazily (e.g., process one log entry at a time) instead of loading everything into memory.

In the below example, we will simulate a logging system in which log entries are processed using unsafe code in an iterator. The unsafe code will manipulate memory directly and yield log entries lazily.

namespace CSharp13NewFeatures
{
    public class Program
    {
        static void Main()
        {
            try
            {
                Console.WriteLine("Unsafe Log Data from Iterator:");
                foreach (var value in GetUnsafeLogData())
                {
                    Console.WriteLine($"Log Entry: {value}");
                }
            }
            catch (Exception ex)
            {
                Console.Error.WriteLine($"Error: {ex.Message}");
            }
        }

        // Iterator that processes log data using unsafe code
        public static unsafe IEnumerable<string> GetUnsafeLogData()
        {
            string[] logs = { "Error: File not found", "Info: Process started", "Warning: Low memory" };

            string[] buffer;

            unsafe
            {
                // Allocate memory using stackalloc for unsafe buffer manipulation
                fixed (char* ptr = logs[0])
                {
                    buffer = new string[logs.Length];

                    // Copy data into the unsafe buffer
                    for (int i = 0; i < logs.Length; i++)
                    {
                        // Simulating unsafe data processing (you can access `logs[i]` using pointer manipulation here)
                        buffer[i] = logs[i];  // For simplicity, we're copying the string directly
                    }
                }
            }

            // Now that the unsafe block is closed, it's safe to yield the data
            foreach (var log in buffer)
                yield return log;  // Yield each log entry lazily
        }
    }
}

Explanation of the Code:

• Unsafe Buffer Allocation: We use the stackalloc keyword inside the unsafe block to allocate memory on the stack. This memory is used to process log entries efficiently without heap allocation overhead. 
• Copying Data to Unsafe Memory: The data from the logs array is copied into the buffer array inside the unsafe block. While the code here simulates unsafe operations (we’re not doing pointer arithmetic on the strings, but this can be expanded for more complex data manipulation).
• Yielding Log Entries: After the unsafe block has been completed and memory manipulation is done, the log entries are yielded lazily from the buffer array. The yield return ensures that data is returned one at a time, which is efficient when dealing with large datasets.

Output:

Real-Time Scenarios for ref and unsafe in Async Methods and Iterators

In log file processing or data stream processing systems, you may need to process large datasets like logs, which could span multiple gigabytes. Using unsafe code in iterators ensures that you can directly manipulate memory, reducing overhead by not having to copy large datasets. Additionally, lazy evaluation (via yield return) allows each log entry to be processed individually without loading the entire dataset into memory.

• Use Case 1: High-Performance Computing: In real-time systems (e.g., video or signal processing), you may need to process large data buffers asynchronously while maintaining memory efficiency. Unsafe code allows for low-level memory manipulation, and ref parameters enable data to be modified to avoid copying large arrays or objects.
• Use Case 2: Network Data Processing: When processing data from network streams or large files, unsafe code can manipulate memory directly, improving performance by eliminating unnecessary copies. Using ref parameters helps modify data asynchronously without creating intermediate copies.
• Use Case 3: Memory-Intensive Iterations: When iterating over large data sets (e.g., in data streaming applications), unsafe iterators can manage memory efficiently while lazily returning data, thus optimizing both memory usage and performance.

This C# 13 enhancement, which allows unsafe code in iterators, provides a great way to handle large data efficiently, especially in scenarios like log processing. You can now directly manipulate memory using pointers in an iterator and lazily return results, making processing huge datasets more memory-efficient and faster.

In the next article, I will discuss Interface Support for ref struct Types in C# with Examples. In this article, I explain Improved Support for ref and unsafe in async methods and Iterators in C# with Examples. I would like your feedback. Please post your feedback, questions, or comments about this article.