When you execute the console application, the three chunks of five integers will be displayed in the console window as shown in Figure 1.

IDG

Using Chunk to split a list of strings in C#

You can also use the Chunk method to split a list of strings as shown in the code snippet given below.

List numbers = new List
{ "USA","UK", "India", "France", "Australia", "Brazil"};
var chunks = numbers.Chunk(3);
int counter = 0;
foreach (var chunk in chunks)
{
    Console.WriteLine($"Chunk #{++counter}");
    Console.WriteLine(string.Join(", ", chunk));
}

When you run the preceding code example, two chunks of string data each containing three elements will be created and the text stored in each chunk will be displayed at the console window as shown in Figure 2.

IDG

Using Chunk to process large files in C#

Chunking is beneficial whenever you’re handling large data sets or massive volumes of data. Chunking also comes in handy when processing large files, saving a lot of resources and time. You can take advantage of the Chunk extension method to split the contents of a file and then process it one chunk at a time. The following code snippet shows how you can split a large file into chunks of 100 bytes each and then display the file content at the console.

int size = 100;
var data = File.ReadLines(@"D:largetextfile.txt");

foreach (var chunk in data.Chunk(size))
{
    Console.WriteLine($"Number of lines in the file is: {chunk.Count()}");
    Console.WriteLine("Displaying the file content:-");

    DisplayText(chunk);
}
void DisplayText(IEnumerable fileContent)
{
    foreach (var text in fileContent)
    {
        Console.WriteLine(text);
    }
}

Here the File.ReadLines method reads a file one line at a time without loading the entire file into the memory at once. By using the Chunk method, you can process the contents of the file one chunk at a time. In this example, the text contained in each chunk will be displayed at the console window as shown in Figure 3.

IDG

Working with Chunk

The Chunk extension method in LINQ enables you to split a data source into bite-sized chunks for processing, thereby improving resource utilization and application performance. However, you should keep the following points in mind:

• If the collection being chunked is empty, the Chunk extension method will return an empty chunk but no exception will be thrown.
• If the number of elements in the collection is not exactly divisible by the size of the chunk, the last chunk will contain the remaining data of the collection.
• Additionally, if the size of the chunk is greater than or equal to the size of the collection being chunked, the Chunk method will return only one chunk of data.

If you plan on working with large data sets, or even the occasional large file, Chunk would be a very handy addition to your toolkit.