DataCollection | ComponentOne
Work with DataCollection / Virtualization
In This Topic
    Virtualization
    In This Topic

    Data Virtualization enables efficient processing of data by allowing large amounts of data to be loaded in lesser period of time. C1DataCollection enables data virtualization for larger data sets with enhanced performance. It provides a powerful feature, known as On-demand or Incremental Loading that helps data to be fetched and loaded in chunks as the user scrolls down a list in real time.

    The image shows cursor scrolling down the scroll-bar of application and the data loading after each scroll.

    In most scenarios, the data comes from different remote source systems. This impacts the time required to show the data, as well as brings an inherent cost in terms of network usage. In order to solve this problem, the data is divided and sent to the client in chunks or pages. There are two widely adopted approaches for data virtualization, pagination and cursor. In Pagination approach, a limit and offset parameter are send to the remote source to specify which portion of the data is requested. This approach returns the total amount of items available, so that the client may know how to retrieve the rest of the items. While, in Cursor approach, a token is send initially, which is null at first, to fetch the pages sequentially. In this case, the received page contains the token to the next one.

    C1DataCollection namespace provides base classes such as C1VirtualDataCollection and C1CursorDataCollection to implement pagination and cursor approach respectively. Both classes have an abstract GetPageAsync() method that must be implemented to return the items that will populate the collection. The base classes take the responsibility of caching the data as well as dispatching the requests of the pages according to a series of parameters that prevents too many pages to be requested together.

    The GetPageAsync method of C1VirtualDataCollection returns the items in the page as well as a token to the next page. This method uses pageIndex, startingIndex, count, sortDescriptions, filterExpression and cancellation Token as parameters. The parameters pageIndex denotes the index of the requesting page, startingIndex denotes the index where the returned items will be inserted and count denotes the number of items to be returned. The GetPageAsync method returns total number of items (TotalCount) along with a list of the requested number of items (count) starting from an index (startingIndex). To know in detail about the parameters of the method, refer this topic.

    The following code snippet depicts the use of C1VirtualDataCollection class:

    This is the C# code snippet for WinForms application:

    C#
    Copy Code 
    public class VirtualModeCollectionView : C1VirtualDataCollection<Customer> 
   {   
        public int TotalCount  { get; set; } = 1_000;
    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//Simulates network traffic.     
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
   }

    This is the VB code snippet for WinForms application:

    VB
    Copy Code 
    Public Class VirtualModeCollectionView
    Inherits C1VirtualDataCollection(Of Customer)
    Public Property TotalCount As Integer = 1_000

    Protected Overrides Async Function GetPageAsync(ByVal pageIndex As Integer, ByVal startingIndex As Integer, ByVal count As Integer, ByVal Optional sortDescriptions As IReadOnlyList(Of SortDescription) = Nothing, ByVal Optional filterExpression As FilterExpression = Nothing, ByVal Optional cancellationToken As CancellationToken = Nothing) As Task(Of Tuple(Of Integer, IReadOnlyList(Of Customer)))
        Await Task.Delay(500, cancellationToken)
        Return New Tuple(Of Integer, IReadOnlyList(Of Customer))(TotalCount, Enumerable.Range(startingIndex, count).[Select](Function(i) New Customer(i)).ToList())
    End Function

End Class

    This is the C# code snippet for WPF application:

    C#
    Copy Code 
    public class VirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
    public int TotalCount { get; set; } = 1_000;

    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//Simulates network traffic.
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
}

    This is the VB code snippet for WPF application:

    VB
    Copy Code 
    Public Class VirtualModeDataCollection
    Inherits C1VirtualDataCollection(Of Customer)
    Public Property TotalCount As Integer = 1_000
    Protected Overrides Async Function GetPageAsync(pageIndex As Integer, startingIndex As Integer, count As Integer, Optional sortDescriptions As IReadOnlyList(Of SortDescription) = Nothing, Optional filterExpression As FilterExpression = Nothing, Optional cancellationToken As CancellationToken = Nothing) As Task(Of Tuple(Of Integer, IReadOnlyList(Of Customer)))
        Await Task.Delay(500, cancellationToken)
        Return New Tuple(Of Integer, IReadOnlyList(Of Customer))(TotalCount, Enumerable.Range(startingIndex, count).[Select](Function(i) New Customer(i)).ToList())
    End Function
End Class

    This is the code snippet for Xamarin Forms:

    Xamarin Forms
    Copy Code 
    public class VirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
    public int TotalCount { get; set; } = 1_000;

    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//Simulates network traffic.
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
}

    This is the code snippet for Android application:

    Android
    Copy Code 
    public class VirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
    public int TotalCount { get; set; } = 1_000;

    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//Simulates network traffic.
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
}

    This is the code snippet for iOS application:

    iOS
    Copy Code 
    public class VirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
    public int TotalCount { get; set; } = 1_000;

    protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        await Task.Delay(500, cancellationToken);//Simulates network traffic.
        return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
    }
}

    The GetPageAsync method of C1CursorDataCollection returns the items in the page as well as a token to the next page. This method uses startingIndex, pageToken, count, sortDescriptions and filterExpression as the parameters. The parameters startingIndex denotes the index where the returned items will be inserted, pageToken denotes the token of the requesting page and count denotes the desired number of items to be returned. To know more about the method and its parameters, refer this topic in detail. The GetPageAsync method returns pages with starting index (startingIndex) along with a list of the requested number of items (PageCount) starting from an index (startingIndex).

    The following code snippet depicts the use of C1CursorDataCollection class:

    This is the C# code snippet for WinForms application:

    C#
    Copy Code 
    public class MyCursorDataCollection : C1CursorDataCollection<Customer>
{
    protected override async Task<Tuple<string, IReadOnlyList<Customer>>> GetPageAsync(

        int startingIndex,

        string pageToken,

        int? count = null,

        IReadOnlyList<SortDescription> sortDescriptions = null,

        FilterExpression filterExpresssion = null,

        CancellationToken cancellationToken = default(CancellationToken));
    {
        var client = new HttpClient();

    var(items, token) = await client.GetAsync("http://www.contoso.com/myservice");

        return new Tuple<string, IReadOnlyList<Customer>>(token, items);
    }
}

    This is the VB code snippet for WinForms application:

    VB
    Copy Code 
    Public Class MyCursorDataCollection
    Inherits C1CursorDataCollection(Of Customer)

    Protected Overrides Async Function GetPageAsync(ByVal startingIndex As Integer, ByVal pageToken As String, ByVal Optional count As Integer? = Nothing, ByVal Optional sortDescriptions As IReadOnlyList(Of SortDescription) = Nothing, ByVal Optional filterExpresssion As FilterExpression = Nothing, ByVal Optional cancellationToken As CancellationToken = Nothing) As Task(Of Tuple(Of String, IReadOnlyList(Of Customer)))


        Dim client = New HttpClient()
    End Function
    Private Sub New(ByVal _ As items, ByVal _ As token)
    End Sub

    Private Function GetAsync() As await
        Return New Tuple(Of String, IReadOnlyList(Of Customer))(token, items)
    End Function

End Class

    This is the C# code snippet for WPF applications:

    C#
    Copy Code 
    public class CursorDataCollection : C1CursorDataCollection<Customer>
{      
    public int PageCount { get; set; } = 50;
    protected override async Task<Tuple<string, IReadOnlyList<Customer>>> GetPageAsync(int startingIndex, string pageToken, int? count = null, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default)
    {
        await Task.Delay(500, cancellationToken);//Simulates network traffic.
        return new Tuple<string, IReadOnlyList<Customer>>(startingIndex.ToString(), Enumerable.Range(startingIndex, PageCount).Select(i => new Customer(i)).ToList());
    }       
}

    This is the VB code snippet for WPF applications:

    VB
    Copy Code 
    Public Class CursorDataCollection
    Inherits C1CursorDataCollection(Of Customer)
    Public Property PageCount As Integer = 50
    Protected Overrides Async Function GetPageAsync(startingIndex As Integer, pageToken As String, Optional count As Integer? = Nothing, Optional sortDescriptions As IReadOnlyList(Of SortDescription) = Nothing, Optional filterExpression As FilterExpression = Nothing, Optional cancellationToken As CancellationToken = Nothing) As Task(Of Tuple(Of String, IReadOnlyList(Of Customer)))
        Await Task.Delay(500, cancellationToken)
        Return New Tuple(Of String, IReadOnlyList(Of Customer))(startingIndex.ToString(), Enumerable.Range(startingIndex, PageCount).[Select](Function(i) New Customer(i)).ToList())
    End Function
End Class

    This is the code snippet for Xamarin Forms:

    Xamarin Forms
    Copy Code 
    public class SimpleOnDemandDataCollection : C1CursorDataCollection<MyDataItem>
{
    public SimpleOnDemandDataCollection()
    {
        PageSize = 30;
    }

    public int PageSize { get; set; }

    protected override async Task<Tuple<string, IReadOnlyList<MyDataItem>>> GetPageAsync(int startingIndex, string pageToken, int? count = null, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpresssion = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        return await Task.Run(() =>
        {
            // create new page of items
            var newItems = new List<MyDataItem>();
            for (int i = 0; i < this.PageSize; i++)
            {
                newItems.Add(new MyDataItem(startingIndex + i));
            }

            return new Tuple<string, IReadOnlyList<MyDataItem>>("token not used", newItems);
        });
    }
}

    This is the code snippet for Android applications:

    Android
    Copy Code 
    public class OnDemandDataCollection : C1CursorDataCollection<MyDataItem>
{
    private bool hasMoreItems = true;

    public override bool HasMoreItems
    { get { return hasMoreItems; } }

    const int MAX = 100;
    int current;
    public int PageSize { get; set; }
    public OnDemandDataCollection()
    {
        PageSize = 20;
    }
    protected override async Task<Tuple<string, IReadOnlyList<MyDataItem>>>
    GetPageAsync(int startingIndex, string pageToken, int? count = null,
    IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression
            filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        if (startingIndex + PageSize >= MAX)
        {
            hasMoreItems = false;
        }

        var newItems = new List<MyDataItem>();
        await Task.Run(() =>
        {
            for (int i = 0; i < this.PageSize; i++)
            {
                //To mimic a long operation,e.g fetching data from a web service
                Thread.Sleep(100);

                newItems.Add(new MyDataItem(startingIndex + i));
                current++;
            }
        });
        return new Tuple<string, IReadOnlyList<MyDataItem>>("token not used", newItems);
    }

}

    This is the code snippet for iOS applications:

    iOS
    Copy Code 
    public class OnDemandDataCollection : C1CursorDataCollection<MyDataItem>
{
    private bool hasMoreItems = true;

    public override bool HasMoreItems
    { get { return hasMoreItems; } }

    const int MAX = 100;
    int current;
    public int PageSize { get; set; }
    public OnDemandDataCollection()
    {
        PageSize = 20;
    }
    protected override async Task<Tuple<string, IReadOnlyList<MyDataItem>>> GetPageAsync(int startingIndex, string pageToken, int? count = null, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
    {
        //No items to load further
        if (startingIndex + PageSize >= MAX)
        {
            hasMoreItems = false;
        }

        var newItems = new List<MyDataItem>();
        await Task.Run(() =>
        {
            for (int i = 0; i < this.PageSize; i++)
            {
                //To mimick a long operation such as fetching data from a web-service
                Thread.Sleep(100);
                newItems.Add(new MyDataItem(startingIndex + i));
                current++;
            }
        });
        return new Tuple<string, IReadOnlyList<MyDataItem>>("token not used", newItems);
    }
}

    Similarly, when the user clicks the Cursor button in the application, it gets a cursor-based data collection. The user can set a cursor-based data collection as the grid's ItemsSource, and call the LoadMoreItemsAsync method of C1CursorDataCollection to asynchronously load the records.

    Private Sub btn_Virtualization_Click(sender As Object, e As RoutedEventArgs)
    cursorDataCollection = Nothing

    virtualDataCollection = New VirtualModeDataCollection() With {
        .Mode = VirtualDataCollectionMode.Manual
    }
    grid.ItemsSource = New C1CollectionView(virtualDataCollection)
    virtualDataCollection.LoadAsync(0, 0)
End Sub
Private Sub btn_Cursor_Click(sender As Object, e As RoutedEventArgs)
    virtualDataCollection = Nothing
    cursorDataCollection = New CursorDataCollection()
    grid.ItemsSource = New C1CollectionView(cursorDataCollection)
    cursorDataCollection.LoadMoreItemsAsync()
End Sub

    private void btn_Virtualization_Click(object sender, RoutedEventArgs e)
{
    cursorDataCollection = null;
    virtualDataCollection = new VirtualModeDataCollection() { Mode = VirtualDataCollectionMode.Manual, PageSize = 50 };
    grid.ItemsSource = new C1CollectionView(virtualDataCollection);
    _ = virtualDataCollection.LoadAsync(0, 10);
}     
private void btn_Cursor_Click(object sender, RoutedEventArgs e)
{
    virtualDataCollection = null;
    cursorDataCollection = new CursorDataCollection();
    grid.ItemsSource = new C1CollectionView(cursorDataCollection);
    cursorDataCollection.LoadMoreItemsAsync();
}

    In WPF application, the MS DataGrid control iterates the collection, which breaks the virtualization of C1VirtualDataCollection. In this case, the user has to handle the ScrollChanged event to load more pages when scrolling to the bottom. In the code below, the ScrollChanged event is called, such that if the grid is bound to a pagination-based data virtualization collection, the next record for the new viewport range are loaded asynchronously using the LoadAsync method of C1VirtualDataCollection. Likewise, if the grid is bound to a cursor-based data virtualization collection and the new viewport range is greater than the current scroll viewer area, then more records for the new viewport are loaded asynchronously using the LoadMoreItemsAsync method of C1CursorDataCollection.

    Private Sub grid_ScrollChanged(sender As Object, e As ScrollChangedEventArgs)
    If virtualDataCollection IsNot Nothing Then
        virtualDataCollection.LoadAsync(CInt(e.VerticalOffset), CInt((e.VerticalOffset + e.ViewportHeight)))
    End If

    If cursorDataCollection IsNot Nothing AndAlso e.VerticalOffset + e.ViewportHeight >= e.ExtentHeight Then
        cursorDataCollection.LoadMoreItemsAsync(CInt(e.ViewportHeight))
    End If
End Sub

    private void grid_ScrollChanged(object sender, ScrollChangedEventArgs e)
{
    if (virtualDataCollection != null)
    {
        _ = virtualDataCollection.LoadAsync((int)e.VerticalOffset, (int)(e.VerticalOffset + e.ViewportHeight));
    }

    if (cursorDataCollection != null && e.VerticalOffset + e.ViewportHeight >= e.ExtentHeight)
    {
        _ = cursorDataCollection.LoadMoreItemsAsync((int)e.ViewportHeight);
    }
}

    Data Loading Message

    DataCollection allows you to display a loading message as the data loads asynchronously. It provides you IsLoading property in the C1VirtualDataCollection class that helps to show a "Loading..." message as demonstrated in the following code:

    C#
    Copy Code 
    collection.PropertyChanged += (s, e) =>
{
  if (e.PropertyName == nameof(C1EntityFrameworkCoreVirtualDataCollection<Person>.IsLoading))
LoadingMessage.Text = collection.IsLoading ? AppResources.LoadingMessage : "";
};