A Guide to Data Virtualization in Xamarin Apps

C1CollectionView is a data management component able to perform the most common data transformations, compositions and virtualization operations (like sorting, filtering, grouping, editing, incremental-loading and refreshing). This component can be downloaded as part of GrapeCity’s ComponentOne Studio for Xamarin or from Nuget.org.

This articles discusses the different benefits and techniques of incremental-loading (data virtualization) and the types of collection views supported by C1CollectionView. We'll also go over the basic steps to get started using C1CollectionView in a Xamarin app.

Why Data Virtualization?

In most scenarios visualizing data on mobile and web, the data comes from remote sources. This introduces a cost in terms of network usage as well as a cost in the time needed to load all the data.

In order to solve this problem, we divide the data and send it to the client in chunks, or pages, each containing a portion of the data. This process is what we call data virtualization (also known as incremental-loading or on-demand loading).

There are two common approaches to provide data virtualization: pagination and cursor.

Pagination consists in sending a limit and offset parameters to the remote source to specify which portion of the data is requested.

Pagination has two main problems. It needs to know the total number of items available so that the client knows how to retrieve the rest of the items. Calculating the total number of items is costly and ensuring the consistency of the data between requests is complicated in scenarios where the data is constantly changing.

The cursor approach works by sending a token to fetch the pages sequentially. The received page contains the token to the next one.

This approach does not require the server to calculate the total number of items and works better with data that is constantly changing. However, because each page needs the token from the previous one, the pages must be fetched sequentially one after the other. With pagination the pages can be requested in parallel.

Pagination vs. Cursor-based Data Virtualization

For example, think about the Google web search versus image search.

While we can’t say for certain exactly how Google implemented these searching strategies, they’re a fair representation of what the pagination (web search on the left) and cursor (image search on the right) virtualization look like. With the paginated web search, you can jump immediately to page 5. With the cursor-based image search the chunks of images are loaded sequentially as you scroll down through the results. Another thing to notice is that the web search tells you exactly how many results there are whereas the image search does not. Is that a coincidence?

But pagination does not require the data be displayed as literal pages with next & previous buttons. It just helps to understand how it works. You can implement a scrolling solution with pagination as well, in fact most data grids with data virtualization use the pagination method because that’s how the scroll thumb knows how big it should be and where it should be placed relative to the entire number of items.

Pagination typically allows faster search of the items by dragging the scrollbar, whereas cursors need activity indicators (progress ring) at the bottom of the data to let users know the data is being pulled.

Pagination vs. Cursor Data Virtualization: Which is Better?

There is no definite winner with pagination versus cursor virtualization. It depends on the kind of data and preferred UI. For larger data sets, pagination is better because it allows searching through results more efficiently. The restriction in searching for items in the cursor approach can be resolved by adding filtering. But luckily for everyone, the C1CollectionView supports both approaches to data virtualization for any business need.

What is in C1CollectionView?

The ComponentOne CollectionView for Xamarin supports virtualization features along with several collection views designed for specific scenarios. There are two base classes that provide data virtualization:

• C1CursorCollectionView – the abstract base class implementation of ICollectionView for remote-source collection views. It provides cursor-based data virtualization and can be used to display information from rest-services, or public services like Facebook, Twitter, DropBox, OneDrive, Google Drive etc.
• C1VirtualCollectionView – the abstract base class to provide page-like data virtualization. This collection view also resolves the problem of synchronizing requests and avoiding duplicates.

In both instances, they provide the GetPageAsync() method that populates the collection view page by page. These can be referred to as data-providing collection views.
Those are the main data virtualization base classes. The other collection views within C1.CollectionView are transforming views. They are all based on the C1WrapCollectionView, which is designed take another collection view as a source to perform some specific transformation.

• C1CollectionView – this is the main marketed type of collection view in within C1.CollectionView. It combines the most common transformation into one such as sorting, filtering and grouping.
• C1FilterCollectionView & C1SortCollectionView – special collection view just for filtering and sorting respectively. To support grouping, the C1CollectionView does not preserve the generic type, so even if you wrap a List collection, it will become an IEnumerable
• C1GroupCollectionView – supports grouping in memory while the collection continues to receive data on demand. The level of grouping support depends on the control, for instance, ListView can only display one-level of grouped data, whereas FlexGrid can show multiple levels.
• C1PagedCollectionView - provides a paged collection view that supports the ISupportPaging interface that allows controls like C1DataPager to change the pages. This collection is a transforming collection, like C1SortCollectionView or C1FilterCollectionView.
• C1EditableCollectionView – with this collection view changes made to the collection are tracked locally and aren’t committed to the source immediately. It’s similar to the .NET DataTable.
• C1SelectCollectionView – allows transforming every item into another item. This collection view is useful to create a view-model collection of items.
• C1SelectManyCollectionView - this is like C1SelectCollectionView, but every item of the original collection is converted to a range of items in the resulting collection.
• C1CacheCollectionView - this collection keeps a reference to the items when they are accessed. It can be used together with C1SelectCollectionView to avoid calling the selector repeatedly for the same item, and thus improve performance.
• C1SequenceCollectionView - this collection allows composing a range of collection into one sequentially to be exposed as a single collection.

Typically, you combine one data-providing collection view with one transforming view to create the scenario you want. For example, you can implement paged data virtualization by wrapping a C1VirtualCollectionView with a C1PagedCollectionView. If you create a view with a List

By providing all of the above mentioned types of collection views, the C1.CollectionView library has you covered for any number of scenarios.

But to make your simple scenario a little easier, there are some special collection views that already combine the data virtualization technique along with some basic transforming features. These take advantage of the entity framework data context features.

• EntityFrameworkCursorCollectionView – implementation of C1CursorCollectionView that provides sorting, filtering, editing, on-demand loading, and refreshing.
• EntityFrameworkVirtualCollectionView – implementation of C1VirtualCollectionView that provides sorting, filtering, editing, on-demand loading, and refreshing.

How to Use C1CollectionView

The simplest way to use C1CollectionView is to use the main C1CollectionView class all by itself. This class provides basic sorting, filtering, and grouping in memory.

C1CollectionView<MyObject> _collectionView;  
var items = new ObservableCollection<MyObject>(());  
_collectionView = new C1CollectionView<MyObject>(items);  
listView.ItemsSource = _collectionView;  

Check out the samples and documentation for more details on the basic features. C1CollectionView alone does not support data virtualization.

So to get back on topic, let’s see the main options we have for this.

Extend the C1CursorCollectionView or C1VirtualCollectionView

By extending the C1CursorCollectionView or C1VirtualCollectionView, and overriding the GetPageAsync method, you can handle the specific loading of items from your data source page by page. The basic template is below, but you should check out the CollectionView101 samples for the full implementation.

public class SimpleOnDemandCollectionView : C1CursorCollectionView<MyDataItem>  
{  
    public SimpleOnDemandCollectionView()  
    {  
        PageSize = 30;  
    }

    public int PageSize { get; set; }

    protected override async Task<Tuple<string, IReadOnlyList<MyDataItem>>> GetPageAsync(int startingIndex, string pageToken, int? count = null, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpresssion = null, CancellationToken cancellationToken = default(CancellationToken))  
    {  
        return await Task.Run(() =>  
        {  
            // create new page of items  
            var newItems = new List<MyDataItem>();  
            for (int i = 0; i < this.PageSize; i++)  
            {  
                newItems.Add(new MyDataItem(startingIndex + i));  
            }

            return new Tuple<string, IReadOnlyList<MyDataItem>>("token not used", newItems);  
        });  
    }  
}  

In a previous article, I discussed incremental loading in the common ListView. You can see the full example of this data virtualization feature here.

Combine Virtualization View with Transforming View

As mentioned, another way to use C1CollectionView for a custom virtualization scenario is combining a data-providing view with one that transforms. To follow this example, first start by extending C1CursorCollectionView or C1VirtualCollectionView and then wrap it with C1GroupCollectionView (for grouping) or C1PagedCollectionView (for paging).

// initialize our Extended C1CursorCollectionView or C1VirtualCollectionView  
_collectionView = new SimpleOnDemandCollectionView() { PageSize = 25 };  
// create group collection view to wrap grouping into our virtualized view.  
var grouping = new C1GroupCollectionView<MyObject>(_collectionView, false);  
await grouping.GroupAsync("ObjectGroup");  
// populate data grid

## grid.ItemsSource = grouping;

You can check out this sample under Xamarin.Forms FlexGrid101/OnDemand. When you install Studio for Xamarin, all samples are installed in your documents folder under ComponentOne Samples.

The Entity Framework Data Virtualization Views

A third option is to use the specialized Entity Framework collection views that have built-in data virtualization and transforming features (sort, filter, group, etc). This is the easiest solution if your scenario meets the requirements of using an entity framework supported data source.

It’s very easy to initialize a C1 EF collection view, all you need to do is define your DbContext object and pass it to the collection view. The C1 EF collection view works with the DbContext object which represents the session with the data base.

Remember, there are two C1 EF collection views, each supporting a different data virtualization technique: EntityFrameworkCursorCollectionView and EntityFrameworkVirtualCollectionView. These are both within the C1.CollectionView.EntityFramework library.

Here is the code snippet to create an EntityFrameworkVirtualCollectionView:

var cv = new EntityFrameworkVirtualCollectionView<MyObject>(dbContext);  
grid.ItemsSource = cv;  

That’s it. To see the full code, including the dbContext, check out the SQLiteDataBase sample included with Studio for Xamarin Samples.

Conclusion

Working with data in a mobile environment requires latency. While users may be OK waiting for data to be processed, they are normally not OK waiting for the data to initially load. That’s where incremental loading (data virtualization) plays a big part in the C1CollectionView usability.

C1CollectionView supports asynchronous interfaces that allow delayed responses for sorting, grouping, and filtering large data sets. This resolves the problem that traditionally was left to the developers, which had to deal with complicated synchronization mechanisms or ended up blocking the user interface.

In addition to supporting two different data virtualization techniques, the entire C1CollectionView library provides many other data-related features. This article touches on some of this, but for more, download Studio for Xamarin to see more of C1CollectionView in action.