Cloud Storage - CRUD Operations using RESTFul C1 WebAPI Services
The advent of cloud computing has ensured round-the-clock data accessibility for users. With the emergence of various service providers, it has become easier for users to store data with permanent availabity.
Here are some of the Major Cloud Storage providers:
- AWS (Amazon Web Service)
- Azure
- DropBox
- GoogleDrive
- OneDrive
Data on these storages can be accessed by users either via web-based clients or APIs exposed by these providers.
Let’s say we created a Web Application that has a feature to convert PDF files to image files. C1 Web API already has the built-in service to convert them to different file formats (JPEG, PNG, TIFF, BMP, GIF) to make our task much easier.
Now, end users of this application may want to upload PDF files from their cloud storage providers and then store the converted files back to the cloud service provider.
Since each of these providers would have their own implementations for an API and different authentication mechanisms, we would need to write separate code to implement CRUD operations for each of these cloud providers. It would become cumbersome to maintain the code for all of them.
A prudent approach would be to write the CRUD & authentication mechanisms for all services inside a single service and then expose common Restful API for all the Cloud Storages.
In 2019 v2, we have added Cloud Storage support to ComponentOne WebAPI, which provides common RESTAPI methods to list, upload, delete and download files separately for each cloud storage provider.
To perform operations on Cloud Storage, C1 WEBAPI would act as the middleware between the Client and Cloud Service Storage APIs. C1 WebAPI would abstract the different CRUD & authentication implementation for cloud storage providers and provide common REST implementation to clients.
Let's have a look at C1 Cloud Service Architecture:
Let’s review the implementation steps using C1 Web API
- Adding C1.Web.Api.Cloud DLL/package
- Cloud storage registration & authentication (AWS, Azure, DropBox, Google Drive, OneDrive)
- CRUD operations on cloud storage
Create WebAPI project using C1 Template
ComponentOne provides a built-in template for C1 WebAPI, which can be used to create a WebAPI project with C1 services.
To begin, open Visual Studio | New | Project | C1 | WebAPI which will open the following template:
Here, we would fill the project name and select the project location, then click OK.
Once you click OK, another dialog opens, which can be used to select the required service. If it is not in the list, we can later add the required DLL’s from Nuget or add from a local machine.
Adding C1.Web.Api.Cloud Dll/package
To use the Cloud Storage, the C1.Web.Api.Cloud DLL or package is required to added an existing or a new WebApi project using the NuGet Package Manager. It installs the required dependencies to interact with Cloud Storages.
Cloud storage registration & authentication
Each Cloud Storage has its own authentication method to perform an operation by authorized users. We need to add the registration code inside the Configuration method of the Startup class.
Startup.cs
public class Startup
{
private readonly HttpConfiguration config = GlobalConfiguration.Configuration;
public void Configuration(IAppBuilder app)
{
app.UseCors(CorsOptions.AllowAll);
// c1 cloud service code here, according to suitable service
...
}
}
Here is the list of authentication mechanisms for various APIs:
AWS (Amazon Web Service)
AWS requires three credentials for authentication:
- Bucket name
- Access token
- Secret key
A Bucket can be created here. Access Token and Secret Key are available at the following link.
Region should also be set to use AWS.
Required DLL(s):
- AWSSDK.Core
- AWSSDK.S3
AWS Registration
app.UseStorageProviders().AddAWSStorage("AWS", AWSAccessTocken, AWSSecretKey, AWSBucketName, region);
AZURE
To use Azure Storage, we need to create a storage account and get the connection string by following this tutorial.
Once the connection string is retrieved, we need to create the container to store the files by following this tutorial.
Required DLL(s):
- WindowsAzure.Storage
Now that we have the required credentials and DLL to use Azure storage inside this project, we need to register Azure Storage with the required credentials using the following code snippet:
Azure Registration
app.UseStorageProviders().AddAzureStorage("Azure", AzureStorageConnectionString);
DropBox
To use the Dropbox Cloud Storage, we need to create an application to store the files. Click here to build your application.
After creating the application, we need an access token for authentication. Click here to generate the access token.
Required DLL(s):
- Dropbox.Api
DropBox Registration
app.UseStorageProviders().AddDropBoxStorage("DropBox", DropboxAppAccessToken, ApplicationName);
GoogleDrive
To work with Google Drive cloud storage, we are required to create an application and generate the credentials.json by following this link.
After generating the credentials.json, include it in the project.
Required DLL(s):
- Google.Apis
- Google.Apis.Auth
- Google.Apis.Core
- Google.Apis.Drive.v3
To access the GoogleDrive API, the credentials stored in the json file should be extracted based on the Scope of the Application. Please use the following code snippet to get user credentials.
private UserCredential GetUserCredential(string[] scopes)
{
UserCredential credential;
using (var fileStream =
new
FileStream(HttpContext.Current.Server.MapPath("credentials.json"), FileMode.Open, FileAccess.Read))
{
// The file token.json stores the user's access and refresh tokens, and is created
// automatically when the authorization flow completes for the first time.
string credPath = "token.json";
credential = GoogleWebAuthorizationBroker.AuthorizeAsync(GoogleClientSecrets.Load(fileStream).Secrets,
scopes,
"user",
CancellationToken.None,
new
FileDataStore(HttpContext.Current.Server.MapPath(credPath), true)).Result;
}
return credential;
}
Google Drive Registration
string[] scopes = { DriveService.Scope.Drive };
app.UseStorageProviders().AddGoogleDriveStorage("GoogleDrive", GetUserCredential(scopes), applicationName);
OneDrive
OneDrive provides the access token upon registering the app successfully. Get started here.
OneDrive Registration
app.UseStorageProviders().AddOneDriveStorage("OneDrive", OneDriveAccessToken);
CRUD operations on cloud storage
Read files from Cloud Storage
Once the Storage is registered, the Cloud API is accessible here.
Now that our API is ready, we can fetch the file list at the provided location. To fetch the files and folder from the targeted location, the URL will be as follows:
var baseUrl = “http://localhost /api/storage/";
var cloudStorage = "Azure/"; // Cloud Storage name from where the data should be fetched
var initPath = "apitest/Test/"; // Initial Path at the CloudStrorage where the files should be list
var fetchListUrl = _baseUrl + "List/" + _cloudStorage + initPath;
Now that the URL is ready to fetch the list from CloudStorage, we will make an ajax call to list the Data and display inside Wijmo 5 TreeView.
Ajax call to fetch the list
<script>
function fetchList() {
$.ajax({
url: fetchListUrl,
success: function (data) {
data.forEach(function (item) {
if (item.Type == 0) {
item.Childs = [];
item.ParentPath = item.Name + "/";
}
});
tree.itemsSource = data;
}
});
}
</script>
Create TreeView to show the file list
HTML: <div id="tree"></div>
JavaScript:
<script>
onload = function () {
tree = new wijmo.nav.TreeView("#tree", {
displayMemberPath: 'Name',
childItemsPath: 'Childs',
lazyLoadFunction: lazyLoadFunction
});
fetchList();
}
</script>
Reading files from sub folders
Since Cloud Storage can contain files with a subfolder, we may need to load the file list from the sub folders separately, by specifying the folder path.
<script>
function lazyLoadFunction(node, callback) {
var item = node.dataItem;
var subPath = item.ParentPath;
var url = fetchListUrl + subPath;
// if CloudStrorage is GoogleDrive
if (item.ItemID) {
url += "?itemid=" + item.ItemID;
}
$.ajax({
url: url
}).then(function (data) {
data.forEach(function (item) {
if (item.Type == 0) {
item.Childs = [];
item.ParentPath = subPath + item.Name;
} else {
item.ParentPath = subPath + "/";
}
});
callback(data);
});
}
</script>
Upload file
Note: The file is uploaded to the API using FormData. While creating FormData, we need to set the FormData parameter name as “file” to upload files correctly.
To upload the file from the Client to Cloud Storage,the following code should be used.
<script>
var operationUrl = _baseUrl + _cloudStorage + initPath;
function UploadFile() {
var item = tree.selectedItem;
if (!item) {
alert("No Item selected to uplaod path");
}
var file = document.getElementById("file").files[0];
var data = new FormData();
data.append("file", file);
var _url = operationUrl + "?subpath=" + item.ParentPath + file.name;
// if CloudStrorage is GoogleDrive
if (item.ItemID) {
_url += "?itemid=" + item.ItemID;
}
$.ajax({
url: _url,
type: 'POST',
data: data,
cache: false,
contentType: false,
processData: false
}).then(function (res) {
alert("Uploaded")
})
}
</script>
Delete File
The following code should be used to delete the file from the servers.
function DeleteFile() {
var item = tree.selectedItem;
if (!item) {
alert("No item to delete");
} else {
var path = item.ParentPath ? item.ParentPath : "";
var url = operationUrl + path + "?subpath=" + item.Name;
// if CloudStrorage is GoogleDrive
if (item.ItemID) {
url += "?itemid=" + item.ItemID;
}
$.ajax({
url: url,
type: 'DELETE',
cache: false,
contentType: false,
processData: false,
success: function (data, success, obj) {
alert("Deleted");
}
});
}
}
Download file
The following code should be used to download the file from the server.
function DownloadFile() {
var item = tree.selectedItem;
if (!item) {
alert("No item to download");
} else {
var path = item.ParentPath ? item.ParentPath : "";
var url = operationUrl + path + "?subpath=" + item.Name; var elem = wijmo.createElement("[](" + url + ")"); elem.click();
}
}
Refer to the C1WebApiExplorer demo for reference here. Get the sample application here. The GIF below can also be used as a helpful demo.
Manish Gupta