Backend Platforms - REST

Overview

OneCompute provides the ability to invoke plain REST services as part of a computational work flow. This is supported by the concepts of WorkerHosts & Workers.

This means that you can take the same worker that you have running on Azure Batch or Console and move it onto a WebAPI endpoint.

There are two approaches to this, traditional REST WebAPI endpoint and the alternative of the "Serverless" code architecture.

OneCompute aware REST WebAPI endpoint

The most flexible way of using a REST endpoint as a means of hosting a OneCompute workload is by using an Azure App Service. In this situation, when scheduling a work unit for execution, OneCompute would call the REST service. It would pass the data of the work unit as the message body to the web service and post the response from the web service as the result.

For this to happen, the REST endpoint has to understand OneCompute concepts such as WorkUnit.

At the moment, we do not have good deployment tooling around this area - this is something we believe that can be improved in the future. We recommend creating an Azure AppService and using the ASP.NET Core (.NET 4.6.1) REST Worker Host application we have created to run your workers.

You can obtain this source code here: https://dnvgl-one.visualstudio.com/OneFoundation/One%20Compute%20Team/_git/OneCompute?path=%2Fsrc%2FWorkerHosts%2FRest%2FDNVGL.One.Compute.WorkerHost.Rest&version=GBmaster

Please contact one of the team for support in doing this.

This provides you with a REST endpoint that can accept OneCompute WorkUnits and execute them using your worker.

For the most part, your client code will be similar to the Azure Batch code (which is described in detail here. We will only describe where the process differs.

Under the REST WorkerHost you do not necessarily need:-

• An Azure Batch account (although you will if you want to switch between REST & Azure Batch runs).
• Not applicable TASK "Worker Project: Deploy Worker packages to Azure Batch"
• Not applicable TASK "Client Application: Specify the Application Packages that the job will utilize"
• Not applicable TASK "Client Application: Configure Azure Batch Service Credentials"

Storage Services

The REST worker host requires Azure Storage as the storage service. It only supports the environment variable based approach to obtaining the storage connection string. If hosting on an Azure App Service, define this in the Application Settings blade, in the App Settings section (rather than Connection Strings section) as property 'OneComputeStorageConnectionString'.

Client Application: Create a OneCompute Scheduler for interaction with the REST Host

OneCompute has a scheduler class RestWorkItemScheduler that will schedule units of work on a OneCompute REST worker host service.

/// <summary>
/// Initializes a new instance of the <see cref="RestWorkItemScheduler" /> class.
/// </summary>
/// <param name="workItemStorageService">The work item storage service.</param>
/// <param name="workItemStatusService">The task status service.</param>
/// <param name="resultStorageService">The result storage service.</param>
/// <param name="urlEndPoint">Url of the endpoint</param>
public RestWorkItemScheduler(IFlowModelStorageService<WorkItem> workItemStorageService, IWorkItemStatusService workItemStatusService, IFlowModelStorageService<Result> resultStorageService, string urlEndPoint)

The first 3 parameters relate to storage services. For detail on this please see read help specific to storage services here.

The 4th parameter is the URL of the endpoint.

var url = "http://myworkerhostapi.azurewebsites.net/api/v1/OneCompute/RunWorkUnit";
var restScheduler = new RestWorkItemScheduler(
                        workItemStorageService,
                        workItemStatusService,
                        resultStorageService,
                        url)
                        {
                            Options = RestWorkItemScheduler.RequestOptions.PassWorkUnit
                        };

Note that we set the Options setting to PassWorkUnit. This informs the scheduler that we are calling a OneCompute aware end point as opposed to a "Serverless" one.

Serverless REST endpoint

Azure Functions & AWS Lambda provide the ability to execute "lightweight" code blocks via web hooks. There is no necessity to manage a host for it, the computing resources are provided on demand. These code blocks can be written in various languages C#, Python, JavaScript etc.

OneCompute has limited support for using these as hosts for your workloads.

There is less up front configuration capability with serverless hosts so your endpoints are not OneCompute aware e.g. they have no understanding of a WorkUnit.

Again you would use the RestWorkItemScheduler with Azure Storage services. But this time you would set the Options setting to PassContent.

var url="";
var restScheduler = new RestWorkItemScheduler(
                            workItemStorageService,
                            workItemStatusService,
                            resultsStorageService,
                            url)
                            {
                                Options = RestWorkItemScheduler.RequestOptions.PassContent
                            };