Backend Platforms - Azure Batch

Caution

Cost Implications - Azure Batch nodes incur costs all the time they are running, even when they are not running your jobs. OneCompute does not manage your compute nodes & pools.

Please take care to read the Microsoft Azure Batch documentation to understand the principles of the service and how costs are accrued.

What is Azure Batch ?

As this is a Azure service it is best described by Microsoft themselves...

Azure Batch is a platform service for running large-scale parallel and high-performance computing (HPC) applications efficiently in the cloud. Azure Batch schedules compute-intensive work to run on a managed collection of virtual machines, and can automatically scale compute resources to meet the needs of your jobs.

With Azure Batch, you can easily define Azure compute resources to execute your applications in parallel, and at scale. There's no need to manually create, configure, and manage an HPC cluster, individual virtual machines, virtual networks, or a complex job and task scheduling infrastructure. Azure Batch automates or simplifies these tasks for you.

Batch is a managed Azure service that is used for batch processing or batch computing--running a large volume of similar tasks for a desired result. Batch computing is most commonly used by organizations that regularly process, transform, and analyze large volumes of data.

Azure Batch Common Workflow

Batch works well with intrinsically parallel (also known as "embarrassingly parallel") applications and workloads. Intrinsically parallel workloads are those that are easily split into multiple tasks that perform work simultaneously on many computers.

-- Microsoft Azure Batch documentation

The Azure Batch service is being continually extended and improved by Microsoft and regular updates are bringing online new features and capabilities. You can follow these here: https://azure.microsoft.com/en-gb/updates/

Using the Azure Batch APIs you can run your compute workload on the Azure Batch service but this then tightly couples your application code to Azure Batch.

OneCompute enables you to run your compute code on Azure Batch without this tight dependency. Sure, there is work to switch your code to another platform but a great deal less than you would have ordinarily. OneCompute takes away a lot of the plumbing that otherwise you would need to write yourself.

Manage Azure Batch Resources

This documentation will highlight areas of particular concern to using Azure Batch as your compute platform. However, it is very important that you have a good understanding of the concepts of the Azure Batch service in order to understand what you are doing.

You can manage your Azure Batch account and resources via the Azure Portal.

An excellent alternative is to download and use the Azure Batch Labs tool from here - a client tool to help create, debug and monitor Azure Batch Applications.

Note

Batch Labs is a tool to manage your Azure Batch accounts. The goal is to implement a great user experience that will help you debug, monitor and manage your pools, jobs and tasks.
Azure Batch Labs Tool

Using Azure Batch as the backend platform for OneCompute.

Note

Click here for the details on how to use OneCompute to create and run a workload using Azure Batch as the back-end platform