Reduction

Overview

OneCompute provides the ability to apply a reduction step after a set of dependent work items have been processed. This is useful when you have a number of tasks that can be executed in parallel, but require a final step to process the results. OneCompute will ensure that all of the dependent work items are processed in parallel first before executing the reduction step.

Creating a reduction task

The following code snippet shows how you can create a reduction task, set its dependencies and associate it with a set of parallel work:

// Define the work to be done.
List<WorkItem> workItems = DefineWork();

// Create a reduction task.
var reductionTask = new WorkUnit();

// Configure the work items to run in parallel first,
// and then run the reduction task at the end.
var work = new ParallelWork
{
    WorkItems = workItems,
    ReductionTask = reductionTask
};

Notice that dependencies will be inferred by OneCompute. The dependencies of a reduction task of a ParallelWork are the WorkItems of the ParallelWork.

Processing a reduction task in a Worker

The WorkItemScheduler will schedule the reduction task to run after all the dependent parallel work items have been processed. The Worker will be called with the reduction task and the results of all of its dependent tasks. The Worker is responsible for processing the dependent task results in whatever manner makes sense for your scenario and producing a "reduced" result.

Integrating reduction in the ExecuteAsync method

The following code snippet shows how you might create a Worker implementation that processes a reduction task in the ExecuteAsync method:

public async Task<object> ExecuteAsync(
    IWorkerExecutionStatusNotificationService statusNotificationService,
    IWorkUnit workUnit, IEnumerable<IResult> dependencyResults)
{
    ReducedResult reducedResult = null;

    if (workUnit.IsReductionTask)
    {
        // Do some meaningful reduction with the dependency results.
        reducedResult = Reduce(dependencyResults);
    }
    else
    {
        await Logger.LogWarningAsync("MyReductionWorker", "Work unit is not a reduction task.");
    }

    return reducedResult;
}

In this case, the Worker is only responsible for processing reduction tasks, but this is not a requirement. An alternative approach would be to process reduction and non-reduction tasks in the same Worker implementation.

Support reduction by implementing the ISupportReduction interface

Another way to implement a worker with reduction support is to implement the ISupportReduction interface:

public async Task<object> ReduceAsync(
    IWorkerExecutionStatusNotificationService statusNotificationService,
    IWorkUnit workUnit, IEnumerable<IResult> dependencyResults)
{
    ReducedResult reducedResult = null;

    // Do some meaningful reduction with the dependency results.
    return Reduce(dependencyResults);
}

This approach separates the reduction functionality from the normal execution.