OneCompute Release Notes
Known issues
Release History
- Version 3.0 - April 1, 2020
- Version 2.2.2 - November 24, 2019
- Version 2.2.1 - October 14, 2019
- Version 2.2 - July 12, 2019
- Version 2.1 - April 30, 2019
- Version 2.0 - February 13, 2019
- Version 1.2.2 - June 30, 2018
- Version 1.2.1 - April 5, 2018
- Version 1.2 - March 30, 2018
- Version 1.1.1 - February 5, 2018
- Version 1.1 - November 22, 2017
Version 3.0 - April 1, 2020
The 3.0 release of OneCompute focused on enhancements to OneCompute Platform:
- Accounts and consumption control
- Improved consumption reports
- Result Lake: providing scalable storage of result files
- Miscellaneous usability enhancements to the OneCompute Portal
Changes
All NuGet package references have been upgraded to their latest versions in this release. Microsoft.WindowsAzure. Storage has changed to Microsoft.Azure.Storage.DataMovement & Microsoft.Azure.Storage.Blob. As of Microsoft.Azure.Storage v9.4 Table API support has been dropped. Table API support is now provided by the Microsoft.Azure.Cosmos.Table library. Applications using Azure storage may need to upgrade to these libraries.
There are a few breaking changes to the OneComputePlatformClient. See list of Enhancements below.
Enhancements
- New home page with usage data.
- Provide link from job directly to corresponding storage folder
- Scalable results storage, see Result Lake
- General Portal Improvements
- Implementation of Accounts
- Limit Credit consumption
- Limit Storage access
- Changes to the OC platforms
- Storage usage visible for users
- Extend portal reports and UI to account for consumption limits
- Add transactions into One Compute
- Access to Results Lake output files, see Using the OneCompute Result Lake
- The WorkUnit.Data property was removed. Its usage was to provide access to application input, which now can retrieved by using the WorkUnit.GetInput method.
- The asynchronous methods of OneComputePlatformClient was renamed with the Async posfix to give room for corresponding syncrhonous methods, in preparation for clients that require synchronous methods.
Defects
- Usage report error when user has no usage
- Portal: "Jobs" table does not show properly
- Unhandled authentication exception for external user
- Portal: Security; Penetration testing vulnerability 2.4.1.1 - W - A2-Broken Authentication and Session Management - Incorrect session management
- Portal: Size sorting is not working properly
- Portal: After "Log out" the user is not logged out nor redirected to home page
- OneCompute portal- Operation: Wrong VM sizes displayed
- Batch: File transfer to node fails occasionally
- Access denied to job details through developer portal
- Portal: Run time is not updated in Jobs view while job is running
- Portal: Applications; Upload fails when package version contains a space
- Exception for a non-Verit user when accepting terms and conditions in the OCP portal
- Uploading big application package fails with HTTP error code 500
- Exception in JobScheduler when updating work item properties
- Credits are not properly accounted for in a faulted job
- West Europe production portal fails
- Errors experienced by a user that has no jobs or container content.
- Cloud storage navigation fails for users with high storage
- The home page is crashing in the production North-Europe environment.
- Dev+Test Prevent web jobs running on Structure compute portal resources
- Jobs getting stuck in states instead of moving to completed
- CSV report has column header Cost instead of Credits
- The Add-Application page fails to validate form inputs
- Pools listed in Application Default Pool drop down wrong
Version 2.2.2 - November 24, 2019
This is a patch release of version 2.2.
Changes
Defects fixed
-Job must be initialized before scheduled
Version 2.2.1 - October 14, 2019
This is a patch release of version 2.2.
Changes
Defects fixed
-Exception in JobScheduler when updating work item properties
Version 2.2 - July 12, 2019
The 2.2 release of OneCompute focused on
- Portal accessibility
- Following up on findings from penetration tests
- Following up on findings from security review
Changes
Enhancements
- Azure Batch should wait for start task to finish
- Make job upload more resilient against network errors
- Implement support for Dockerised worker containers
- Add time zone to 'last activation time' when adding / modifying package.
- Security: Fill out DTC's ASSA (Application Security Self-Assessment) form prior to external penetration test
- PoC: Calling OneCompute API from a service
- Penetration testing follow up
- Security review follow up
- Increase ingress and egress limits for storage accounts
- Remove user storage when removing a user
- Improve input files mechanism to match output files
- Job preparation task
- Job release task
Defects
- The WorkerHost project depends on Azure
- AuthRedirectUri app setting not set in deployment scripts
- Question mark link in header is not working
- JobScheduler crashes when the size of the resource files collection is > 32K
- Account admins not assigned to any account can see all data
- User search box is not available anymore
- Should prevent unauthorized access to blob container
- Access to other users' storage through editing user in portal
- Prevent XSS attack in blob names
- Deleting multiple files from the cloud container fails
- OneCompute Platform API URL is wrong
- User registration fails after adding service principal user
- Sesam regression tests fail due to failure to authenticate the test user
- Job serialization problem
- Data transfer Including filters is picking up all files rather than the filtered
- Prevent worker host service reporting missing optional "ITelemetryClient"
- Delete selected blob storage fails
- Changing default version leads to error message
- Error when nothing is typed for the new application id
- Unable to delete selected content when "sibling", unselected content is present
Version 2.1 - April 30, 2019
The 2.1 release of OneCompute focused on
- Consumption reporting in the OneCompute Portal
- Ability to specify default pools for Batch applications
- Enhanced pool management from the OneCompute portal
- Application properties on jobs and work items
Changes
Enhancements
- Application properties for jobs and work items
- Report data storage used per company and per user
- It should be possible for the users to see usage for their account (CPU hours and data storage) per month
- Report 'all companies' should show usage per company
- Accounting Report: Display credits value in the Usage table
- Change "created" status to "waiting to start" or similar
- Specify default pool per application
- Allow updating Terms of Use / Click To Accept without deploying portal
- Create "Company accountant" role
- It should be possible for the users to see usage for their account (Credits) per job
- Provide a dashboard for the user showing the company's usage so far this month, as well as their (company's) consumption limits.
- Portal: Jobs: Bring back show run/simulation time for the job
- Report more details for usage per company
- Add more options to the Operation tab
- Show current status of cloud nodes
- Store and retrieve job information
- Store and retrieve Work Item properties
- Portal: Jobs: Show overview of current and recently completed jobs
- Portal: Jobs: Administrator should be able to view jobs across all users
Defects fixed
- OneComputePlatform test portal: "Show user storage per user" gives server error
- The current consumption .CSV file can not be opened in Excel
- The reduction task of a ParallelWork fails to run in Azure Batch
- Authorization not working in swagger
Version 2.0 - February 13, 2019
The 2.0 release of OneCompute focused on
- Establishing OneCompute Platform, a shared, scalable compute service running in Azure.
- Increasing scalability and robustness of OneCompute components.
Features of OneCompute Platform
- OneCompute API with secure access through Veracity
- Schedule jobs for execution
- Monitor job execution
- Get job status and progress
- Provide secure access to a private blob container for every user
- OneCompute Portal with secure login through Veracity and role based security
- Every user can browse their own blob container and download files through the OneCompute Portal
- Every user can list their jobs
- Every user can get a consumption report
- Accountants can get consumption reports for companies and users
- Operators can manage compute pools
- Application Owners can manage applications
- User Managers can manage user access to applications
Changes
Enhancements
- Orchestration options on Job for error handling - Implementation Azure batch and console
- Extend WorkItem with description
- Allow jobs to run to completion even if one or more of the work units fail
- Get work items without deserializing json content
- Graceful shutdown and restart of Job Scheduler web job
- Application ownership implementation
- Assign the job to the pool
- Add ability to delete application packages in OneCompute portal
- Only send activation emails to user management role
- Create a user management role
- Also send activation emails to a specified additional non-user email address
- Remove multiple workspace at once
- Add async bulk insert method to AzureStorageTableHelper
- Allow files to be excluded from file upload after the work unit completes
- BlobStorageFileTransferService should invoke events on the original thread
- Accounting reports should report consumption in the form of credits used, not Core-hours
- Show credits for old jobs
- Store CPU consumption for on-going jobs, not just per finished job
- Application Owner can edit credit per consumption unit for an application
- Operator can edit CPU cost factor on a pool
- Aggregation of consumption and credits per job
- List jobs per user
Defects fixed
- Cancelling an Azure Batch job does not work
- Job occasionally fails to complete even if all work units are completed
- Exception while monitoring Batch tasks terminates job
- The OneCompute storage connection string is exposed in the console output from the WorkerHost
- BlobFileTransferService fails to transfer files at high loads
- File transfer occasionally hangs when running at high load
- Unclear error message when trying to run in cloud without account
- Make AzureBatchWorkItemScheduler more resilient when scheduling a large job
- Status updates are lost in the WorkerExecutor
- When Azure Batch fails to run a CloudTask, the error information is not stored in the work item status table
- Fix Azure application package deployment script
- Job scheduling: Input files with space are not copied to compute node
- Task that fails to upload all files to cloud storage should fail
- Count time during copying files to Storage as part of task duration
Version 1.2.2 - June 30, 2018
This is a patch release of version 1.2.
Changes
Defects fixed
- Aggregate CPU time as long
- WorkUnit is completed before files are uploaded and result stored
- Log the Job.ServiceName to the JobStatus record
- Add a PoolId property on the job to allow the client to choose backend pool to run on
- When a work unit fails, the file transfer is cancelled
- Log which pool is used
Version 1.2.1 - April 5, 2018
This is a patch release of version 1.2.
Changes
Enhancements
Version 1.2 - March 30, 2018
The 1.2 release has focused on:
- Fixing defects
- Minor enhancements
- Enhancing the production environment of OneCompute
- Adding automatic regression testing of the supported back-ends
- Automatic build of examples and VSIX projects
- Standardizing on .NET Standard 2.0
- Standardizing on the new C# project format that simplifies NuGet packaging and avoids package dependency inconsistencies
Changes
Enhancements
- Scheduling a job with no user Id specified should throw a descriptive exception
- Reading job status for a non-existing job should return null
- Upgrade solution to .NET Standard
- Update third party NuGet dependencies to latest versions
- Correlate Azure Batch job id with 1C job id
- Support custom Azure Batch Worker Hosts
- Azure Batch Deployment Tools
- Allow user identities which are not email addresses. This enables an application to replace the default user id validator, which requires the user id to be a valid email address, with an alternative id validator. The application may provide a custom validator. OneCompute 1.2 provides a GUID validator, which requires the user id to be a GUID.
Defects fixed
- BlobStorageFileTransferService generates incorrect blob names
- Job with WorkUnit that fails is recorded with status = Completed. Should be Faulted
Version 1.1.1 - February 5, 2018
This is a patch (bug fix release of version 1.1).
Changes
Version 1.1 - November 22, 2017
This was the initial release of OneCompute.