A task is the object you use to describe a computation on Qarnot's computing platform.
A task is described by the following properties:
- a UUID, to uniquely idenfity the task
- a name, so you can easily identify your tasks when you use the API, one of our SDKs, or the console
- a profile describing the nature of the task (e.g Blender, Docker)
- a pool, it is an open session with its own profile and a set of instances where the task will be run. A task with a pool will use the pool profile and does not need a personal profile.
- constants definitions (see profiles), used to parametrize the profile (e.g the blender file to render, the docker image to use, the command to execute, or a pool UUID, see profiles, …)
- a certain number of instances or a given range, describing how many instances of the program should be run
- input resources (one or several API bucket), that will be available for your program to use (see environment for further details on how to access them from your program)
- a job so you can group a set of tasks into a particular object. It allows the tasks to be dependent on one another and use the task dependencies to manage the flow of your tasks.
- task dependencies, a list of task to wait before running the actual task. It links task work when a task needs values calculated by other tasks to compute. To be used, all the tasks must belong to the same job with the "use dependencies" flag activated.
During the execution, the task will have several other properties:
- a state, among the following values. The order in which they are represented here also reflects the life cycle of the task.
||The task has been submitted|
||Some instances have been dispatched to computing nodes, but none is running yet|
||All your instances have been dispatched, but none is running yet|
||Some instances of the task are running, but some are still in queue|
||All the remaining instances of the task are running (some may have completed already)|
||The task is complete and results are being uploaded into a bucket|
||The task is currently in the cancellation process.|
||The task has been canceled by the user|
||The task has finished and every instance have exited with a zero (0) exit code|
||The task has finished but at least one instances has exited with a non-zero exit code|
||The task is currently in the deletion process.|
- the progress of the computations: a percentage, which instances have completed, have failed, are running, are waiting
- some details about the running instances, like:
- CPU usage, nominal and actual frequency (see environment)
- RAM size and usage
- the cumulated CPU time (the one that is invoiced), and a human wall time from task creation
When the task has completed, you'll have access to your results, in the form of a bucket, and the execution time of the task will be definitely invoiced.
A task is composed of one or several instances. Each instance will execute on its own computing node, and every instance will have the same profile, input data and constants, to the exception of:
INSTANCE_IDconstant, which is set to the instance number (see profiles for details on constants)
INSTANCE_IDenvironment variable (see environment for details on the execution environment)
When creating a task, you can specify a number of instances, or you can be more specific on which values
INSTANCE_ID will take
by using a range.
Parallelism between instances
As much as possible, the instances of your tasks will run in parallel on different nodes. If not enough compute nodes are available or if your maximum simultaneous CPU limit has been reached, the remaining instances are put in queue by the Qarnot scheduler. They will be dispatched when free compute nodes become available, or some of your other frames complete.
Instances are running on different geographical locations, so running communicating applications such as MPI is not possible out of the box. MPI tasks are supported but have not been made available for public use. You can contact us if you need to run this kind of tasks.
Failure or success is determined by your program's return code: zero (0) for success, non-zero for failure.
You can access your application standard output and error through our REST API.
Please note that they are reported with a best effort logic. If the log rate is too high,
some parts will be truncated and replaced by
REST API error codes: If you encounter a REST API error, you can refer to the common errors page.
- Read the pools section.