Profiles and constants
A profile can be viewed as a template for tasks, that can be parametrized by constants. Every task has a profile, and the basic thing you'll do when creating a task is to select a profile and adjust some constants.
Let's take some examples:
- you want to run some Python script of yours on
Qarnot's computing platform. After uploading your Python files and input
data to a bucket, you will select one of our
pythonprofiles, and set in the
PYTHON_SCRIPTconstant the arguments you want to pass to the
- If you want to run a container from the Docker hub, you'll select one of
dockerprofiles, and use the
DOCKER_TAGconstants to select the Docker image and tag to use.
DOCKER_CMDwill let you specify the command that you want to run in your containers.
Batch and worker profiles
There are two major kinds of profiles that you can choose from: batch profiles, and worker profiles. They differ in what they allow to do, and in the way they are invoiced when one of your instances is terminated before completion.
Termination before completion
Most of the time, Qarnot's computing nodes are located in one of our computing heaters, that are used to provide free domestic heat. We try to pay an equal attention to people we heat and to people that compute on our service, hence we try to meet temperature requests with the minimal possible impact on your computing tasks. Most of the time, this is handled by our scheduler that targets in priority the nodes that need the most heat, and then, during the task's execution, by regulating the frequency of the CPU. Sometimes however, a given computing node needs to completely stop heating. This happens mostly for tasks that run for a very long time. In those rare cases, your instance will be terminated and re-launched on another node. One of the main differences between batch and worker profiles is the way you are invoiced in such a case.
A batch profile is made to execute a program on a set of inputs (a resource bucket) and produce a set of outputs (a result bucket). Examples of this would be a blender rendering, machine learning on a fixed data set, … With such a kind of task, an instance terminated before completion did not output any result, so it produced no value for you. Hence, it is not invoiced.
For instance, let's take the following situation:
- you launched a blender rendering (only one instance for the sake of simplicity)
- it computed 2 hours on a node, then we needed to terminate it
- it was re-launched on another node, where it completed in 3 hours.
Your task has used 5 hours of computation time on our farm, 3 of which produced valuable output for you. You will only be invoiced 3 hours.
There is a limitation to benefit from such a behaviour: batch tasks do not have network access
A worker profiles is designed to let you start instances running a worker, that pulls work from and pushes results to sources external to Qarnot's computing service.
For example, you could have several instances of your application that pull work items from an instance of RabbitMQ that you host, then pull associated data form an Amazon S3 bucket, perform some computation, upload results to S3 and notify your RabbitMQ that the task is completed.
Tasks based on a worker profile have full internet access. However, as we cannot distinguish "useful" from "useless" work, everything second of computation time will be invoiced.