Learn the basics of Qarnot API and python SDK through a small batch application.

We'll look at how the script call the Qarnot's computing service to start a parallel payload contained in a basic Docker container, send and retrieve data from the payload as well as standard outputs.



  • Qarnot account: If you don't already have a Qarnot account, create one for free Qarnot account.
  • API token: Once you have a Qarnot account, retrieve your API token from the Qarnot console.

Python environment

We recommend you to setup a Python virtual environment with virtualenv. To install it, use one of the following commands for Python2:

$ apt-get install python-virtualenv 
$ easy_install virtualenv 
$ pip install virtualenv

Or for Python3:

$ apt-get install python3-venv
$ pip3 install virtualenv

Once virtualenv is installed, you can create your own environment by running the following commands in the project directory.

$ virtualenv venv 
New python executable in venv/bin/python 
Installing setuptools, pip, wheel...done.

Or with Python3:

$ python3 -m venv venv

Then each time you want to use your virtual environment you have to activate it by running this command:

$ . venv/bin/activate

Finally you have to install in your environment the Qarnot SDK:

$ pip install qarnot

If you plan to send large files to the API, we advise you to install the optional requests-toolbelt dependency in order not to overuse your memory:

$ pip install requests-toolbelt

You are now ready to use the Qarnot SDK.

Code sample

Batch Python tutorial code sample

Create a file with the following 6 lines code sample.

import qarnot
conn = qarnot.connection.Connection(client_token="xxxx_mytoken_xxxx")
task = conn.create_task('helloworld', 'docker-batch', 4)
task.constants['DOCKER_CMD'] = 'echo hello world from node ${INSTANCE_ID}!'

Prepare client script

conn = qarnot.connection.Connection(client_token='xxxx_mytoken_xxxx')

You have to replace xxxx-mytoken-xxxx by your authentication token.

Running the sample

Launch the script :


Understanding the sample

  • 1 - First, we must import the Qarnot Python SDK,
import qarnot
  • 2 - Then, we have to create a connection to Qarnot computing,
conn = qarnot.connection.Connection(client_token='xxxx_mytoken_xxxx')

Don't forget to replace xxxx-mytoken-xxxx by your own authentication API token.

  • 3 - Create a new task with,
task = conn.create_task('hello world', 'docker-batch', 1)

The created task is named 'hello world', is a 'docker-batch' application of one instance.

  • 4 - Assign the docker application constant DOCKER_CMD,
task.constants['DOCKER_CMD'] = 'echo hello world!'

The command echo hello world! will be executed on a compute node. Not locally.

  • 5 - Submit the task and wait it to complete,

The .run() method perform a blocking call.
If you want a non blocking call, use the .submit() method and .wait(1) to periodicaly check your task status.

  • 6 - Display the task standard output

Use .stderr() to display task standard error.
In the non blocking version, you can use .freshstdout() and .freshstderr() to display new fresh outpout only.