Developers
menu
menu

Learn the basics of Qarnot API and python SDK through a small batch application.

We'll look at how the script call the Qarnot's computing service to start a parallel payload contained in a basic Docker container, send and retrieve data from the payload as well as standard outputs.

Prerequisites

Accounts

  • Qarnot account: If you don't already have a Qarnot account, create one for free Qarnot account.
  • API token: Once you have a Qarnot account, retrieve your API token from the Qarnot console.

Download and install Qarnot python SDK

Go to your favorite tmp folder and clone the Python SDK:

$ git clone https://github.com/qarnot/qarnot-sdk-python.git

Go to the Python SDK directory:

$ cd qarnot-sdk-python/ 

Python environment

We recommend you to setup a Python virtual environment with virtualenv. To install it, use one of the following commands:

$ apt-get install python-virtualenv 
$ easy_install virtualenv 
$ pip install virtualenv

Once virtualenv is installed, you can create your own environment by running the following commands in the project directory.

$ virtualenv venv 
New python executable in venv/bin/python 
Installing setuptools, pip, wheel...done.

Then each time you want to use your virtual environment you have to activate it by running this command:

$ . venv/bin/activate

Finally you have to install in your environment the required dependencies running:

$ pip install -r requirements.txt

If you plan to send large files to the API, we advise you to install the optional requests-toolbelt dependency in order not to overuse your memory:

$ pip install -r requirements-optional.txt

You are now ready to use the Qarnot SDK.

Code sample

Batch Python tutorial code sample

Create a helloworld.py file with the following 6 lines code sample.

import qarnot
conn = qarnot.connection.Connection(client_token="xxxx_mytoken_xxxx")
task = conn.create_task('helloworld', 'docker-batch', 4)
task.constants['DOCKER_CMD'] = 'echo hello world from node ${INSTANCE_ID}!'
task.run()
print(task.stdout())

Prepare client script

conn = qarnot.connection.Connection(client_token='xxxx_mytoken_xxxx')

You have to replace xxxx-mytoken-xxxx by your authentication token.

Running the sample

Launch the script :

python helloworld.py

Understanding the sample

  • 1 - First, we must import the Qarnot Python SDK,
import qarnot
  • 2 - Then, we have to create a connection to Qarnot computing,
conn = qarnot.connection.Connection(client_token='xxxx_mytoken_xxxx')

Don't forget to replace xxxx-mytoken-xxxx by your own authentication API token.

  • 3 - Create a new task with,
task = conn.create_task('hello world', 'docker-batch', 1)

The created task is named 'hello world', is a 'docker-batch' application of one instance.

  • 4 - Assign the docker application constant DOCKER_CMD,
task.constants['DOCKER_CMD'] = 'echo hello world!'

The command echo hello world! will be executed on a compute node. Not locally.

  • 5 - Submit the task and wait it to complete,
task.run()

The .run() method perform a blocking call.
If you want a non blocking call, use the .submit() method and .wait(1) to periodicaly check your task status.

  • 6 - Display the task standard output
print(task.stdout())

Use .stderr() to display task standard error.
In the non blocking version, you can use .fresh_stdout() and .fresh_stderr() to display new fresh outpout only.