Connection

Module describing a connection.

class qarnot.connection.Connection(fileconf=None, client_token=None, cluster_url=None, cluster_unsafe=False, cluster_timeout=None, storage_url=None, storage_unsafe=False, retry_count=5, retry_wait=1.0)[source]

Bases: object

Represents the couple cluster/user to which submit tasks.

__init__(fileconf=None, client_token=None, cluster_url=None, cluster_unsafe=False, cluster_timeout=None, storage_url=None, storage_unsafe=False, retry_count=5, retry_wait=1.0)[source]

Create a connection to a cluster with given config file, options or environment variables. Available environment variable are QARNOT_CLUSTER_URL, QARNOT_CLUSTER_UNSAFE, QARNOT_CLUSTER_TIMEOUT and QARNOT_CLIENT_TOKEN.

Parameters:
  • fileconf (str or dict) – path to a qarnot configuration file or a corresponding dict
  • client_token (str) – API Token
  • cluster_url (str) – (optional) Cluster url.
  • cluster_unsafe (bool) – (optional) Disable certificate check
  • cluster_timeout (int) – (optional) Timeout value for every request
  • storage_url (str) – (optional) Storage service url.
  • storage_unsafe (bool) – (optional) Disable certificate check
  • retry_count (int) – (optional) ConnectionError retry count. Default to 5.
  • retry_wait (float) – (optional) Retry on error wait time, progressive. (wait * (retry_count - retry_num). Default to 1s

Configuration sample:

[cluster]
# url of the REST API
url=https://localhost
# No SSL verification ?
unsafe=False
[client]
# auth string of the client
token=login
[storage]
url=https://storage
unsafe=False
s3client

Pre-configured s3 client object.

Return type:list(S3.Client)
Returns:A list of ObjectSummary resources
s3resource

Pre-configured s3 resource object.

Return type:list(S3.ServiceResource)
Returns:A list of ObjectSummary resources
user_info

Get information of the current user on the cluster.

Return type:

UserInfo

Returns:

Requested information.

Raises:
buckets()[source]

Get the list of buckets.

Return type:list(class:~qarnot.bucket.Bucket).
Returns:List of buckets
disks()[source]

Get the list of disks on this cluster for this user.

Return type:

list(Disk).

Returns:

Disks on the cluster owned by the user.

Raises:
pools(summary=True)[source]

Get the list of pools stored on this cluster for this user.

Return type:

List of Pool.

Returns:

Pools stored on the cluster owned by the user.

Raises:
tasks(tags=None, summary=True)[source]

Get the list of tasks stored on this cluster for this user.

:param List of str tags: Desired filtering tags :rtype: List of Task. :returns: Tasks stored on the cluster owned by the user.

Raises:
retrieve_pool(uuid)[source]

Retrieve a qarnot.pool.Pool from its uuid

Parameters:

uuid (str) – Desired pool uuid

Return type:

Pool

Returns:

Existing pool defined by the given uuid

Raises:
retrieve_task(uuid)[source]

Retrieve a qarnot.task.Task from its uuid

Parameters:

uuid (str) – Desired task uuid

Return type:

Task

Returns:

Existing task defined by the given uuid

Raises:
retrieve_or_create_bucket(uuid)[source]

Retrieve a Bucket from its description, or create a new one.

Parameters:uuid (str) – the bucket uuid (name)
Return type:Bucket
Returns:Existing or newly created bucket defined by the given name
retrieve_or_create_disk(description)[source]

Retrieve a Disk from its description, or create a new one.

Note

Description are not unique, if multiple description match, an exception will be raised

Parameters:

description (str) – a short description of the disk

Return type:

Disk

Returns:

Existing or newly created disk defined by the given description

Raises:
retrieve_bucket(uuid)[source]

Retrieve a Bucket from its uuid (name)

Parameters:uuid (str) – Desired disk uuid (name)
Return type:Bucket
Returns:Existing bucket defined by the given uuid (name)
Raises:botocore.exceptions.ClientError: Bucket does not exist, or invalid credentials
retrieve_disk(uuid)[source]

Retrieve a Disk from its uuid

Parameters:

uuid (str) – Desired disk uuid

Return type:

Disk

Returns:

Existing disk defined by the given uuid

Raises:
create_disk(description, lock=False, tags=None)[source]

Create a new Disk.

Parameters:
  • description (str) – a short description of the disk
  • lock (bool) – prevents the disk to be removed accidentally
  • tags (list(str)) – custom tags
Return type:

qarnot.disk.Disk

Returns:

The created Disk.

Raises:
create_pool(name, profile, instancecount=1, shortname=None)[source]

Create a new Pool.

Parameters:
  • name (str) – given name of the pool
  • profile (str) – which profile to use with this pool
  • instancecount (int) – number of instances to run for the pool
  • shortname (str) – optional unique friendly shortname of the pool
Return type:

Pool

Returns:

The created Pool.

Note

See available profiles with profiles().

create_task(name, profile_or_pool, instancecount_or_range=1, shortname=None)[source]

Create a new Task.

Parameters:
  • name (str) – given name of the task
  • profile_or_pool (str or Pool) – which profile to use with this task, or which Pool to run task
  • instancecount_or_range (int or str) – number of instances, or ranges on which to run task. Defaults to 1.
  • shortname (str) – optional unique friendly shortname of the task
Return type:

Task

Returns:

The created Task.

Note

See available profiles with profiles().

submit_tasks(tasks)[source]

Submit a list of Task.

:param List of Task. :raises qarnot.exceptions.QarnotGenericException: API general error, see message for details

Note

Will ensure all added files are on the resource disk regardless of their uploading mode.

profiles()[source]

Get list of profiles available on the cluster.

Return type:

list of Profile

Raises:
retrieve_profile(name)[source]

Get details of a profile from its name.

Return type:

Profile

Raises:
create_bucket(name)[source]

Create a new Bucket.

Parameters:name (str) – bucket name
Return type:qarnot.bucket.Bucket
Returns:The created Bucket.
class qarnot.connection.UserInfo(info)[source]

Bases: object

Information about a qarnot user.

email = None
Type:str

User email address.

disk_count = None
Type:int

Number of disks owned by the user.

max_disk = None
Type:int

Maximum number of disks allowed (resource and result disks).

max_bucket = None
Type:int

Maximum number of buckets allowed (resource and result buckets).

quota_bytes_disk = None
Type:int

Total storage space allowed for the user’s disks (in Bytes).

quota_bytes_bucket = None
Type:int

Total storage space allowed for the user’s buckets (in Bytes).

used_quota_bytes_disk = None
Type:int

Total storage space used by the user’s disks (in Bytes).

used_quota_bytes_bucket = None
Type:int

Total storage space used by the user’s buckets (in Bytes).

task_count = None
Type:int

Total number of tasks belonging to the user.

max_task = None
Type:int

Maximum number of tasks the user is allowed to create.

running_task_count = None
Type:int

Number of tasks currently in ‘Submitted’ state.

max_running_task = None
Type:int

Maximum number of running tasks.

max_instances = None
Type:int

Maximum number of instances.

class qarnot.connection.Profile(info)[source]

Bases: object

Information about a profile.

name = None
Type:str

Name of the profile.

constants = None
Type:List of (str, str)

List of couples (name, value) representing constants for this profile and their default values.