chris

ChRIS Python client library built on aiohttp (async HTTP client) and pyserde (dataclasses deserializer).

Installation

Requires Python 3.10 or 3.11.

pip install aiochris
# or
poetry add aiochris

Brief Example

from chris import ChrisClient

chris = await ChrisClient.from_login(
    username='chris',
    password='chris1234',
    url='https://cube.chrisproject.org/api/v1/'
)
dircopy = await chris.search_plugins(name_exact='pl-brainmgz', version='2.0.3').get_only()
plinst = await dircopy.create_instance(compute_resource_name='host')
await plinst.set(title="copies brain image files into feed")

Introduction

aiochris provides three core classes: AnonChrisClient, ChrisClient, and ChrisAdminClient. These clients differ in permissions.

Methods are only defined for what the client has permission to see or do.

anon_client = await AnonChrisClient.from_url('https://cube.chrisproject.org/api/v1/')
# ok: can search for plugins without logging in...
plugin = await anon_client.search_plugins(name_exact='pl-mri10yr06mo01da_normal').first()
# IMPOSSIBLE! AnonChrisClient.create_instance not defined...
await plugin.create_instance()

# IMPOSSIBLE! authentication required for ChrisClient
authed_client = await ChrisClient.from_url('https://cube.chrisproject.org/api/v1/')
authed_client = await ChrisClient.from_login(
    url='https://cube.chrisproject.org/api/v1/',
    username='chris',
    password='chris1234'
)
# authenticated client can also search for plugins
plugin = await authed_client.search_plugins(name_exact='pl-mri10yr06mo01da_normal').first()
await plugin.create_instance()  # works!

Client Constructors

Working with aiohttp

aiochris hides the implementation detail that it is built upon aiohttp, however one thing is important to keep in mind: be sure to call ChrisClient.close at the end of your program.

chris = await ChrisClient.from_login(...)
# -- snip --
await chris.close()

You can also use an asynchronous context manager.

async with (await ChrisClient.from_login(...)) as chris:
    await chris.upload_file('./something.dat', 'something.dat')
    ...

Efficiency with Multiple Clients

If using more than one aiohttp client in an application, it's more efficient to use the same connector. One connector instance should be shared among every client object, including all aiochris clients and other aiohttp clients.

Example: efficiently using multiple aiohttp clients

import aiohttp
from chris import ChrisClient

with aiohttp.TCPConnector() as connector:
    chris_client1 = await ChrisClient.from_login(
        url='https://example.com/cube/api/v1/',
        username='user1',
        password='user1234',
        connector=connector,
        connector_owner=False
    )
    chris_client2 = await ChrisClient.from_login(
        url='https://example.com/cube/api/v1/',
        username='user2',
        password='user4321',
        connector=connector,
        connector_owner=False
    )
    plain_http_client = aiohttp.ClientSession(connector=connector, connector_owner=False)

Advice for Getting Started

Searching for things (plugins, plugin instances, files) in CUBE is a common task, and CUBE often returns multiple items per response. Hence, it is important to understand how the Search helper class works. It simplifies how we interact with paginated collection responses from CUBE.

When performing batch operations, use asyncio.gather to run async functions concurrently.

aiochris uses many generic types, so it is recommended you use an IDE with good support for type hints, such as PyCharm or VSCodium with Pylance configured.

Examples

Create a client given username and password

from chris import ChrisClient

chris = await ChrisClient.from_login(
    url='https://cube.chrisproject.org/api/v1/',
    username='chris',
    password='chris1234'
)

Search for a plugin

# it's recommended to specify plugin version
plugin = await chris.search_plugins(name_exact="pl-dcm2niix", version="0.1.0").get_only()

# but if you don't care about plugin version...
plugin = await chris.search_plugins(name_exact="pl-dcm2niix").first()

Create a feed by uploading a file

uploaded_file = await chris.upload_file('./brain.nii', 'my_experiment/brain.nii')
dircopy = await chris.search_plugins(name_exact='pl-dircopy', version="2.1.1").get_only()
plinst = await dircopy.create_instance(dir=uploaded_file.parent)
feed = await plinst.get_feed()
await feed.set(name="An experiment on uploaded file brain.nii")

Run a ds-type ChRIS plugin

# search for plugin to run
plugin = await chris.search_plugins(name_exact="pl-dcm2niix", version="0.1.0").get_only()

# search for parent node 
previous = await chris.plugin_instances(id=44).get_only()

await plugin.create_instance(
    previous=previous,               # required. alternatively, specify previous_id
    title="convert DICOM to NIFTI",  # optional
    compute_resource_name="galena",  # optional
    memory_limit="2000Mi",           # optional
    d=9,                             # optional parameter of pl-dcm2niix
)

Search for plugin instances

finished_freesurfers = chris.plugin_instances(
    plugin_name_exact='pl-fshack',
    status='finishedSuccessfully'
)
async for p in finished_freesurfers:
    print(f'"{p.title}" finished on date: {p.end_date}')

Delete all plugin instances with a given title, in parallel

import asyncio
from chris import acollect

search = chris.plugin_instances(title="delete me")
plugin_instances = await acollect(search)
await asyncio.gather(*(p.delete() for p in plugin_instances))

Enable Debug Logging

A log message will be printed to stderr before every HTTP request is sent.

import logging

logging.basicConfig(level=logging.DEBUG)
 1"""
 2.. include:: ./home.md
 3
 4.. include:: ./examples.md
 5"""
 6
 7import chris.client
 8import chris.models
 9import chris.util
10from chris.util.search import Search, acollect
11from chris.client.normal import ChrisClient
12from chris.client.anon import AnonChrisClient
13from chris.client.admin import ChrisAdminClient
14from chris.models.enums import Status, ParameterTypeName
15
16__all__ = [
17    "AnonChrisClient",
18    "ChrisClient",
19    "ChrisAdminClient",
20    "Search",
21    "acollect",
22    "Status",
23    "ParameterTypeName",
24    "client",
25    "models",
26    "util",
27]
13class AnonChrisClient(BaseChrisClient[AnonymousCollectionLinks, "AnonChrisClient"]):
14    """
15    An anonymous ChRIS client. It has access to read-only GET resources,
16    such as being able to search for plugins.
17    """
18
19    @classmethod
20    async def from_url(
21        cls,
22        url: str,
23        max_search_requests: int = 100,
24        connector: Optional[aiohttp.BaseConnector] = None,
25        connector_owner: bool = True,
26    ) -> "AnonChrisClient":
27        """
28        Create an anonymous client.
29
30        See `chris.client.base.BaseChrisClient.new` for parameter documentation.
31        """
32        return await cls.new(
33            url=url,
34            max_search_requests=max_search_requests,
35            connector=connector,
36            connector_owner=connector_owner,
37        )
38
39    @http.search("plugins")
40    def search_plugins(self, **query) -> Search[PublicPlugin]:
41        """
42        Search for plugins.
43
44        Since this client is not logged in, you cannot create plugin instances using this client.
45        """
46        ...

An anonymous ChRIS client. It has access to read-only GET resources, such as being able to search for plugins.

@classmethod
async def from_url( cls, url: str, max_search_requests: int = 100, connector: Optional[aiohttp.connector.BaseConnector] = None, connector_owner: bool = True) -> chris.AnonChrisClient:
19    @classmethod
20    async def from_url(
21        cls,
22        url: str,
23        max_search_requests: int = 100,
24        connector: Optional[aiohttp.BaseConnector] = None,
25        connector_owner: bool = True,
26    ) -> "AnonChrisClient":
27        """
28        Create an anonymous client.
29
30        See `chris.client.base.BaseChrisClient.new` for parameter documentation.
31        """
32        return await cls.new(
33            url=url,
34            max_search_requests=max_search_requests,
35            connector=connector,
36            connector_owner=connector_owner,
37        )

Create an anonymous client.

See chris.client.base.BaseChrisClient.new for parameter documentation.

@http.search('plugins')
def search_plugins( self, **query) -> chris.Search[chris.models.public.PublicPlugin]:
39    @http.search("plugins")
40    def search_plugins(self, **query) -> Search[PublicPlugin]:
41        """
42        Search for plugins.
43
44        Since this client is not logged in, you cannot create plugin instances using this client.
45        """
46        ...

Search for plugins.

Since this client is not logged in, you cannot create plugin instances using this client.

Inherited Members
chris.link.collection_client.CollectionJsonApiClient
CollectionJsonApiClient
url
chris.client.base.BaseChrisClient
new
close
chris.link.linked.Linked
max_search_requests
15class ChrisClient(AuthenticatedClient[CollectionLinks, "ChrisClient"]):
16    """
17    A normal user *ChRIS* client, who may upload files and create plugin instances.
18    """
19
20    @classmethod
21    async def create_user(
22        cls,
23        url: ChrisURL | str,
24        username: Username | str,
25        password: Password | str,
26        email: str,
27        session: Optional[aiohttp.ClientSession],
28    ) -> UserData:
29        payload = {
30            "template": {
31                "data": [
32                    {"name": "email", "value": email},
33                    {"name": "username", "value": username},
34                    {"name": "password", "value": password},
35                ]
36            }
37        }
38        headers = {
39            "Content-Type": "application/vnd.collection+json",
40            "Accept": "application/json",
41        }
42        async with _optional_session(session) as session:
43            res = await session.post(url + "users/", json=payload, headers=headers)
44            await raise_for_status(res)
45            return from_json(UserData, await res.text())

A normal user ChRIS client, who may upload files and create plugin instances.

@classmethod
async def create_user( cls, url: Union[chris.models.types.ChrisURL, str], username: Union[chris.models.types.Username, str], password: Union[chris.models.types.Password, str], email: str, session: Optional[aiohttp.client.ClientSession]) -> chris.models.data.UserData:
20    @classmethod
21    async def create_user(
22        cls,
23        url: ChrisURL | str,
24        username: Username | str,
25        password: Password | str,
26        email: str,
27        session: Optional[aiohttp.ClientSession],
28    ) -> UserData:
29        payload = {
30            "template": {
31                "data": [
32                    {"name": "email", "value": email},
33                    {"name": "username", "value": username},
34                    {"name": "password", "value": password},
35                ]
36            }
37        }
38        headers = {
39            "Content-Type": "application/vnd.collection+json",
40            "Accept": "application/json",
41        }
42        async with _optional_session(session) as session:
43            res = await session.post(url + "users/", json=payload, headers=headers)
44            await raise_for_status(res)
45            return from_json(UserData, await res.text())
Inherited Members
chris.link.collection_client.CollectionJsonApiClient
CollectionJsonApiClient
url
chris.client.authed.AuthenticatedClient
from_login
from_token
search_feeds
search_plugins
plugin_instances
upload_file
user
username
search_compute_resources
get_all_compute_resources
chris.client.base.BaseChrisClient
new
close
chris.link.linked.Linked
max_search_requests
 31class ChrisAdminClient(AuthenticatedClient[AdminCollectionLinks, "ChrisAdminClient"]):
 32    """
 33    A client who has access to `/chris-admin/`. Admins can register new plugins and
 34    add new compute resources.
 35    """
 36
 37    @http.post("admin")
 38    async def _register_plugin_from_store_raw(
 39        self, plugin_store_url: str, compute_names: str
 40    ) -> Plugin:
 41        ...
 42
 43    async def register_plugin_from_store(
 44        self, plugin_store_url: PluginUrl, compute_names: Iterable[ComputeResourceName]
 45    ) -> Plugin:
 46        """
 47        Register a plugin from a ChRIS Store.
 48        """
 49        return await self._register_plugin_from_store_raw(
 50            plugin_store_url=plugin_store_url, compute_names=",".join(compute_names)
 51        )
 52
 53    async def add_plugin(
 54        self,
 55        plugin_description: str | dict,
 56        compute_resources: str
 57        | ComputeResource
 58        | Iterable[ComputeResource | ComputeResourceName],
 59        fname: str = "Plugin.json",
 60    ) -> Plugin:
 61        """
 62        Add a plugin to *CUBE*.
 63
 64        Examples
 65        --------
 66
 67        ```python
 68        cmd = ['docker', 'run', '--rm', 'fnndsc/pl-mri-preview', 'chris_plugin_info']
 69        output = subprocess.check_output(cmd, text=True)
 70        desc = json.loads(output)
 71        desc['name'] = 'pl-mri-preview'
 72        desc['public_repo'] = 'https://github.com/FNNDSC/pl-mri-preview'
 73        desc['dock_image'] = 'fnndsc/pl-mri-preview'
 74
 75        await chris_admin.add_plugin(plugin_description=desc, compute_resources='host')
 76        ```
 77
 78        The example above is just for show. It's not a good example for several reasons:
 79
 80        - Calls blocking function `subprocess.check_output` in asynchronous context
 81        - It is preferred to use a versioned string for `dock_image`
 82        - `host` compute environment is not guaranteed to exist. Instead, you could
 83          call `chris.client.authed.AuthenticatedClient.search_compute_resources`
 84          or `chris.client.authed.AuthenticatedClient.get_all_compute_resources`:
 85
 86        ```python
 87        all_computes = await chris_admin.get_all_compute_resources()
 88        await chris_admin.add_plugin(plugin_description=desc, compute_resources=all_computes)
 89        ```
 90
 91        Parameters
 92        ----------
 93        plugin_description: str | dict
 94            JSON description of a plugin.
 95            [spec](https://github.com/FNNDSC/CHRIS_docs/blob/5078aaf934bdbe313e85367f88aff7c14730a1d4/specs/ChRIS_Plugins.adoc#descriptor_file)
 96        compute_resources
 97            Compute resources to register the plugin to. Value can be either a comma-separated `str` of names,
 98            a `chris.models.public.ComputeResource`, a sequence of `chris.models.public.ComputeResource`,
 99            or a sequence of compute resource names as `str`.
100        fname: str
101            File name to send along in the multi-part POST request. Not important.
102        """
103        compute_names = _serialize_crs(compute_resources)
104        if not isinstance(plugin_description, str):
105            plugin_description = json.dumps(plugin_description)
106        data = aiohttp.FormData()
107        data.add_field("fname", io.StringIO(plugin_description), filename=fname)
108        data.add_field("compute_names", compute_names)
109        async with self.s.post(self.collection_links.admin, data=data) as res:
110            await raise_for_status(res)
111            return deserialize_linked(self, Plugin, await res.json())
112
113    async def create_compute_resource(
114        self,
115        name: str | ComputeResourceName,
116        compute_url: str | PfconUrl,
117        compute_user: str,
118        compute_password: str,
119        description: str = None,
120        compute_auth_url: str = None,
121        compute_auth_token: str = None,
122        max_job_exec_seconds: str = None,
123    ) -> ComputeResource:
124        """
125        Define a new compute resource.
126        """
127        return await (await self._admin).create_compute_resource(
128            name=name,
129            compute_url=compute_url,
130            compute_user=compute_user,
131            compute_password=compute_password,
132            description=description,
133            compute_auth_url=compute_auth_url,
134            compute_auth_token=compute_auth_token,
135            max_job_exec_seconds=max_job_exec_seconds,
136        )
137
138    @async_cached_property
139    async def _admin(self) -> _AdminApiClient:
140        """
141        Get a (sub-)client for `/chris-admin/api/v1/`
142        """
143        res = await self.s.get(self.collection_links.admin)
144        body = await res.json()
145        links = from_dict(AdminApiCollectionLinks, body["collection_links"])
146        return _AdminApiClient(
147            url=self.collection_links.admin,
148            s=self.s,
149            collection_links=links,
150            max_search_requests=self.max_search_requests,
151        )

A client who has access to /chris-admin/. Admins can register new plugins and add new compute resources.

async def register_plugin_from_store( self, plugin_store_url: chris.models.types.PluginUrl, compute_names: Iterable[chris.models.types.ComputeResourceName]) -> chris.models.logged_in.Plugin:
43    async def register_plugin_from_store(
44        self, plugin_store_url: PluginUrl, compute_names: Iterable[ComputeResourceName]
45    ) -> Plugin:
46        """
47        Register a plugin from a ChRIS Store.
48        """
49        return await self._register_plugin_from_store_raw(
50            plugin_store_url=plugin_store_url, compute_names=",".join(compute_names)
51        )

Register a plugin from a ChRIS Store.

async def add_plugin( self, plugin_description: str | dict, compute_resources: Union[str, chris.models.public.ComputeResource, Iterable[Union[chris.models.public.ComputeResource, chris.models.types.ComputeResourceName]]], fname: str = 'Plugin.json') -> chris.models.logged_in.Plugin:
 53    async def add_plugin(
 54        self,
 55        plugin_description: str | dict,
 56        compute_resources: str
 57        | ComputeResource
 58        | Iterable[ComputeResource | ComputeResourceName],
 59        fname: str = "Plugin.json",
 60    ) -> Plugin:
 61        """
 62        Add a plugin to *CUBE*.
 63
 64        Examples
 65        --------
 66
 67        ```python
 68        cmd = ['docker', 'run', '--rm', 'fnndsc/pl-mri-preview', 'chris_plugin_info']
 69        output = subprocess.check_output(cmd, text=True)
 70        desc = json.loads(output)
 71        desc['name'] = 'pl-mri-preview'
 72        desc['public_repo'] = 'https://github.com/FNNDSC/pl-mri-preview'
 73        desc['dock_image'] = 'fnndsc/pl-mri-preview'
 74
 75        await chris_admin.add_plugin(plugin_description=desc, compute_resources='host')
 76        ```
 77
 78        The example above is just for show. It's not a good example for several reasons:
 79
 80        - Calls blocking function `subprocess.check_output` in asynchronous context
 81        - It is preferred to use a versioned string for `dock_image`
 82        - `host` compute environment is not guaranteed to exist. Instead, you could
 83          call `chris.client.authed.AuthenticatedClient.search_compute_resources`
 84          or `chris.client.authed.AuthenticatedClient.get_all_compute_resources`:
 85
 86        ```python
 87        all_computes = await chris_admin.get_all_compute_resources()
 88        await chris_admin.add_plugin(plugin_description=desc, compute_resources=all_computes)
 89        ```
 90
 91        Parameters
 92        ----------
 93        plugin_description: str | dict
 94            JSON description of a plugin.
 95            [spec](https://github.com/FNNDSC/CHRIS_docs/blob/5078aaf934bdbe313e85367f88aff7c14730a1d4/specs/ChRIS_Plugins.adoc#descriptor_file)
 96        compute_resources
 97            Compute resources to register the plugin to. Value can be either a comma-separated `str` of names,
 98            a `chris.models.public.ComputeResource`, a sequence of `chris.models.public.ComputeResource`,
 99            or a sequence of compute resource names as `str`.
100        fname: str
101            File name to send along in the multi-part POST request. Not important.
102        """
103        compute_names = _serialize_crs(compute_resources)
104        if not isinstance(plugin_description, str):
105            plugin_description = json.dumps(plugin_description)
106        data = aiohttp.FormData()
107        data.add_field("fname", io.StringIO(plugin_description), filename=fname)
108        data.add_field("compute_names", compute_names)
109        async with self.s.post(self.collection_links.admin, data=data) as res:
110            await raise_for_status(res)
111            return deserialize_linked(self, Plugin, await res.json())

Add a plugin to CUBE.

Examples
cmd = ['docker', 'run', '--rm', 'fnndsc/pl-mri-preview', 'chris_plugin_info']
output = subprocess.check_output(cmd, text=True)
desc = json.loads(output)
desc['name'] = 'pl-mri-preview'
desc['public_repo'] = 'https://github.com/FNNDSC/pl-mri-preview'
desc['dock_image'] = 'fnndsc/pl-mri-preview'

await chris_admin.add_plugin(plugin_description=desc, compute_resources='host')

The example above is just for show. It's not a good example for several reasons:

all_computes = await chris_admin.get_all_compute_resources()
await chris_admin.add_plugin(plugin_description=desc, compute_resources=all_computes)
Parameters
  • plugin_description (str | dict): JSON description of a plugin. spec
  • compute_resources: Compute resources to register the plugin to. Value can be either a comma-separated str of names, a chris.models.public.ComputeResource, a sequence of chris.models.public.ComputeResource, or a sequence of compute resource names as str.
  • fname (str): File name to send along in the multi-part POST request. Not important.
async def create_compute_resource( self, name: Union[str, chris.models.types.ComputeResourceName], compute_url: Union[str, chris.models.types.PfconUrl], compute_user: str, compute_password: str, description: str = None, compute_auth_url: str = None, compute_auth_token: str = None, max_job_exec_seconds: str = None) -> chris.models.public.ComputeResource:
113    async def create_compute_resource(
114        self,
115        name: str | ComputeResourceName,
116        compute_url: str | PfconUrl,
117        compute_user: str,
118        compute_password: str,
119        description: str = None,
120        compute_auth_url: str = None,
121        compute_auth_token: str = None,
122        max_job_exec_seconds: str = None,
123    ) -> ComputeResource:
124        """
125        Define a new compute resource.
126        """
127        return await (await self._admin).create_compute_resource(
128            name=name,
129            compute_url=compute_url,
130            compute_user=compute_user,
131            compute_password=compute_password,
132            description=description,
133            compute_auth_url=compute_auth_url,
134            compute_auth_token=compute_auth_token,
135            max_job_exec_seconds=max_job_exec_seconds,
136        )

Define a new compute resource.

Inherited Members
chris.link.collection_client.CollectionJsonApiClient
CollectionJsonApiClient
url
chris.client.authed.AuthenticatedClient
from_login
from_token
search_feeds
search_plugins
plugin_instances
upload_file
user
username
search_compute_resources
get_all_compute_resources
chris.client.base.BaseChrisClient
new
close
chris.link.linked.Linked
max_search_requests
async def acollect(async_iterable: AsyncIterable[~T]) -> list[~T]:
202async def acollect(async_iterable: AsyncIterable[T]) -> list[T]:
203    """
204    Simple helper to convert a `Search` to a [`list`](https://docs.python.org/3/library/stdtypes.html#list).
205
206    Using this function is not recommended unless you can assume the collection is small.
207    """
208    # nb: using tuple here causes
209    #     TypeError: 'async_generator' object is not iterable
210    # return tuple(e async for e in async_iterable)
211    return [e async for e in async_iterable]

Simple helper to convert a Search to a list.

Using this function is not recommended unless you can assume the collection is small.

class Status(enum.Enum):
 5class Status(enum.Enum):
 6    """
 7    Possible statuses of a plugin instance.
 8    """
 9
10    created = "created"
11    waiting = "waiting"
12    scheduled = "scheduled"
13    started = "started"
14    registeringFiles = "registeringFiles"
15    finishedSuccessfully = "finishedSuccessfully"
16    finishedWithError = "finishedWithError"
17    cancelled = "cancelled"

Possible statuses of a plugin instance.

created = <Status.created: 'created'>
waiting = <Status.waiting: 'waiting'>
scheduled = <Status.scheduled: 'scheduled'>
started = <Status.started: 'started'>
registeringFiles = <Status.registeringFiles: 'registeringFiles'>
finishedSuccessfully = <Status.finishedSuccessfully: 'finishedSuccessfully'>
finishedWithError = <Status.finishedWithError: 'finishedWithError'>
cancelled = <Status.cancelled: 'cancelled'>
Inherited Members
enum.Enum
name
value
class ParameterTypeName(enum.Enum):
20class ParameterTypeName(enum.Enum):
21    """
22    Plugin parameter types.
23    """
24
25    string = "string"
26    integer = "integer"
27    float = "float"
28    boolean = "boolean"

Plugin parameter types.

string = <ParameterTypeName.string: 'string'>
integer = <ParameterTypeName.integer: 'integer'>
float = <ParameterTypeName.float: 'float'>
boolean = <ParameterTypeName.boolean: 'boolean'>
Inherited Members
enum.Enum
name
value