aiochris

ChRIS Python client library built on aiohttp (async HTTP client) and pyserde (dataclasses deserializer).

Installation

Requires Python 3.11 or 3.12.

pip install aiochris
# or
rye add aiochris

Brief Example

from aiochris import ChrisClient

chris = await ChrisClient.from_login(
    username='chris',
    password='chris1234',
    url='https://cube.chrisproject.org/api/v1/'
)
dircopy = await chris.search_plugins(name_exact='pl-brainmgz', version='2.0.3').get_only()
plinst = await dircopy.create_instance(compute_resource_name='host')
await plinst.set(title="copies brain image files into feed")

Introduction

aiochris provides three core classes: AnonChrisClient, ChrisClient, and ChrisAdminClient. These clients differ in permissions.

Methods are only defined for what the client has permission to see or do.

anon_client = await AnonChrisClient.from_url('https://cube.chrisproject.org/api/v1/')
# ok: can search for plugins without logging in...
plugin = await anon_client.search_plugins(name_exact='pl-mri10yr06mo01da_normal').first()
# IMPOSSIBLE! AnonChrisClient.create_instance not defined...
await plugin.create_instance()

# IMPOSSIBLE! authentication required for ChrisClient
authed_client = await ChrisClient.from_url('https://cube.chrisproject.org/api/v1/')
authed_client = await ChrisClient.from_login(
    url='https://cube.chrisproject.org/api/v1/',
    username='chris',
    password='chris1234'
)
# authenticated client can also search for plugins
plugin = await authed_client.search_plugins(name_exact='pl-mri10yr06mo01da_normal').first()
await plugin.create_instance()  # works!

Client Constructors

aiochris in Jupyter Notebook

Jupyter and IPython support top-level await. This, in conjunction with ChrisClient.from_chrs, make aiochris a great way to use _ChRIS_ interactively with code.

For a walkthrough, see https://github.com/FNNDSC/aiochris/blob/master/examples/aiochris_as_a_shell.ipynb

Working with aiohttp

aiochris hides the implementation detail that it is built upon aiohttp, however one thing is important to keep in mind: be sure to call ChrisClient.close at the end of your program.

chris = await ChrisClient.from_login(...)
# -- snip --
await chris.close()

You can also use an asynchronous context manager.

async with (await ChrisClient.from_login(...)) as chris:
    await chris.upload_file('./something.dat', 'something.dat')
    ...

Efficiency with Multiple Clients

If using more than one aiohttp client in an application, it's more efficient to use the same connector. One connector instance should be shared among every client object, including all aiochris clients and other aiohttp clients.

Example: efficiently using multiple aiohttp clients

import aiohttp
from aiochris import ChrisClient

with aiohttp.TCPConnector() as connector:
    chris_client1 = await ChrisClient.from_login(
        url='https://example.com/cube/api/v1/',
        username='user1',
        password='user1234',
        connector=connector,
        connector_owner=False
    )
    chris_client2 = await ChrisClient.from_login(
        url='https://example.com/cube/api/v1/',
        username='user2',
        password='user4321',
        connector=connector,
        connector_owner=False
    )
    plain_http_client = aiohttp.ClientSession(connector=connector, connector_owner=False)

Advice for Getting Started

Searching for things (plugins, plugin instances, files) in CUBE is a common task, and CUBE often returns multiple items per response. Hence, it is important to understand how the Search helper class works. It simplifies how we interact with paginated collection responses from CUBE.

When performing batch operations, use asyncio.gather to run async functions concurrently.

aiochris uses many generic types, so it is recommended you use an IDE with good support for type hints, such as PyCharm or VSCodium with Pylance configured.

Examples

Create a client given username and password

from aiochris import ChrisClient

chris = await ChrisClient.from_login(
    url='https://cube.chrisproject.org/api/v1/',
    username='chris',
    password='chris1234'
)

Search for a plugin

# it's recommended to specify plugin version
plugin = await chris.search_plugins(name_exact="pl-dcm2niix", version="0.1.0").get_only()

# but if you don't care about plugin version...
plugin = await chris.search_plugins(name_exact="pl-dcm2niix").first()

Create a feed by uploading a file

uploaded_file = await chris.upload_file('./brain.nii', 'my_experiment/brain.nii')
dircopy = await chris.search_plugins(name_exact='pl-dircopy', version="2.1.1").get_only()
plinst = await dircopy.create_instance(dir=uploaded_file.parent)
feed = await plinst.get_feed()
await feed.set(name="An experiment on uploaded file brain.nii")

Run a ds-type ChRIS plugin

# search for plugin to run
plugin = await chris.search_plugins(name_exact="pl-dcm2niix", version="0.1.0").get_only()

# search for parent node 
previous = await chris.plugin_instances(id=44).get_only()

await plugin.create_instance(
    previous=previous,               # required. alternatively, specify previous_id
    title="convert DICOM to NIFTI",  # optional
    compute_resource_name="galena",  # optional
    memory_limit="2000Mi",           # optional
    d=9,                             # optional parameter of pl-dcm2niix
)

Search for plugin instances

finished_freesurfers = chris.plugin_instances(
    plugin_name_exact='pl-fshack',
    status='finishedSuccessfully'
)
async for p in finished_freesurfers:
    print(f'"{p.title}" finished on date: {p.end_date}')

Delete all plugin instances with a given title, in parallel

import asyncio
from aiochris import ChrisClient, acollect

chris = ChrisClient.from_login(...)
search = chris.plugin_instances(title="delete me")
plugin_instances = await acollect(search)
await asyncio.gather(*(p.delete() for p in plugin_instances))

Enable Debug Logging

A log message will be printed to stderr before every HTTP request is sent.

import logging

logging.basicConfig(level=logging.DEBUG)
 1"""
 2.. include:: ./home.md
 3
 4.. include:: ./examples.md
 5"""
 6
 7import aiochris.client
 8import aiochris.models
 9import aiochris.util
10from aiochris.util.search import Search, acollect
11from aiochris.client.normal import ChrisClient
12from aiochris.client.anon import AnonChrisClient
13from aiochris.client.admin import ChrisAdminClient
14from aiochris.enums import Status, ParameterTypeName
15
16__all__ = [
17    "AnonChrisClient",
18    "ChrisClient",
19    "ChrisAdminClient",
20    "Search",
21    "acollect",
22    "Status",
23    "ParameterTypeName",
24    "client",
25    "models",
26    "util",
27    "errors",
28    "types",
29]
13class AnonChrisClient(BaseChrisClient[AnonymousCollectionLinks, "AnonChrisClient"]):
14    """
15    An anonymous ChRIS client. It has access to read-only GET resources,
16    such as being able to search for plugins.
17    """
18
19    @classmethod
20    async def from_url(
21        cls,
22        url: str,
23        max_search_requests: int = 100,
24        connector: Optional[aiohttp.BaseConnector] = None,
25        connector_owner: bool = True,
26    ) -> "AnonChrisClient":
27        """
28        Create an anonymous client.
29
30        See `aiochris.client.base.BaseChrisClient.new` for parameter documentation.
31        """
32        return await cls.new(
33            url=url,
34            max_search_requests=max_search_requests,
35            connector=connector,
36            connector_owner=connector_owner,
37        )
38
39    @http.search("plugins")
40    def search_plugins(self, **query) -> Search[PublicPlugin]:
41        """
42        Search for plugins.
43
44        Since this client is not logged in, you cannot create plugin instances using this client.
45        """
46        ...

An anonymous ChRIS client. It has access to read-only GET resources, such as being able to search for plugins.

@classmethod
async def from_url( cls, url: str, max_search_requests: int = 100, connector: Optional[aiohttp.connector.BaseConnector] = None, connector_owner: bool = True) -> AnonChrisClient:
19    @classmethod
20    async def from_url(
21        cls,
22        url: str,
23        max_search_requests: int = 100,
24        connector: Optional[aiohttp.BaseConnector] = None,
25        connector_owner: bool = True,
26    ) -> "AnonChrisClient":
27        """
28        Create an anonymous client.
29
30        See `aiochris.client.base.BaseChrisClient.new` for parameter documentation.
31        """
32        return await cls.new(
33            url=url,
34            max_search_requests=max_search_requests,
35            connector=connector,
36            connector_owner=connector_owner,
37        )

Create an anonymous client.

See aiochris.client.base.BaseChrisClient.new for parameter documentation.

@http.search('plugins')
def search_plugins( self, **query) -> Search[aiochris.models.public.PublicPlugin]:
39    @http.search("plugins")
40    def search_plugins(self, **query) -> Search[PublicPlugin]:
41        """
42        Search for plugins.
43
44        Since this client is not logged in, you cannot create plugin instances using this client.
45        """
46        ...

Search for plugins.

Since this client is not logged in, you cannot create plugin instances using this client.

Inherited Members
aiochris.link.collection_client.CollectionJsonApiClient
CollectionJsonApiClient
url
aiochris.client.base.BaseChrisClient
new
close
aiochris.link.linked.Linked
s
max_search_requests
15class ChrisClient(AuthenticatedClient[CollectionLinks, "ChrisClient"]):
16    """
17    A normal user *ChRIS* client, who may upload files and create plugin instances.
18    """
19
20    @classmethod
21    async def create_user(
22        cls,
23        url: ChrisURL | str,
24        username: Username | str,
25        password: Password | str,
26        email: str,
27        session: Optional[aiohttp.ClientSession] = None,
28    ) -> UserData:
29        payload = {
30            "template": {
31                "data": [
32                    {"name": "email", "value": email},
33                    {"name": "username", "value": username},
34                    {"name": "password", "value": password},
35                ]
36            }
37        }
38        headers = {
39            "Content-Type": "application/vnd.collection+json",
40            "Accept": "application/json",
41        }
42        async with _optional_session(session) as session:
43            res = await session.post(url + "users/", json=payload, headers=headers)
44            await raise_for_status(res)
45            return from_json(UserData, await res.text())

A normal user ChRIS client, who may upload files and create plugin instances.

@classmethod
async def create_user( cls, url: Union[aiochris.types.ChrisURL, str], username: Union[aiochris.types.Username, str], password: Union[aiochris.types.Password, str], email: str, session: Optional[aiohttp.client.ClientSession] = None) -> aiochris.models.data.UserData:
20    @classmethod
21    async def create_user(
22        cls,
23        url: ChrisURL | str,
24        username: Username | str,
25        password: Password | str,
26        email: str,
27        session: Optional[aiohttp.ClientSession] = None,
28    ) -> UserData:
29        payload = {
30            "template": {
31                "data": [
32                    {"name": "email", "value": email},
33                    {"name": "username", "value": username},
34                    {"name": "password", "value": password},
35                ]
36            }
37        }
38        headers = {
39            "Content-Type": "application/vnd.collection+json",
40            "Accept": "application/json",
41        }
42        async with _optional_session(session) as session:
43            res = await session.post(url + "users/", json=payload, headers=headers)
44            await raise_for_status(res)
45            return from_json(UserData, await res.text())
Inherited Members
aiochris.link.collection_client.CollectionJsonApiClient
CollectionJsonApiClient
url
aiochris.client.authed.AuthenticatedClient
from_login
from_token
from_chrs
search_feeds
search_plugins
plugin_instances
upload_file
user
username
search_compute_resources
get_all_compute_resources
search_pacsfiles
aiochris.client.base.BaseChrisClient
new
close
aiochris.link.linked.Linked
s
max_search_requests
 33class ChrisAdminClient(AuthenticatedClient[AdminCollectionLinks, "ChrisAdminClient"]):
 34    """
 35    A client who has access to `/chris-admin/`. Admins can register new plugins and
 36    add new compute resources.
 37    """
 38
 39    @http.post("admin")
 40    async def _register_plugin_from_store_raw(
 41        self, plugin_store_url: str, compute_names: str
 42    ) -> Plugin: ...
 43
 44    async def register_plugin_from_store(
 45        self, plugin_store_url: PluginUrl, compute_names: Iterable[ComputeResourceName]
 46    ) -> Plugin:
 47        """
 48        Register a plugin from a ChRIS Store.
 49        """
 50        return await self._register_plugin_from_store_raw(
 51            plugin_store_url=plugin_store_url, compute_names=",".join(compute_names)
 52        )
 53
 54    async def add_plugin(
 55        self,
 56        plugin_description: str | dict,
 57        compute_resources: str
 58        | ComputeResource
 59        | Iterable[ComputeResource | ComputeResourceName],
 60    ) -> Plugin:
 61        """
 62        Add a plugin to *CUBE*.
 63
 64        Examples
 65        --------
 66
 67        ```python
 68        cmd = ['docker', 'run', '--rm', 'fnndsc/pl-mri-preview', 'chris_plugin_info']
 69        output = subprocess.check_output(cmd, text=True)
 70        desc = json.loads(output)
 71        desc['name'] = 'pl-mri-preview'
 72        desc['public_repo'] = 'https://github.com/FNNDSC/pl-mri-preview'
 73        desc['dock_image'] = 'fnndsc/pl-mri-preview'
 74
 75        await chris_admin.add_plugin(plugin_description=desc, compute_resources='host')
 76        ```
 77
 78        The example above is just for show. It's not a good example for several reasons:
 79
 80        - Calls blocking function `subprocess.check_output` in asynchronous context
 81        - It is preferred to use a versioned string for `dock_image`
 82        - `host` compute environment is not guaranteed to exist. Instead, you could
 83          call `aiochris.client.authed.AuthenticatedClient.search_compute_resources`
 84          or `aiochris.client.authed.AuthenticatedClient.get_all_compute_resources`:
 85
 86        ```python
 87        all_computes = await chris_admin.get_all_compute_resources()
 88        await chris_admin.add_plugin(plugin_description=desc, compute_resources=all_computes)
 89        ```
 90
 91        Parameters
 92        ----------
 93        plugin_description: str | dict
 94            JSON description of a plugin.
 95            [spec](https://github.com/FNNDSC/CHRIS_docs/blob/5078aaf934bdbe313e85367f88aff7c14730a1d4/specs/ChRIS_Plugins.adoc#descriptor_file)
 96        compute_resources
 97            Compute resources to register the plugin to. Value can be either a comma-separated `str` of names,
 98            a `aiochris.models.public.ComputeResource`, a sequence of `aiochris.models.public.ComputeResource`,
 99            or a sequence of compute resource names as `str`.
100        """
101        compute_names = _serialize_crs(compute_resources)
102        if not isinstance(plugin_description, str):
103            plugin_description = json.dumps(plugin_description)
104        data = aiohttp.FormData()
105        data.add_field(
106            "fname",
107            io.StringIO(plugin_description),
108            filename="aiochris_add_plugin.json",
109        )
110        data.add_field("compute_names", compute_names)
111        async with self.s.post(self.collection_links.admin, data=data) as res:
112            await raise_for_status(res)
113            return deserialize_linked(self, Plugin, await res.json())
114
115    async def create_compute_resource(
116        self,
117        name: str | ComputeResourceName,
118        compute_url: str | PfconUrl,
119        compute_user: str,
120        compute_password: str,
121        compute_innetwork: bool = None,
122        description: str = None,
123        compute_auth_url: str = None,
124        compute_auth_token: str = None,
125        max_job_exec_seconds: str = None,
126    ) -> ComputeResource:
127        """
128        Define a new compute resource.
129        """
130        return await (await self._admin).create_compute_resource(
131            name=name,
132            compute_url=compute_url,
133            compute_user=compute_user,
134            compute_password=compute_password,
135            compute_innetwork=compute_innetwork,
136            description=description,
137            compute_auth_url=compute_auth_url,
138            compute_auth_token=compute_auth_token,
139            max_job_exec_seconds=max_job_exec_seconds,
140        )
141
142    @async_cached_property
143    async def _admin(self) -> _AdminApiClient:
144        """
145        Get a (sub-)client for `/chris-admin/api/v1/`
146        """
147        res = await self.s.get(self.collection_links.admin)
148        body = await res.json()
149        links = from_dict(AdminApiCollectionLinks, body["collection_links"])
150        return _AdminApiClient(
151            url=self.collection_links.admin,
152            s=self.s,
153            collection_links=links,
154            max_search_requests=self.max_search_requests,
155        )

A client who has access to /chris-admin/. Admins can register new plugins and add new compute resources.

async def register_plugin_from_store( self, plugin_store_url: aiochris.types.PluginUrl, compute_names: Iterable[aiochris.types.ComputeResourceName]) -> aiochris.models.logged_in.Plugin:
44    async def register_plugin_from_store(
45        self, plugin_store_url: PluginUrl, compute_names: Iterable[ComputeResourceName]
46    ) -> Plugin:
47        """
48        Register a plugin from a ChRIS Store.
49        """
50        return await self._register_plugin_from_store_raw(
51            plugin_store_url=plugin_store_url, compute_names=",".join(compute_names)
52        )

Register a plugin from a ChRIS Store.

async def add_plugin( self, plugin_description: str | dict, compute_resources: Union[str, aiochris.models.public.ComputeResource, Iterable[Union[aiochris.models.public.ComputeResource, aiochris.types.ComputeResourceName]]]) -> aiochris.models.logged_in.Plugin:
 54    async def add_plugin(
 55        self,
 56        plugin_description: str | dict,
 57        compute_resources: str
 58        | ComputeResource
 59        | Iterable[ComputeResource | ComputeResourceName],
 60    ) -> Plugin:
 61        """
 62        Add a plugin to *CUBE*.
 63
 64        Examples
 65        --------
 66
 67        ```python
 68        cmd = ['docker', 'run', '--rm', 'fnndsc/pl-mri-preview', 'chris_plugin_info']
 69        output = subprocess.check_output(cmd, text=True)
 70        desc = json.loads(output)
 71        desc['name'] = 'pl-mri-preview'
 72        desc['public_repo'] = 'https://github.com/FNNDSC/pl-mri-preview'
 73        desc['dock_image'] = 'fnndsc/pl-mri-preview'
 74
 75        await chris_admin.add_plugin(plugin_description=desc, compute_resources='host')
 76        ```
 77
 78        The example above is just for show. It's not a good example for several reasons:
 79
 80        - Calls blocking function `subprocess.check_output` in asynchronous context
 81        - It is preferred to use a versioned string for `dock_image`
 82        - `host` compute environment is not guaranteed to exist. Instead, you could
 83          call `aiochris.client.authed.AuthenticatedClient.search_compute_resources`
 84          or `aiochris.client.authed.AuthenticatedClient.get_all_compute_resources`:
 85
 86        ```python
 87        all_computes = await chris_admin.get_all_compute_resources()
 88        await chris_admin.add_plugin(plugin_description=desc, compute_resources=all_computes)
 89        ```
 90
 91        Parameters
 92        ----------
 93        plugin_description: str | dict
 94            JSON description of a plugin.
 95            [spec](https://github.com/FNNDSC/CHRIS_docs/blob/5078aaf934bdbe313e85367f88aff7c14730a1d4/specs/ChRIS_Plugins.adoc#descriptor_file)
 96        compute_resources
 97            Compute resources to register the plugin to. Value can be either a comma-separated `str` of names,
 98            a `aiochris.models.public.ComputeResource`, a sequence of `aiochris.models.public.ComputeResource`,
 99            or a sequence of compute resource names as `str`.
100        """
101        compute_names = _serialize_crs(compute_resources)
102        if not isinstance(plugin_description, str):
103            plugin_description = json.dumps(plugin_description)
104        data = aiohttp.FormData()
105        data.add_field(
106            "fname",
107            io.StringIO(plugin_description),
108            filename="aiochris_add_plugin.json",
109        )
110        data.add_field("compute_names", compute_names)
111        async with self.s.post(self.collection_links.admin, data=data) as res:
112            await raise_for_status(res)
113            return deserialize_linked(self, Plugin, await res.json())

Add a plugin to CUBE.

Examples
cmd = ['docker', 'run', '--rm', 'fnndsc/pl-mri-preview', 'chris_plugin_info']
output = subprocess.check_output(cmd, text=True)
desc = json.loads(output)
desc['name'] = 'pl-mri-preview'
desc['public_repo'] = 'https://github.com/FNNDSC/pl-mri-preview'
desc['dock_image'] = 'fnndsc/pl-mri-preview'

await chris_admin.add_plugin(plugin_description=desc, compute_resources='host')

The example above is just for show. It's not a good example for several reasons:

all_computes = await chris_admin.get_all_compute_resources()
await chris_admin.add_plugin(plugin_description=desc, compute_resources=all_computes)
Parameters
async def create_compute_resource( self, name: Union[str, aiochris.types.ComputeResourceName], compute_url: Union[str, aiochris.types.PfconUrl], compute_user: str, compute_password: str, compute_innetwork: bool = None, description: str = None, compute_auth_url: str = None, compute_auth_token: str = None, max_job_exec_seconds: str = None) -> aiochris.models.public.ComputeResource:
115    async def create_compute_resource(
116        self,
117        name: str | ComputeResourceName,
118        compute_url: str | PfconUrl,
119        compute_user: str,
120        compute_password: str,
121        compute_innetwork: bool = None,
122        description: str = None,
123        compute_auth_url: str = None,
124        compute_auth_token: str = None,
125        max_job_exec_seconds: str = None,
126    ) -> ComputeResource:
127        """
128        Define a new compute resource.
129        """
130        return await (await self._admin).create_compute_resource(
131            name=name,
132            compute_url=compute_url,
133            compute_user=compute_user,
134            compute_password=compute_password,
135            compute_innetwork=compute_innetwork,
136            description=description,
137            compute_auth_url=compute_auth_url,
138            compute_auth_token=compute_auth_token,
139            max_job_exec_seconds=max_job_exec_seconds,
140        )

Define a new compute resource.

Inherited Members
aiochris.link.collection_client.CollectionJsonApiClient
CollectionJsonApiClient
url
aiochris.client.authed.AuthenticatedClient
from_login
from_token
from_chrs
search_feeds
search_plugins
plugin_instances
upload_file
user
username
search_compute_resources
get_all_compute_resources
search_pacsfiles
aiochris.client.base.BaseChrisClient
new
close
aiochris.link.linked.Linked
s
max_search_requests
async def acollect(async_iterable: collections.abc.AsyncIterable[~T]) -> list[~T]:
204async def acollect(async_iterable: AsyncIterable[T]) -> list[T]:
205    """
206    Simple helper to convert a `Search` to a [`list`](https://docs.python.org/3/library/stdtypes.html#list).
207
208    Using this function is not recommended unless you can assume the collection is small.
209    """
210    # nb: using tuple here causes
211    #     TypeError: 'async_generator' object is not iterable
212    # return tuple(e async for e in async_iterable)
213    return [e async for e in async_iterable]

Simple helper to convert a Search to a list.

Using this function is not recommended unless you can assume the collection is small.

class Status(enum.Enum):
 5class Status(enum.Enum):
 6    """
 7    Possible statuses of a plugin instance.
 8    """
 9
10    created = "created"
11    waiting = "waiting"
12    scheduled = "scheduled"
13    started = "started"
14    registeringFiles = "registeringFiles"
15    finishedSuccessfully = "finishedSuccessfully"
16    finishedWithError = "finishedWithError"
17    cancelled = "cancelled"

Possible statuses of a plugin instance.

created = <Status.created: 'created'>
waiting = <Status.waiting: 'waiting'>
scheduled = <Status.scheduled: 'scheduled'>
started = <Status.started: 'started'>
registeringFiles = <Status.registeringFiles: 'registeringFiles'>
finishedSuccessfully = <Status.finishedSuccessfully: 'finishedSuccessfully'>
finishedWithError = <Status.finishedWithError: 'finishedWithError'>
cancelled = <Status.cancelled: 'cancelled'>
Inherited Members
enum.Enum
name
value
class ParameterTypeName(enum.Enum):
20class ParameterTypeName(enum.Enum):
21    """
22    Plugin parameter types.
23    """
24
25    string = "string"
26    integer = "integer"
27    float = "float"
28    boolean = "boolean"

Plugin parameter types.

string = <ParameterTypeName.string: 'string'>
integer = <ParameterTypeName.integer: 'integer'>
float = <ParameterTypeName.float: 'float'>
boolean = <ParameterTypeName.boolean: 'boolean'>
Inherited Members
enum.Enum
name
value