Skip to content

Open In Colab

Plugins Development

Preliminary

This notebook will teach you how to build your first plugin.

A plugin is an uploaded Python script triggered by an event that you define.

For instance, you can trigger a plugin when a labeler clicks on Submit with the on_submit handler.

The plugin should have different methods for the different types of events:

  • on_submit
  • on_review

These methods have a predefined set of parameters:

  • the label submitted
  • the asset_id of the asset labeled

Some attributes are available in the class:

  • self.kili
  • self.project_id

Therefore, the skeleton of the plugin should look like this:

from kili.plugins import PluginCore
from kili.types import Label
import numpy as np

class PluginHandler(PluginCore):
    """Custom plugin"""

    def on_review(self, label: Label, asset_id: str) -> None:
        """Dedicated handler for Review action"""
        # Do something...

    def on_submit(self, label: Label, asset_id: str) -> None:
        """Dedicated handler for Submit action"""
        # Do something...

Do not hesitate to reach out to us if you need more.

NB: The plugin capabilities of Kili are under active development, and compatible with version 2.125.2 and later of Kili. Don't hesitate to reach out via Github or Kili support to provide feedback.

Instantiate Kili

!pip install --upgrade kili

from kili.client import Kili
import os

kili = Kili()

Develop your plugin

The first step is to define the functions that will be called when the event is triggered. You will be able to iterate on these functions locally (more on that in the next section).

This cell should be the contents of the .py file that you will upload as a plugin at the end. This file should define the PluginHandler class that will contain the proper methods. We recommend using a modern IDE like VScode to get type hints and autocompletion on the methods.

from kili.plugins import PluginCore
from kili.types import Label
import numpy as np


def custom_function(label: Label):
    label_id = label.get("id")
    print(f"My custom function for review of label with id {label_id}")


class PluginHandler(PluginCore):
    """
    Custom plugin instance
    """

    def custom_method(self, project_id, label_id):
        print(f"custom_method called for label {label_id}")
        random_seed = np.random.random(1)[0]
        if random_seed > 0.5:
            self.logger.warning("Generating issue")
            # Use kili for actions with self.kili
            self.kili.append_to_issues(
                label_id=label_id,
                project_id=project_id,
                text="Random issue generated for this label",
            )

    def on_review(self, label: Label, asset_id: str) -> None:
        """
        Dedicated handler for Review action
        """
        custom_function(label)

    def on_submit(self, label: Label, asset_id: str) -> None:
        """
        Dedicated handler for Submit action
        """
        print("On submit called")

        project_id = self.project_id
        label_id = label.get("id")

        self.custom_method(project_id, label_id)

Testing the plugin locally

In this we will show you how to test your plugin locally before uploading it.

project_id = "<PROJECT ID>"

Instantiate the plugin:

my_plugin_instance = PluginHandler(kili, project_id)


def get_label(label_id, project_id):
    """
    Function to get the object Label with the same keys as it will be in the plugin
    """
    label = list(
        kili.labels(
            project_id=project_id,
            label_id=label_id,
            fields=["id", "jsonResponse", "author.id", "labelType", "createdAt", "secondsToLabel"],
        )
    )[0]

    label["authorId"] = label["author"]["id"]
    del label["author"]
    return label

Test the plugin run

If you already have a test project with labels added, you can directly use the IDs of these labels (see the following cell). Otherwise, you can follow the plugins_example.ipynb notebook to create a new project and then upload an asset with an associated label.

asset_id = "<YOUR_ASSET_ID>"
label_id = "<YOUR_LABEL_ID>"

label = get_label(label_id=label_id, project_id=project_id)

my_plugin_instance.on_submit(label=Label(**label), asset_id=asset_id)

On submit called
custom_method called for label clcyqwn5a2gyk0lpn8d7w486h

Test the plugin run on Kili

When you finish debugging the code, you may want to upload it directly into Kili.

Note that you might get an error if the plugin name already exists in your Kili organization.

path_to_plugin = "path/to/my/plugin.py"
plugin_name = "My first kili plugin"

from kili.exceptions import GraphQLError

try:
    kili.upload_plugin(path_to_plugin, plugin_name)
except GraphQLError as error:
    print(str(error))

Hint: A plugin with this name already exist, if you want to override it you can use the command kili.update_plugin(plugin_path="plugin.py", plugin_name="My first kili plugin")
error: "[pluginsError] An error occured handling your plugin -- This can be due to: 400: Bad Request: createPlugin: an entity Plugin already exists with value "My first kili plugin" for field 'name' | trace : false"

Plugins must be activated in the project that you want them to run in. Be careful with production projects: your custom workflows or rules will also be applied

kili.activate_plugin_on_project(plugin_name, project_id=project_id);

Plugin with name "My first kili plugin" activated on project "clcyr3xsz2e8j0lrehb1ufte9"

Monitoring the plugin

Plugin creation takes some time (around 5 minutes). The plugin will begin to run only after it's been fully created (if labeling events are to be triggered on this project).

Additionally, you can get the logs of the runs:

kili.get_plugin_logs(project_id=project_id, plugin_name=plugin_name)

You can set custom date rules for filtering your logs:

from datetime import date
from datetime import datetime

dt = date.today()  # You can change this date if needed
start_date = datetime.combine(dt, datetime.min.time())

kili.get_plugin_logs(project_id=project_id, plugin_name=plugin_name, start_date=start_date)

Managing your plugin

Here are several other methods to manage your plugins and their lifecycle:

Get the list of all uploaded plugins in your organization:

plugins = kili.list_plugins()
print([plugin for plugin in plugins if plugin["name"] == plugin_name])

[{'name': 'My first kili plugin', 'projectIds': ['clcyoj8s129ap0krfd5k2cjvl', 'clcyqw5m42e380lredqtzh4tx'], 'id': 'clb12ceii05to019g5wkh9rvz', 'createdAt': '2022-11-28T17:27:35.802Z', 'updatedAt': '2023-01-16T11:00:36.691Z'}]

Update a plugin with new source code:

updated_path = "plugin.py"

if updated_path != path_to_plugin:
    kili.update_plugin(plugin_name=plugin_name, plugin_path=updated_path)

Deactivate the plugin on a certain project (the plugin can still be active for other projects):

kili.deactivate_plugin_on_project(plugin_name=plugin_name, project_id=project_id);

Plugin My first kili plugin deactivated on project clcyr3xsz2e8j0lrehb1ufte9

Delete the plugin completely (deactivates automatically the plugin from all projects):

if delete_plugin_from_org:
    kili.delete_plugin(plugin_name=plugin_name)