How to export data from a Kili project

Outline

This tutorial explains the multiple ways to export a Kili project. It describes:

• Methods to export the labels one by one, after filtering
• The solutions for performing a full-project export

The methods are illustrated with code snippets.

Export methods

With Kili, once you have annotated enough assets, you can export the data programmatically to train a machine learning algorithm with it. There are several ways to do it:

• Fetch the assets and/or the labels one by one using .assets or .labels, perform the data transformation yourself and then write the data to one or several output files.
• Export the whole project as a dataset. To do that, use the .export_labels method that creates an archive containing the labels in your chosen format.

Preliminary steps

1) Fetch the project ID from the Kili UI (in Settings / Admin):

2) Ensure that your Kili API key has been set as an environment variable:

export KILI_API_KEY=<YOUR_API_KEY>

3) If Kili has not been installed yet, install Kili.

%pip install  kili

4) Import packages and instantiate Kili:

from pathlib import Path

from kili.client import Kili

kili = Kili(
    # api_endpoint="https://cloud.kili-technology.com/api/label/v2/graphql",
    # the line above can be uncommented and changed if you are working with an on-premise version of Kili
)

Exporting assets and labels one by one

To retrieve all assets of a project one by one, perform the following steps:

Exporting the latest labels per asset

First, fetch the assets:

assets = kili.assets(
    your_project_id,
    fields=["externalId", "latestLabel.jsonResponse"],
    label_output_format="parsed_label",
)

Now if you print an asset, you will see that you can access its latestLabel:

print(assets[0])

{'latestLabel': {'jsonResponse': {'JOB_0': {'annotations': [{'categories': [{'name': 'OBJECT_A'}], 'mid': '20230111125258113-44528', 'type': 'rectangle', 'boundingPoly': [{'normalizedVertices': [{'x': 0.6101435505380516, 'y': 0.7689773770786136}, {'x': 0.6101435505380516, 'y': 0.39426226491370664}, {'x': 0.8962087421313937, 'y': 0.39426226491370664}, {'x': 0.8962087421313937, 'y': 0.7689773770786136}]}], 'polyline': [], 'children': {}}]}}}, 'externalId': 'car_1'}

You can now get your label, and write the category name into a text file for example:

for asset in assets:
    if asset["latestLabel"]:  # check if asset has annotations
        class_ = asset["latestLabel"].jobs["JOB_0"].annotations[0].category.name
        with Path(asset["externalId"] + ".txt").open("w", encoding="utf-8") as f:
            f.write(class_)

Filtering specific labels per asset through the method filters

You can specify label filters directly in the .assets and the .labels methods. The available filters are listed in the arguments list for each one of these methods.

When done, you can write the conversion code to get the data in the format that you need.

Get only the assets with a consensus mark of at least 0.5:

assets = kili.assets(
    your_project_id, fields=["externalId", "id", "consensusMark"], consensus_mark_gte=0.5
)
print(assets)
# + asset conversion code

[{'externalId': 'car_1', 'id': 'clcyuykzd0000bgvze2z3wk81', 'consensusMark': 0.6504290982818591}]

Get all the labels with a honeypot mark above 0.1:

labels = kili.labels(
    your_project_id,
    fields=["labelOf.externalId", "honeypotMark", "author.email", "id"],
    honeypot_mark_gte=0.1,
)
print(labels)
# + label conversion code

[{'labelOf': {'externalId': 'car_1'}, 'author': {'email': 'john.doe@kili-technology.com'}, 'honeypotMark': 0.16527040499137607, 'id': 'clcyuynri2fnl0krf0d7pgabo'}, {'labelOf': {'externalId': 'car_1'}, 'author': {'email': 'john.smith@kili-technology.com'}, 'honeypotMark': 0.20754115450190522, 'id': 'clcyuynri2fnm0krfhx934jee'}]

Get all the labels added by a specific project member:

labels = kili.labels(
    your_project_id, fields=["labelOf.externalId", "author.email", "id"], user_id=john_doe_id
)
print(labels)
# + label conversion code

[{'labelOf': {'externalId': 'car_1'}, 'author': {'email': 'john.doe@kili-technology.com'}, 'id': 'clcyuynri2fnl0krf0d7pgabo'}]

This code will return a list of labels authored by John Doe.

You can also use the author_in parameter to filter by name directly.

Filtering specific labels per asset through the label properties

You can also look for specific labels, for example the last "review" status label per user, and dump the result into a json file. You can use the field "labels.isLatestReviewLabelForUser" to check if the label is the latest per user.

import json

assets = kili.assets(
    your_project_id,
    fields=["externalId", "labels.jsonResponse", "labels.isLatestReviewLabelForUser"],
)

for asset in assets:
    if asset["labels"]:  # check if asset has annotations
        for label in asset["labels"]:
            if label["isLatestReviewLabelForUser"] and "JOB_0" in label["jsonResponse"]:
                annotation = label["jsonResponse"]["JOB_0"]
                with Path(asset["externalId"] + ".json").open("w", encoding="utf-8") as f:
                    f.write(json.dumps(annotation))
                break  # once we find a latest label done by a reviewer, we move on to the next asset.

Filtering the latest label per annotator

When working on a project with consensus enabled, it can be useful to export the latest label made by each annotator:

import tempfile
from collections import defaultdict

assets = kili.assets(
    your_project_id,
    fields=[
        "externalId",
        "labels.author.email",
        "labels.createdAt",
        "labels.labelType",
        "labels.jsonResponse",
    ],
)

for asset in assets:
    if asset["labels"]:
        latest_label_by_user = defaultdict(list)
        for label in asset["labels"]:
            if label["labelType"] == "DEFAULT":
                latest_label_by_user[label["author"]["email"]].append(label)
        latest_label_per_user = {
            email: max(labels, key=lambda x: x["createdAt"])
            for email, labels in latest_label_by_user.items()
        }
        with (Path(tempfile.gettempdir()) / (asset["externalId"] + ".json")).open(
            "w", encoding="utf-8"
        ) as f:
            f.write(json.dumps(latest_label_per_user))

Exporting a whole project

You can export your project data from the Kili UI (see documentation), but Kili SDK also enables you to export your labels and assets into several export formats.

Available formats

Format	UI	Python Client	Command Line Interface
Kili (raw)	✅	✅	✅
YOLO	✅	✅	✅
YOLO V4, V5, V7, V8 (deprecated)	❌	✅	✅
Pascal VOC	✅	✅	✅
COCO	✅	✅	✅
GeoJSON	✅	✅	✅

The .export_labels method

The .export_labels method enables the export of a full project. It does the following preprocessing:

• Only fetches the labels of types "DEFAULT" and "REVIEW" (see the label types explanations).
• If specified, selects a subset of asset ids.
• Exports labels to one of the standard formats (only available for a restricted set of ML tasks).
• Using various method arguments, you can decide:
  • Whether or not to include the assets in the export
  • Whether to export just the latest label or all the labels
  • Whether to create one folder for all the jobs or one folder per job
  • Whether or not to export the label-related data into one single file

Note that some formats are by default single-file, while others use many files:

Format	Single file	Multiple files
Kili	✅	✅
Yolo	❌	✅
Pascal VOC	❌	✅
COCO	✅	❌

For all the formats, in the output archive, a README.kili.txt file is also created. Here is an example of its contents:

Exported Labels from KILI
=========================

- Project name: Awesome annotation project
- Project identifier: abcdefghijklmnop
- Project description: This project contains labels, most of which are awesome.
- Export date: 20221125-093324
- Exported format: kili
- Exported labels: latest

Kili format, one file per asset

The following code snippet exports the whole asset payload and the associated labels, with one json file per asset, into the /tmp/export.zip folder.

kili.export_labels(
    project_id=your_project_id,
    filename="/tmp/export.zip",
    fmt="kili",
)

Fetching assets...
/tmp/export.zip

Kili format, one file for the whole project

This code snippet exports the whole asset payload and the associated labels as one file for the whole project, into the /tmp/export.zip folder.

kili.export_labels(
    project_id=your_project_id,
    filename="/tmp/export.zip",
    fmt="kili",
    single_file=True,
)

Fetching assets...
/tmp/export.zip

YOLO formats

When you have at least one object detection job with a bounding box, polygon or semantic tool, you can also export to the "yolo" format, the YOLO format the Kili app exports: the Ultralytics dataset format, which every YOLO version from v5 to YOLO26 reads. "yolo_v4", "yolo_v5", "yolo_v7" and "yolo_v8" are deprecated and will be removed in a future release: they still work, each as before, with a DeprecationWarning. "yolo_v7" and "yolo_v8" write the files of "yolo", "yolo_v5" another structure of the metadata YAML file (its names as a map), and "yolo_v4" the bounding boxes alone, for Detect, with a classes.txt. In all the cases, a label file is produced per asset and label, containing the labels the export type selects: with several, each file is suffixed _label<n>, beside the asset's image, written once. A video gets a label file per frame holding a shape; frames without any are left out. Each dataset's data.yaml lists its classes and its images for training and validation. With a folder per task or per job, the images are shared in images/ at the root: to train on one dataset, put its labels/ beside images/.

The export writes a dataset for each Ultralytics task the project's jobs give:

Task	Written when the project has	One line per
The bounding boxes' task, by bbox_as	a bounding box job	bounding box
Segment	a polygon or semantic job	polygon and semantic segmentation part

bbox_as chooses how "yolo" writes the bounding boxes:

bbox_as	Task	A bounding box is written as
"box" (default)	Detect	the upright box around it: a rotated box loses its angle
"obb"	OBB	its four corners: a rotated box keeps its angle
"polygon"	Segment, beside the polygons	its four corners

With layout="split" (one folder per job), each job's folder is written for its own task: Segment for a job with a polygon or semantic tool, its bounding boxes as polygons; the bounding boxes' task otherwise. With layout="merged" and two tasks, each dataset goes in its own folder, detect/ or obb/, and segment/, each with its labels/ and metadata file, beside the shared images/. With one task, it is at the root of the archive. Points, lines, pose and classification are not exported.

For bouding boxes, each YOLO label has the following shape:

2        0.25 0.67 0.26 0.34
^        ^    ^    ^    ^
class    x    y    w    h

where:

• class is the class index in the classes list contained in the YOLO metadata file.
• x is the x-coordinate relative to the image width (between 0.0 and 1.0) of the center of the bounding box.
• y is the y-coordinate relative to the image height (between 0.0 and 1.0) of the center of the bounding box.
• w is the width relative to the image width (between 0.0 and 1.0) of the bounding box.
• h is the height relative to the image height (between 0.0 and 1.0) of the bounding box.

For polygons or segmentations, each YOLO label has the following shape:

2        0.25 0.67 0.26 0.34 0.4 0.5 0.6 0.7  ...
^        ^    ^    ^    ^    ^   ^   ^   ^
class    x1   y1   x2   y2   x3  y3  x4  y4   ...

where:

• class is the class index in the classes list contained in the YOLO metadata file.
• xi is the x-coordinate relative to the image width (between 0.0 and 1.0) of the i-th point of the polygon.
• yi is the y-coordinate relative to the image height (between 0.0 and 1.0) of the i-th point of the polygon.

For oriented bounding boxes (bbox_as="obb"), each YOLO label holds the box's four corners, in the same shape with exactly 4 points: class x1 y1 x2 y2 x3 y3 x4 y4.

Here is an example of a YOLO annotation over an image:

Here is how to export to YOLO:

kili.export_labels(
    project_id=your_project_id,
    filename="/tmp/export.zip",
    fmt="yolo",
)

Fetching assets...
/tmp/export.zip

Note that a standard YOLO file format must also include:

• The path root to the assets
• The train, val and test subfolders

Placing specific data in specific folders is the decision of an ML engineer or a Data scientist, so we are not providing a code snippet here.

COCO format

To export your data into the COCO format, run the following code:

kili.export_labels(
    project_id=your_project_id,
    filename="/tmp/export.zip",
    fmt="coco",
)

Fetching assets...
Convert to coco format: 1it [00:00, 54.94it/s]
/tmp/export.zip

This will create an archive containing both:

• The COCO annotation file
• The data/ folder with all the assets

Cleanup

We can remove the project that we created:

kili.delete_project(your_project_id)

Summary

In this tutorial, we have seen several ways to export labels from a Kili project:

• Using .assets and .labels and their filtering arguments, a subset of assets or labels can be selected and then exported.
• Using .export_labels, the whole project can be exported into a standard output format.