Label Utils module

This module provides a set of helper functions to read, modify and create labels.

Points

Helpers to create point annotations.

`normalized_point_to_point(point, img_width=None, img_height=None)`

Converts a Kili label format normalized point to a 2D point.

It is the inverse of the method point_to_normalized_point.

A point is a dict with keys 'x' and 'y', and corresponding values in pixels (int or float).

Conventions for the input point:

The origin is the top left corner of the image.
x-axis is horizontal and goes from left to right.
y-axis is vertical and goes from top to bottom.

Conventions for the output point:

The origin is the bottom left corner of the image.
x-axis is horizontal and goes from left to right.
y-axis is vertical and goes from bottom to top.

If the image width and height are provided, the point coordinates will be scaled to the image size. If not, the method will keep the normalized coordinates.

Parameters:

Name	Type	Description	Default
`point`	`Dict[str, float]`	Point to convert.	required
`img_width`	`Union[int, float]`	Width of the image the point is defined in.	`None`
`img_height`	`Union[int, float]`	Height of the image the point is defined in.	`None`

Returns:

Type	Description
`Dict[str, float]`	A dict with keys 'x' and 'y', and corresponding values in pixels.

Source code in kili/utils/labels/point.py

def normalized_point_to_point(
    point: Dict[str, float],
    img_width: Optional[Union[int, float]] = None,
    img_height: Optional[Union[int, float]] = None,
) -> Dict[str, float]:
    # pylint: disable=line-too-long
    """Converts a Kili label format normalized point to a 2D point.

    It is the inverse of the method `point_to_normalized_point`.

    A point is a dict with keys 'x' and 'y', and corresponding values in pixels (int or float).

    Conventions for the input point:

    - The origin is the top left corner of the image.
    - x-axis is horizontal and goes from left to right.
    - y-axis is vertical and goes from top to bottom.

    Conventions for the output point:

    - The origin is the bottom left corner of the image.
    - x-axis is horizontal and goes from left to right.
    - y-axis is vertical and goes from bottom to top.

    If the image width and height are provided, the point coordinates will be scaled to the image size.
    If not, the method will keep the normalized coordinates.

    Args:
        point: Point to convert.
        img_width: Width of the image the point is defined in.
        img_height: Height of the image the point is defined in.

    Returns:
        A dict with keys 'x' and 'y', and corresponding values in pixels.
    """
    if (img_width is None) != (img_height is None):
        raise ValueError("img_width and img_height must be both None or both not None.")

    img_height = img_height or 1
    img_width = img_width or 1

    return {
        "x": point["x"] * img_width,
        "y": (1 - point["y"]) * img_height,
    }

`point_to_normalized_point(point, img_width=None, img_height=None)`

Converts a 2D point to a Kili label format normalized point.

The output can be used to create object detection annotations. See the documentation for more details.

A point is a dict with keys 'x' and 'y', and corresponding values in pixels (int or float).

Conventions for the input point:

The origin is the bottom left corner of the image.
x-axis is horizontal and goes from left to right.
y-axis is vertical and goes from bottom to top.

Conventions for the output point:

The origin is the top left corner of the image.
x-axis is horizontal and goes from left to right.
y-axis is vertical and goes from top to bottom.

If the image width and height are provided, the point coordinates will be normalized to [0, 1]. If not, the method expects the point coordinates to be already normalized.

Parameters:

Name	Type	Description	Default
`point`	`Dict[str, Union[int, float]]`	Point to convert.	required
`img_width`	`Union[int, float]`	Width of the image the point is defined in.	`None`
`img_height`	`Union[int, float]`	Height of the image the point is defined in.	`None`

Returns:

Type	Description
`Dict[str, float]`	A dict with keys 'x' and 'y', and corresponding normalized values.

Example

from kili.utils.labels.point import point_to_normalized_point

normalized_point = point_to_normalized_point({"x": 5, "y": 40}, img_width=100, img_height=100)

json_response = {
    "OBJECT_DETECTION_JOB": {
        "annotations": [
            {
                "point": normalized_point,
                "categories": [{"name": "CLASS_A"}],
                "type": "marker",
            }
        ]
    }
}

Source code in kili/utils/labels/point.py

def point_to_normalized_point(
    point: Dict[str, Union[int, float]],
    img_width: Optional[Union[int, float]] = None,
    img_height: Optional[Union[int, float]] = None,
) -> Dict[str, float]:
    # pylint: disable=line-too-long
    """Converts a 2D point to a Kili label format normalized point.

    The output can be used to create object detection annotations. See the [documentation](https://docs.kili-technology.com/reference/export-object-entity-detection-and-relation) for more details.

    A point is a dict with keys 'x' and 'y', and corresponding values in pixels (int or float).

    Conventions for the input point:

    - The origin is the bottom left corner of the image.
    - x-axis is horizontal and goes from left to right.
    - y-axis is vertical and goes from bottom to top.

    Conventions for the output point:

    - The origin is the top left corner of the image.
    - x-axis is horizontal and goes from left to right.
    - y-axis is vertical and goes from top to bottom.

    If the image width and height are provided, the point coordinates will be normalized to [0, 1].
    If not, the method expects the point coordinates to be already normalized.

    Args:
        point: Point to convert.
        img_width: Width of the image the point is defined in.
        img_height: Height of the image the point is defined in.

    Returns:
        A dict with keys 'x' and 'y', and corresponding normalized values.

    !!! Example
        ```python
        from kili.utils.labels.point import point_to_normalized_point

        normalized_point = point_to_normalized_point({"x": 5, "y": 40}, img_width=100, img_height=100)

        json_response = {
            "OBJECT_DETECTION_JOB": {
                "annotations": [
                    {
                        "point": normalized_point,
                        "categories": [{"name": "CLASS_A"}],
                        "type": "marker",
                    }
                ]
            }
        }
        ```
    """
    if (img_width is None) != (img_height is None):
        raise ValueError("img_width and img_height must be both None or both not None.")

    if img_width is not None and img_height is not None:
        point = {
            "x": point["x"] / img_width,
            "y": point["y"] / img_height,
        }

    assert 0 <= point["x"] <= 1, f"Point x coordinate {point['x']} should be in [0, 1]."
    assert 0 <= point["y"] <= 1, f"Point y coordinate {point['y']} should be in [0, 1]."

    return {"x": point["x"], "y": 1 - point["y"]}

Bounding boxes

Helpers to create boundingPoly rectangle annotations.

`bbox_points_to_normalized_vertices(*, bottom_left, bottom_right, top_right, top_left, img_width=None, img_height=None)`

Converts a bounding box defined by its 4 points in pixels coordinates to normalized vertices.

The output can be used to create a boundingPoly rectangle annotation. See the documentation for more details.

A point is a dict with keys 'x' and 'y', and corresponding values in pixels (int or float).

Conventions for the input points:

The origin is the bottom left corner of the image.
x-axis is horizontal and goes from left to right.
y-axis is vertical and goes from bottom to top.

Conventions for the output vertices:

The origin is the top left corner of the image.
x-axis is horizontal and goes from left to right.
y-axis is vertical and goes from top to bottom.

If the image width and height are provided, the point coordinates will be normalized to [0, 1]. If not, the method expects the points' coordinates to be already normalized.

Parameters:

Name	Type	Description	Default
`bottom_left`	`Dict[str, Union[int, float]]`	Bottom left point of the bounding box.	required
`bottom_right`	`Dict[str, Union[int, float]]`	Bottom right point of the bounding box.	required
`top_right`	`Dict[str, Union[int, float]]`	Top right point of the bounding box.	required
`top_left`	`Dict[str, Union[int, float]]`	Top left point of the bounding box.	required
`img_width`	`Union[int, float]`	Width of the image the bounding box is defined in.	`None`
`img_height`	`Union[int, float]`	Height of the image the bounding box is defined in.	`None`

Returns:

Type	Description
`List[Dict[str, float]]`	A list of normalized vertices.

Example

from kili.utils.labels.bbox import bbox_points_to_normalized_vertices

inputs = {
    bottom_left = {"x": 0, "y": 0},
    bottom_right = {"x": 10, "y": 0},
    top_right = {"x": 10, "y": 10},
    top_left = {"x": 0, "y": 10},
    img_width = 100,
    img_height = 100,
}
normalized_vertices = bbox_points_to_normalized_vertices(**inputs)
json_response = {
    "OBJECT_DETECTION_JOB": {
        "annotations": [
            {
                "boundingPoly": [{"normalizedVertices": normalized_vertices}],
                "categories": [{"name": "CLASS_A"}],
                "type": "rectangle",
            }
        ]
    }
}

Source code in kili/utils/labels/bbox.py

def bbox_points_to_normalized_vertices(
    *,
    bottom_left: Dict[str, Union[int, float]],
    bottom_right: Dict[str, Union[int, float]],
    top_right: Dict[str, Union[int, float]],
    top_left: Dict[str, Union[int, float]],
    img_width: Optional[Union[int, float]] = None,
    img_height: Optional[Union[int, float]] = None,
) -> List[Dict[str, float]]:
    # pylint: disable=line-too-long
    """Converts a bounding box defined by its 4 points in pixels coordinates to normalized vertices.

    The output can be used to create a boundingPoly rectangle annotation. See the [documentation](https://docs.kili-technology.com/reference/export-object-entity-detection-and-relation#standard-object-detection) for more details.

    A point is a dict with keys 'x' and 'y', and corresponding values in pixels (int or float).

    Conventions for the input points:

    - The origin is the bottom left corner of the image.
    - x-axis is horizontal and goes from left to right.
    - y-axis is vertical and goes from bottom to top.

    Conventions for the output vertices:

    - The origin is the top left corner of the image.
    - x-axis is horizontal and goes from left to right.
    - y-axis is vertical and goes from top to bottom.

    If the image width and height are provided, the point coordinates will be normalized to [0, 1].
    If not, the method expects the points' coordinates to be already normalized.

    Args:
        bottom_left: Bottom left point of the bounding box.
        bottom_right: Bottom right point of the bounding box.
        top_right: Top right point of the bounding box.
        top_left: Top left point of the bounding box.
        img_width: Width of the image the bounding box is defined in.
        img_height: Height of the image the bounding box is defined in.

    Returns:
        A list of normalized vertices.

    !!! Example
        ```python
        from kili.utils.labels.bbox import bbox_points_to_normalized_vertices

        inputs = {
            bottom_left = {"x": 0, "y": 0},
            bottom_right = {"x": 10, "y": 0},
            top_right = {"x": 10, "y": 10},
            top_left = {"x": 0, "y": 10},
            img_width = 100,
            img_height = 100,
        }
        normalized_vertices = bbox_points_to_normalized_vertices(**inputs)
        json_response = {
            "OBJECT_DETECTION_JOB": {
                "annotations": [
                    {
                        "boundingPoly": [{"normalizedVertices": normalized_vertices}],
                        "categories": [{"name": "CLASS_A"}],
                        "type": "rectangle",
                    }
                ]
            }
        }
        ```
    """
    assert bottom_left["x"] <= bottom_right["x"], "bottom_left.x must be <= bottom_right.x"
    assert top_left["x"] <= top_right["x"], "top_left.x must be <= top_right.x"
    assert bottom_left["y"] <= top_left["y"], "bottom_left.y must be <= top_left.y"
    assert bottom_right["y"] <= top_right["y"], "bottom_right.y must be <= top_right.y"

    if (img_width is None) != (img_height is None):
        raise ValueError("img_width and img_height must be both None or both not None.")

    vertices = [
        point_to_normalized_point(top_left, img_width=img_width, img_height=img_height),
        point_to_normalized_point(bottom_left, img_width=img_width, img_height=img_height),
        point_to_normalized_point(bottom_right, img_width=img_width, img_height=img_height),
        point_to_normalized_point(top_right, img_width=img_width, img_height=img_height),
    ]

    return vertices

`normalized_vertices_to_bbox_points(normalized_vertices, img_width=None, img_height=None)`

Converts a rectangle normalizedVertices annotation to its 4 points in pixels or in normalized coordinates depending on the image width and height arguments.

It is the inverse of the method bbox_points_to_normalized_vertices.

A point is a dict with keys 'x' and 'y', and corresponding values in pixels (int or float).

Conventions for the input vertices:

The origin is the top left corner of the image.
x-axis is horizontal and goes from left to right.
y-axis is vertical and goes from top to bottom.

Conventions for the output points (top_left, bottom_left, bottom_right, top_right):

The origin is the bottom left corner of the image.
x-axis is horizontal and goes from left to right.
y-axis is vertical and goes from bottom to top.

If the image width and height are provided, the point coordinates will be scaled to the image size. If not, the method will keep the normalized coordinates.

Parameters:

Name	Type	Description	Default
`normalized_vertices`	`List[Dict[str, float]]`	A list of normalized vertices.	required
`img_width`	`Union[int, float]`	Width of the image the bounding box is defined in.	`None`
`img_height`	`Union[int, float]`	Height of the image the bounding box is defined in.	`None`

Returns:

Type	Description
`Dict[str, Dict[str, float]]`	A dict with keys 'top_left', 'bottom_left', 'bottom_right', 'top_right', and corresponding points.

Example

from kili.utils.labels.bbox import normalized_vertices_to_bbox_points

normalized_vertices = label["jsonResponse"]["OBJECT_DETECTION_JOB"]["annotations"][0]["boundingPoly"][0]["normalizedVertices"]
img_height, img_width = 1080, 1920
bbox_points = normalized_vertices_to_bbox_points(normalized_vertices, img_width, img_height)

Source code in kili/utils/labels/bbox.py

def normalized_vertices_to_bbox_points(
    normalized_vertices: List[Dict[str, float]],
    img_width: Optional[Union[int, float]] = None,
    img_height: Optional[Union[int, float]] = None,
) -> Dict[str, Dict[str, float]]:
    # pylint: disable=line-too-long
    """Converts a rectangle normalizedVertices annotation to its 4 points in pixels or in normalized coordinates depending on the image width and height arguments.

    It is the inverse of the method `bbox_points_to_normalized_vertices`.

    A point is a dict with keys 'x' and 'y', and corresponding values in pixels (int or float).

    Conventions for the input vertices:

    - The origin is the top left corner of the image.
    - x-axis is horizontal and goes from left to right.
    - y-axis is vertical and goes from top to bottom.

    Conventions for the output points (top_left, bottom_left, bottom_right, top_right):

    - The origin is the bottom left corner of the image.
    - x-axis is horizontal and goes from left to right.
    - y-axis is vertical and goes from bottom to top.

    If the image width and height are provided, the point coordinates will be scaled to the image size.
    If not, the method will keep the normalized coordinates.

    Args:
        normalized_vertices: A list of normalized vertices.
        img_width: Width of the image the bounding box is defined in.
        img_height: Height of the image the bounding box is defined in.

    Returns:
        A dict with keys 'top_left', 'bottom_left', 'bottom_right', 'top_right', and corresponding points.

    !!! Example
        ```python
        from kili.utils.labels.bbox import normalized_vertices_to_bbox_points

        normalized_vertices = label["jsonResponse"]["OBJECT_DETECTION_JOB"]["annotations"][0]["boundingPoly"][0]["normalizedVertices"]
        img_height, img_width = 1080, 1920
        bbox_points = normalized_vertices_to_bbox_points(normalized_vertices, img_width, img_height)
        ```
    """
    if len(normalized_vertices) != 4:
        raise ValueError("normalized_vertices must have length 4.")

    if (img_width is None) != (img_height is None):
        raise ValueError("img_width and img_height must be both None or both not None.")

    img_height = img_height or 1
    img_width = img_width or 1

    top_left = {}
    bottom_left = {}
    bottom_right = {}
    top_right = {}

    for vertex, point in zip(normalized_vertices, (top_left, bottom_left, bottom_right, top_right)):
        point["x"] = vertex["x"] * img_width
        point["y"] = (1 - vertex["y"]) * img_height

    return {
        "top_left": top_left,
        "bottom_left": bottom_left,
        "bottom_right": bottom_right,
        "top_right": top_right,
    }

Polygon and segmentation masks

Helpers to create boundingPoly polygon and semantic annotations.

`mask_to_normalized_vertices(image)`

Converts a binary mask to a list of normalized vertices using OpenCV cv2.findContours.

The output can be used to create "boundingPoly" polygon or semantic annotations. See the documentation for more details.

Parameters:

Name	Type	Description	Default
`image`	`ndarray`	Binary mask. Should be an array of shape (height, width) with values 0 and 255.	required

Returns:

Type	Description
`Tuple`	A tuple containing a list of normalized vertices and the hierarchy of the contours (see documentation).

Example

import urllib.request
import cv2
from kili.utils.labels.image import mask_to_normalized_vertices

mask_url = "https://raw.githubusercontent.com/kili-technology/kili-python-sdk/master/recipes/img/HUMAN.mask.png"
urllib.request.urlretrieve(mask_url, "mask.png")

img = cv2.imread("mask.png")[:, :, 0]  # keep only height and width
img[200:220, 200:220] = 0  # add a hole in the mask to test the hierarchy

contours, hierarchy = mask_to_normalized_vertices(img)
# hierarchy tells us that the first contour is the outer contour
# and the second one is the inner contour

json_response = {
    "OBJECT_DETECTION_JOB": {
        "annotations": [
            {
                "boundingPoly": [
                    {"normalizedVertices": contours[0]},  # outer contour
                    {"normalizedVertices": contours[1]},  # inner contour
                ],
                "categories": [{"name": "A"}],
                "type": "semantic",
            }
        ]
    }
}

Source code in kili/utils/labels/image.py

def mask_to_normalized_vertices(
    image: np.ndarray,
) -> Tuple[List[List[Dict[str, float]]], np.ndarray]:
    # pylint: disable=line-too-long
    """Converts a binary mask to a list of normalized vertices using OpenCV [cv2.findContours](https://docs.opencv.org/4.7.0/d3/dc0/group__imgproc__shape.html#gadf1ad6a0b82947fa1fe3c3d497f260e0).

    The output can be used to create "boundingPoly" polygon or semantic annotations.
    See the [documentation](https://docs.kili-technology.com/reference/export-object-entity-detection-and-relation#standard-object-detection) for more details.

    Args:
        image: Binary mask. Should be an array of shape (height, width) with values 0 and 255.

    Returns:
        Tuple: A tuple containing a list of normalized vertices and the hierarchy of the contours (see [documentation](https://docs.opencv.org/4.7.0/d9/d8b/tutorial_py_contours_hierarchy.html)).

    !!! Example
        ```python
        import urllib.request
        import cv2
        from kili.utils.labels.image import mask_to_normalized_vertices

        mask_url = "https://raw.githubusercontent.com/kili-technology/kili-python-sdk/master/recipes/img/HUMAN.mask.png"
        urllib.request.urlretrieve(mask_url, "mask.png")

        img = cv2.imread("mask.png")[:, :, 0]  # keep only height and width
        img[200:220, 200:220] = 0  # add a hole in the mask to test the hierarchy

        contours, hierarchy = mask_to_normalized_vertices(img)
        # hierarchy tells us that the first contour is the outer contour
        # and the second one is the inner contour

        json_response = {
            "OBJECT_DETECTION_JOB": {
                "annotations": [
                    {
                        "boundingPoly": [
                            {"normalizedVertices": contours[0]},  # outer contour
                            {"normalizedVertices": contours[1]},  # inner contour
                        ],
                        "categories": [{"name": "A"}],
                        "type": "semantic",
                    }
                ]
            }
        }
        ```
    """
    if image.ndim > 2:
        raise ValueError(f"Image should be a 2D array, got {image.ndim}D array")

    unique_values = np.unique(image).tolist()
    if not all(value in [0, 255] for value in unique_values):
        raise ValueError(f"Image should be binary with values 0 and 255, got {unique_values}")

    img_height, img_width = image.shape
    # pylint:disable=no-member
    contours, hierarchy = cv2.findContours(image, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

    contours = [
        _opencv_contour_to_normalized_vertices(contour, img_width, img_height)
        for contour in contours
    ]
    hierarchy = hierarchy[0]

    return contours, hierarchy

`normalized_vertices_to_mask(normalized_vertices, img_width, img_height)`

Converts a Kili label with normalized vertices to a binary mask.

It is the inverse of the method mask_to_normalized_vertices.

Parameters:

Name	Type	Description	Default
`normalized_vertices`	`List[Dict[str, float]]`	A list of normalized vertices.	required
`img_width`	`Union[int, float]`	Width of the image the segmentation is defined in.	required
`img_height`	`Union[int, float]`	Height of the image the segmentation is defined in.	required

Returns:

Type	Description
`ndarray`	A numpy array of shape (height, width) with values 0 and 255.

Example

from kili.utils.labels.image import normalized_vertices_to_mask

normalized_vertices = label["jsonResponse"]["OBJECT_DETECTION_JOB"]["annotations"][0]["boundingPoly"][0]["normalizedVertices"]
img_height, img_width = 1080, 1920
mask = normalized_vertices_to_mask(normalized_vertices, img_width, img_height)
plt.imshow(mask)
plt.show()

Source code in kili/utils/labels/image.py

def normalized_vertices_to_mask(
    normalized_vertices: List[Dict[str, float]],
    img_width: Union[int, float],
    img_height: Union[int, float],
) -> np.ndarray:
    # pylint: disable=line-too-long
    """Converts a Kili label with normalized vertices to a binary mask.

    It is the inverse of the method `mask_to_normalized_vertices`.

    Args:
        normalized_vertices: A list of normalized vertices.
        img_width: Width of the image the segmentation is defined in.
        img_height: Height of the image the segmentation is defined in.

    Returns:
        A numpy array of shape (height, width) with values 0 and 255.

    !!! Example
        ```python
        from kili.utils.labels.image import normalized_vertices_to_mask

        normalized_vertices = label["jsonResponse"]["OBJECT_DETECTION_JOB"]["annotations"][0]["boundingPoly"][0]["normalizedVertices"]
        img_height, img_width = 1080, 1920
        mask = normalized_vertices_to_mask(normalized_vertices, img_width, img_height)
        plt.imshow(mask)
        plt.show()
        ```
    """
    mask = np.zeros((img_height, img_width), dtype=np.uint8)
    polygon = [
        [
            int(round(vertice["x"] * img_width)),
            int(round(vertice["y"] * img_height)),
        ]
        for vertice in normalized_vertices
    ]
    polygon = np.array([polygon])
    cv2.fillPoly(img=mask, pts=polygon, color=255)  # pylint:disable=no-member
    return mask