Skip to content

tracks

sleap.gui.overlays.tracks

Track trail and track list overlays.

Classes:

Name Description
TrackListOverlay

Class to show track number and names in overlay.
TrackTrailOverlay

Class to show track trails as overlay on video frame.

TrackListOverlay

Bases: BaseOverlay

Class to show track number and names in overlay.

Methods:

Name Description
add_to_scene

Adds track list as overlay on video.

Attributes:

Name Type Description
visible

Gets or set whether overlay is visible.
Source code in sleap/gui/overlays/tracks.py 
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
@attr.s(auto_attribs=True)
class TrackListOverlay(BaseOverlay):
    """Class to show track number and names in overlay."""

    text_box: Optional[QtTextWithBackground] = None

    def add_to_scene(self, video: Video, frame_idx: int):
        """Adds track list as overlay on video."""

        html = "Tracks:"
        num_to_show = min(9, len(self.labels.tracks))

        for i, track in enumerate(self.labels.tracks[:num_to_show]):
            idx = i + 1

            if html:
                html += "<br />"
            color = self.player.color_manager.get_track_color(track)
            html_color = f"#{color[0]:02X}{color[1]:02X}{color[2]:02X}"
            track_text = f"<b>{track.name}</b>"
            if str(idx) != track.name:
                track_text += f" ({idx})"
            html += f"<span style='color:{html_color}'>{track_text}</span>"

        text_box = QtTextWithBackground()
        text_box.setDefaultTextColor(QtGui.QColor("white"))
        text_box.setHtml(html)
        text_box.setOpacity(0.7)

        self.text_box = text_box
        self.visible = False

        self.player.scene.addItem(self.text_box)

    @property
    def visible(self):
        """Gets or set whether overlay is visible."""
        if self.text_box is None:
            return False
        return self.text_box.isVisible()

    @visible.setter
    def visible(self, val):
        if self.text_box is None:
            return
        if val:
            pos = self.player.view.mapToScene(10, 10)
            if pos.x() > 0:
                self.text_box.setPos(pos)
            else:
                self.text_box.setPos(10, 10)
        self.text_box.setVisible(val)

visible property writable

Gets or set whether overlay is visible.

add_to_scene(video, frame_idx)

Adds track list as overlay on video.

Source code in sleap/gui/overlays/tracks.py 
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
def add_to_scene(self, video: Video, frame_idx: int):
    """Adds track list as overlay on video."""

    html = "Tracks:"
    num_to_show = min(9, len(self.labels.tracks))

    for i, track in enumerate(self.labels.tracks[:num_to_show]):
        idx = i + 1

        if html:
            html += "<br />"
        color = self.player.color_manager.get_track_color(track)
        html_color = f"#{color[0]:02X}{color[1]:02X}{color[2]:02X}"
        track_text = f"<b>{track.name}</b>"
        if str(idx) != track.name:
            track_text += f" ({idx})"
        html += f"<span style='color:{html_color}'>{track_text}</span>"

    text_box = QtTextWithBackground()
    text_box.setDefaultTextColor(QtGui.QColor("white"))
    text_box.setHtml(html)
    text_box.setOpacity(0.7)

    self.text_box = text_box
    self.visible = False

    self.player.scene.addItem(self.text_box)

TrackTrailOverlay

Bases: BaseOverlay

Class to show track trails as overlay on video frame.

Initialize this object with both its data source and its visual output scene, and it handles both extracting the relevant data for a given frame and plotting it in the output.

Attributes:

Name Type Description
labels Labels | None

The :class:Labels dataset from which to get overlay data.
player QtVideoPlayer | None

The video player in which to show overlay.
trail_length int

The maximum number of frames to include in trail.
trail_shade str

A literal "Dark", "Normal", or "Bright" which determines the shade of the trail color.
Usage

After class is instantiated, call :meth:add_to_scene(frame_idx) to plot the trails in scene.

Methods:

Name Description
__attrs_post_init__

Initialize the shade options attribute after initalizing the instance.
add_to_scene

Plot the trail on a given frame.
get_frame_selection

Return LabeledFrame objects to include in trail for specified frame.
get_shade_options

Return a dictionary with values to multiply each RGB value by.
get_track_trails

Get data needed to draw track trail.
get_tracks_in_frame

Returns list of tracks that have instance in specified frame.
map_to_qt_path

Converts a list of (x, y)-tuples to a QPainterPath.
Source code in sleap/gui/overlays/tracks.py 
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
@attr.s(auto_attribs=True)
class TrackTrailOverlay(BaseOverlay):
    """Class to show track trails as overlay on video frame.

    Initialize this object with both its data source and its visual output
    scene, and it handles both extracting the relevant data for a given
    frame and plotting it in the output.

    Attributes:
        labels: The :class:`Labels` dataset from which to get overlay data.
        player: The video player in which to show overlay.
        trail_length: The maximum number of frames to include in trail.
        trail_shade: A literal "Dark", "Normal", or "Bright" which determines the shade
            of the trail color.

    Usage:
        After class is instantiated, call :meth:`add_to_scene(frame_idx)`
        to plot the trails in scene.
    """

    trail_length: int = 0
    trail_shade: str = attr.ib(
        default="Normal", validator=attr.validators.in_(["Dark", "Normal", "Light"])
    )
    show: bool = True
    max_node_count: Optional[int] = None

    def __attrs_post_init__(self):
        """Initialize the shade options attribute after initalizing the instance."""

        self.shade_options = self.get_shade_options()

    @classmethod
    def get_length_options(cls):
        if prefs["trail length"] != 0:
            return (0, 10, 50, 100, 250, 500, prefs["trail length"])
        return (0, 10, 50, 100, 250, 500)

    @classmethod
    def get_shade_options(cls):
        """Return a dictionary with values to multiply each RGB value by."""

        return {"Dark": 0.6, "Normal": 1.0, "Light": 1.25}

    def get_track_trails(
        self, frame_selection: Iterable["LabeledFrame"]
    ) -> Optional[Dict[Track, List[List[Tuple[float, float]]]]]:
        """Get data needed to draw track trail.

        Args:
            frame_selection: an iterable with the :class:`LabeledFrame`
                objects to include in trail.

        Returns:
            Dictionary keyed by track, value is list of lists of (x, y) tuples
                i.e., for every node in instance, we get a list of positions
        """

        all_track_trails = dict()

        if not frame_selection:
            return

        nodes = self.labels.skeletons[0].nodes
        max_node_count = self.max_node_count or prefs["trail node count"]
        if len(nodes) > max_node_count:
            nodes = nodes[:max_node_count]

        for frame in frame_selection:
            # Prefer user instances over predicted instances
            for inst in get_instances_to_show(frame):
                if inst.track is not None:
                    if inst.track not in all_track_trails:
                        all_track_trails[inst.track] = [[] for _ in range(len(nodes))]

                    # loop through all nodes
                    for node_i, node in enumerate(nodes):
                        if (
                            node.name in inst.points["name"]
                            and inst.points["visible"][node_i]
                        ):
                            point = (
                                inst.points["xy"][node_i][0],
                                inst.points["xy"][node_i][1],
                            )

                        # Add last location of node so that we can easily
                        # calculate trail length (since we adjust opacity).
                        elif len(all_track_trails[inst.track][node_i]):
                            point = all_track_trails[inst.track][node_i][-1]
                        else:
                            point = None

                        if point is not None:
                            all_track_trails[inst.track][node_i].append(point)

        return all_track_trails

    def get_frame_selection(self, video: Video, frame_idx: int):
        """Return `LabeledFrame` objects to include in trail for specified frame."""

        frame_selection = self.labels.find(video, range(0, frame_idx + 1))
        frame_selection.sort(key=lambda x: x.frame_idx)

        return frame_selection[-self.trail_length :]

    def get_tracks_in_frame(
        self, video: Video, frame_idx: int, include_trails: bool = False
    ) -> List[Track]:
        """Returns list of tracks that have instance in specified frame.

        Args:
            video: Video for which we want tracks.
            frame_idx: Frame index for which we want tracks.
            include_trails: Whether to include tracks which aren't in current
                frame but would be included in trail (i.e., previous frames
                within trail_length).
        Returns:
            List of tracks.

        """

        if include_trails:
            lfs = self.get_frame_selection(video, frame_idx)
        else:
            lfs = self.labels.find(video, frame_idx)

        tracks_in_frame = [inst.track for lf in lfs for inst in lf]

        return tracks_in_frame

    def add_to_scene(self, video: Video, frame_idx: int):
        """Plot the trail on a given frame.

        Args:
            video: current video
            frame_idx: index of the frame to which the trail is attached

        """
        self.items = []

        if not self.show or self.trail_length == 0:
            return

        frame_selection = self.get_frame_selection(video, frame_idx)

        all_track_trails = self.get_track_trails(frame_selection)
        if all_track_trails is None:
            return

        for track, trails in all_track_trails.items():
            trail_color = tuple(
                min(c * self.shade_options[self.trail_shade], 255)
                for c in self.player.color_manager.get_track_color(track)
            )
            color = QtGui.QColor(*trail_color)
            pen = QtGui.QPen()
            pen.setCosmetic(True)
            pen.setColor(color)

            seg_count = 2 if self.trail_length <= 50 else 3
            seg_len = self.trail_length // seg_count

            for trail in trails:
                if not trail:
                    continue

                # Break list into fixed length segments so that shorter trails
                # will still have the same number of frames in the earlier
                # segments and will just have shorter or missing later segments.

                segments = []
                for seg_idx in range(seg_count):
                    start = max(0, len(trail) - (seg_idx + 1) * seg_len)
                    end = min(len(trail), 1 + len(trail) - seg_idx * seg_len)
                    segments.append(trail[start:end])
                    if start == 0:
                        break

                # Draw each segment, which each later segment (i.e., the part of
                # trail further back from current frame) with a thinner line.

                width = prefs["trail width"]
                for segment in segments:
                    pen.setWidthF(width)
                    path = self.map_to_qt_path(segment)
                    item = self.player.scene.addPath(path, pen)
                    self.items.append(item)
                    width /= 2

    @staticmethod
    def map_to_qt_path(point_list):
        """Converts a list of (x, y)-tuples to a `QPainterPath`."""
        if not point_list:
            return QtGui.QPainterPath()

        path = QtGui.QPainterPath(QtCore.QPointF(*point_list[0]))
        for point in point_list:
            path.lineTo(*point)
        return path

__attrs_post_init__()

Initialize the shade options attribute after initalizing the instance.

Source code in sleap/gui/overlays/tracks.py 
44
45
46
47
def __attrs_post_init__(self):
    """Initialize the shade options attribute after initalizing the instance."""

    self.shade_options = self.get_shade_options()

add_to_scene(video, frame_idx)

Plot the trail on a given frame.

Parameters:

Name Type Description Default
video Video

current video

 required
frame_idx int

index of the frame to which the trail is attached

 required
Source code in sleap/gui/overlays/tracks.py 
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def add_to_scene(self, video: Video, frame_idx: int):
    """Plot the trail on a given frame.

    Args:
        video: current video
        frame_idx: index of the frame to which the trail is attached

    """
    self.items = []

    if not self.show or self.trail_length == 0:
        return

    frame_selection = self.get_frame_selection(video, frame_idx)

    all_track_trails = self.get_track_trails(frame_selection)
    if all_track_trails is None:
        return

    for track, trails in all_track_trails.items():
        trail_color = tuple(
            min(c * self.shade_options[self.trail_shade], 255)
            for c in self.player.color_manager.get_track_color(track)
        )
        color = QtGui.QColor(*trail_color)
        pen = QtGui.QPen()
        pen.setCosmetic(True)
        pen.setColor(color)

        seg_count = 2 if self.trail_length <= 50 else 3
        seg_len = self.trail_length // seg_count

        for trail in trails:
            if not trail:
                continue

            # Break list into fixed length segments so that shorter trails
            # will still have the same number of frames in the earlier
            # segments and will just have shorter or missing later segments.

            segments = []
            for seg_idx in range(seg_count):
                start = max(0, len(trail) - (seg_idx + 1) * seg_len)
                end = min(len(trail), 1 + len(trail) - seg_idx * seg_len)
                segments.append(trail[start:end])
                if start == 0:
                    break

            # Draw each segment, which each later segment (i.e., the part of
            # trail further back from current frame) with a thinner line.

            width = prefs["trail width"]
            for segment in segments:
                pen.setWidthF(width)
                path = self.map_to_qt_path(segment)
                item = self.player.scene.addPath(path, pen)
                self.items.append(item)
                width /= 2

get_frame_selection(video, frame_idx)

Return LabeledFrame objects to include in trail for specified frame.

Source code in sleap/gui/overlays/tracks.py 
115
116
117
118
119
120
121
def get_frame_selection(self, video: Video, frame_idx: int):
    """Return `LabeledFrame` objects to include in trail for specified frame."""

    frame_selection = self.labels.find(video, range(0, frame_idx + 1))
    frame_selection.sort(key=lambda x: x.frame_idx)

    return frame_selection[-self.trail_length :]

get_shade_options() classmethod

Return a dictionary with values to multiply each RGB value by.

Source code in sleap/gui/overlays/tracks.py 
55
56
57
58
59
@classmethod
def get_shade_options(cls):
    """Return a dictionary with values to multiply each RGB value by."""

    return {"Dark": 0.6, "Normal": 1.0, "Light": 1.25}

get_track_trails(frame_selection)

Get data needed to draw track trail.

Parameters:

Name Type Description Default
frame_selection Iterable[LabeledFrame]

an iterable with the :class:LabeledFrame objects to include in trail.

 required

Returns:

Type Description
Optional[Dict[Track, List[List[Tuple[float, float]]]]]

Dictionary keyed by track, value is list of lists of (x, y) tuples i.e., for every node in instance, we get a list of positions
Source code in sleap/gui/overlays/tracks.py 
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
def get_track_trails(
    self, frame_selection: Iterable["LabeledFrame"]
) -> Optional[Dict[Track, List[List[Tuple[float, float]]]]]:
    """Get data needed to draw track trail.

    Args:
        frame_selection: an iterable with the :class:`LabeledFrame`
            objects to include in trail.

    Returns:
        Dictionary keyed by track, value is list of lists of (x, y) tuples
            i.e., for every node in instance, we get a list of positions
    """

    all_track_trails = dict()

    if not frame_selection:
        return

    nodes = self.labels.skeletons[0].nodes
    max_node_count = self.max_node_count or prefs["trail node count"]
    if len(nodes) > max_node_count:
        nodes = nodes[:max_node_count]

    for frame in frame_selection:
        # Prefer user instances over predicted instances
        for inst in get_instances_to_show(frame):
            if inst.track is not None:
                if inst.track not in all_track_trails:
                    all_track_trails[inst.track] = [[] for _ in range(len(nodes))]

                # loop through all nodes
                for node_i, node in enumerate(nodes):
                    if (
                        node.name in inst.points["name"]
                        and inst.points["visible"][node_i]
                    ):
                        point = (
                            inst.points["xy"][node_i][0],
                            inst.points["xy"][node_i][1],
                        )

                    # Add last location of node so that we can easily
                    # calculate trail length (since we adjust opacity).
                    elif len(all_track_trails[inst.track][node_i]):
                        point = all_track_trails[inst.track][node_i][-1]
                    else:
                        point = None

                    if point is not None:
                        all_track_trails[inst.track][node_i].append(point)

    return all_track_trails

get_tracks_in_frame(video, frame_idx, include_trails=False)

Returns list of tracks that have instance in specified frame.

Parameters:

Name Type Description Default
video Video

Video for which we want tracks.

 required
frame_idx int

Frame index for which we want tracks.

 required
include_trails bool

Whether to include tracks which aren't in current frame but would be included in trail (i.e., previous frames within trail_length).

 False
Source code in sleap/gui/overlays/tracks.py 
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
def get_tracks_in_frame(
    self, video: Video, frame_idx: int, include_trails: bool = False
) -> List[Track]:
    """Returns list of tracks that have instance in specified frame.

    Args:
        video: Video for which we want tracks.
        frame_idx: Frame index for which we want tracks.
        include_trails: Whether to include tracks which aren't in current
            frame but would be included in trail (i.e., previous frames
            within trail_length).
    Returns:
        List of tracks.

    """

    if include_trails:
        lfs = self.get_frame_selection(video, frame_idx)
    else:
        lfs = self.labels.find(video, frame_idx)

    tracks_in_frame = [inst.track for lf in lfs for inst in lf]

    return tracks_in_frame

map_to_qt_path(point_list) staticmethod

Converts a list of (x, y)-tuples to a QPainterPath.

Source code in sleap/gui/overlays/tracks.py 
207
208
209
210
211
212
213
214
215
216
@staticmethod
def map_to_qt_path(point_list):
    """Converts a list of (x, y)-tuples to a `QPainterPath`."""
    if not point_list:
        return QtGui.QPainterPath()

    path = QtGui.QPainterPath(QtCore.QPointF(*point_list[0]))
    for point in point_list:
        path.lineTo(*point)
    return path