The SDK comes with a `ThumbnailsPlugin` that has the ability to return a _thumbnail_ image for a given timestamp.

![](img/thumbs.png)

## Basic Usage

Ensure the `ThumbnailsPlugin` is enabled when registering the SDK, e.g.

```swift
import PRESTOplay
import CastlabsThumbnails

let res = PRESTOplaySDK.shared.setup("LICENSE", [ThumbnailsPlugin()])
```

Once the player session is created, load the thumbnail images and timing information:

```swift
PRESTOplaySDK.shared.thumbnails(for: player)?.loadThumbnailsFrom(webVTTtrack: url) { error in
    if let error = error {
        // Handle error
    } else {
    // Thumbnails are ready to use
    }
}
```

Currently the thumbnails can be loaded by a WebVTT file or Master Grid Image.

Retrieve the image for a particular point in time:

```swift
let thumbnails = PRESTOplaySDK.shared.thumbnails(for: player)
thumbnails?.getThumbnail(webVTTtrack: url) { error in
    // Handle the error
}

...
if let thumbnail = PRESTOplaySDK.shared.thumbnails(for: player)?.getThumbnail(atTimestamp: timestamp)
// Use thumbnail image data
```

The client code could then, for example, take this data to construct a UIImageView object and then display it on the UI when the player seekbar is being touched.

## WebVTT track

You can encode your thumbnail images using the WebVTT format.

```
WEBVTT

00:00:00.000 --> 00:00:09.999
thumb1.png

00:00:10.000 --> 00:00:19.999
thumb2.png

00:00:20.000 --> 00:00:29.999
thumb3.png
```

The timestamp needs to be in the exact _(HH:)MM:SS.MMM_ format.

We also support the usage of grid images where you can store more than one image at once. You need to append the coordinate to each thumbnail in the WebVTT file

```
WEBVTT

00:00:00.000 --> 00:00:09.999
thumbsGrid.png#0,0,430,242

00:00:10.000 --> 00:00:19.999
thumbsGrid.png#430,0,430,242

00:00:20.000 --> 00:00:29.999
thumbsGrid.png#0,242,430,242
```

![](img/thumbs_grid.jpg)

You can pass a local or remote url to the player.

For bundled local WebVTT file please make sure all the grid images are also bundled at the same path as WebVTT file (or relative to):

![](img/bundled_webvtt.png)

or are a valid Url to the image:

```
WEBVTT

00:00:00.000 --> 00:00:10.000
https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=0,0,262,180

00:00:10.000 --> 00:00:20.000
https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=262,0,262,180

00:00:20.000 --> 00:00:30.000
https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=524,0,262,180
```

## Master grid images

We provide a custom way to generate grid master images.
This is done by initializing a `GridThumbnail` with a few details and passing it to the `loadThumbnailsFrom()` method. E.g.

```swift
let gridInfo = GridThumbnail(
    baseUrl: "https://demo.cf.castlabs.com/media/Route66/dash/thumbs",
    pathTemplate: "$index$_7.jpg",
    gridHeight: 10,
    gridWidth: 10,
    durationMs: 1000,
    maxIndex: 7)

// or
let gridInfo = GridThumbnail(
    baseUrl: "https://demo.cf.castlabs.com/media/Route66/dash/thumbs/$index$_7.jpg",
    gridHeight: 10,
    gridWidth: 10,
    durationMs: 1000)

PRESTOplaySDK.shared.thumbnails(for: player)?.loadThumbnailsFrom(gridThumbnail: gridInfo) { error in
    if let error = error {
        // Handle error
    } else {
    // Thumbnails are ready to use
    }
}
```

Please refer to `GridThumbnail.swift` for details on how to configure this.

## Caching

When `loadThumbnailsFrom()` is called it will store the downloaded thumbnail images (and webVTT file) on the device filesystem. They can then be used while offline in future playback sessions i.e. for offline playback.
This local storage cache can be cleaned up by calling:

```swift
PRESTOplaySDK.shared.thumbnails(for: player)?.deleteLocalCache()
```

## Apple TV player - ApplePlayerViewController

To display thumbnails on the Apple TV player, I-Frames are required in the HLS manifest file.

```
# I-Frame Playlists
#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=241218,BANDWIDTH=1074106,CODECS="avc1.4D401F",RESOLUTION=1280x720,URI="media-1/iframes.m3u8"
```

## Native Apple Thumbnails - Experimental!

Based on `AVPlayer` track selection - `AVPlayer` decides whether to use I-Frames tracks or current playback track to form thumbnails.

To display thumbnails a dedicated `UIView` is needed.

```swift
let thumbnails = PRESTOplaySDK.shared.thumbnails(for: player, engine: .apple)

DispatchQueue.main.async {
    thumbnails?.getThumbnail(
        atTimestamp: position,
        renderOn: thumbnailsView)

    /// animate thumbnails view
}
```

IFrame track requirements are described in the following documents:

1. https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc8216bis#section-4.4.6.3
2. https://developer.apple.com/documentation/http-live-streaming/hls-authoring-specification-for-apple-devices#Trick-Play
