# Thumbnails

The SDK comes with a `ThumbnailsPlugin` that has the ability to return a thumbnail image for a given timestamp.

## Basic Usage

Ensure the `ThumbnailsPlugin` is enabled when registering the SDK, e.g.

    import PRESTOplay
    import CastlabsThumbnails

    let res = PRESTOplaySDK.shared.setup("LICENSE", [ThumbnailsPlugin()])

Once the player session is created, load the thumbnail images and timing information:

    PRESTOplaySDK.shared.thumbnails(for: player)?.loadThumbnailsFrom(webVTTtrack: url) { error in
        if let error = error {
            // Handle error
        } else {
        // Thumbnails are ready to use
        }
    }

Currently the thumbnails can be loaded by a WebVTT file or Master Grid Image.

Retrieve the image for a particular point in time:

    let thumbnails = PRESTOplaySDK.shared.thumbnails(for: player)
    thumbnails?.getThumbnail(webVTTtrack: url) { error in
        // Handle the error
    }

    ...
    if let thumbnail = PRESTOplaySDK.shared.thumbnails(for: player)?.getThumbnail(atTimestamp: timestamp)
    // Use thumbnail image data

The client code could then, for example, take this data to construct a UIImageView object and then display it on the UI when the player seekbar is being touched.

## WebVTT track

You can encode your thumbnail images using the WebVTT format.

    WEBVTT

    00:00:00.000 --> 00:00:09.999
    thumb1.png

    00:00:10.000 --> 00:00:19.999
    thumb2.png

    00:00:20.000 --> 00:00:29.999
    thumb3.png

The timestamp needs to be in the exact (HH:)MM:SS.MMM format.

We also support the usage of grid images where you can store more than one image at once. You need to append the coordinate to each thumbnail in the WebVTT file

    WEBVTT

    00:00:00.000 --> 00:00:09.999
    thumbsGrid.png#0,0,430,242

    00:00:10.000 --> 00:00:19.999
    thumbsGrid.png#430,0,430,242

    00:00:20.000 --> 00:00:29.999
    thumbsGrid.png#0,242,430,242

You can pass a local or remote url to the player.

For bundled local WebVTT file please make sure all the grid images are also bundled at the same path as WebVTT file (or relative to):

or are a valid Url to the image:

    WEBVTT

    00:00:00.000 --> 00:00:10.000
    https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=0,0,262,180

    00:00:10.000 --> 00:00:20.000
    https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=262,0,262,180

    00:00:20.000 --> 00:00:30.000
    https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=524,0,262,180

## Master grid images

We provide a custom way to generate grid master images. This is done by initializing a `GridThumbnail` with a few details and passing it to the `loadThumbnailsFrom()` method. E.g.

    let gridInfo = GridThumbnail(
        baseUrl: "https://demo.cf.castlabs.com/media/Route66/dash/thumbs",
        pathTemplate: "$index$_7.jpg",
        gridHeight: 10,
        gridWidth: 10,
        durationMs: 1000,
        maxIndex: 7)

    // or
    let gridInfo = GridThumbnail(
        baseUrl: "https://demo.cf.castlabs.com/media/Route66/dash/thumbs/$index$_7.jpg",
        gridHeight: 10,
        gridWidth: 10,
        durationMs: 1000)

    PRESTOplaySDK.shared.thumbnails(for: player)?.loadThumbnailsFrom(gridThumbnail: gridInfo) { error in
        if let error = error {
            // Handle error
        } else {
        // Thumbnails are ready to use
        }
    }

Please refer to `GridThumbnail.swift` for details on how to configure this.

## Caching

When `loadThumbnailsFrom()` is called it will store the downloaded thumbnail images (and webVTT file) on the device filesystem. They can then be used while offline in future playback sessions i.e. for offline playback. This local storage cache can be cleaned up by calling:

    PRESTOplaySDK.shared.thumbnails(for: player)?.deleteLocalCache()

## Apple TV player - ApplePlayerViewController

To display thumbnails on the Apple TV player, I-Frames are required in the HLS manifest file.

    # I-Frame Playlists
    #EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=241218,BANDWIDTH=1074106,CODECS="avc1.4D401F",RESOLUTION=1280x720,URI="media-1/iframes.m3u8"

## Native Apple Thumbnails - Experimental!

Based on `AVPlayer` track selection - `AVPlayer` decides whether to use I-Frames tracks or current playback track to form thumbnails.

To display thumbnails a dedicated `UIView` is needed.

    let thumbnails = PRESTOplaySDK.shared.thumbnails(for: player, engine: .apple)

    DispatchQueue.main.async {
        thumbnails?.getThumbnail(
            atTimestamp: position,
            renderOn: thumbnailsView)

        /// animate thumbnails view
    }

IFrame track requirements are described in the following documents:

1. https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc8216bis#section-4.4.6.3
2. https://developer.apple.com/documentation/http-live-streaming/hls-authoring-specification-for-apple-devices#Trick-Play