Thumbnails

The SDK provides thumbnail preview support through ThumbnailsAPI.

Engine overview

ThumbnailsAPI supports two thumbnail engines:

  • .castlabs: Uses ThumbnailsPlugin and supports WebVTT/grid loading with local cache.
  • .apple: Uses native AVPlayer-based preview rendering (renderOn).

PRESTOplaySDK.shared.thumbnails(for:player) defaults to .castlabs.

Setup

Enable plugins during SDK setup.

For castlabs thumbnails:

import PRESTOplay
import CastlabsThumbnails

_ = PRESTOplaySDK.shared.setup("LICENSE", [ThumbnailsPlugin()])

For Apple native player thumbnails, ensure HLSPlugin is enabled.

Basic usage (.castlabs)

Create a player, get thumbnails API, load data, then query images by timestamp:

let player = PRESTOplaySDK.shared.player()
guard let thumbnails = PRESTOplaySDK.shared.thumbnails(for: player, engine: .castlabs),
      let vttURL = URL(string: "https://example.com/thumbs.vtt")
else { return }

thumbnails.loadThumbnailsFrom(webVTTtrack: vttURL) { error in
    guard error == nil else { return }

    if let thumb = thumbnails.getThumbnail(atTimestamp: 12.0) {
        // thumb.imageData, thumb.timeStampStart, thumb.timeStampEnd
    }
}

getThumbnail(atTimestamp: CMTime) is deprecated. Use Double seconds.

WebVTT thumbnails

WebVTT thumbnail tracks can reference single images or grid regions.

Single image entries:

WEBVTT

00:00:00.000 --> 00:00:09.999
thumb1.png

00:00:10.000 --> 00:00:19.999
thumb2.png

The timestamp needs to be in the exact (HH:)MM:SS.MMM format.

Grid-region entries:

WEBVTT

00:00:00.000 --> 00:00:09.999
thumbsGrid.png#0,0,430,242

00:00:10.000 --> 00:00:19.999
thumbsGrid.png#430,0,430,242

Absolute URLs are also supported in WebVTT entries. For bundled local WebVTT file please make sure all the grid images are also bundled at the same path as WebVTT file (or relative to):

or are a valid Url to the image:

WEBVTT

00:00:00.000 --> 00:00:10.000
https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=0,0,262,180

00:00:10.000 --> 00:00:20.000
https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=262,0,262,180

00:00:20.000 --> 00:00:30.000
https://demo.castlabs.com/media/Route66/dash/thumbs/1_7.jpg#xywh=524,0,262,180

Master grid thumbnails

You can define grid-based thumbnails without WebVTT by using GridThumbnail.

Explicit template:

let grid = GridThumbnail(
    baseUrl: URL(string: "https://demo.cf.castlabs.com/media/Route66/dash/thumbs")!,
    pathTemplate: "$index$_7.jpg",
    gridHeight: 10,
    gridWidth: 10,
    durationMs: 10000,
    maxIndex: 7
)

Single URL pattern (max index derived from .../$index$_N.ext):

let grid = GridThumbnail(
    baseUrl: URL(string: "https://demo.cf.castlabs.com/media/Route66/dash/thumbs/$index$_7.jpg")!,
    gridHeight: 10,
    gridWidth: 10,
    durationMs: 10000
)

Load grid thumbnails:

thumbnails.loadThumbnailsFrom(gridThumbnail: grid) { error in
    // Handle completion
}

Cache management

Loaded WebVTT/grid resources are cached on device storage for reuse (including offline reuse).

thumbnails.deleteLocalCache()

I-Frame thumbnails for Apple HLS playback

HLS streams can provide an I-Frame rendition for efficient trick-play and thumbnail previews. The master playlist declares it with #EXT-X-I-FRAME-STREAM-INF:

#EXT-X-I-FRAME-STREAM-INF:AVERAGE-BANDWIDTH=241218,BANDWIDTH=1074106,CODECS="avc1.4D401F",RESOLUTION=1280x720,URI="media-1/iframes.m3u8"

PRESTOplay supports two separate ways to use these thumbnails with Apple HLS playback:

  1. ApplePlayerViewController uses the thumbnail experience provided by AVPlayerViewController. The application does not request or render individual preview images.
  2. The .apple thumbnails engine lets an application render a preview into its own UIView by calling ThumbnailsAPI.

Both approaches rely on AVFoundation to select the preview rendition. Provide a compatible I-Frame playlist for efficient previews. Without one, AVFoundation can instead obtain a frame from a regular video rendition.

ApplePlayerViewController (tvOS)

Use ApplePlayerViewController when the standard player controls and their system-managed thumbnail UI are sufficient. No thumbnail API calls or custom thumbnail view are needed. Include an I-Frame playlist in the HLS master playlist so AVFoundation can present efficient scrub previews.

Native thumbnails client (.apple) — Experimental!

Use the native thumbnails client when the application owns the seek bar or needs to position and style a thumbnail preview itself. It creates a dedicated, muted AVPlayer from the main HLS player’s AVAsset, seeks it to the requested time, and renders the result into the supplied UIView through an AVPlayerLayer.

let thumbnails = PRESTOplaySDK.shared.thumbnails(for: player, engine: .apple)

DispatchQueue.main.async {
    thumbnails?.getThumbnail(atTimestamp: 10.0, renderOn: thumbnailView) { error in
        // Optional completion
    }
}

The native .apple engine supports only getThumbnail(atTimestamp:renderOn:completionHandler:).

Seek tolerance

Configure nativeThumbnailsSeekTolerance on HLSSettings to control how accurately the Apple native thumbnails engine seeks when rendering thumbnails.

Set nativeThumbnailsSeekTolerance to 0 to request exact frame extraction from the video data. Use a non-zero tolerance, such as the default 0.5, to allow AVPlayer to seek to a nearby frame. For HLS streams with I-Frame playlists, this lets native thumbnails use the I-Frame track from the HLS playlist.

let hlsSettings = HLSSettings(nativeThumbnailsSeekTolerance: 0.5)
_ = PRESTOplaySDK.shared.setup("LICENSE", [HLSPlugin(hlsSettings)])

For native preview rendering:

let thumbnails = PRESTOplaySDK.shared.thumbnails(for: player, engine: .apple)

DispatchQueue.main.async {
    thumbnails?.getThumbnail(atTimestamp: 10.0, renderOn: thumbnailView) { error in
        // Optional completion
    }
}

Playback, DRM, and network behavior

The native client thumbnails use a second AVPlayer. The thumbnail player is created when the application requests the .apple thumbnails API, and it shares the main player’s AVAsset. Preview seeks therefore cause AVFoundation to fetch an additional HLS playlist or rendition and the media segments needed for the preview. AVFoundation chooses the rendition, it is commonly a low-bitrate rendition or the I-Frame track, but this is not guaranteed. Set nativeThumbnailsPreferredPeakBitRate to express a maximum desired bitrate for the thumbnail player.

The extra traffic is expected while native thumbnails are in use. Account for it in startup, concurrent-playback, and CDN-request estimates.

For FairPlay-protected HLS, the thumbnail player shares the main player’s AVAsset. It uses the FairPlay content key already acquired for main playback, so requesting native thumbnails does not cause a second FairPlay license request.

I-Frame authoring references