Subtitles

The SDK comes with a SubtitlesPlugin that adds support for sideloading custom subtitle tracks from a url as well as enabling subtitles rendering on DASH streams (or any other content played using the castlabs PlayerEngine). It supports the formats: TTML v1, IMSC1, WebVTT and SRT.

Setup plugin

Ensure the SubtitlesPlugin is enabled when registering the SDK, e.g.

import PRESTOplay
import CastlabsSubtitles

let res = PRESTOplaySDK.shared.register("LICENSE", [SubtitlesPlugin()])

Side-load subtitle tracks

Subtitle tracks will be automatically generated from the streaming manifests. However you can also use the following method to manually side-load custom tracks:

    var subtitles = PRESTOplaySDK.shared.subtitles(for: player)
    subtitles?.subtitlesStyle = SubtitlesStyle(fromSystemSettings: true)

    let subtitlesTrack = TextTrack(
        id: UUID().uuidString,
        format: .web_vtt,
        isPlatformRendered: false,
        label: "sideloaded",
        language: "en",
        sourceUrl: url)

    if let track = subtitlesTrack {
        subtitles?.addTextTrack(track) { _, error in
            if let error { print(error) } else { print("subtitles track loaded") }
            self.player.setTextTrack(track)
        }
    }

This method downloads and parses the subtitles content. Then it creates a new subtitle track appended after any existing subtitle tracks.

Styling

We support overriding the text rendering style specified by the subtitle content with a custom user specified style in compliance with FCC requirements. This is done by configuring a SubtitlesStyle object and assigning it to the subtitlesStyle property of the Player. This property is by default nil, which indicates that the style is not overridden and the style defined in the media content will be used.

The style can be overwritten as follows:

let subtitlesStyle = SubtitlesStyle(fromSystemSettings: false)
subtitlesStyle.foregroundColor = .red
subtitlesStyle.edgeType = .dropShadow
subtitlesStyle.edgeColor = .black
subtitles?.subtitlesStyle = subtitlesStyle

In addition, the SubtitlesStyle object can be pre-populated by reading the values that are currently stored in the iOS/iPadOS system settings (Settings > Accessibility > Subtitles & Captioning).

let subtitlesStyle = SubtitlesStyle(fromSystemSettings: true)
subtitles?.subtitlesStyle = subtitlesStyle

NOTE: Subtitles styling may appear differently when using the Apple player engine and some of the styling options may not be available there. See SubtitlesStyle.swift for details

Styling priority

In order to understand how the style is eventually rendered on the screen, please consider that the style rules can come from three places:

the content (WebVTT CSS, TTML etc..)
the iOS/iPadOS Device Accessibility settings
programmatically set with our SDK

When programmatically setting a custom style with our SDK

for DASH streams and sideloaded tracks
- every setting will be applied and it will take priority on the style defined by the content and the device settings
for HLS streams
- only a subset of the style settings are programmable in AVPlayer. The settings available there are: fontSizeScale, foregroundColor, backgroundColor, windowColor, typeface and edgeType.
- if in the Device Settings the Video Override flag is disabled for a certain style property, the value of this property cannot be overwritten
- if Video Override is enabled the priority is: content, SDK, system

As an example, this is the result of applying the priority rules above for the foregroundColor property of an HLS stream

Content	SDK	Device	Video Override	Result
White	Red	Yellow	YES	White
White	Red	Yellow	NO	Yellow
Undefined	Red	Yellow	YES	Red
Undefined	Red	Yellow	NO	Yellow

Preview

To make it easier to preview the effect of style changes and to enable creating a UI for style settings the SubtitlesAPI includes the ability to show a preview.

First, load a subtitle track to display in the preview:

    let subtitlesTrack = TextTrack(
        id: UUID().uuidString,
        format: .web_vtt,
        isPlatformRendered: false,
        label: "sideloaded",
        language: "en",
        sourceUrl: url)

    if let track = subtitlesTrack {
        subtitles?.addTextTrack(track) { _, error in
            if let error { print(error) } else { print("subtitles track loaded") }
            self.player.setTextTrack(track)
        }
    }

Then draw the preview by passing the style and a UIView object where it should be shown.

subtitles?.previewSubtitles(withStyle: subtitlesStyleUserPreferences(), onView: view)

Only the first cue in the subtitle file will be displayed. To change the style simply call this function again with a new style setting.

See SubtitlePreviewViewController.swift in the PlaymakerApp example which comes with the SDK package for an example.

Modification

There are cases when you want to modify subtitles track content before you pass it to the CastlabsSubtitles parser or renderer. For example:

You want to experiment on the client-side, before you modify the backend content
Text is embedded in segments and it is hard to modify them on the server side
The text is not standard compliant

SubtitlesAPI exposes a modification method:

subtitles?.onTextSegmentContent = { content, format in
    guard let data = content.data(using: .utf8) else { return content }

    if format == .ttml {
        let parser = FixIMSC1Subtitles(content: content)
        let xml = XMLParser(data: data)
        xml.delegate = parser
        xml.parse()

        return parser.getContent()
    }

    return content
}

Formats

Supported subtitles formats:

Type	Supported
SRT	Yes
WebVTT	Yes
TTML	Yes
IMSC1	Yes*
SMPTE-TT	Yes*

Embedded tracks in HLSPlayer (CastlabsApple) streams with IMSC1 support text profile only. All side-loaded track types are supported.

Native subtitles rendering can be styled via Media Accessibility system settings (https://developer.apple.com/documentation/mediaaccessibility).

TTML Variants

TTML (Timed Text Markup Language), IMSC1 (IMSC 1 - TTML Profiles for Internet Media Subtitles and Captions 1.0), and SMPTE-TT (Society of Motion Picture and Television Engineers - Timed Text) are all related to subtitle and caption formats used in digital media. While they share similarities, there are some differences between them:

TTML (Timed Text Markup Language): TTML is a standardized XML-based markup language used for authoring and distributing timed text content like subtitles, captions, and descriptions across various media delivery platforms. It’s quite versatile and can adapt to different types of media. TTML provides a framework for expressing timed text, but it can have different profiles or subsets to suit specific applications.
IMSC1 (IMSC 1 - TTML Profiles for Internet Media Subtitles and Captions 1.0): IMSC1 is a specific profile of TTML designed for internet media. It was created to address the need for a standardized format for subtitles and captions in online content. IMSC1 focuses on interoperability across different devices and platforms for delivering captions over the internet.
SMPTE-TT (Society of Motion Picture and Television Engineers - Timed Text): SMPTE-TT is a standard developed by the Society of Motion Picture and Television Engineers. It’s based on TTML and is specifically designed for use in the professional media production and distribution domain. SMPTE-TT aims to ensure interoperability and consistency across various tools and systems used in the broadcasting and film industry.

Key differences between these formats include their specific use cases, profiles, and target industries. While TTML serves as a broader framework, IMSC1 focuses on internet media delivery, and SMPTE-TT is tailored for professional media production and distribution, emphasizing interoperability within that domain.