# Subtitles

The SDK comes with a `SubtitlesPlugin` that adds support for sideloading custom subtitle tracks from a url as well as enabling subtitles rendering on DASH streams (or any other content played using the castlabs PlayerEngine). It supports the formats: TTML v1, IMSC1, WebVTT and SRT.

## Setup plugin

Ensure the `SubtitlesPlugin` is enabled when registering the SDK, e.g.

    import PRESTOplay
    import CastlabsSubtitles

    let res = PRESTOplaySDK.shared.setup("LICENSE", [SubtitlesPlugin()])

## Side-load subtitle tracks

Subtitle tracks will be automatically generated from the streaming manifests. However you can also use the following method to manually side-load custom tracks:

        var subtitles = PRESTOplaySDK.shared.subtitles(for: player)
        subtitles?.subtitlesStyle = SubtitlesStyle(fromSystemSettings: true)

        let subtitlesTrack = TextTrack(
            id: UUID().uuidString,
            format: .web_vtt,
            isPlatformRendered: false,
            label: "sideloaded",
            language: "en",
            sourceUrl: url)

        if let track = subtitlesTrack {
            subtitles?.addTextTrack(track) { _, error in
                if let error { print(error) } else { print("subtitles track loaded") }
                self.player.setTextTrack(track)
            }
        }

This method downloads and parses the subtitles content. Then it creates a new subtitle track appended after any existing subtitle tracks.

## Styling

We support overriding the text rendering style specified by the subtitle content with a custom user specified style in compliance with FCC requirements. This is done by configuring a `SubtitlesStyle` object and assigning it to the `subtitlesStyle` property of the Player. This property is by default `nil`, which indicates that the style is not overridden and the style defined in the media content will be used.

The style can be overwritten as follows:

    let subtitlesStyle = SubtitlesStyle(fromSystemSettings: false)
    subtitlesStyle.foregroundColor = .red
    subtitlesStyle.edgeType = .dropShadow
    subtitlesStyle.edgeColor = .black
    subtitles?.subtitlesStyle = subtitlesStyle

In addition, the `SubtitlesStyle` object can be pre-populated by reading the values that are currently stored in the iOS/iPadOS system settings (Settings > Accessibility > Subtitles & Captioning).

    let subtitlesStyle = SubtitlesStyle(fromSystemSettings: true)
    subtitles?.subtitlesStyle = subtitlesStyle

NOTE: Subtitles styling may appear differently when using the Apple player engine and some of the styling options may not be available there. See SubtitlesStyle.swift for details

## Styling priority

In order to understand how the style is eventually rendered on the screen, please consider that the style rules can come from three places:

- the content (WebVTT CSS, TTML etc..)
- the iOS/iPadOS Device Accessibility settings
- programmatically set with our SDK

When programmatically setting a custom style with our SDK

- for DASH streams and sideloaded tracks
  - every setting will be applied and it will take priority on the style defined by the content and the device settings
- for HLS streams
  - only a subset of the style settings are programmable in AVPlayer. The settings available there are: fontSizeScale, foregroundColor, backgroundColor, windowColor, typeface and edgeType.
  - if in the Device Settings the `Video Override` flag is disabled for a certain style property, the value of this property cannot be overwritten
  - if `Video Override` is enabled the priority is: content, SDK, system

As an example, this is the result of applying the priority rules above for the `foregroundColor` property of an HLS stream

Content | SDK | Device | Video Override | Result
--- | --- | --- | --- | ---
White | Red | Yellow | YES | White
White | Red | Yellow | NO | Yellow
Undefined | Red | Yellow | YES | Red
Undefined | Red | Yellow | NO | Yellow

## Preview

To make it easier to preview the effect of style changes and to enable creating a UI for style settings the SubtitlesAPI includes the ability to show a preview.

First, load a subtitle track to display in the preview:

        let subtitlesTrack = TextTrack(
            id: UUID().uuidString,
            format: .web_vtt,
            isPlatformRendered: false,
            label: "sideloaded",
            language: "en",
            sourceUrl: url)

        if let track = subtitlesTrack {
            subtitles?.addTextTrack(track) { _, error in
                if let error { print(error) } else { print("subtitles track loaded") }
                self.player.setTextTrack(track)
            }
        }

Then draw the preview by passing the style and a UIView object where it should be shown.

    subtitles?.previewSubtitles(withStyle: subtitlesStyleUserPreferences(), onView: view)

Only the first cue in the subtitle file will be displayed. To change the style simply call this function again with a new style setting.

See `SubtitlePreviewViewController.swift` in the PlaymakerApp example which comes with the SDK package for an example.

## Modification

There are cases when you want to modify subtitles track content before you pass it to the `CastlabsSubtitles` parser or renderer. For example:

- You want to experiment on the client-side, before you modify the backend content
- Text is embedded in segments and it is hard to modify them on the server side
- The text is not standard compliant

`SubtitlesAPI` exposes a modification method:

    subtitles?.onTextSegmentContent = { content, format in
        guard let data = content.data(using: .utf8) else { return content }

        if format == .ttml {
            let parser = FixIMSC1Subtitles(content: content)
            let xml = XMLParser(data: data)
            xml.delegate = parser
            xml.parse()

            return parser.getContent()
        }

        return content
    }

## Formats

Supported subtitles formats:

Type | Supported
--- | ---
SRT | Yes
WebVTT | Yes
TTML | Yes
IMSC1 | Yes*
SMPTE-TT | Yes*

- Embedded tracks in `HLSPlayer` (`CastlabsApple`) streams with `IMSC1` support text profile only. All side-loaded track types are supported.

Native subtitles rendering can be styled via Media Accessibility system settings (https://developer.apple.com/documentation/mediaaccessibility).

## TTML Variants

TTML (Timed Text Markup Language), IMSC1 (IMSC 1 - TTML Profiles for Internet Media Subtitles and Captions 1.0), and SMPTE-TT (Society of Motion Picture and Television Engineers - Timed Text) are all related to subtitle and caption formats used in digital media. While they share similarities, there are some differences between them:

1. TTML (Timed Text Markup Language): TTML is a standardized XML-based markup language used for authoring and distributing timed text content like subtitles, captions, and descriptions across various media delivery platforms. It’s quite versatile and can adapt to different types of media. TTML provides a framework for expressing timed text, but it can have different profiles or subsets to suit specific applications.

2. IMSC1 (IMSC 1 - TTML Profiles for Internet Media Subtitles and Captions 1.0): IMSC1 is a specific profile of TTML designed for internet media. It was created to address the need for a standardized format for subtitles and captions in online content. IMSC1 focuses on interoperability across different devices and platforms for delivering captions over the internet.

3. SMPTE-TT (Society of Motion Picture and Television Engineers - Timed Text): SMPTE-TT is a standard developed by the Society of Motion Picture and Television Engineers. It’s based on TTML and is specifically designed for use in the professional media production and distribution domain. SMPTE-TT aims to ensure interoperability and consistency across various tools and systems used in the broadcasting and film industry.

Key differences between these formats include their specific use cases, profiles, and target industries. While TTML serves as a broader framework, IMSC1 focuses on internet media delivery, and SMPTE-TT is tailored for professional media production and distribution, emphasizing interoperability within that domain.