New Closed Caption Features for BrightScript Developers

Over the past few months, Roku has added a number of improvements to closed caption support for its non-legacy devices. First, the Roku platform recently added a new settings UI screen for controlling closed captions across all channels installed on user’s Roku player. The intention of these new global settings is to provide a single place for enabling or disabling closed captions and for setting caption rendering preferences such as caption text font, size, color, and the like. It is Roku’s position that individual channels will respect the global caption settings, thus eliminating the need for channel specific UI to control closed captions. The overall Roku closed caption user experience will be vastly improved as users can set their caption settings in one place and have these settings reflected in all of their channels.

In addition to the global close caption settings, the Roku platform now supports caption rendering in the roVideoPlayer component. Previously captions could only be used in the roVideoScreen. Now developers who build channels with custom UI and video playback using roVideoPlayer can include closed captions as part of their video playback experience.

The Roku SDK now includes new BrightScript components and APIs to let developers take advantage of these features which are described in this article. This article also includes a sample BrightScript channel demonstrating how to use the new APIs. This sample can be downloaded from this link.

Global Closed Caption Settings

Roku supports various industry standard closed caption delivery formats. See this document for details. The 608 and TTML formats also allow content providers to include text styling information in the caption data to control the visual appearance of the caption text on screen. The new Roku closed captions settings UI lets users control turning captions on and off, and also lets them override the caption styling properties provided by the content. The global closed caption settings screen is shown here:

BrightScript developers can query these various global setting values from their channels. The roDeviceInfo component supports two closed caption related functions, GetCaptionsMode and GetCaptionsOption:

    GetCaptionsMode() as String
    GetCaptionsOption(option as String) as String

The first of these functions simply returns a string On, Off, or Instant Replay, indicating the global caption state. In the Instant Replay state, captions are only displayed during playback for the section of video replayed after pressing the instant replay remote button. The GetCaptionsOption function is used to query the state of style settings such as caption text and background color, opacity, etc. A full set of option names that can be passed to this function can be found in the Roku SDK documentation at this link.

The Roku SDK offers several options for supporting closed captions in your BrightScript channels. These options range from doing nothing but configuring your content meta data to point to your caption source and playing content in roVideoScreen, to rendering closed captions completely from scratch within the roVideoPlayer. Let’s look at the simple case first.

Rendering Captions in roVideoScreen

Most BrightScript channels use the roVideoScreen component for video playback. This component provides the fully featured default Roku playback experience, including built in trick play support, so developers need to do very little to play content. roVideoScreen provides closed caption support with the same content meta data driven approach as the other features of the component.

roVideoScreen can render captions delivered as either external files or embedded directly in dedicated tracks in the content stream. For example, a smooth stream with TTML closed captions in the stream can be played with the following code:

    videoScreen = CreateObject("roVideoScreen")
    metadata = {
	Stream : { url: "someurl" }
        StreamFormat : "ism"
        TrackIDAudio: "audio_eng"
        TrackIDSubtitle: "ism/textstream_eng"
  }
    port = CreateObject("roMessagePort")
    videoScreen.SetMessagePort(port)
    videoScreen.SetContent(metaData)
    videoScreen.Show()

If the user has set the global closed caption setting to On, the code above will play the specified video along with the specified closed captions. If the user turns global closed captions to Off, the same channel code will play the video without the captions, even though the caption metadata is specified. BrightScript can override the global closed caption setting if desired by calling the roVideoScreen function ShowSubtitle. This function takes one Boolean argument which specifies whether to show or hide closed captions. There is a similar function, ShowSubtitleOnReplay, that controls whether captions are displayed when the instant replay button is pressed on the remote. Channels generally won’t need to override these global settings. In fact, Roku recommends that channels respect the global settings set by users with the global settings screen. If a channel does for some reason need to override the settings, any changes should only be for that channel session. Once the user exists the channel and later starts it up again, the default caption behavior should be to respect the global settings.

There is currently one case in which your channel does need to call ShowSubtitle and ShowSubtitleOnReplay. In the case of closed captions that are not part of the video stream (such as SRT or TTML captions loaded via the SubtitleUrl metadata value) these functions do need to be called in order for captions to render. In these cases, a function like the one below is helpful:

    Function EnableCaptions(player as object) as void
            di = CreateObject("roDeviceInfo")
            captionsMode = di.GetCaptionsMode()

            if (captionsMode = "On")
                player.ShowSubtitle(true)
                player.ShowSubtitleOnReplay(false)
            else if (captionsMode = "Off")
                player.ShowSubtitle(false)
                player.ShowSubtitleOnReplay(false)
            else if (captionsMode = "Instant Replay")
                player.ShowSubtitle(false)
                player.ShowSubtitleOnReplay(true)
            endif
    End Function

The Roku firmware provides a shortcut to the global closed caption settings from within BrightScript channels. Pressing the * key on the remote during playback in an roVideoScreen will display a dialog like the one shown below:

This interface allows users to control captions within the channel. The last item in the dialog lets the user select the caption track for content that supports multiple languages or audio tracks. Changes made in this dialog are only kept for the current channel session. The global defaults can only be controlled with the global caption settings screen.

Rendering Captions in roVideoPlayer

Although the majority of BrightScript channels use roVideoScreen for content playback, more and more developers are interested in building their own custom Roku playback experiences. Custom video players are built by embedding an roVideoPlayer inside an roScreen or roImageCanvas. Closed captions can be rendered in roVideoPlayer with the help of the new roCaptionRenderer component. This component is documented here.

roCaptionRenderer gives BrightScript developers a lot of flexibility in how to render captions. In some cases, developers may be using roVideoPlayer because they want to render their own custom playback controls, but want the Roku firmware to provide the default closed caption rendering. In other cases, developers may be using roVideoPlayer to completely customize the way closed captions are displayed, from custom placement to custom fonts. Below is a screen shot from the sample channel showing caption rendering with a custom font:

The different levels of caption renderer support are called modes. Mode 1 rendering relies on the Roku firmware to do caption drawing. Mode 2 however requires the BrightScript code to implement all aspects of closed caption rendering within the channel. The mode is set by calling the roCaptionRenderer SetMode function:

    SetMode(mode as Integer) as Void

If the mode is 1, the BrightScript must call the roCaptionRenderer function SetScreen to tell the firmware where to draw:

    SetScreen(screen as Object) as Void

Additionally, if the channel is rendering captions in an roScreen instance (as opposed to an roImageCanvas instance) the script must also call the roCaptionRenderer UpdateCaption() function to tell the firmware to draw the caption text. An roScreen should call UpdateCaption() in response to the isCaptionUpdateRequest event. The table below summarizes the steps required to render captions with a caption renderer in mode 1.

roScreen roImageCanvas
Call SetScreen() Call SetScreen()
Call UpdateCaption()

If the mode is 2, regardless of whether captions are to be rendered in an roScreen or an roImageCanvas instance, the BrightScript code is responsible for doing all of the caption text rendering. In this case, the BrightScript event loop will receive isCaptionText events for each caption string. The channel is then responsible for doing all of the caption string drawing.

BrightScript channels do not directly create instances of roCaptionRenderer. When a channel creates an roVideoPlayer and assigns it to a screen, the Roku firmware creates an roCaptionRenderer. The channel can then get a reference to this caption renderer by calling the new roVideoPlayer GetCaptionRenderer function:

    GetCaptionRenderer() as Object

With this reference, your BrightScript code can set the caption mode, call UpdateCaption, etc. The code below shows how to use the roCaptionRenderer to display TTML captions embedded in a smooth stream using an roVideoPlayer embedded in an roScreen. In this example, we are using roCaptionRenderer mode 1 so the caption string drawing is done in firmware.

    Function Main() as void
        screen = CreateObject("roScreen")
        player = CreateObject("roVideoPlayer")
        port = CreateObject("roMessagePort")
        player.SetMessagePort(port)
        player.SetContentList([
        {
            Stream : {
                url: http://...
            }
           StreamFormat : "ism"
           TrackIDAudio: "audio_eng"
           TrackIDSubtitle: "ism/textstream_eng"
        }
      ])
      captions = player.GetCaptionRenderer()
      captions.SetScreen(screen)
      captions.SetMode(1) ‘Not strictly required as mode 1 is the default
      captions.SetMessagePort(port)
  captions.ShowSubtitle(true)
  player.Play()
  screen.SwapBuffers()
  
  while true
      msg = wait(250, port)
      if (msg  invalid)
	      if type(msg) = "roCaptionRendererEvent"
	          if (msg.isCaptionUpdateRequest())
                      screen.Clear(&h00)
                      captions.UpdateCaption()
                      screen.SwapBuffers()
	          endif
	      endif
      endif
  end while
End Function

The video player content meta data is configured as in any other video channel, then the caption renderer instance is grabbed from the player with the GetCaptionRenderer call. The channel then sets the screen associated with the renderer. This tells the Roku firmware which screen to draw caption strings in for mode 1 rendering. The caption renderer is also assigned a message port so that the BrightScript event loop can receive roCaptionRendererEvents. In the mode 1 case above, the channel will receive isCaptionUpdateRequest events. In response to this event, the channel calls the roCaptionRender UpdateCaption() function, which tells the firmware to draw the specific caption string.

To learn more about Roku closed caption support download the sample channel. The sample is a more fully featured example, showing how to render both mode 1 (firmware drawn) and mode 2 (channel drawn) captions. Happy coding!

About Robert Burdick

Roku Developer Support Manager
This entry was posted in Uncategorized. Bookmark the permalink.
  • Mike

    How is the implemented when using category/categoryLeaf xml files?

  • Joe

    I am using this feature with subtitles part of the stream. However, if the captions are turned off, pressing the info button results in a “No tracks available error” and I can not turn them on. If I have the default set to on, everything works fine

    Why would this be the case?

    • Robert Burdick

      It sounds like you are using captions contained in a separate file. Currently captions that aren’t in stream are not detected by the captions dialog that comes up when pressing the * key during video playback, and instead you see the dialog you are getting. This is being fixed in our next software update.