Netflix Explains The Streaming Closed Caption Problem

Neil Hunt, Netflix's Chief Product Officer, on the Netflix blog explains why it's so hard for Netflix to add closed captions to the streaming titles:

"You might be asking how it could be so hard, since we already subtitle foreign language streams with English subtitles. These subtitles are "burned in" to the video stream at the time of encoding - they are so-called "open captions" that cannot be turned on and off by the viewer. The majority of viewers would object to English captions on English content, so we have to figure out how to let individual viewers turn them on and off.

Encoding a separate stream for each title is not an option - it takes us about 500 processor-months to make one encode through the entire library, and for this we would have to re-encode four different formats. Duplicating the encoded streams is prohibitive in space too.

So we are working on optionally delivering the SAMI file (Synchronized Accessible Media Interchange), or similar, to the client, and having it render the text and then overlay it on the video at playback time. Unfortunately, the tools for rendering SAMI files in Silverlight, or in CE (Consumer Electronics) devices, are weak or non-existent, and there is some technology development required.

I would expect to deliver subtitles or captions to Silverlight clients sometime in 2010, and roll the same technology out to each CE device as we are able to migrate the technology, and work with the CE manufacturer to deliver firmware updates for each player."

Comments

[Complication]

You would think that they would encode at least one low quality version with the burned in subtitles. This would give a solution for the hearing impaired will they work on the selectable subtitle problem.

[Hunter McDaniel]

net is this. Microsoft spent so much of there Silverlight budget on copy-protection that they didn't have any time left for basic playback features.

[CyberPhoenix]

When is Netflix going to dump Microsoft's low quality Silverblight infection?

"Watch Out. Moonlight/Silverlight Knock on Your Door. All Your Media Mustn’t Belongs to Microsoft."

"Microsoft hopes to essentially ‘hijack’ the media industry. Let’s just see how"

http://boycottnovell.com/2008/01/27/media-domination-strategy/

[duracalle]

2010? I've been using divx with added subtitles (just an overlayed text file) working perfeclty for 10 years. How can it be so hard? It's infurating.

[CyberPhoenix]

@Hunter McDaniel -

"Microsoft spent so much of there Silverlight budget on copy-protection that they didn't have any time left for basic playback features"

That's correct!!!

"Silver Lie: The New Music DRM"

"Microsoft was unable to leverage Silverlight and elevate XAML as an alternative to HTML which is encumbered with patents, DRM, and other nasties"

"Much like music DRM, no matter how hard Microsoft fights for it, it doesn’t get any real traction. Microsoft, in turn, blames the economy"

"Silverlight had failed to gain momentum and there were also substantiated accusations that Microsoft was faking acceptance. Exceptions appear to be Microsoft’s media allies"

http://boycottnovell.com/2009/01/28/silverlight-fails/

Netflix should try creating video software that doesn't require low quality vendor lock-in Microsoft trash.

[Gaurav]

How do Hulu and Youtube do it?

[colin]

"How do Hulu and Youtube do it?"

They both transport video as Flash and appear to use Synchronized Multimedia Integration Language (SMIL) for CC. This SAMI issue truly is a self imposed problem created by Netflix's decision to use a half-baked Microsoft technology.

I initially accepted at face value Netflix's contention they had to use Silverlight (and whatever their previous Microsoft only technology was) due to studio licensing restrictions.

Now that I think about it, the fact Hulu has to negotiate with the same studios and was able to use a more user sympathetic technology suggests Netflix simply took the path of least resistance for them in choosing Silverlight. This whole problem of unprepared CC technology could have been avoided.

[Mr Nethead]

*sigh*

Hulu has captions on some videos. NBC, ABC and FOX have captions on some videos. Even CNET has captions on many of their videos. Of course, none of those other companies are using Silverlight. :|

Oh well. Hopefully they really will have this problem fixed in 2010. I'm not counting on it though.

[Wayno]

The captioning ALREADY exists on DVD'S - it's already there, and encoded!

No encoding required. I recognise BS when I see it. But you guys deliberately strip it off. If I rent the DVD version, I have closed captioning available.

I think you need to dump $ilverlight/micro$soft and get a format that works.

Wayno

[Seth]

Reed, I appreciate your commitment to the problem. I know you're sincere.

But I think in the future you'll come to realize Silverlight was an expensive detour for Netflix. In addition to the CC problem, Siverlight suffers from all the typical Microsoft problems. As you know, it's a delivery system that has gained little widespread support. And its evolution has been slow.

You'd almost be better off creating another contest for an open source system that meets your needs. It almost already exists. The community just needs a little nudge.

I have always been a huge fan of instant viewing and since the switch to Silverlight I have found my enjoyment has been encumbered by Silverlight's slow loading and glitches.

Ya gotta take a big bite on this. It's just going to put Netflix behind other companies in the future. Hopefully you're already developing something in parallel?

[JJ]

I hope Netflix doesn't adopt Hulu's technology. I (along with many other Hulu users) have noticed choppy playback and it makes the Hulu experience something to desire. It is hard to understand why Hulu's site has problems since I had no problems with video playback about 2-3 months ago.

[Mr. Nethead]

Isn't Reed Hastings on the board of directors at Microsoft? Considering that, is it any surprise that Netflix uses Silverlight?

[Scrammm]

The captioning ALREADY exists on DVD'S - it's already there, and encoded!

Yes, in a separate file that can be overlaid by the DVD player. That you would compare a physical disc with many files to a single stream from the net is truly baffling.

I've been using divx with added subtitles (just an overlayed text file) working perfeclty for 10 years.

Divx is a container format. They can embed a separate cpations file (XSUB). It's not so easy with a streamed file.

[Hunter McDaniel]

"Divx is a container format. They can embed a separate cpations file (XSUB). It's not so easy with a streamed file."

I don't really buy that. They already have to match up two streams (audio and video). Subtitles are just a third stream.

[El Moon]

ROKU players or streaming to a PC or now DVD players is worthless with CC - there allot of people (young and old) that cannot hear well and rely on CC to catch everthing.

NETFLIX cannot tells if it will ever be available. It also doesn't work from other providers like AMAZON.

No wonder NETFLIX gives it aware (instant watch).

[ncmacasl]

Here is a list of Netflix InstantWatch titles with English Subtitles (now available on the Wii & PS3 as well as PC/Macs -- currently at 125 titles):
http://ncmacasl.blogspot.com/2010/10/netflix-instantwatch-titles-with.html

[Eric]

I thought it was just me! I'm so glad to see I'm not alone in being pissed off at netflix for not having cc available. Haha so it's all in the name of copyright protection? Christ.. Makes me not want to do business with them.

[Dave Sills]

My wife is deaf. We recently signed up with Netflix to enjoy family movies with our 3 kids. Unfortunately, you don't offer closed captioning. Why not!!!?