niamu

Brendon Walsh
1080p Video on the iPad... Sort of

1080p Video on the iPad... Sort of

2010-04-18

It has been 2 weeks since the iPad launch, and now that most of the hysteria has subsided and most people have stopped asking whether or not they can touch it, it's time we got down to business and really stress what this device is capable of. For me, the iPad is my portable media center. I sync all of my TV and podcasts to it so I can stay current and entertained when I'm away from my Boxee setup at home and have some time to kill.

Unfortunately, this process is not without limitations. iTunes will not sync media that does not meet Apple's defined requirements as outlined on the iPad specs page. So all of my gorgeous Blu-Ray rips must be transcoded to a lower bitrate and down to 720p to be accepted by iTunes for transfer to the iPad. This is understandable because the screen is just shy of 720p capable to begin with, so there's really no reason to want a 1080p video on the iPad save for the extra time needed to transcode to a compliant 720p file.

The solution? Stream the video using Apple's HTTP Live Streaming protocol. Yes, I know I could make use of Air Video, but I'm a DIY kind of guy, and don't take kindly to paying for something I can do myself, or cluttering up my devices with applications that are superfluous.

So what's it take to pull this off? (I'm gonna make you click the "Read More" link now because it's fun.)

The Tools

The Commands

Now that we have all of our tools ready, it's time to start swinging some hammers and bashing walls.

ffmpeg -i video.mp4 \
	 -acodec libmp3lame \
	 -b 128kb \
	 -vcodec libx264 \
	 -vpre hq \
	 -vb 2500 \
	 -s 1280x720 \
	 -f mpegts - | \
	 mediastreamsegmenter -O -b http://192.168.0.2/stream/temp \
	 -f /stream/temp/ -t 10 -s 4 -D

The above command is doing a lot of work, so let's break it down.

ffmpeg is taking the input of video.mp4, then transcoding it's audio component to mp3 with a maximum bitrate of 128kbps. It is also transcoding the video component to H.264 with a maximum bitrate of 2500kbps and using a built-in preset for high quality limited to dimensions that the iPad can handle. Finally, it passes both of these new audio and video streams into an MPEGTS container for what mediastreamsegmenter is about to do to it.

The handoff happens at this point: - |

This makes ffmpeg "pipe" it's output to mediastreamsegmenter for it to handle the splitting of the tracks into more digestible portions for streaming. And that's where the magic happens.

mediasteasegmenter optimizes the stream, specifies the base URL of where the video files will be held for browser access. In this case it's, http://192.168.0.2/stream/temp. You will have to make adjustments based on where your media files will be hosted. The next step of the segmenter is to define where in the file system these encoded bit size pieces will be stored. Then we've told it to cut each section into 10 second chunks, keep 4 of those chunks in the index file at a time and delete the rest as you go to save space.

Hopefully you got all that. That was my attempt to save you the time of reading through documentation and man pages.

Of course, even though we have all of these perfectly formatted video segments available through a web browser now, we still can't do much with them. Apache doesn't understand what to do with these file types yet and we haven't wrapped them in a nice HTML5 page either.

So first, let's add some MIME types to Apache. Add the following to the end of /etc/apache2/mime.types:

application/x-mpegURL m3u8
video/MP2T ts

Now restart Apache for the changes to take effect.

apachectl restart

We're almost home free! All that's left is to wrap it all up in a nice little HTML5 web page that Safari can read on the iPad and we're good to run our first test.

<!DOCTYPE html>

<html>
<head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
 <title>HTML5 Streaming Video</title>
</head>
<body>

<video controls>
 <source src="temp/prog_index.m3u8" />
</video>

</body>
</html>

I've saved this just in the directory above where mediastreamsegmenter saved the video chunks so that when I navigate to http://192.168.0.2/stream/ on the iPad, the video stream is available in a nice HTML5 player that QuickTime can handle.

The Fun Part

What we've just completed is the hard way of live streaming a video and transcoding it on the fly to an iPad. The fun parts happen when you get lazy like me and decide not to transcode the video on the fly as I, unfortunately, do not have a super computer or beowulf cluster which can handle real-time transcoding of HD video so I tried something I didn't expect to work...

ffmpeg -i video.mp4 -acodec copy -vcodec copy -vbsf h264_mp4toannexb -f mpegts - | mediastreamsegmenter -O -b http://192.168.0.2/stream/temp -f /stream/temp/ -t 10 -s 4 -D

What we're doing this time is simply using ffmpeg as a pass-through device to the segmenter. It's taking both the audio and video stream of a relatively standard AVCHD 1080p video stream and passed it directly through to the segmenter. All the segmenter is doing is serving up 10 second chunks at a time of what should be an unsupported video stream. Imagine my surprise when it actually started playing. Not only can the iPad playback 1080p, but it can also handle a vast array of video codecs and higher level profiles than just the baseline profile.

As you might have guessed, this little hack is not at all without it's flaws. I tried my crazy 15MBps 1080p rip of The Dark Knight, and it played for about 7 minutes before crashing Safari. I imagine this is largely a result of the 256 of RAM in the iPad but it could also be related to the incredible strain on the A4 processor pulling in all that data over WiFi and pushing it to the screen. On more than one occasion I tapped the screen to adjust volume or with the intent to scrub a little, and Safari immediately crashed. This demonstrates that the A4 chip can't handle a stream that large and handle a quick overlay fade in of controls at the same time. This gives a sense of just how much strain this puts on the device.

720p is another story entirely. At no point did I ever experience a crash watching a 720p stream. Even at a higher profile and bitrate than Apple defines as supported. So sure, this isn't the easiest thing in the world to implement, but depending on your computer's horsepower, it may beat transcoding the video and syncing through iTunes if you intend to watch it within your LAN (or VPN if you're crazy). And no, you're not going to be able to stream Blu-Ray discs to your iPad (at least no Blu-Ray that's longer than a few minutes).

For me, this proved just what a killer app Safari really is on Apple mobile devices. Safari seems to be the loophole for just about anything a developer or consumer doesn't like about the iPad, iPhone, or iPod Touch. Case and point: Google Voice.

Aside Re: The App Store

Thank sweet zombie Jesus that Apple is controlling their application store. Sure, we all know how many fart joke applications are out there, but I can only imagine how much crap would be there on top of all that if Apple had a "free market". I'm damn glad Flash isn't on my mobile devices. I'm happy that Apple is blocking cross-platform applications. None of this would work well on these devices. Hell, it barely works now on a Mac. Ask any Mac user whether they're happy with the performance of Adobe applications that have been cross-compiled for the Mac (that is to say, all Adobe applications). Take a few minutes and listen to this apt clip from last week's MacBreak Weekly.

The app store is a little broken. I won't deny it. But it would be far worse if Apple wasn't playing quality control. Too often I see people discussing the prospects of making a quick buck in the App store with a second or third rate app.

The Last Word

Please don't misunderstand, I am an amateur when it comes to live transcoding of media. If you have some brilliant way to make ffmpeg perform a little faster in it's transcoding (as I know Air Video does), please share it in the comments.