Why is video compression such a big deal?

A place to discuss the science of computers and programs, from algorithms to computability.

Formal proofs preferred.

Moderators: phlip, Moderators General, Prelates

User avatar
King Author
Posts: 736
Joined: Sun Apr 12, 2009 12:30 pm UTC
Location: Pennsylvania, USA

Why is video compression such a big deal?

Postby King Author » Thu Feb 19, 2015 4:52 am UTC

Why is good, high-quality video compression so elusive? What about video makes it so...bloated and gig-consuming?

I specifically ask 'cause anytime I load up a video in Windows Movie Maker, even if I significantly shrink the size of it, the saved file ends up being ridiculously enormous. Like, I can load up a 250 MB, ~5:00 minute video clip and trim it down to ~45 seconds, and when I go to save it, it'll be 1.3 Gigabytes. Not an exaggeration, in fact, I've had worse.

Granted I assume Windows Movie Maker is the crappiest, lowest-quality video editing program they could possibly've churned out; they just slapped it together and threw it on their OS to be able to say "Windows comes pre-packaged with video-editing software!" But that just raises the point -- if WMM produces such horrifically bloated files, how do good video editing programs produce quality, small-ish size clips? If two programs are doing the exact same thing - effectively, showing a series of images while playing an audio track in sync - how can their file sizes vary so drastically?

What's the science going on here?
I have signitures disabled. If you do, too...you can't read this, so nevermind >_>

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Why is video compression such a big deal?

Postby EvanED » Thu Feb 19, 2015 5:53 am UTC

I don't know much about the algorithms... but if all you're doing is cropping out a segment, you can use something like Avidemux or Virtualdub to do so without reencoding. Not only is this much, much faster, but you won't have quality loss from re-encoding.

User avatar
Mokele
Posts: 775
Joined: Fri Aug 21, 2009 8:18 pm UTC
Location: Atlanta, GA

Re: Why is video compression such a big deal?

Postby Mokele » Thu Feb 19, 2015 2:45 pm UTC

Two words: Virtualdub & Xvid

I use a LOT of videos for my scientific work, and annoyingly, one of the tools I use for analysis only accepts uncompressed AVIs, and I need them at full 1080p. A 10 second clip is around 4GB for my stuff, even with audio stripped out. But once I run it through Virtualdub using the Xvid compression codec at the default settings, that 4GB video is reduced to 4-8 MB, with no perceptible quality deterioration. And for a clip that length, compression usually takes less than a minute.
"With malleus aforethought, mammals got an earful of their ancestor's jaw" - J. Burns, Biograffiti

User avatar
King Author
Posts: 736
Joined: Sun Apr 12, 2009 12:30 pm UTC
Location: Pennsylvania, USA

Re: Why is video compression such a big deal?

Postby King Author » Thu Feb 19, 2015 11:17 pm UTC

EvanED wrote:I don't know much about the algorithms... but if all you're doing is cropping out a segment, you can use something like Avidemux or Virtualdub to do so without reencoding. Not only is this much, much faster, but you won't have quality loss from re-encoding.


I actually found that while googling yesterday, it works nice, but unfortunately only with a small selection of file types. And annoyingly, only certain types of flv. It opens and works with any flv just fine, but unless it's a certain type I guess, when I save it, it saves as a black screen with just the audio.

Is that what it is, though? Re-encoding is what produces such ridiculously bloated file sizes?

Mokele wrote:Two words: Virtualdub & Xvid

I use a LOT of videos for my scientific work, and annoyingly, one of the tools I use for analysis only accepts uncompressed AVIs, and I need them at full 1080p. A 10 second clip is around 4GB for my stuff, even with audio stripped out. But once I run it through Virtualdub using the Xvid compression codec at the default settings, that 4GB video is reduced to 4-8 MB, with no perceptible quality deterioration. And for a clip that length, compression usually takes less than a minute.


O_o

See, this is only compounding my "wtf, video?" confusion. 4 GB to 4 MB?! How is that possible? What in the world is going on with video technology that that sort of thing is possible?

(Sidebar: What kind of work do you do that you're anaylzing 10 second, 4GB video clips?)
I have signitures disabled. If you do, too...you can't read this, so nevermind >_>

Derek
Posts: 2181
Joined: Wed Aug 18, 2010 4:15 am UTC

Re: Why is video compression such a big deal?

Postby Derek » Fri Feb 20, 2015 8:57 am UTC

The answer here is that Windows Movie Maker is shit. In fact, from what you're describing I don't even know if it's doing compression at all, 1.3 gigs for 45 seconds is awful. When I'm doing video editing (which is very rarely) I use VirtualDub and encode to h.264, which gives me good results.

From my experience, Vdub should open pretty much any video format, but it will only save as avi. I have never tried using it with flv, but all the formats I've used have worked fine.

Some examples from the videos I've made:

42 minutes of 1080p60, h.264: 2.18 GB
6 hours of 480p30, DivX: 3 GB
40 seconds of 1920x1200x60, h.264: 58.2 MB

User avatar
Xenomortis
Not actually a special flower.
Posts: 1446
Joined: Thu Oct 11, 2012 8:47 am UTC

Re: Why is video compression such a big deal?

Postby Xenomortis » Fri Feb 20, 2015 10:18 am UTC

1.3g for 45 seconds does seem to be on the same order of magnitude as the raw outputs I got from Fraps, back when I was toying with that.
Image

Ubik
Posts: 1016
Joined: Thu Oct 18, 2007 3:43 pm UTC

Re: Why is video compression such a big deal?

Postby Ubik » Fri Feb 20, 2015 10:52 am UTC

Movie editing software might intentionally be working with uncompressed* files, because otherwise you'd need to encode and encode and encode. I believe the programs that don't require re-encoding have limitations on where cuts can be made, depending on the contents of the compressed file. Anyway, cutting stuff is not the only thing more full-blown movie editors do.

I don't know Windows Movie Maker, but maybe it has separate feature to export a movie into one of the space-efficient formats? Like photo editing software that have save as to make a file in the native format of the program and "export to web" to make an optimized JPEG/PNG/whatever.

Disclaimer: It's been a while since I had the basic education on these things.

* Even the "uncompressed" files may actually be lossily compressed, like throwing bits off some channels or having some channels stored at half resolution or something similar.

User avatar
King Author
Posts: 736
Joined: Sun Apr 12, 2009 12:30 pm UTC
Location: Pennsylvania, USA

Re: Why is video compression such a big deal?

Postby King Author » Fri Feb 20, 2015 10:57 am UTC

Derek wrote:The answer here is that Windows Movie Maker is shit. In fact, from what you're describing I don't even know if it's doing compression at all, 1.3 gigs for 45 seconds is awful. When I'm doing video editing (which is very rarely) I use VirtualDub and encode to h.264, which gives me good results.

From my experience, Vdub should open pretty much any video format, but it will only save as avi. I have never tried using it with flv, but all the formats I've used have worked fine.

Some examples from the videos I've made:

42 minutes of 1080p60, h.264: 2.18 GB
6 hours of 480p30, DivX: 3 GB
40 seconds of 1920x1200x60, h.264: 58.2 MB


I've got no prob with AVIs, but can Vdub edit? I just want to grab out tiny clips from larger files.
I have signitures disabled. If you do, too...you can't read this, so nevermind >_>

User avatar
Mokele
Posts: 775
Joined: Fri Aug 21, 2009 8:18 pm UTC
Location: Atlanta, GA

Re: Why is video compression such a big deal?

Postby Mokele » Fri Feb 20, 2015 3:05 pm UTC

King Author wrote:See, this is only compounding my "wtf, video?" confusion. 4 GB to 4 MB?! How is that possible? What in the world is going on with video technology that that sort of thing is possible?

(Sidebar: What kind of work do you do that you're anaylzing 10 second, 4GB video clips?)


In all fairness, my compression is probably extreme because I have high frame rates (70+ fps) causing initial large size, but also an almost totally single-color background that takes up 95%+ of the frame in large, uniform areas that don't move from frame to frame - the actual videos are overhead views of sidewinder rattlesnakes moving through obstacles, so the vast majority of the video is just a flat expanse of unicolor sand.

Also, Vdub can indeed export smaller sequences of longer files, as well as some basic video manipulations like cropping, black-and-white, contrast/brightness/gamma, appending multiple AVIs, downsampling, changing frame rate, etc.
"With malleus aforethought, mammals got an earful of their ancestor's jaw" - J. Burns, Biograffiti

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: Why is video compression such a big deal?

Postby EvanED » Fri Feb 20, 2015 3:07 pm UTC

King Author wrote:I've got no prob with AVIs, but can Vdub edit? I just want to grab out tiny clips from larger files.
It's a codec thing. Without some added plugin or something I'm not sure I ever got working (Avidemux I think just worked on the box and is similarly capable), it doesn't handle MPEG. That will exclude a lot of videos, including I think most camera output. Could be why it didn't work for you.

[Disclaimer: the above information could be out of date or just plain wrong and isn't intended to slander virtualdub. :-)]

User avatar
Mokele
Posts: 775
Joined: Fri Aug 21, 2009 8:18 pm UTC
Location: Atlanta, GA

Re: Why is video compression such a big deal?

Postby Mokele » Fri Feb 20, 2015 7:14 pm UTC

EvanED wrote:
King Author wrote:I've got no prob with AVIs, but can Vdub edit? I just want to grab out tiny clips from larger files.
It's a codec thing. Without some added plugin or something I'm not sure I ever got working (Avidemux I think just worked on the box and is similarly capable), it doesn't handle MPEG. That will exclude a lot of videos, including I think most camera output. Could be why it didn't work for you.

[Disclaimer: the above information could be out of date or just plain wrong and isn't intended to slander virtualdub. :-)]


Here are some plugins for virtual dub that will let you edit MOVs and MP4s - http://www.videohelp.com/tools/VirtualDub
"With malleus aforethought, mammals got an earful of their ancestor's jaw" - J. Burns, Biograffiti

Derek
Posts: 2181
Joined: Wed Aug 18, 2010 4:15 am UTC

Re: Why is video compression such a big deal?

Postby Derek » Fri Feb 20, 2015 8:16 pm UTC

Yeah codecs are always a bitch. That's the number one reason I hate messing with video editing. I eventually was able to get the right ones that I needed, but I don't remember from where.

User avatar
Xanthir
My HERO!!!
Posts: 5400
Joined: Tue Feb 20, 2007 12:49 am UTC
Location: The Googleplex
Contact:

Re: Why is video compression such a big deal?

Postby Xanthir » Sat Feb 21, 2015 3:49 am UTC

King Author wrote:See, this is only compounding my "wtf, video?" confusion. 4 GB to 4 MB?! How is that possible? What in the world is going on with video technology that that sort of thing is possible?

A video, taken literally, is just a series of images. A whole bunch of images. If your video is 60fps and 45s long, you're looking at approximately 2700 images. Go look at the average size of an image on your computer and multiply that by 2700; you'll probably get a number around a few gigs.

Video compression works by not literally saving entire images for each frame; instead they only save parts of the frame that changed relative to the previous. (And a bunch more complicated compression stuff.) Imagine the frames all stacked on top of each other; it's the same as image compression, just in 3d rather than 2d. Since most videos are not randomly changing every pixel on every frame, this usually lets you cut out the vast majority of the weight and easily get a hundred- or thousand-fold reduction in size.
(defun fibs (n &optional (a 1) (b 1)) (take n (unfold '+ a b)))

User avatar
WanderingLinguist
Posts: 237
Joined: Tue May 22, 2012 5:14 pm UTC
Location: Seoul
Contact:

Re: Why is video compression such a big deal?

Postby WanderingLinguist » Mon Feb 23, 2015 2:47 am UTC

What Xanthir said, but I'll go a little deeper, in case you're interested. (If not, feel free to ignore this post; it gets a bit technical)

Video compression is an extremely complicated field, and the compression ratios that are possible these days are quite impressive, but there are a lot of tradeoffs involved. I'll try to explain a little bit about how those impressive compression ratios are achieved, and then you might understand why it's not always desirable/possible in some cases.

Let's consider utterly uncompressed video for a moment (nobody actually works with it like this, but just for comparison). 1080p video is 1920 x 1080, or about 2 million pixels. Each of those has red, green and blue values, so multiple by three. 6MB for a totally uncompressed frame. (30 frames per second) x (5 seconds) x (6MB) = almost 1GB for every five seconds of video (if I did my math right). Clearly, compression is needed.

You could apply still image compression (JPEG or PNG) but if you do the math, the size per frame is still too big, and there are other issues with still image compression (see below).

Here's (roughly) how modern video codecs work:

1) Humans see color at a much lower resolution than intensity (brightness), so RGB is converted to YUV (where Y is intensity, and the color is encoded in the U and V channels). U and V channels are then stored at 1/4 the resolution (1/2 width and 1/2 height). This means there's less detail in the color data, but you won't be able to see the difference, and it immediately cuts the size of the compressed video in half.

2) Using similar tricks based on human perception, it's possible to throw away data describing details that humans might not notice making it easier to compress.

(So far, this is the same as how still image compression such as JPEG works, except that data that can be safely thrown away without being noticeable to a human observer is different when dealing with motion, which is why you can't just use a series of JPEG images for video—it's been tried and it looks terrible; the general theory is the same but the specifics are necessarily different because of how human vision works)

3) As Xanthir said, storing the difference from one frame to another reduces the amount of data significantly. But this is where things get problematic. What if I have a 1-hour video and I want to seek to the middle and start playing? The frame I'm seeking to in the middle of the video is dependent on the frame before it (it has only the differences), and that frame in turn is dependent on the frame before and so on, back to the beginning of the video. This means seeking to the middle of a 1-hour video could take several minutes (it depends on how fast the video can be decoded compared to the frame rate, but on a low-powered mobile device, it could conceivably take 30 minutes just to seek to the middle of a movie, which is clearly unacceptable). So what's done to fix that is to store the entire frame periodically. In encoded video, these are called "i-frames", and the frames that store only differences from the previous frame are called "p-frames"). So you might have one i-frame per second. More i-frames means it's faster to seek, but the file is bigger; less means the file is smaller but seeking takes longer or is not possible (I have seen files with just a single i-frame at the beginning and nothing but p-frames for the rest of the file).

4) Storing pixel differences between frames is not always good enough; if something moves in the video, lots of pixels will have changed, but they are often quite similar to pixels nearby (at a fixed offset) in the previous frame. So storing differences can also mean storing some motion data too.

5) A more recent advancement is to actually look at the last few frames and predict what the next frame is going to look like (in particular measuring motion between previous frames and predicting what the next frame might look like). If the encoder and decoder are making the same predictions, it's only necessary to store the difference between the predicting and actual frame. If the prediction is good, this dramatically reduces the size of the compressed video.

6) If there's an upcoming i-frame, it's possible to store the difference between the previous frame AND the next frame. This is called a b-frame, and can also significantly reduce the compressed video size.

Why you sometimes want larger video...

There are a lot of reasons why it's sometimes necessary or desirable (or unavoidable) to have a larger video file.

A) Quality is lost in re-compression, so for a video you're editing, you want lower compression on your source material.

B) It's very hard to do real-time edits if there aren't enough i-frames, so for editing, it's common to use lots and lots of i-frames.

C) It takes significantly more processor power to do motion prediction, and the decoder has to do the same work as the encoder. For mobile use, for example, sometimes it's better to trade off larger video/more data usage for something that's computationally less expensive because it can significantly extend battery life (and some lower-end phones might not even be able to handle video that uses motion prediction).

D) Some of the techniques of analyzing video for compression are patented and the license fees can be expensive.

E) Better compression is not always compatible with existing decoders in the field.

For streaming video, (C) and (E) can be addressed by compressing different versions of the same file and having the server choose the best one for the receiving device.

One of the tricky parts is that when you see reference to a format such as "H.264", it doesn't tell the whole story about the compressed data. H.264 has a LOT of optional features (such as motion prediction, etc.) that an encoder and decoder can choose to implement or not to implement. There are dozens of these features, which may or may not be supported depending on the video resolution and other criteria. Obviously, this is a compatibility nightmare, so the H.264 standard defines several different fixed sets of features, which are group into profiles and levels. So if you have an H.264 video clip and you want to play it on, say, your phone, you need to know what profile the phone supports, and what profile the H.264 video was encoded with. (The safest and most widely supported is baseline profile but it also results in larger files). This applies not just to H.264 but other formats as well; I mention H.264 because it's the most common format these days, and I'm most familiar with it.

Even worse, people talk about "mpeg files" or "avi files" and so on, but these have very little to do with compression: They are all container formats, and the video inside them can be encoded with any of quite a few different codecs; these days, quite commonly, some kind of H.264 encoding, but others are quite possible too.

So, there are a lot of reasons why the same video, encoded in different ways, can be wildly different in size.

(Apologizes in advance if I made any technical mistakes here... I wrote most of it from memory and didn't consult any reference documents, so I specific details or terms may be wrong, but hopefully you get the general picture).

User avatar
Dthen
Still hasn't told us what comes after D
Posts: 553
Joined: Sat Jan 02, 2010 6:35 pm UTC
Location: Ayrshire, Scotland

Re: Why is video compression such a big deal?

Postby Dthen » Mon Feb 23, 2015 1:45 pm UTC

Thank you for the informative and interesting post, WanderingLinguist.
Dthen wrote:I AM NOT A CAT.

User avatar
Wildcard
Candlestick!
Posts: 253
Joined: Wed Jul 02, 2008 12:42 am UTC
Location: Outside of the box

Re: Why is video compression such a big deal?

Postby Wildcard » Wed Jun 10, 2015 7:55 am UTC

Dthen wrote:Thank you for the informative and interesting post, WanderingLinguist.

Seconded.

I personally dived as deep as I wanted into this area (read: not very deep) when I worked out how to put my DVD collection on my computer for easier access. Handbrake + VLC + a lot of fiddling around and googling instructions = I can now do it easily, but have no idea what is going on in the background while I do it. Add in MetaZ and Subler for DVD chapter titles and other tagging data, and my video collection is actually easy to browse. But, that's from a usability perspective, not an understanding perspective.

Thanks to your post I now actually understand the terms i-frames and p-frames, which I had encountered before without knowing any referent.
There's no such thing as a funny sig.


Return to “Computer Science”

Who is online

Users browsing this forum: No registered users and 5 guests