• Home
• Site Map
• About Us
• Contact Us
• Links  
• 

• Search Data
• Climate Change
• Conceptual Models
• Coastal Indicators
• Habitat Mapping
• Natural Resource Management
• Landform & Stability Maps
• Tropical Rivers

Underwater photography of the seabed - codecs

Codec Comparison by James Wise

Purpose:

To analyse a range of popular codecs and assess their suitability for use in a variety of applications. Also to explore and understand issues related to the different types of artefacts that var ious codecs and codec families produce, how the MPEG compression scheme works and which codec/codec families are best suited for use in specific applications.

Target Uses / Environments:

1. Use in an application for computer aided digital manipulation and processing.
2. Long term archival use.

Imagery Sources:

Still Images

• Lossless formats: BMP, RAW, PNG
• Lossy formats: JPEG, GIF

• DV Cameras
• NTSC - 640x480 29.97 fps interlaced
• PAL - 720x576 25 fps interlaced
• Analogue Video
• Captured via SVideo – dependent on capture format, assume PAL DV as default.
• Captured via firewire – this provides a standard DV source as listed above.
• Digital Frame Grabber System
• A direct to disk progressive scan system, using mega pixel digital cameras

About Compression and compression artefacts:

Almost all computer images are compressed. The only types of imagery that are not compressed at all are RAW, PNG and BMP (although BMP does support compression). However, rather than focusing on compressed versus uncompressed, it is more important to look at whether a compression scheme is lossless or lossy. A lossless compression scheme means that no information is lost during the process of compression and decompression. When a lossless image is viewed later, the image/image stream is reproduced with 100% accuracy. The goal of lossless compression is to make file sizes smaller without any loss of quality. In contrast a lossy compression scheme actually throws away some of the information in the compression/decompression process, however most of the time this change represents minimal changes in an image, and is barely noticeable to the human eye. The goal of a lossy compression scheme is to minimize the file size of the image as much as possible without any noticeable loss in quality. Hence the selection of the most appropriate compression scheme is a constant trade-off between file size and image quality.

Each different lossy compression scheme will introduce small artefacts into the resultant image.

An artefact is a noticeable difference between the original image and the compressed version. Some examples of compression artefacts are seen in figures 1 and 2.

JPEG compression artefacts:

Figure 1. Original image on top, same image after JPEG compression on the bottom

These artefacts are typical of jpeg compression. The artefacts exhibit losses in clarity and certain areas of the image suffer from blocking, and intensity jumps.

Some more facts about JPEG from: http://graphicssoft.about.com/od/formatsjpeg/a/jpegmythsfacts_2.html

"JPEG is best suited for large photographic images where file size is the most important consideration, such as images that will be posted on the Web or transmitted via email and FTP. JPEG is not suitable for most small images under a few hundred pixels in dimension, and it is not suitable for screen shots, images with text, images with sharp lines and large blocks of color, and images that will be edited repeatedly."

GIF compression artefacts:

The GIF format works by reducing the total number of colours present in an image.

This is very effective for images that have large areas of the same colour, such as cartoon drawings.

However the compression effects quickly become noticeable in images that have areas of soft transitions or gradients within the image. However once reduced to a 256 colour palette the GIF format offers lossless compression.

Figure 2. Original image on left, GIF compressed version on the right.

The most obvious compression artefact here is the banding caused by the reduction in the number of colours present in the image. This is caused by the GIF compression method, which reduces the number of colours in an image to only 256 colours.

PNG:

PNG (Portable Network Graphics) is a file format that was developed as a successor to the GIF format. Capable of 8, 16, 32 and even 64 bit palettes, and featuring lossless compression, the PNG format also offers support for other advanced features such as alpha values, gamma correction, and advanced interlacing. The compression methods offered by the PNG format are superior to that of the GIF format and often result in smaller file sizes whilst retaining the same image quality. However this added complexity means that there can be considerable variation within the resulting file size after saving an image as a PNG. This is caused by differing implementations of the PNG compression scheme across various applications. PNG format files are often larger than the same image saved as a JPEG this is due to JPEG being a lossy format, while PNG is lossless.

Interlacing:

Interlacing is a method of capturing video frames that involves capturing 2 half size frames for every real frame displayed. This offers the benefit of a higher actual frame rate, with only half of the information being displayed each frame. In situations of fast movement within the video stream interlacing creates a more natural flowing image (less stuttering) at the cost of an ugly image if a user tries to view a single frozen frame.

Figure 3. Example of a still frame captured from an interlaced movie.

The interlacing is particularly noticeable on the brown fish as it is swimming past the red fish in the background, and also on the white leading edge of the red fish’s tail. One of the biggest problems with interlacing is that its effects are very disruptive for any images that require further processing. Interlacing can be overcome, either by using a progressive scan camera system, or by use of one of many de-interlacing filters. Progressive scan cameras record a movie such that the fields are not interlaced. De-interlacing can easily be applied to an interlaced video stream, however the process usually introduces some slight blurring, especially when compared to a progressive scan camera.

Figure 4. Image from figure 3 after application of a de-interlacing filter.

Notice that whilst the straight-line artefacts are gone, the overall image is more blurred.

However this image is much better for further digital image processing.

Inter-frame encoding versus Intra-frame encoding, and key frames:

Inter-frame encoding refers to any method that uses parts of previous frames to predict the values for the current frame. Intra-frame encoding refers to any method that only uses information from the current frame to predict parts of the given frame. Obviously inter-frame encoding is only valid for image streams, where as intra-frame encoding can be applied to both still images and image streams.

A key frame is any frame in a compressed image sequence that can be decompressed and displayed using only information from that frame. A key frame does not have any dependency on frames that come before or after the key frame. Key frames can be accessed faster than non key frames in an image sequence, thus an image sequence with many key frames is comparatively faster to navigate – especially if the user is moving through the image stream in a random fashion. An image sequence with many key frames will often look better than a sequence with few key frames, especially if there is a lot of variation from frame to frame. It is possible to compress an image sequence such that every frame is a key frame; this creates a stream that is very quick to browse through regardless of direction or how far you want to move from one frame to another, however this obviously increases the resulting file size.

DV Compression:

DV compression is a type of MPEG2 encoding that fits certain specific guidelines for frame rate and image size, as well as data rate and compression methods. These guidelines are specified by the Moving Picture Experts Group; hence the title MPEG(2) – as this is the second iteration of the standard. http://www.chiariglione.org/mpeg/

Figure 5. Examples of two common DV/MPEG compression errors.

The top image has blurred and overly dark lines above the top of the white page, this over contrast between areas of great difference is typical of a DV compression scheme and is due to the loss in luminance and chroma information inherent in the DV codec. The second image is an obvious example of "stair step" artefacts within an image; although rare in DV encoded footage this is another possible artefact of the DV compression.

MPEG compression:

MPEG compression is based upon a simple concept. This concept is to identify what is different in frame n+1 when compared to frame n. This enables the system to isolate parts of the foreground that may be moving or changing rapidly and isolate these parts from the non-moving "background" . This allows the codec to only transmit/include the information about parts of an image that have changed between the frames. To do this for every pixel would be inefficient, so an MPEG codec breaks an image up into several "slices" across the screen, it then breaks up these slices into square "macro blocks" , each of which contains 4 "blocks" . Where a "block" is a square group of pixels (usually 8x8), the intention is that most background blocks will be the same from 1 frame to the next, thus by signalling no change in a block or even whole macroblock from 1 frame to the next, considerable information is saved/minimized.

Figure 6. The MPEG compression process, complete with GOPs, slices, macro blocks and blocks.

This system of compression causes the "blocking" type of artefacts that are most often associated with low bit rate MPEG2 streams. An example of these blocking artefacts can be seen in the mage below.

Figure 7. Original "Lena" test image on left, low-bit rate MPEG2 compressed version on right.

The blocking artefact is caused by the compression and decompression of that specific block, not being close enough when compared to the original image, and when compared to the adjacent blocks.

Compression within a block is done by a method known as Discrete Cosine Transfer (DCT). DCT essentially transforms the spatial information into frequency information and then takes advantage of some properties of the resulting frequency information to encode the DCT information into as few bytes as possible.

GOPs and I, B, and P frames:

Now that we understand a bit about the compression scheme used for blocks, macro blocks and slices is it time to understand how whole frames are compressed. This is actually relatively simple, as all MPEG2 frames are organized in GOPs (Groups Of Pictures), these GOPs are made up by a combination of either I, B, or P frames.

• An I frame or a reference frame is similar to a key frame, in that it holds all of the information to decode that frame within that frame itself. Intra or I frames, are single compressed frames which contain all of the spatial information of a video frame
• P frames or predicted frames use a combination of motion vectors and block information from I frames and other B frames to predict/generate the image for any given B frame. Predictive frames are computed based on the nearest previous I or P frame. P frames are more highly compressed than I frames and provide a reference for the calculation of B frames.
• B frames or Bi-directional frames uses both I and P frames from ahead and behind in the video stream to predict the current frame. Bi-directional frames use both a past and subsequent frames as a reference to calculate the compressed frame data.

Most common MPEG2 streams have GOP of at least 15 pictures, with 1 I frame and a varying amount of B and P frames. It is possible to control most of these parameters before encoding a video. For example it is possible to force an I frame every x frames, or create a stream with only P frames, and I but no B frames. Each I, B, or P frame contains information for every slice within the image, which in turn contains information about the macro blocks and blocks within that slice. At the lowest level the blocks contain information that indicates if a macro block or block has changed from the previous frame and if the block has changed it may contain information about how the block has changed or a reference to a future or previous frame or a different block that is a better representation for the current block. Obviously I frames only contain information about that specific frame.

Constant Bit rate versus Variable Bit Rate:

Constant bit rate streams have a constant data rate throughout the image stream, where as variable bit rate allows for the data rate to vary according to the complexity of the scenes in the image stream.

Variable bit rate is a more complex technology and is not as widely supported as constant bit rate streams, however most modern MPEG4 codecs offer support for both methods. Variable bit rate methods allow for lower bit rates when the scene in question is one that is easily encoded, allowing this extra bandwidth to be used in other parts of the image stream that require more information to give an equal quality output.

Single Pass versus Multi Pass Encoding:

With most of the modern codecs it is possible to encode files in either a single pass, or multiple passes.

With single pass encoding, the encoder looks at every frame of the image stream approximately in order, sometimes looking at a group of up to 11 or 12 frames and encoding them together, the window of frames to look at can be user specified in most codecs. In multi pass encoding the image stream is effectively analysed twice or more. In the second and subsequent passes the encoder can look ahead to the results of frames in the future as encoded by the first pass. Multi pass encoding results in better compression and higher quality, at the cost of increased compression time. Unfortunately multi pass encoding is not possible on live image streams, which make it inappropriate for some tasks, such as on the fly compression of video from cameras. However for long term archival of footage multi pass encoding is highly suitable. Multi-pass encoding can be especially useful with variable bit rate encoding methods, as the second or nth pass of the encoding process already has access to information about the remaining portion of the file.

MPEG4 / DivX / Xvid - All H.263 formats:

All of these codecs are essentially evolutions of the basic MPEG2 concepts. Many of the features introduced in the MPEG2 standard have been refined or optimised, to create this new family of codecs that offer similar quality to previous MPEG2 codecs but at a much reduced bit rate. One of the biggest changes is allowing multiple sizes for blocks and macro blocks, instead of just 8x8 pixels to a block or 8x8 blocks to a macro block, blocks and macro blocks may now be 16x16, 8x16, 8x8, or 16x8. By allowing for different shapes and sizes within blocks and macro blocks, the codec is able to adapt to the shape of non-square elements/objects within a picture.

The MPEG4 standard also introduced the concept of video objects. Video objects are simply groups of macroblocks that are designated to belong to a single foreground object – or a background object. This has enabled the MPEG4 and superseding codec families much greater control over complex and detailed foreground objects that may be moving fast throughout an image sequence. Another major change has been to allow the codec to use different size blocks and macro blocks for the luminance (brightness) information when compared to the chroma (colour) information. Again this allows the codec to provide more detail in areas of the image that require more information, and less detail in parts of the image that are similar or do not feature fast moving objects. The method of encoding these macroblocks has also changed from MPEG2, which has increased the efficiency of the compression system. Another change has been to allow B frames to refer to up to 5 or more other frames in order to do the prediction on that frame as well as motion compensation/prediction for macro blocks and blocks within that frame. Motion estimation and prediction can be used to guess where a macroblock or block should be in a B frame, this allows the compression system to only encode the macroblock and the direction it is moving. Again improvements have been made in the eventual encoding of all of the macroblocks for a frame.

WMP9 / AVC – All H.264 formats:

The improvements from H.263, to H.264 are relatively minor, mostly with the H.264 family providing more advanced variables and algorithms for motion estimation, different encoding schemes, and with H.264 providing a de-blocking filter as part of the codec system itself. The DCT computing and encoding system has been advanced considerably in H.264 and this has had the effect of reducing blocking errors whilst also increasing the quality of the decompressed image. Some of the H.264 codecs provide special settings for encoding of interlaced video – this does not create better output when the uncompressed imagery is de-interlaced, but provides for more efficient encoding within the codec. Many H.264 codecs are currently available, however they are only available as commercial packages, costing between $50 and $500.

Confused about DivX, H.263, H.264, and WMV?

• Is DivX5/6 H.263 or H.264?
• What about WMV9 H.263 or H.264?

Neither and both depending on how you look at it. H.263 is a low-motion codec specification, which was a derivative of H.261. It offered fairly good quality at low bit rates but degraded badly if the video has a lot of motion. The original "DivX:-)" codec (the hack of the MS WMV proprietary MPEG4 codec) used H.263 at least partially. The original DivX codec was actually 2 separate codecs, a low-motion and a high-motion codec the low-motion codec was H.263.

H.264 (or AVC as it is now called) is a much more recent standard. What people currently call DivX (or Xvid) is MPEG4. MPEG4 is an umbrella term for a group of different encoding technologies (or layers), one of which is now H.264 (referred to as MEPG4 part 10). Originally, H.264 was a separate and distinct standard, but was absorbed by the MPEG Consortium for inclusion in the MPEG4 standard. Most if not all modern MPEG4 derivative codecs are implementing support for it, DivX included. So a DivX file could be H.264, but it doesn’t have to be.

WMV9, however, is a proprietary format and while it may use similar techniques to encode video it is not H.263, H.264, MPEG4, or any other standards based codec, it is simply WMV9.

So which codec is right for me?

Well first one needs to understand that selecting the correct codec is about understanding your source footage and making a trade-off between file size, quality, and speed of compression.

For still images I would recommend saving and working with all images as PNG files. This ensures that the files can be read by a variety of applications, across multiple platforms, whilst also supporting compression for large files but in a loss-less format. BMP could also be considered a useful format for permanent storage of still imagery, but BMP offers fewer features and larger file sizes than PNG.

*Apparently TIF files using jpeg compression are not lossless, but still feature superior output when compared to pure jpeg files, and my own test did not find any difference between jpeg compressed TIF files and the uncompressed BMP versions.

Note: If your original image files are compressed then selecting a new format will not give you any better image quality. For example, going from a poorly compressed JPEG to a PNG will not magically give you an improvement in quality.

For long term archival of video footage I would recommend either multi pass variable bit rate DivX 6.x, or WMV9. In the future a (free) H.264 based codec will definitely offer the best combination of image quality and file size, but these codecs are not widely in use at the moment, and as such make them unsuitable for use in a professional capture, compression and archival system to be used by many people. I think it is also important to note that whilst the WMV codec provides very few artefacts in an image stream, it is a closed format, and can be difficult to access within other (non MS) applications.

Recommended compression format for footage that originates as DV would be:

For real-time/non archival compression:

I would suggest compressing to a lossless Mjpeg codec first, this requires little CPU power, but will massively reduce the HD requirements when compared to raw or uncompressed formats and should be easily done in real time. Of course if the capture system is capable of capturing the footage in a RAW format and maintaining the frame rate then this is preferred. Then later on compress to DivX 6 using a 2-pass method. This should reduce the file size considerable compared to the lossless Mjpeg or RAW image stream. However if this sounds like too much effort, capturing directly to DivX 6.x should still provide an adequate quality compression balance.

Since the recent release of the DivX codec version 6, this codec has shown improved compression quality, small file sizes, and improves compression speed over previous versions and when compared to

• Doom9.org (www.doom9.org) have just published their most recent codec review. Their review covers many codecs discussed here and more including a few currently unreleased codecs. There is also LOTS of useful codec related information at this site.
• Transcoding of various footage. Transcoding or converting between formats, or re-encoding a file can cause a massive decrease in the quality of the image.
• A very Good website with information on MPEG is at http://www.fh-friedberg.de/fachbereiche/e2/telekom-labor/zinke/mk/mpeg2beg/beginnzi.htm
• 16/6/2005 DivX 6.0 Released. – Still only h.263, and h.263 advanced – no h.264.

Standard profiles are Home profile, and HD profile. Encode times do not vary to a significant degree between the Home and HD profiles.

• Now offers much better quality in the extreme and insane modes.
• "Insane Mode" offers the highest Quality, @ approx 6-7 Fps encode rate, on a PAL DV file.
• "Extreme Mode" offers 8-10fps encode rate, on same PAL-DV file. With very little noticeable drop in quality.
• "Fastest Mode" Offers 33-40 fps encode rate on a PAL-DV file, however considerable less quality than either of the 2 quality modes.
• "Balanced Mode" Offers 25-30fps encode rate on PAL-DV file, better resulting image than fastest
• Support for de-interlacing of files, is now built into codec and has been much improved.
• HD – extreme quality offers approx same (or better )quality @ 10% of the file size.

Other comments:

• This is a list of comments I have compiled from various sources on the Internet:

From http://www.gromkov.com/faq/general/divx_quicktime_wmv9.html

"The best codecs are DivX 5.x (now DivX 6.x) and WMV9 -- it all depends on your priorities. If you're going to stick to computers as playback devices, the faster speed of DivX is welcome. If you want to play your stuff on the PDAs, portable video players, and DVD players, WMV9 has broader industry support and is worth the extra encoding time. Both codecs delivered quite impressive image quality.

When it comes to visual quality, it's difficult to say whether DivX 5.x or Windows Media Video 9 looks better. It's safe to make the generalization that the DivX encoded clips tend to have a touch more detail, but also a few more compression artefacts, than the WMV9 video. On the whole, watching the clips in motion and scrutinizing details over and over, it's hard to recommend one over the other. DivX certainly encodes a whole lot faster, which can be a real concern when compressing large video clips. WMV9, on the other hand, has found its way into several consumer electronics devices, with a great many more on the way. The DivX codec is also available in a few devices, with more on the way, but support for WMV9 in DVD players, portable video players, and home media gateways is certainly stronger."

1MBit Encoding Times (Pentium 4C 2.4GHz CPU), Clip Duration: 4:16min

WMV9 17:22
DivX 13:15
Apple Quicktime Sorenson3 08:37
Apple Quicktime MPEG4 11:50

From: http://bg.broadcastengineering.com/ar/broadcasting_mpeg_avc/

The image below gives a graphical comparison of MPEG-4 AVC and MPEG-2 quality. It compares the performance of state-of-the-art encoders encoding a 30-frame-per-second, CIF-resolution video of a tennis match.

LIST of MPEG-4 AVC/H.264 VIDEO CODECS

List from http://forum.doom9.org/showthread.php?t=95939

Ateme H.264/MPEG-4 AVC Codec (Ateme)
No download on original site:
http://www.ateme.com/products/h264.php
See Nero Recode from ftp://ftp.nero.com/
30 day trial of Nero Digital (with Nero Recode) available from:
http://www.nerodigital.com/eng/index.html

VSS H.264 Video Codec (Vanguard Software Solutions, Inc.)
Trial available, VfW.
http://www.vsofts.com/h264/codecs.html

Mainconcept H.264 (MainConcept AG)
Trial available
http://www.mainconcept.com/h264_encoder.shtml
See download page and SDK page
Mainconcept declare partnership with Elecard in H.264 in March 2005.

Elecard OneClick Compressor & Elecard AVC/H.264 Decoder Package (Elecard)
21-day free trial. Have H.264 SDK
http://www.elecard.com/products/oneclick.shtml

Moonlight H.264 Video Codec (Moonlight Cordless LTD.)
21-day evaluation, DirectX filter.
http://www.moonlight.co.il/
This codec was developed with Elecard team. Now Elecard states they have fully another codec, but they are looks quite similar with Moonlight.

QuickTime 7 H.264 (Apple Computer, Inc.)
Integrated. Who test it? Any remarks welcome!
http://www.apple.com/quicktime/technologies/h264/

FastVDO H.264 (FastVDO LLC)
Demo has decode time limit of 5 min.
http://www.fastvdo.com/H.264.html

LEAD H.264 Video codec (LEAD Technologies)
Only demo, but you can buy codec online (100$). Who test it?
http://www.leadcodecs.com/Codecs/LEAD-H264.htm

Compression Master 3 (Popwire/Teleca AB)
H264 encoder is all a part of Popwire's Compression Master. Demo soft available (encoding is limited to 20 seconds).
http://www.popwire.com

SVM H.264 Decoder Kit & MKi DVD Converter (Pegasus Information Technology Inc.)
H.264 encoder and decoder with non standart format.
http://www.h-264.com/downloads.htm

AVC Alliance free demo player (AVC Alliance, written by Philips Electronics)
Poor but free.
http://www.avc-alliance.nl/main/downloads.htm

Intel IPP H.264 codec (Intel Corporation)
Intel IPP Library containe now H.264 encoder and decoder. Higher quality codecs are promised in 5.0 version (~Q3.2005, available as beta now)
http://www.intel.com

ATI H.264 (ATI Technologies Inc.)
No download, codec will be software only and soft+hardware support
http://www.ati.com/technology/h264.html

Mpegable AVC Codec (dicas digital image coding GmbH)
No download, VfW
http://www.mpegable.com/show/mpegableavc.html
Old version from free-codecs
http://www.codec-download.com

Expert H.264 (PixelTools Corporation)
Download on request
http://www.pixeltools.com/experth264.html

Softstream H.264/MPEG-4 (Media Excel)
Audio & Video encoders ane decoders. Evaluation copy on request
http://www.mediaexcel.com

MPEG-2/HDV/H.264 software (KDDI R&D Labs. Inc.)
No download. MPEG related products also with H.264 support
http://avs.kddilabs.jp/

Fraunhofer IIS H.264 Codec (Fraunhofer IIS)
No download.
http://www.iis.fraunhofer.de/amm/download/mpeg4/

UBLive-264-C64 (UB Video Incorporated)
Demo available on request.
http://www.ubvideo.com/mainmenu.html

Sorenson Squeeze 4 Compression Suite (Sorenson) o download. Converter.
http://www.sorensonmedia.com/

Sonic’s HD-Series AVC encoder (Sonic Solutions)
No download. Only press release now.
http://www.sonic.com

Enchansed sklmp4 (Pascal Massimino)
No download. Announce of 264 development.
http://skal.planet-d.net/coding/mpeg4codec.html

NEX VISION H.264 (NEX VISION)
No download.
http://www.nexvision.fr/

Hughes Network Systems H.264 (Hughes Network Systems, LLC)
No download. Own Win&Linux codec for internal usage
http://www.hns.com/HNS/Doc/0/MGD6UC...03-04_IPoS.html