ffmpeg.git
8 years agoFix out of bound reads/writes in the TIFF decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:45:03 +0000 (00:45 +0200)]
Fix out of bound reads/writes in the TIFF decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ca5d432e028ffdd4067b87aed6702168c3207b6)

8 years agoCheck for out of bound writes in the QDM2 decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:45:05 +0000 (00:45 +0200)]
Check for out of bound writes in the QDM2 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a7876c6e4e62e94d51e364ba99aae4da7671238)

8 years agoFix out of bound reads in the QDM2 decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:45:04 +0000 (00:45 +0200)]
Fix out of bound reads in the QDM2 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 491eaf35ae1f9b619441314bec33766e31580184)

8 years agoFix out of bound reads due to integer overflow in the ADPCM IMA Electronic Arts EACS...
Laurent Aimar [Fri, 30 Sep 2011 22:45:02 +0000 (00:45 +0200)]
Fix out of bound reads due to integer overflow in the ADPCM IMA Electronic Arts EACS decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 346876ec168affe7c21be88d8f1acf1a75cc8409)

8 years agoCheck for out of bound reads in the Electronic Arts CMV decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:45:01 +0000 (00:45 +0200)]
Check for out of bound reads in the Electronic Arts CMV decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a5d46235f3f70f0b620f8e54649ece45ecc5b170)

8 years agoPrevent NULL dereferences when missing the reference frame in the Electronic Arts...
Laurent Aimar [Fri, 30 Sep 2011 22:45:00 +0000 (00:45 +0200)]
Prevent NULL dereferences when missing the reference frame in the Electronic Arts CMV decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 113d7be62497c4e59db8f224fdb7f0a90cf17d03)

8 years agoFix potential pointer arithmetic overflows in the Electronic Arts CMV decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:44:59 +0000 (00:44 +0200)]
Fix potential pointer arithmetic overflows in the Electronic Arts CMV decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e9064c9ce8ed18c3a3aab61e58e663b8f5b0c551)

8 years agoPrevent infinite loop in the ANM decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:44:58 +0000 (00:44 +0200)]
Prevent infinite loop in the ANM decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39993860e1525ca5d1b07521865b7e9e1b294ed7)

8 years agoFix double free on error in Deluxe Paint Animation demuxer.
Laurent Aimar [Fri, 30 Sep 2011 22:44:57 +0000 (00:44 +0200)]
Fix double free on error in Deluxe Paint Animation demuxer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d39d7122e34d2cf994d6dc474fe0c8bee2f7fcfd)

8 years agoCheck for out of bound reads in AVS decoder.
Laurent Aimar [Fri, 30 Sep 2011 22:44:56 +0000 (00:44 +0200)]
Check for out of bound reads in AVS decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7afe9e5638242a3210a0fc378e34e3af41e29176)

8 years agoCheck for out of bound writes in the avs demuxer.
Laurent Aimar [Fri, 30 Sep 2011 22:44:55 +0000 (00:44 +0200)]
Check for out of bound writes in the avs demuxer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5d44c061cf511d97be5fac8d76be2f3915c6e798)

8 years agoCheck for corrupted data in avs demuxer.
Laurent Aimar [Fri, 30 Sep 2011 22:44:54 +0000 (00:44 +0200)]
Check for corrupted data in avs demuxer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1cce7def0a8eff2e7db294b7d195a0fb1a5043b0)

8 years agolavf: Avoid using av_malloc(0) in av_dump_format
Martin Storsjö [Fri, 30 Sep 2011 17:30:35 +0000 (20:30 +0300)]
lavf: Avoid using av_malloc(0) in av_dump_format

On OS X, av_malloc(0) returns pointers that cause crashes when
freed.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit e81e5e8ad2bb5746df0c343c396019aca165cf66)

8 years agoavcodec: reject audio packets with NULL data and non-zero size
Justin Ruggles [Sat, 24 Sep 2011 01:54:44 +0000 (21:54 -0400)]
avcodec: reject audio packets with NULL data and non-zero size

There is no valid reason the user should ever send such packets in the
first place, but the documentation for CODEC_CAP_DELAY states that the
codec is guaranteed not to get a NULL packet unless that capability is
set. That isn't true without preventing this case.
(cherry picked from commit 6326afd5e90cfed9df08b652a1cd6f6a948c239a)

8 years agoFix out of bound writes in fix_bitshift() of the shorten decoder.
Laurent Aimar [Thu, 29 Sep 2011 22:05:53 +0000 (00:05 +0200)]
Fix out of bound writes in fix_bitshift() of the shorten decoder.

The data pointers s->decoded[*] already take into account s->nwrap.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f42b3195d3f2692a4dfc0a8668bb4ac35301f2ed)

8 years agoCheck for out of bound reads in the Tiertex Limited SEQ decoder.
Laurent Aimar [Thu, 29 Sep 2011 22:05:47 +0000 (00:05 +0200)]
Check for out of bound reads in the Tiertex Limited SEQ decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5d7e3d71673d64a16b58430a0027afadb6b3a54e)

8 years agoFix the size of workspace buffers in the motion pixels decoder.
Laurent Aimar [Thu, 29 Sep 2011 22:05:49 +0000 (00:05 +0200)]
Fix the size of workspace buffers in the motion pixels decoder.

Some buffers must be mod 4 in width and/or height.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 62234a4d3a30f3949694781ef8a941ef55b210fa)

8 years agoClear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffer used...
Laurent Aimar [Thu, 29 Sep 2011 22:05:48 +0000 (00:05 +0200)]
Clear FF_INPUT_BUFFER_PADDING_SIZE bytes at the end of the temporary buffer used in motion pixels decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e60619f9b4bdaf9af46887cdb2c86454567b4a61)

8 years agoCheck for out of bounds writes in the Delphine Software International CIN decoder.
Laurent Aimar [Thu, 29 Sep 2011 22:05:51 +0000 (00:05 +0200)]
Check for out of bounds writes in the Delphine Software International CIN decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3035c4034b6af3ad47f921e3385196e1b9d44ddf)

8 years agoCheck for out of bounds reads in the Delphine Software International CIN decoder.
Laurent Aimar [Thu, 29 Sep 2011 22:05:50 +0000 (00:05 +0200)]
Check for out of bounds reads in the Delphine Software International CIN decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8e5f093c2cf13eab3d68d893bf8f30c56ba4e733)

8 years agoCheck for out of bound reads in the QuickDraw decoder.
Laurent Aimar [Thu, 29 Sep 2011 22:05:46 +0000 (00:05 +0200)]
Check for out of bound reads in the QuickDraw decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 44e2f0c3cd2df68659e07ed3d5eab4974794eb33)

8 years agomov: Only touch extradata in mov_read_extradata() if codec_id is what we expect
Tomas Härdin [Wed, 28 Sep 2011 08:34:51 +0000 (10:34 +0200)]
mov: Only touch extradata in mov_read_extradata() if codec_id is what we expect

Extradata should only be parsed from the avss, fiel, jp2h and alac atoms for
AVS, MJPEG, Motion JPEG 2000 and ALAC respectively.
This also fixes the mov demuxer coming up with bogus extradata for some
AVC-Intra samples due to the presence of fiel atoms.
(cherry picked from commit e571305a71494af195891e314b05936f040f89d3)

8 years agoCheck for out of bound reads in xan_huffman_decode() of the xan decoder.
Laurent Aimar [Tue, 27 Sep 2011 22:45:54 +0000 (00:45 +0200)]
Check for out of bound reads in xan_huffman_decode() of the xan decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c8b835954ae4aef797112afda3b52f8dfe3c7b74)

8 years agodca: clear inactive subbands only once in qmf_32_subbands()
Mans Rullgard [Wed, 28 Sep 2011 20:00:35 +0000 (21:00 +0100)]
dca: clear inactive subbands only once in qmf_32_subbands()

Writing zeros to the high entries in the array need only be
done once as the cutoff position is constant throughout the
loop.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit bf00a73ace9b1aba790b75dcb26d43adfceb769f)

8 years agovf_unsharp: set default chroma size value to 5x5
Stefano Sabatini [Sat, 13 Aug 2011 14:30:44 +0000 (16:30 +0200)]
vf_unsharp: set default chroma size value to 5x5

The previous default value 0x0 was not good, since it is not even
valid.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 1ee20141900c98f9dc25eca121c66c3ff468c1e4)

8 years agovf_unsharp: fix out-of-buffer read
Stefano Sabatini [Fri, 12 Aug 2011 06:42:35 +0000 (08:42 +0200)]
vf_unsharp: fix out-of-buffer read

In apply_unsharp(), when y is >= height, prevent out-of-buffer reading
from src, read from the last buffer line in src2 instead.

The check was implemented in the original unsharp libmpcodecs code and
lost in the port.

This also fixes output discrepancy between the two filters.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 998e8519efbc772994c5ba19c0d39573998be9db)

8 years agoCheck for unsupported parameters in ff_j2k_dwt_init()
Laurent Aimar [Wed, 28 Sep 2011 23:04:54 +0000 (01:04 +0200)]
Check for unsupported parameters in ff_j2k_dwt_init()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b4483a531a139f304f4370f470325facb107202f)

8 years agoCheck for out of bound reads in jpeg 2000 decoder.
Laurent Aimar [Wed, 28 Sep 2011 23:04:53 +0000 (01:04 +0200)]
Check for out of bound reads in jpeg 2000 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 02660a871301adada14b0e0fe64c66f73c2e4541)

8 years agoPrevent calling init_vlc() with invalid parameters in motionpixels decoder.
Laurent Aimar [Wed, 28 Sep 2011 23:04:52 +0000 (01:04 +0200)]
Prevent calling init_vlc() with invalid parameters in motionpixels decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 41b7389cade702383e59343561776f83bb26e17f)

8 years agoPrevent NULL dereference when the palette is missing in the xan decoder.
Laurent Aimar [Wed, 28 Sep 2011 23:04:51 +0000 (01:04 +0200)]
Prevent NULL dereference when the palette is missing in the xan decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 56ee5a9ad1b385129c40ba4773f99bc5943ae8af)

8 years agoFixed out of bound accesses in xan_unpack() of the xan decoder.
Laurent Aimar [Wed, 28 Sep 2011 23:04:49 +0000 (01:04 +0200)]
Fixed out of bound accesses in xan_unpack() of the xan decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5279141c1d7a72c467b7e0666fc2275cbcec4668)

8 years agomovenc: Replace av_realloc by av_realloc_f when relevant.
Nicolas George [Wed, 28 Sep 2011 15:16:11 +0000 (17:16 +0200)]
movenc: Replace av_realloc by av_realloc_f when relevant.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 194c2432eecd97c36914956f3bf7781ac4fc6f3d)

8 years agogxfenc: Replace av_realloc by av_realloc_f when relevant.
Nicolas George [Wed, 28 Sep 2011 15:16:11 +0000 (17:16 +0200)]
gxfenc: Replace av_realloc by av_realloc_f when relevant.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit af84d9bb9e8a69a715fc7596d6cbaa00ad00dc29)

8 years agoaviobuf: Replace av_realloc by av_realloc_f when relevant.
Nicolas George [Wed, 28 Sep 2011 15:16:11 +0000 (17:16 +0200)]
aviobuf: Replace av_realloc by av_realloc_f when relevant.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 247a1dc84716cb033b538a5bd5ba8b33de0e8260)

8 years agoavienc: Replace av_realloc by av_realloc_f when relevant.
Nicolas George [Wed, 28 Sep 2011 15:16:11 +0000 (17:16 +0200)]
avienc: Replace av_realloc by av_realloc_f when relevant.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e47cfe9e5c10eee3c8d0b6aff81792c0f10e66e1)

8 years agoavidec: Replace av_realloc by av_realloc_f when relevant.
Nicolas George [Wed, 28 Sep 2011 15:16:11 +0000 (17:16 +0200)]
avidec: Replace av_realloc by av_realloc_f when relevant.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 71e23d39a396f45bbdf258735b02a4bd5e25fd49)

8 years ago4xm: Replace av_realloc by av_realloc_f when relevant.
Nicolas George [Wed, 28 Sep 2011 15:16:11 +0000 (17:16 +0200)]
4xm: Replace av_realloc by av_realloc_f when relevant.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0cc44facf17153454727c26f2f40ee2f77b90df5)

8 years agolibvpxenc: Replace av_realloc by av_realloc_f when relevant.
Nicolas George [Wed, 28 Sep 2011 15:16:11 +0000 (17:16 +0200)]
libvpxenc: Replace av_realloc by av_realloc_f when relevant.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 769298a6869c3b16557280a63f6502409d1b5e49)

8 years agobitstream: Replace av_realloc by av_realloc_f when relevant.
Nicolas George [Wed, 28 Sep 2011 15:16:11 +0000 (17:16 +0200)]
bitstream: Replace av_realloc by av_realloc_f when relevant.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 198ed6474d603f930430067b8b56955d443e821c)

8 years agoIntroduce av_realloc_f.
Nicolas George [Sun, 20 Mar 2011 18:39:20 +0000 (19:39 +0100)]
Introduce av_realloc_f.

av_realloc_f helps avoiding memory-leaks in typical uses of realloc.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5cd754bca290775ec2dbbf88597ab58e0482efca)

8 years agoIntroduce av_size_mult.
Nicolas George [Sun, 20 Mar 2011 18:39:20 +0000 (19:39 +0100)]
Introduce av_size_mult.

av_size_mult helps checking for overflow when computing the size of a memory
area.

Signed-off-by: Nicolas George <nicolas.george@normalesup.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b2600509fef4d77645491f208b8113c372a32110)

8 years agoCheck for out of bound reads in the flic decoder.
Laurent Aimar [Tue, 27 Sep 2011 20:05:15 +0000 (22:05 +0200)]
Check for out of bound reads in the flic decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1f024b882094b26c85e87698faa002b8713b5f88)

8 years agoPrevent out of bound accesses in the xan decoder.
Laurent Aimar [Tue, 27 Sep 2011 21:53:47 +0000 (23:53 +0200)]
Prevent out of bound accesses in the xan decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit feca3ba053499e0d38f48910cef9bc431036956f)

8 years agoCheck for invalid/corrupted bitstream in sun raster decoder.
Laurent Aimar [Tue, 27 Sep 2011 19:24:03 +0000 (21:24 +0200)]
Check for invalid/corrupted bitstream in sun raster decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b9596a503760ccbd82ca3c03d1c0d5b9449be12b)

8 years agoPrevent NULL dereferences when missing the reference frame in the xan decoder.
Laurent Aimar [Tue, 27 Sep 2011 21:43:57 +0000 (23:43 +0200)]
Prevent NULL dereferences when missing the reference frame in the xan decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 19e95b88459e879d3e67a66350d937c32ed762ca)

8 years agoCheck for out of bounds reads in sun rasterfile decoder.
Laurent Aimar [Tue, 27 Sep 2011 21:43:53 +0000 (23:43 +0200)]
Check for out of bounds reads in sun rasterfile decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 039f3c33fffd2f4ae376b662ea4ec67c1d6a4c04)

8 years agoCheck for corrupted extra data in wmavoice decoder.
Laurent Aimar [Tue, 27 Sep 2011 21:43:52 +0000 (23:43 +0200)]
Check for corrupted extra data in wmavoice decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61930119cbff7572ebe7ade6cd9268becbec37f4)

8 years agoCheck for out of bound writes in the wmavoice decoder.
Laurent Aimar [Tue, 27 Sep 2011 21:43:51 +0000 (23:43 +0200)]
Check for out of bound writes in the wmavoice decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e09ae22ab7d9af7f1cbfd2445fa71ad9e7c28ee3)

8 years agoPrevent NULL dereferences when missing the reference frame in the bink decoder.
Laurent Aimar [Tue, 27 Sep 2011 18:22:13 +0000 (20:22 +0200)]
Prevent NULL dereferences when missing the reference frame in the bink decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 66aae97a60fcd8658f18c484b5af898a48d0e3f9)

8 years agoCheck for out of bound writes when building tree in bink decoder.
Laurent Aimar [Mon, 26 Sep 2011 23:02:17 +0000 (01:02 +0200)]
Check for out of bound writes when building tree in bink decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 254af56dd101bc756194dd080bb99e8f123500dd)

8 years agoCheck for various out of bound writes in the bink decoder.
Laurent Aimar [Mon, 26 Sep 2011 23:02:16 +0000 (01:02 +0200)]
Check for various out of bound writes in the bink decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 36bf135d4c32973933000a819208df7da9dd3e16)

8 years agoReset internal state on corrupted blocks in wavpack decoder.
Laurent Aimar [Mon, 26 Sep 2011 21:37:30 +0000 (23:37 +0200)]
Reset internal state on corrupted blocks in wavpack decoder.

wavpack_decode_block() supposes that it is called back with the exact
same buffer unless it has returned with an error. With multi-channels
files, wavpack_decode_frame() was breaking this assumption.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c2a016ad4d9c29285813ba5806189e63e063e0fb)

8 years agoValidate the number of audio channels before using it in wmapro decoder.
Laurent Aimar [Mon, 26 Sep 2011 21:37:29 +0000 (23:37 +0200)]
Validate the number of audio channels before using it in wmapro decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc64434030a8e214fed4179733e49026e7550da5)

8 years agows_snd: make sure number of channels is 1
Justin Ruggles [Mon, 12 Sep 2011 13:44:21 +0000 (09:44 -0400)]
ws_snd: make sure number of channels is 1
(cherry picked from commit 6a818cb3ff2056d43361e5fd09e318cd2ca2a7b4)

8 years agows_snd: add some checks to prevent buffer overread or overwrite.
Justin Ruggles [Mon, 12 Sep 2011 13:41:06 +0000 (09:41 -0400)]
ws_snd: add some checks to prevent buffer overread or overwrite.
(cherry picked from commit 417364ce1f979031ef6fee661fc15e1869bdb1b4)

8 years agows_snd: decode to AV_SAMPLE_FMT_U8 instead of S16.
Justin Ruggles [Mon, 12 Sep 2011 12:55:43 +0000 (08:55 -0400)]
ws_snd: decode to AV_SAMPLE_FMT_U8 instead of S16.

8-bit unsigned is the native sample format.
(cherry picked from commit 2322ced8da990835717a176b8d2c32961cfecd3e)

8 years agoflacdec: fix buffer size checking in get_metadata_size()
Justin Ruggles [Tue, 13 Sep 2011 19:13:44 +0000 (15:13 -0400)]
flacdec: fix buffer size checking in get_metadata_size()

Adds an additional check before reading the next block header and avoids a
potential integer overflow when checking the metadata size against the
remaining buffer size.
(cherry picked from commit 4c5e7b27d57dd2be777780e840eef9be63242158)

8 years agoFix a buffer overflow in libx264 interface to x264 encoder. Previous code ignored...
Mike Scheutzow [Mon, 26 Sep 2011 14:57:53 +0000 (10:57 -0400)]
Fix a buffer overflow in libx264 interface to x264 encoder. Previous code ignored the compressed buffer size passed in. This change returns as many complete NALs as can fit in the buffer, and logs an error message.

Signed-off-by: Mike Scheutzow <mike.scheutzow@alcatel-lucent.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e2dae1faa84ada5746ac2114de7eb68abd824131)

8 years agolibaac+ support
tipok [Mon, 26 Sep 2011 00:28:21 +0000 (02:28 +0200)]
libaac+ support

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
8 years agoCheck for out of bound bands limit in mpc v8 decoder.
Laurent Aimar [Sun, 25 Sep 2011 11:43:38 +0000 (13:43 +0200)]
Check for out of bound bands limit in mpc v8 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 508e47a5751b063e5b3d1d6aceda8a19ad8b1d37)

8 years agoFix return value on EOF in mpc v8 demuxer.
Laurent Aimar [Sun, 25 Sep 2011 11:43:37 +0000 (13:43 +0200)]
Fix return value on EOF in mpc v8 demuxer.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7ec5ea437fc88ca58d7ac3cd12dfa2f0fbd4011f)

8 years agoh264: ff_h264_decode_extradata: check buffer args
Alexander Strasser [Sat, 24 Sep 2011 16:00:16 +0000 (18:00 +0200)]
h264: ff_h264_decode_extradata: check buffer args

  The buffer size and pointer were not checked prior to testing the first
byte of the buffer. These were sometimes checked before calling, but it is
better to add it inside the function as it takes buf and size arguments.

Signed-off-by: Alexander Strasser <eclipse7@gmx.net>
(cherry picked from commit 715f259bf949b06df1b5ed0307606dc258754c99)

8 years agoCompile x86/swscale_template with -mno-red-zone.
Reimar Döffinger [Wed, 21 Sep 2011 17:10:58 +0000 (19:10 +0200)]
Compile x86/swscale_template with -mno-red-zone.

Replaces a very hackish hack to fix the same issue (call instruction
overwriting stack variables).

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit 424bcc46b5fb0d662e0fb9ad6319c5b9ef3d770f)

8 years agoffmpeg: increase bit_buffer_size, the header size is clearly too small for rgb48...
Michael Niedermayer [Sun, 25 Sep 2011 14:13:07 +0000 (16:13 +0200)]
ffmpeg: increase bit_buffer_size, the header size is clearly too small for rgb48 raw based formats

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d8289ff9a9a3d0af416e70a1c864e1c6ce095656)

8 years agoAdd av_calloc() helper.
Laurent Aimar [Sat, 24 Sep 2011 16:39:13 +0000 (18:39 +0200)]
Add av_calloc() helper.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ccecab4a0d7f3f4f296551c2e22bbf12af7d14e8)

8 years agoFix potential pointer arithmetic overflows in rle_unpack() of vmd video decoder.
Laurent Aimar [Sat, 24 Sep 2011 22:08:51 +0000 (00:08 +0200)]
Fix potential pointer arithmetic overflows in rle_unpack() of vmd video decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 35cb6854bb76b4a5b6f2aea2dce81e18d7ab61cd)

8 years agoFix out of bound reads in rle_unpack() of vmd video decoder.
Laurent Aimar [Sat, 24 Sep 2011 22:08:50 +0000 (00:08 +0200)]
Fix out of bound reads in rle_unpack() of vmd video decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4749e074987d45cb98935a683a7ee8e1ee376d86)

8 years agoCheck for out of bound reads in vmd_decode() of vmd video decoder.
Laurent Aimar [Sat, 24 Sep 2011 22:08:49 +0000 (00:08 +0200)]
Check for out of bound reads in vmd_decode() of vmd video decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e07377e7366d038d6f28792c46745463a0542650)

8 years agoFix potential pointer arithmetic overflows in lz_unpack of vmd video decoder.
Laurent Aimar [Sat, 24 Sep 2011 21:52:24 +0000 (23:52 +0200)]
Fix potential pointer arithmetic overflows in lz_unpack of vmd video decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 78cb39d2b2ad731dd3b984b0c0711b9f1d6de004)

8 years agoPrevent out of bound read in lz_unpack in vmd video decoder.
Laurent Aimar [Sat, 24 Sep 2011 21:52:23 +0000 (23:52 +0200)]
Prevent out of bound read in lz_unpack in vmd video decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5127f465bd3e2cf9cbf66dea3cf7b481b522d266)

8 years agoPrevent NULL dereferences when the previous frame is missing in vmd video decoder.
Laurent Aimar [Sat, 24 Sep 2011 21:16:19 +0000 (23:16 +0200)]
Prevent NULL dereferences when the previous frame is missing in vmd video decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6a6383bebcb03a785797007031ad1c9786a508a5)

8 years agoCheck for invalid update parameters in vmd video decoder.
Laurent Aimar [Sat, 24 Sep 2011 21:16:18 +0000 (23:16 +0200)]
Check for invalid update parameters in vmd video decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7aed1280ea14b60fceae04d71dfd03e1daf2d04)

8 years agoFix potential overread in vmd audio decoder.
Laurent Aimar [Sat, 24 Sep 2011 21:16:17 +0000 (23:16 +0200)]
Fix potential overread in vmd audio decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 00cbe9e4053fd562b6f21e76aca6636ff926b637)

8 years agovp56:Fix error recovery code on size changes in vp5/6 decoder
Laurent Aimar [Sat, 24 Sep 2011 14:16:40 +0000 (16:16 +0200)]
vp56:Fix error recovery code on size changes in vp5/6 decoder

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1aad9cd9d26adcca1d398a4cb3d67f89a6fa8967)

8 years agovp6:Reset the internal state when aborting key frames header parsing in vp6 decoder.
Laurent Aimar [Sat, 24 Sep 2011 14:19:25 +0000 (16:19 +0200)]
vp6:Reset the internal state when aborting key frames header parsing in vp6 decoder.
It prevents leaving the state only half initialized.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 91f104496bb7632ed5ff03798e06dd8af014f0d9)

8 years agoh264: pass buffer & size to ff_h264_decode_extradata()
Michael Niedermayer [Fri, 23 Sep 2011 03:42:45 +0000 (05:42 +0200)]
h264: pass buffer & size to ff_h264_decode_extradata()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
8 years agoh264: Check for out of bounds reads in ff_h264_decode_extradata().
Laurent Aimar [Sat, 24 Sep 2011 14:44:14 +0000 (16:44 +0200)]
h264: Check for out of bounds reads in ff_h264_decode_extradata().

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57764c699671b32f0c70795feafc0c6d7491f198)

8 years agofft: avoid a signed overflow
Sean McGovern [Tue, 20 Sep 2011 01:32:09 +0000 (21:32 -0400)]
fft: avoid a signed overflow

As a signed integer, 1<<31 overflows, so force it to unsigned.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit c2d3f561072132044114588a5f56b8e1974a2af7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
8 years agotiffenc: initialize forgotten avctx.
Jean First [Fri, 23 Sep 2011 21:39:51 +0000 (23:39 +0200)]
tiffenc: initialize forgotten avctx.
(cherry picked from commit f7e797aa5c987c39b55666a2d41877ef2aec40bc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
8 years agotiffenc: Add forgotten avclass to context.
Jean First [Fri, 23 Sep 2011 21:39:26 +0000 (23:39 +0200)]
tiffenc: Add forgotten avclass to context.
(cherry picked from commit 43c481e56929789883f5b078e0e86d50fcc0025e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
8 years agoaacsbr: add a assert0 to check for a inconsistency that
Michael Niedermayer [Fri, 23 Sep 2011 20:30:27 +0000 (22:30 +0200)]
aacsbr: add a assert0 to check for a inconsistency that
occurd during debug. I dont know if this can happen normally but if so
it would be quite bad.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit abe0dbea2e228621e97184e39159d189b6085fe3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
8 years agopsxstr: improve probe to not misdetect so much.
Michael Niedermayer [Fri, 23 Sep 2011 17:01:12 +0000 (19:01 +0200)]
psxstr: improve probe to not misdetect so much.
The score of 50 can probably be raised if needed
Fixes Ticket490

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3f7dc480c1bf6abf4ac0f633a0c7e63d8eb29a55)

8 years agolavf/utils: only complain about aspect missmatch when the difference is "meassureable"
Michael Niedermayer [Thu, 22 Sep 2011 16:17:28 +0000 (18:17 +0200)]
lavf/utils: only complain about aspect missmatch when the difference is "meassureable"

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e8d8517b160bd2dea1279d19ec0efd83e00c8c6c)

8 years agompeg4videoenc: remove forgotten return -1
Michael Niedermayer [Thu, 22 Sep 2011 16:16:57 +0000 (18:16 +0200)]
mpeg4videoenc: remove forgotten return -1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f9bb7395a10ae44eb0f0f3f070f23124dfaee06e)

8 years agompeg4videoenc: guess a good aspect when we cant store the exact one.
Michael Niedermayer [Thu, 22 Sep 2011 12:08:39 +0000 (14:08 +0200)]
mpeg4videoenc: guess a good aspect when we cant store the exact one.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 394781a89706479aa56749a9d69c4e74b398dd71)

8 years agoavformat_free_context: favor av_freep()
Michael Niedermayer [Fri, 30 Sep 2011 23:29:30 +0000 (01:29 +0200)]
avformat_free_context: favor av_freep()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2a93f28a4b6eef8b93046e0b4e3225f2ff1e7324)

8 years agompegvideo: increase emu edge buffer size
Michael Niedermayer [Fri, 30 Sep 2011 22:58:01 +0000 (00:58 +0200)]
mpegvideo: increase emu edge buffer size
This fixes a crash with 422 H.264

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7322483d72d4abefae9f5c08c611f521de7236a5)

8 years agoh264: fix FIXME and use list_count in ff_h264_fill_mbaff_ref_list()
Michael Niedermayer [Wed, 28 Sep 2011 20:24:05 +0000 (22:24 +0200)]
h264: fix FIXME and use list_count in ff_h264_fill_mbaff_ref_list()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 237d31e0b98b95eef687e612177ca3ea24b709fc)

8 years agoh264: More correct ref_count check in decode_slice_header()
Michael Niedermayer [Wed, 28 Sep 2011 20:22:56 +0000 (22:22 +0200)]
h264: More correct ref_count check in decode_slice_header()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dc9ce40069bde3d28f8d0b3e5bd733ae255fecb5)

8 years agoFix segfault in save_bits:
Sascha Sommer [Sat, 24 Sep 2011 18:56:41 +0000 (20:56 +0200)]
Fix segfault in save_bits:
use put_bits_count to get the buffer fill state instead of
num_saved_bits as num_saved_bits is sometimes reset when
frames are lost
(Ticket 495)
(cherry picked from commit 780d45473c32fa356c8ce385c3ea4692567c3228)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4f6187c7356111540024901932294e9807061dd0)

8 years agoupdate version numbers for 0.8.4 n0.8.4
Michael Niedermayer [Thu, 22 Sep 2011 00:29:11 +0000 (02:29 +0200)]
update version numbers for 0.8.4

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
8 years agovp6: partially propagate huffman tree building errors during coeff model parsing...
Dustin Brody [Tue, 16 Aug 2011 20:46:34 +0000 (16:46 -0400)]
vp6: partially propagate huffman tree building errors during coeff model parsing and fix misspelling

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit f913eeea43078b3b9052efd8d8d29e7b29b39208)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
8 years agoCheck for huffman tree building error in vp6 decoder.
Laurent Aimar [Wed, 21 Sep 2011 18:46:32 +0000 (20:46 +0200)]
Check for huffman tree building error in vp6 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c249d4fbaf4431b20a90a3c942f3370c0039d9e)

8 years agoRelease old pictures after a resolution change in vp5/6 decoder
Laurent Aimar [Wed, 21 Sep 2011 18:46:33 +0000 (20:46 +0200)]
Release old pictures after a resolution change in vp5/6 decoder

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dba20b84784a7931b7eac50ced1d43e86801bde9)

8 years agoCheck for missing reference in vp5/6 decoder.
Laurent Aimar [Wed, 21 Sep 2011 18:46:30 +0000 (20:46 +0200)]
Check for missing reference in vp5/6 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6a0e78929aa7d6b2c6b598c1589fb0e48fccb132)

8 years agoCheck for invalid slices offsets in RV30/40 decoder.
Laurent Aimar [Wed, 21 Sep 2011 18:46:31 +0000 (20:46 +0200)]
Check for invalid slices offsets in RV30/40 decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b64269ce5528bdbec8af671042f97af1242cf044)

8 years agoCheck output buffer size in nellymoser decoder.
Laurent Aimar [Wed, 21 Sep 2011 18:46:29 +0000 (20:46 +0200)]
Check output buffer size in nellymoser decoder.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 741ec30bd2385f794efa9fafa84d39a917f2574e)

8 years agoMerge remote-tracking branch 'khirnov/release/0.7' into release/0.8
Michael Niedermayer [Wed, 21 Sep 2011 23:10:24 +0000 (01:10 +0200)]
Merge remote-tracking branch 'khirnov/release/0.7' into release/0.8

* khirnov/release/0.7: (64 commits)
  rv34: Check for invalid slice offsets
  rv34: Fix potential overreads
  rv34: Avoid NULL dereference on corrupted bitstream
  rv10: Reject slices that does not have the same type as the first one
  lavf: Fix context pointer in av_open_input_stream when avformat_open_input fails
  oggdec: fix out of bound write in the ogg demuxer
  Fixed size given to init_get_bits().
  smacker: fix a few off by 1 errors
  Check for invalid VLC value in smacker decoder.
  Check and propagate errors when VLC trees cannot be built in smacker decoder.
  Fixed off by one packet size allocation in the smacker demuxer.
  Check for invalid packet size in the smacker demuxer.
  ape demuxer: fix segfault on memory allocation failure.
  xan: Add some buffer checks (cherry picked from commit 0872bb23b4bd2d94a8ba91070f706d1bc1c3ced8)
  Fixed size given to init_get_bits() in xan decoder. (cherry picked from commit 393d5031c6aaaf8c2dda4eb5d676974c349fae85)
  smacker demuxer: handle possible av_realloc() failure.
  Fixed segfault with wavpack decoder on corrupted decorrelation terms sub-blocks.
  cljr: init_get_bits size in bits instead of bytes (cherry picked from commit 0c1f5b93d9b97c4cc3684ba91a040e90bfc760d2)
  indeo2: fail if input buffer too small (cherry picked from commit b7ce4f1d1c3add86ece7ca595ea6c4a10b471055)
  indeo2: init_get_bits size in bits instead of bytes (cherry picked from commit 68ca330cbd479111db9cb7649d7530ad59f04cc8)
  ...

Conflicts:
ffmpeg.c
libavdevice/alsa-audio.h
libavformat/gxf.c
libswscale/x86/swscale_template.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
8 years agoHack around gcc 4.6 breaking asm using call.
Reimar Döffinger [Sun, 28 Aug 2011 19:14:13 +0000 (21:14 +0200)]
Hack around gcc 4.6 breaking asm using call.

gcc 4.6 no longer decrements esp to account for local variables.
Thus using call will end up overwriting some local variable.
So add an extra one it can safely clobber.
This is a huge hack because it's basically pure chance it works,
no idea how this is supposed to be done.

Fixes trac ticket #397.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit c928e9172928f3cddab949b107dd53b6bf79555e)

8 years agoFix dxva2 decoding for some H264 samples.
Carl Eugen Hoyos [Wed, 21 Sep 2011 21:47:34 +0000 (23:47 +0200)]
Fix dxva2 decoding for some H264 samples.