ffmpeg.git
7 years agoaacsbr: prevent out of bounds memcpy().
Alex Converse [Tue, 10 Jan 2012 21:07:09 +0000 (13:07 -0800)]
aacsbr: prevent out of bounds memcpy().

Fixes Libav Bug 195.
Fixes CVE-2012-0850

This doesn't make the code handle sample rate or upsample/downsample
change properly but this is still a good sanity check.

Based on change by Michael Niedermayer.

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 17ce52912f59a74ecc265e062578fb1181456e18)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agortpdec_asf: Fix integer underflow that could allow remote code execution
Michael Niedermayer [Wed, 7 Sep 2011 12:12:42 +0000 (14:12 +0200)]
rtpdec_asf: Fix integer underflow that could allow remote code execution

Fixes MSVR-11-0088
Fixes CVE-2011-4031
Credit:  Jeong Wook Oh of Microsoft and Microsoft Vulnerability Research (MSVR)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 5ea091fb5a12dc0210b8efdf30b573b87e21652b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodpcm: ignore extra unpaired bytes in stereo streams.
Alex Converse [Fri, 17 Feb 2012 22:13:40 +0000 (14:13 -0800)]
dpcm: ignore extra unpaired bytes in stereo streams.

Fixes: CVE-2011-3951

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit ce7aee9b733134649a6ce2fa743e51733f33e67e)
(cherry picked from commit eaeaeb265fe46e1d81452960de918227541873b4)

Conflicts:

libavcodec/dpcm.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agotqi: Pass errors from the MB decoder
Michael Niedermayer [Mon, 19 Dec 2011 03:13:37 +0000 (04:13 +0100)]
tqi: Pass errors from the MB decoder

This silences some valgrind warnings.
CC: libav-stable@libav.org
Fixes second half of http://ffmpeg.org/trac/ffmpeg/ticket/794
Bug found by: Oana Stratulat

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f85334f58e1286287d0547a49fa9c93b40cbf48f)
(cherry picked from commit 90290a5150e84fb138ccde57657dc03830f08c1c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 5872580e65aab026b77754eb184f97ba7cc6ea35)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: Add check for invalid chroma_format_idc
Alexander Strange [Sat, 24 Mar 2012 21:32:14 +0000 (17:32 -0400)]
h264: Add check for invalid chroma_format_idc

Fixes a crash when FF_DEBUG_PICT_INFO is used.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 6ef4063957aa5025c8d2cd757b6a537e4b6874df)

Fixes: CVE-2012-0851

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 47132345184dc3d0ff962a57a1225564fe979548)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoadpcm: ADPCM Electronic Arts has always two channels
Janne Grunau [Thu, 5 Jan 2012 19:50:55 +0000 (20:50 +0100)]
adpcm: ADPCM Electronic Arts has always two channels

Fixes half of http://ffmpeg.org/trac/ffmpeg/ticket/794
Adresses CVE-2012-0852

(cherry picked from commit bb5b3940b08d8dad5b7e948e8f3b02cd2eb70716)

Conflicts:

libavcodec/adpcm.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh263dec: Disallow width/height changing with frame threads.
Michael Niedermayer [Fri, 17 Feb 2012 21:35:10 +0000 (13:35 -0800)]
h263dec: Disallow width/height changing with frame threads.

Fixes CVE-2011-3937

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 71db86d53b5c6872cea31bf714a1a38ec78feaba)

Conflicts:

libavcodec/h263dec.c

Signed-off-by: Alex Converse <alex.converse@gmail.com>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 4be63587e110c05cda3101abf2e3745d919f3fae)

Conflicts:

libavcodec/h263dec.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovqavideo: return error if image size is not a multiple of block size
Mans Rullgard [Mon, 23 Apr 2012 12:16:33 +0000 (13:16 +0100)]
vqavideo: return error if image size is not a multiple of block size

The decoder assumes in various places that the image size
is a multiple of the block size, and there is no obvious
way to support odd sizes.  Bailing out early if the header
specifies a bad size avoids various errors later on.

Fixes CVE-2012-0947.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 58b2e0f0f2fc96c1158e04f8aba95cbe6157a1a3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit d5207e2af81580dd5e6277b354c8b459c3624f26)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agocelp filters: Do not read earlier than the start of the 'out' vector.
Alex Converse [Fri, 4 May 2012 17:27:03 +0000 (10:27 -0700)]
celp filters: Do not read earlier than the start of the 'out' vector.

CC: libav-stable@libav.org
(cherry picked from commit 37ddd3833219fa7b913fff3f5cccc6878b047e6b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 9ea94c44b1b414ab3bc6e9220ebb77621423ca38)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomotionpixels: Clip YUV values after applying a gradient.
Alex Converse [Wed, 2 May 2012 19:08:03 +0000 (12:08 -0700)]
motionpixels: Clip YUV values after applying a gradient.

Prevents illegal reads on truncated and malformed input.

CC: libav-stable@libav.org
(cherry picked from commit b5da848facd41169283d7bfe568b83bdfa7fc42e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit aaa6a666774eb02c351c84e80622a5c69e9b642e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh263: more strictly forbid frame size changes with frame-mt.
Ronald S. Bultje [Thu, 29 Mar 2012 19:24:10 +0000 (12:24 -0700)]
h263: more strictly forbid frame size changes with frame-mt.

Prevents crashes because the old check was incomplete.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2d22d4307dcc1461f39a2ffb9c8db6c6b23fd080)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 7fe4c8cb761b0fc8685dacf9f187311b9d124a52)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: additional protection against unsupported size/bitdepth changes.
Ronald S. Bultje [Thu, 29 Mar 2012 23:37:09 +0000 (16:37 -0700)]
h264: additional protection against unsupported size/bitdepth changes.

Fixes crashes in codepaths not covered by original checks.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 732f9fcfe54fc9a0a7bbce53fe86b38744c2d301)

Conflicts:

libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 746f1594d71dece6fd6f786447e19be9c200a07d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoUpdate changelog for 0.7.5 release
Reinhard Tartler [Sun, 1 Apr 2012 17:45:27 +0000 (19:45 +0200)]
Update changelog for 0.7.5 release

7 years agoid3v2: fix skipping extended header in id3v2.4
Anton Khirnov [Sat, 31 Mar 2012 05:52:42 +0000 (07:52 +0200)]
id3v2: fix skipping extended header in id3v2.4

In v2.4, the length includes the length field itself.
(cherry picked from commit ddb4431208745ea270dce8fce4cba999f0ed4303)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agoUpdate RELEASE file for 0.7.5
Reinhard Tartler [Sun, 1 Apr 2012 17:08:06 +0000 (19:08 +0200)]
Update RELEASE file for 0.7.5

7 years agolcl: use AVERROR_INVALIDDATA instead of AVERROR_UNKNOWN
Reinhard Tartler [Sun, 18 Mar 2012 18:08:15 +0000 (19:08 +0100)]
lcl: use AVERROR_INVALIDDATA instead of AVERROR_UNKNOWN

While bogus, this change avoids the necessity to backport
AVERROR_UNKNOWN, which is not entirely trivial.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agokgv1dec: Increase offsets array size so it is large enough.
Michael Niedermayer [Wed, 25 Jan 2012 22:23:35 +0000 (23:23 +0100)]
kgv1dec: Increase offsets array size so it is large enough.

Fixes CVE-2011-3945

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 807a045ab7f51993a2c1b3116016cbbd4f3d20d6)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit a02e8df973f5478ec82f4c507f5b5b191a5ecb6b)
(cherry picked from commit d5f2382d0389ed47a566ea536887af908bf9b14f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agokgv1: use avctx->get/release_buffer().
Ronald S. Bultje [Thu, 29 Dec 2011 17:07:32 +0000 (09:07 -0800)]
kgv1: use avctx->get/release_buffer().

Also fixes crashes on corrupt bitstreams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 33cd32b389864f2437c94e6fd7dc109ff5f0ed06)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit e537dc230b2e123be8aebdaeee5a7d7787328b0b)

Conflicts:

libavcodec/kgv1dec.c
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agokvmc: fix invalid reads
Gaurav Narula [Mon, 12 Dec 2011 14:54:54 +0000 (20:24 +0530)]
kvmc: fix invalid reads

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit ad3161ec1d70291efcf40121d703ef73c0b08e5b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agonsvdec: Propagate error values instead of returning 0 in nsv_read_header().
Diego Biurrun [Thu, 23 Jun 2011 11:27:21 +0000 (13:27 +0200)]
nsvdec: Propagate error values instead of returning 0 in nsv_read_header().

This eliminates a warning about a set-but-unused variable.
(cherry picked from commit 35fa0d47585cef28cd8191dccf0607d90c7667a6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomjpegbdec: Fix overflow in SOS.
Alex Converse [Wed, 25 Jan 2012 21:39:24 +0000 (13:39 -0800)]
mjpegbdec: Fix overflow in SOS.

Based in part by a fix from Michael Niedermayer <michaelni@gmx.at>

Fixes CVE-2011-3947

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit b57d262412204e54a7ef8fa1b23ff4dcede622e5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 083a8a00373b12dc06b8ae4c49eec61fb5e55f4b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoshorten: Use separate pointers for the allocated memory for decoded samples.
Michael Niedermayer [Sun, 25 Dec 2011 11:28:50 +0000 (12:28 +0100)]
shorten: Use separate pointers for the allocated memory for decoded samples.

Fixes invalid free() if any of the buffers are not allocated due to either
not decoding a header or an error prior to allocating all buffers.

Fixes CVE-2012-0858
CC: libav-stable@libav.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 204cb29b3c84a74cbcd059d353c70c8bdc567d98)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 6fc3287b9ccece290c5881b92948772bbf72e68c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoshorten: check for realloc failure (cherry picked from commit 9e5e2c2d010c05c10337e9c...
Justin Ruggles [Thu, 15 Sep 2011 22:08:52 +0000 (18:08 -0400)]
shorten: check for realloc failure (cherry picked from commit 9e5e2c2d010c05c10337e9c1ec9d0d61495e0c9c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoatrac3: Fix crash in tonal component decoding.
Michael Niedermayer [Sat, 17 Dec 2011 02:18:58 +0000 (03:18 +0100)]
atrac3: Fix crash in tonal component decoding.

Add a check to avoid writing past the end of the channel_unit.components[]
array.

Bug Found by: cosminamironesei
Fixes CVE-2012-0853
CC: libav-stable@libav.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit c509f4f74713b035a06f79cb4d00e708f5226bc5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit f43b6e2b1ed47a1254a5d44c700a7fad5e9784be)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agows_snd1: Fix wrong samples count and crash.
Michael Niedermayer [Sat, 24 Dec 2011 23:10:27 +0000 (00:10 +0100)]
ws_snd1: Fix wrong samples count and crash.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9fb7a5af97d8c084c3af2566070d09eae0ab49fc)

Addresses CVE-2012-0848

Reviewed-by: Justin Ruggles <justin.ruggles@gmail.com>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 697a45d861b7cd6a96718383a44f41348487f844)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agows_snd: add some checks to prevent buffer overread or overwrite. (cherry picked from...
Justin Ruggles [Mon, 12 Sep 2011 13:41:06 +0000 (09:41 -0400)]
ws_snd: add some checks to prevent buffer overread or overwrite. (cherry picked from commit 417364ce1f979031ef6fee661fc15e1869bdb1b4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agows_snd: decode to AV_SAMPLE_FMT_U8 instead of S16.
Justin Ruggles [Mon, 12 Sep 2011 12:55:43 +0000 (08:55 -0400)]
ws_snd: decode to AV_SAMPLE_FMT_U8 instead of S16.

8-bit unsigned is the native sample format.
(cherry picked from commit 2322ced8da990835717a176b8d2c32961cfecd3e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodca: include libavutil/mathematics.h for possibly missing M_SQRT1_2
Kostya Shishkov [Wed, 7 Mar 2012 19:07:17 +0000 (20:07 +0100)]
dca: include libavutil/mathematics.h for possibly missing M_SQRT1_2

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: stricter reference limit enforcement.
Ronald S. Bultje [Tue, 13 Mar 2012 23:26:44 +0000 (16:26 -0700)]
h264: stricter reference limit enforcement.

Progressive images can have only 16 references, error out if there are
more, since the data is almost certainly corrupt, and the invalid value
will lead to random crashes or invalid writes later on.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e0febda22d0e0fab094a9c886b0e0f0f662df1ef)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agojvdec: unbreak video decoding
Paul B Mahol [Wed, 14 Mar 2012 03:02:02 +0000 (03:02 +0000)]
jvdec: unbreak video decoding

The safe bitstream reader broke it since the buffer size was specified
in bytes instead of bits.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
CC: libav-stable@libav.org
(cherry picked from commit a1c036e961a32f7208e7315dabfa0ee99d779edb)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoxxan: don't read before start of buffer in av_memcpy_backptr().
Ronald S. Bultje [Fri, 9 Mar 2012 00:32:46 +0000 (16:32 -0800)]
xxan: don't read before start of buffer in av_memcpy_backptr().

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f1279e286b00e99f343adb51e251f036a3df6f32)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodsicinvideo: validate buffer offset before copying pixels.
Ronald S. Bultje [Sun, 11 Mar 2012 14:28:54 +0000 (07:28 -0700)]
dsicinvideo: validate buffer offset before copying pixels.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c95fefa0420be9cc0f09a95041acf11114aaacd0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agohuffyuv: add padding to classic (v1) huffman tables.
Ronald S. Bultje [Thu, 8 Mar 2012 00:29:23 +0000 (16:29 -0800)]
huffyuv: add padding to classic (v1) huffman tables.

We slightly overread the input buffer, so we require
padding at the end of the buffer, as is documented in the
get_bits API. Without padding, we'll read uninitialized
data or beyond the end of the .rodata, which may crash.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 4ffe5e2aa5241f8da9afd2c8fbc854dcc916c5f9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agotiffdec: Prevent illegal memory access caused by recycled pointers.
Alex Converse [Wed, 7 Mar 2012 01:00:29 +0000 (17:00 -0800)]
tiffdec: Prevent illegal memory access caused by recycled pointers.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit fd0be63049ed46660993d0550a4f0847a0b942ea)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowma: fix off-by-one in array bounds check.
Ronald S. Bultje [Wed, 7 Mar 2012 22:18:14 +0000 (14:18 -0800)]
wma: fix off-by-one in array bounds check.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit b4bccf3e4e58f6fe58043791ca09db01a4343fac)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoraw: move buffer size check up.
Ronald S. Bultje [Wed, 7 Mar 2012 00:08:10 +0000 (16:08 -0800)]
raw: move buffer size check up.

This way, it protects against overreads for 4bpp/2bpp content also.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit cc5dd632cecc5114717d0b90f8c2be162b1c6ee8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agosmacker: error out if palette copy-with-offset overruns palette size.
Ronald S. Bultje [Wed, 7 Mar 2012 01:24:20 +0000 (17:24 -0800)]
smacker: error out if palette copy-with-offset overruns palette size.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit a93b572ae4f517ce0c35cf085167c318e9215908)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agosvq3: protect against negative quantizers.
Ronald S. Bultje [Tue, 6 Mar 2012 01:03:32 +0000 (17:03 -0800)]
svq3: protect against negative quantizers.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 11b940a1a8e7e5d5b212935a3ce78aeda577f5f2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomov: Add more HDV and XDCAM FourCCs.
Alex Converse [Tue, 21 Feb 2012 23:37:35 +0000 (15:37 -0800)]
mov: Add more HDV and XDCAM FourCCs.

Reference: VLC
(cherry picked from commit b142496c5630b9bc88fb9eaccae7f6bd62fb23e7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomov: Add support for MPEG2 HDV 720p24 (hdv4)
Alex Converse [Tue, 21 Feb 2012 22:08:02 +0000 (14:08 -0800)]
mov: Add support for MPEG2 HDV 720p24 (hdv4)

(cherry picked from commit 0ad522afb3a3b3d22402ecb82dd4609f7655031b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh263dec: Disallow width/height changing with frame threads.
Michael Niedermayer [Fri, 17 Feb 2012 21:35:10 +0000 (13:35 -0800)]
h263dec: Disallow width/height changing with frame threads.

Fixes CVE-2011-3937

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 71db86d53b5c6872cea31bf714a1a38ec78feaba)

Conflicts:

libavcodec/h263dec.c

Signed-off-by: Alex Converse <alex.converse@gmail.com>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agotiff: Make the TIFF_LONG and TIFF_SHORT types unsigned.
Alex Converse [Thu, 23 Feb 2012 18:22:51 +0000 (10:22 -0800)]
tiff: Make the TIFF_LONG and TIFF_SHORT types unsigned.

TIFF v6.0 (unimplemented) adds signed equivalents.
(cherry picked from commit e32548d1331ce05a054f1028fcdda8823a4f215a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agosvq3: Prevent illegal reads while parsing extradata.
Alex Converse [Fri, 10 Feb 2012 04:21:47 +0000 (20:21 -0800)]
svq3: Prevent illegal reads while parsing extradata.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 9e1db721c4329f4ac166a0bcc002c8d75f831aba)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodv: Fix small overread in audio frequency table.
Alex Converse [Fri, 10 Feb 2012 01:11:55 +0000 (17:11 -0800)]
dv: Fix small overread in audio frequency table.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 0ab3687924457cb4fd81897bd39ab3cc5b699588)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoac3: Do not read past the end of ff_ac3_band_start_tab.
Mans Rullgard [Tue, 31 Jan 2012 18:20:33 +0000 (10:20 -0800)]
ac3: Do not read past the end of ff_ac3_band_start_tab.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 034b03e7a0e8e4f8f66c82b736f2c0aa7c063ec0)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodv: Fix small stack overread related to CVE-2011-3929 and CVE-2011-3936.
Alex Converse [Thu, 26 Jan 2012 23:08:26 +0000 (15:08 -0800)]
dv: Fix small stack overread related to CVE-2011-3929 and CVE-2011-3936.

Found with asan.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 2d1c0dea5f6b91bec7f5fa53ec050913d851e366)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodv: Fix null pointer dereference due to ach=0
Michael Niedermayer [Tue, 24 Jan 2012 16:51:40 +0000 (17:51 +0100)]
dv: Fix null pointer dereference due to ach=0

dv: Fix null pointer dereference due to ach=0

Fixes part2 of CVE-2011-3929

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Roman Shaposhnik <roman@shaposhnik.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 5a396bb3a66a61a68b80f2369d0249729bf85e04)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agodv: check stype
Michael Niedermayer [Tue, 24 Jan 2012 16:48:23 +0000 (17:48 +0100)]
dv: check stype

dv: check stype

Fixes part1 of CVE-2011-3929
Possibly fixes part of CVE-2011-3936

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Roman Shaposhnik <roman@shaposhnik.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 635bcfccd439480003b74a665b5aa7c872c1ad6b)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agonsvdec: Propagate errors
Alex Converse [Fri, 27 Jan 2012 01:23:09 +0000 (17:23 -0800)]
nsvdec: Propagate errors

Related to CVE-2011-3940.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit c898431ca5ef2a997fe9388b650f658fb60783e5)

Conflicts:

libavformat/nsvdec.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agonsvdec: Be more careful with av_malloc().
Alex Converse [Fri, 27 Jan 2012 01:21:46 +0000 (17:21 -0800)]
nsvdec: Be more careful with av_malloc().

Check results for av_malloc() and fix an overflow in one call.

Related to CVE-2011-3940.

Based in part on work from Michael Niedermayer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 8fd8a48263ff1437f9d02d7e78dc63efb9b5ed3a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agonsvdec: Fix use of uninitialized streams.
Michael Niedermayer [Tue, 24 Jan 2012 21:20:26 +0000 (22:20 +0100)]
nsvdec: Fix use of uninitialized streams.

Fixes CVE-2011-3940 (Out of bounds read resulting in out of bounds write)

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c011706bc752d34bc6ada31d7df2ca0c9af7c6b)

Signed-off-by: Alex Converse <alex.converse@gmail.com>
(cherry picked from commit 6a89b41d9780325ba6d89a37f2aeb925aa68e6a3)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFix format string vulnerability detected by -Wformat-security.
Fabian Greffrath [Mon, 5 Mar 2012 15:06:01 +0000 (16:06 +0100)]
Fix format string vulnerability detected by -Wformat-security.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit c9dbac36ad4bac07f6c1d06d465e361ab55bcb95)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: fix mmxext chroma deblock to use correct TC values. (cherry picked from commit...
Ronald S. Bultje [Sun, 26 Feb 2012 16:57:14 +0000 (08:57 -0800)]
h264: fix mmxext chroma deblock to use correct TC values. (cherry picked from commit b0c4f04338234ee011d7b704621347ef232294fe)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agocscd: use negative error values to indicate decode_init() failures.
Ronald S. Bultje [Wed, 29 Feb 2012 21:55:09 +0000 (13:55 -0800)]
cscd: use negative error values to indicate decode_init() failures.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8a9faf33f2b4f40afbc3393b2be49867cea0c92d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoh264: prevent overreads in intra PCM decoding.
Ronald S. Bultje [Wed, 29 Feb 2012 02:48:27 +0000 (18:48 -0800)]
h264: prevent overreads in intra PCM decoding.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d1604b3de96575195b219028e2c4f08b2259aa7d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowmaenc: fix m/s stereo encoding for the first frame
Justin Ruggles [Fri, 2 Mar 2012 22:11:25 +0000 (17:11 -0500)]
wmaenc: fix m/s stereo encoding for the first frame

We need to set ms_stereo in encode_init() in order to avoid incorrectly
encoding the first frame as non-m/s while flagging it as m/s. Fixes an
uncomfortable pop in the left channel at the start of playback.

CC:libav-stable@libav.org
(cherry picked from commit 51ddf35c9017018e58c15275ff5b129647a0c94d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowmaenc: limit allowed sample rate to 48kHz
Justin Ruggles [Fri, 2 Mar 2012 21:27:57 +0000 (16:27 -0500)]
wmaenc: limit allowed sample rate to 48kHz

ff_wma_init() allows up to 50kHz, but this generates an exponent band
size table that requires 65 bands. The code assumes 25 bands in many
places, and using sample rates higher than 48kHz will lead to buffer
overwrites.

CC:libav-stable@libav.org
(cherry picked from commit 1ec075cfecac01f9a289965db06f76365b0b1737)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowmaenc: limit block_align to MAX_CODED_SUPERFRAME_SIZE
Justin Ruggles [Fri, 2 Mar 2012 21:10:00 +0000 (16:10 -0500)]
wmaenc: limit block_align to MAX_CODED_SUPERFRAME_SIZE

This is near the theoretical limit for wma frame size and is the most that
our decoder can handle. Allowing higher bit rates will just end up padding
each frame with empty bytes.

Fixes invalid writes for avconv when using very high bit rates.

CC:libav-stable@libav.org
(cherry picked from commit c2b8dea1828f35c808adcf12615893d5c740bc0a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowmaenc: require a large enough output buffer to prevent overwrites
Justin Ruggles [Fri, 2 Mar 2012 21:33:33 +0000 (16:33 -0500)]
wmaenc: require a large enough output buffer to prevent overwrites

The maximum theoretical frame size is around 17000 bytes. Although in
practice it will generally be much smaller, we require a larger buffer
just to be safe.

CC: libav-stable@libav.org
(cherry picked from commit dfc4fdedf8cfc56a505579b1f2c1c5efbce4b97e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomatroska: check buffer size for RM-style byte reordering.
Ronald S. Bultje [Fri, 2 Mar 2012 01:01:22 +0000 (17:01 -0800)]
matroska: check buffer size for RM-style byte reordering.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9c239f6026a170866a4a0c96908980ac2cfaa8b3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowmadec: Verify bitstream size makes sense before calling init_get_bits.
Alex Converse [Fri, 27 Jan 2012 22:24:07 +0000 (14:24 -0800)]
wmadec: Verify bitstream size makes sense before calling init_get_bits.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 48f1e5212c90b511c90fa0449655abb06a9edda2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.
Alex Converse [Thu, 1 Mar 2012 22:07:22 +0000 (14:07 -0800)]
rv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2f6528537fdd88820f3a4683d5e595d7b3a62689)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agolcl: return negative error codes on decode_init() errors.
Ronald S. Bultje [Thu, 1 Mar 2012 01:50:28 +0000 (17:50 -0800)]
lcl: return negative error codes on decode_init() errors.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd17a40a7e0eba21b5d27c67aff795e2910766e4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agohuffyuv: do not abort on unknown pix_fmt; instead, return an error.
Ronald S. Bultje [Thu, 1 Mar 2012 17:41:22 +0000 (09:41 -0800)]
huffyuv: do not abort on unknown pix_fmt; instead, return an error.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 63c9de6469005974288f4e4d89fc79a590e38c06)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovmnc: return error on decode_init() failure.
Ronald S. Bultje [Wed, 29 Feb 2012 03:00:48 +0000 (19:00 -0800)]
vmnc: return error on decode_init() failure.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 07a180972fb369bb59bf6d4f8edb4598c51e80d2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorpza: error out on buffer overreads.
Ronald S. Bultje [Wed, 29 Feb 2012 01:04:33 +0000 (17:04 -0800)]
rpza: error out on buffer overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 78e9852a2e3b198ecd69ffa0deab3fa22a8e5378)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoqtrle: return error on decode_init() failure.
Ronald S. Bultje [Wed, 29 Feb 2012 03:00:39 +0000 (19:00 -0800)]
qtrle: return error on decode_init() failure.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e54ae60e46f737b8e9a96548971091f7ab6b8f7c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoswscale: fix another integer overflow.
Ronald S. Bultje [Wed, 29 Feb 2012 02:21:31 +0000 (18:21 -0800)]
swscale: fix another integer overflow.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 791de61bbb0d2bceb1037597b310e2a4a94494fd)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovp56: error out on invalid stream dimensions.
Ronald S. Bultje [Thu, 23 Feb 2012 19:19:33 +0000 (11:19 -0800)]
vp56: error out on invalid stream dimensions.

Prevents crashes when playing corrupt vp5/6 streams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8bc396fc0e8769a056375c1c211f389ce0e3ecc5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoasf: don't seek back on EOF.
Ronald S. Bultje [Wed, 29 Feb 2012 00:13:46 +0000 (16:13 -0800)]
asf: don't seek back on EOF.

Seeking back on EOF will reset the EOF flag, causing us to re-enter
the loop to find the next marker in the ASF file, thus potentially
causing an infinite loop.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bb6d5411e1e1a8e0608b1af1c4addee654dcbac5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoasf: error out on ridiculously large minpktsize values.
Ronald S. Bultje [Fri, 17 Feb 2012 20:21:22 +0000 (12:21 -0800)]
asf: error out on ridiculously large minpktsize values.

They cause various issues further down in demuxing.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6e57a02b9f639af53acfa9fc742c1341400818f8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agovorbis: fix overflows in floor1[] vector and inverse db table index.
Ronald S. Bultje [Wed, 11 Jan 2012 01:01:26 +0000 (17:01 -0800)]
vorbis: fix overflows in floor1[] vector and inverse db table index.

(cherry picked from commit 24947d4988012f1f0fd467c83418615adc11c3e8)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoFix parser not to clobber has_b_frames when extradata is set.
Reinhard Tartler [Sun, 26 Feb 2012 09:50:45 +0000 (10:50 +0100)]
Fix parser not to clobber has_b_frames when extradata is set.

Because in contrast to the decoder, the parser does not setup low_delay.
The code in parse_nal_units would always end up setting has_b_frames
to "1", except when stream is explicitly marked as low delay.
Since the parser itself would create 'extradata', simply reopening
the parser would cause this.

This happens for instance in estimate_timings_from_pts(), which causes the
parser to be reopened on the same stream.

This fixes Libav #22 and FFmpeg (trac) #360

CC: libav-stable@libav.org
Based on a patch by Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(commit 31ac0ac29b6bba744493f7d1040757a3f51b9ad7)

Comments and description adapted by Reinhard Tartler.

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 790a367d9ecd04360f78616765ee723f3fe65645)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agorm: prevent infinite loops for index parsing.
Ronald S. Bultje [Wed, 22 Feb 2012 19:33:24 +0000 (11:33 -0800)]
rm: prevent infinite loops for index parsing.

Specifically, prevent jumping back in the file for the next index, since
this can lead to infinite loops where we jump between indexes referring
to each other, and don't read indexes that don't fit in the file.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit aac07a7a4c2c7a4a29cf6dbc88c1b9fdd191b99d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agofraps: release reference buffer on pix_fmt change.
Ronald S. Bultje [Fri, 24 Feb 2012 22:11:04 +0000 (14:11 -0800)]
fraps: release reference buffer on pix_fmt change.

Prevents crash when trying to copy from a non-existing plane in e.g.
a RGB32 reference image to a YUV420P target image

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 830f70442a87a31f7c75565e9380e3caf8333b8a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agokgv1: release reference picture on size change.
Ronald S. Bultje [Sat, 25 Feb 2012 00:27:53 +0000 (16:27 -0800)]
kgv1: release reference picture on size change.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 6c4c27adb61b2881a94ce5c7d97ee1c8adadb5fe)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agolcl: error out if uncompressed input buffer is smaller than framesize.
Ronald S. Bultje [Fri, 24 Feb 2012 00:09:36 +0000 (16:09 -0800)]
lcl: error out if uncompressed input buffer is smaller than framesize.

This prevents crashes when trying to read beyond the end of the buffer
while decoding frame data.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit be129271eac04f91393bf42a490ec631e1a9abea)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agotiff: Prevent overreads in the type_sizes array.
Alex Converse [Thu, 23 Feb 2012 18:47:50 +0000 (10:47 -0800)]
tiff: Prevent overreads in the type_sizes array.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 447363870f2f91e125e07ac2d0820359a5d86b06)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoswf: check return values for av_get/new_packet().
Ronald S. Bultje [Thu, 23 Feb 2012 19:53:27 +0000 (11:53 -0800)]
swf: check return values for av_get/new_packet().

Prevents crashers when using the packet if allocation failed.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 31632e73f47d25e2077fce729571259ee6354854)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agotruemotion2: error out if the huffman tree has no nodes.
Ronald S. Bultje [Wed, 22 Feb 2012 20:19:52 +0000 (12:19 -0800)]
truemotion2: error out if the huffman tree has no nodes.

This prevents crashers and errors further down when reading nodes in the
empty tree.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2b83e8b7005d531bc78b0fd4f699e9faa54ce9bb)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomjpegb: don't return 0 at the end of frame decoding.
Ronald S. Bultje [Sat, 18 Feb 2012 00:27:36 +0000 (16:27 -0800)]
mjpegb: don't return 0 at the end of frame decoding.

Return 0 indicates "please return the same data again", i.e. it causes
an infinite loop. Instead, return that we consumed the buffer if we
finished decoding succesfully, or return an error if an error occurred.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 74699ac8c8b562e9f8d26e21482b89585365774a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoasf: prevent packet_size_left from going negative if hdrlen > pktlen.
Ronald S. Bultje [Fri, 17 Feb 2012 20:21:18 +0000 (12:21 -0800)]
asf: prevent packet_size_left from going negative if hdrlen > pktlen.

This prevents failed assertions further down in the packet processing
where we require non-negative values for packet_size_left.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 41afac7f7a67c634c86b1d17fc930e9183d4aaa0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agohuffyuv: error out on bit overrun.
Ronald S. Bultje [Fri, 17 Feb 2012 23:00:47 +0000 (15:00 -0800)]
huffyuv: error out on bit overrun.

On EOF, get_bits() will continuously return 0, causing an infinite
loop.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 84c202cc37024bd78261e4222e46631ea73c48dd)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoals: prevent infinite loop in zero_remaining().
Ronald S. Bultje [Fri, 17 Feb 2012 20:28:26 +0000 (12:28 -0800)]
als: prevent infinite loop in zero_remaining().

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit af468015d972c0dec5c8c37b2685ffa5cbe4ae87)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agocook: prevent div-by-zero if channels is zero.
Ronald S. Bultje [Fri, 17 Feb 2012 20:10:33 +0000 (12:10 -0800)]
cook: prevent div-by-zero if channels is zero.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 941fc1ea1ed7f7d99a8b9e2607b41f2f2820394a)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoswscale: take first/lastline over/underflows into account for MMX.
Ronald S. Bultje [Thu, 23 Feb 2012 00:48:38 +0000 (16:48 -0800)]
swscale: take first/lastline over/underflows into account for MMX.

Fixes crashes for extremely large resizes (several 100-fold).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1d8c4af396b6ed84c84b5ebf0bf1163c4a7a3017)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoswscale: fix overflows in filterPos[] calculation for large sizes.
Ronald S. Bultje [Thu, 23 Feb 2012 00:46:31 +0000 (16:46 -0800)]
swscale: fix overflows in filterPos[] calculation for large sizes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 19a65b5be47944c607a9e979edb098924d95f2e4)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoswscale: enforce a minimum filtersize.
Ronald S. Bultje [Sat, 11 Feb 2012 16:42:28 +0000 (08:42 -0800)]
swscale: enforce a minimum filtersize.

At very small dimensions, this calculation could lead to zero-sized
filters, which leads to uninitialized output, zero-sized allocations,
loop overflows in SIMD that uses do{..}while(i++<filtersize); instead
of for(i=0;i<filtersize;i++){..} and several other similar failures.
Therefore, require a minimum filtersize of 1.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit dae2ce361a2b5fd9be1d43e5e8c00bdbc5f03e3d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agosmacker: Sanity check huffman tables found in the headers.
Alex Converse [Thu, 26 Jan 2012 00:12:42 +0000 (16:12 -0800)]
smacker: Sanity check huffman tables found in the headers.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 9adf25c1cf78dbf1d71bf386c49dc74cb8a60df0)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomatroska: don't overwrite string values until read/alloc was succesful.
Ronald S. Bultje [Sat, 25 Feb 2012 00:12:18 +0000 (16:12 -0800)]
matroska: don't overwrite string values until read/alloc was succesful.

This prevents certain tags with a default value assigned to them (as per
the EBML syntax elements) from ever being assigned a NULL value. Other
parts of the code rely on these being non-NULL (i.e. they don't check for
NULL before e.g. using the string in strcmp() or similar), and thus in
effect this prevents crashes when reading of such specific tags fails,
either because of low memory or because of targeted file corruption.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit cd40c31ee9ad2cca6f3635950b002fd46be07e98)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agomatroskadec: Pad AAC extradata.
Alex Converse [Wed, 25 Jan 2012 22:34:21 +0000 (14:34 -0800)]
matroskadec: Pad AAC extradata.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d2ee8c17793201ce969afd1f433ba1580c143cd2)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoaac: fix infinite loop on end-of-frame with sequence of 1-bits.
Alex Converse [Wed, 22 Feb 2012 19:05:42 +0000 (11:05 -0800)]
aac: fix infinite loop on end-of-frame with sequence of 1-bits.

Based-on-work-by: Ronald S. Bultje <rsbultje@gmail.com>
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1cd9a6154bc1ac1193c703cea980ed21c3e53792)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agowma: Clip WMA1 and WMA2 frame length to 11 bits.
Alex Converse [Wed, 25 Jan 2012 02:43:43 +0000 (18:43 -0800)]
wma: Clip WMA1 and WMA2 frame length to 11 bits.

The MDCT buffers in the decoder are only sized for up to 11 bits. The
reverse engineered documentation for WMA1/2 headers say that that for
all samplerates above 32kHz 11 bits are used. 12 and 13 bit support
were added for WMAPro. I was unable to make any Microsoft tools generate
a test file at a samplerate above 48kHz.

Discovered by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org
(cherry picked from commit d78bb1a4b2a3a415b68e4e6dd448779eccec64e3)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoflac: fix infinite loops on all-zero input or end-of-stream.
Ronald S. Bultje [Wed, 15 Feb 2012 17:52:11 +0000 (09:52 -0800)]
flac: fix infinite loops on all-zero input or end-of-stream.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 52e4018be47697a60f4f18f83551766df31f5adf)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agogolomb: avoid infinite loop on all-zero input (or end of buffer).
Ronald S. Bultje [Tue, 14 Feb 2012 19:50:57 +0000 (11:50 -0800)]
golomb: avoid infinite loop on all-zero input (or end of buffer).

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c6643fddba73560f26f90d327c84d8832222a720)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoqdm2: Check data block size for bytes to bits overflow.
Alex Converse [Wed, 25 Jan 2012 23:27:11 +0000 (15:27 -0800)]
qdm2: Check data block size for bytes to bits overflow.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit dac56d9ce01eb9963f28f26b97a81db5cbd46c1c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
7 years agoavcodec: Remove a misplaced and useless attribute_deprecated
Martin Storsjö [Wed, 2 Nov 2011 15:54:00 +0000 (17:54 +0200)]
avcodec: Remove a misplaced and useless attribute_deprecated

If attribute_deprecated is used in an enum declaration, it
should follow the 'enum' keyword, otherwise it's ignored
silently. This is the only case of attribute_deprecated for
enum declarations currently.

Currently, this attribute_deprecated doesn't have any effect.
If moved to the right place, it emits a warning every single
time avcodec.h is included, like this:

avcodec.h:2827: warning: ‘AVLPCType’ is deprecated (declared at avcodec.h:543)

There is already a working attribute_deprecated for the
corresponding field in AVCodecContext, so therefore this
one shouldn't be needed.

Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 1b6da627d49e98fe7661c9aa9ec4e16ab04dfda4)

Signed-off-by: Martin Storsjö <martin@martin.st>
7 years agointfloat_readwrite: fix signed addition overflows
Mans Rullgard [Sat, 8 Oct 2011 01:16:29 +0000 (02:16 +0100)]
intfloat_readwrite: fix signed addition overflows

These additions might overflow the signed range for large
input values.  Converting to unsigned before the addition
rather than after avoids such undefined behaviour.  The
result under normal two's complement wraparound remains
unchanged.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 88d1e2b2b0a129365a62efd666db0394e8ffbe08)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agosmacker: validate channels and sample format.
Justin Ruggles [Wed, 21 Sep 2011 15:49:33 +0000 (11:49 -0400)]
smacker: validate channels and sample format.

(cherry picked from commit ff1f89de2da3472d133e2c95bf7c9ad2d88df33d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>
7 years agosmacker: check buffer size before reading output size
Justin Ruggles [Wed, 21 Sep 2011 15:42:55 +0000 (11:42 -0400)]
smacker: check buffer size before reading output size

(cherry picked from commit cf044f8bff0d28dbc34492f18b0d18b3ba8bad9d)

Signed-off-by: Anton Khirnov <anton@khirnov.net>