ffmpeg.git
4 years agoUpdate for 0.7.17 n0.7.17
Michael Niedermayer [Wed, 11 Mar 2015 23:54:14 +0000 (00:54 +0100)]
Update for 0.7.17

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoh264/cabac: check loop index
Michael Niedermayer [Thu, 31 Jan 2013 03:20:24 +0000 (04:20 +0100)]
h264/cabac: check loop index

fix out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cdf0877bc341684c56ac1fe057397adbadf329ee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoroqvideoenc: set enc->avctx in roq_encode_init
Andreas Cadhalpun [Mon, 9 Mar 2015 18:24:09 +0000 (19:24 +0100)]
roqvideoenc: set enc->avctx in roq_encode_init

So far it is only set in roq_encode_frame, but it is used in
roq_encode_end to free the coded_frame. This currently segfaults if
roq_encode_frame is not called between roq_encode_init and
roq_encode_end.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cf82c426fadf90105e1fb9d5ecd267cc3aa2b288)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/tiff: move bpp check to after "end:"
Michael Niedermayer [Sun, 8 Mar 2015 22:27:43 +0000 (23:27 +0100)]
avcodec/tiff: move bpp check to after "end:"

This ensures that all current and future code-pathes get bpp checked

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d5e9fc782150d4596c72440a0aa02b7f4f1254b1)

Conflicts:

libavcodec/tiff.c

4 years agoavcodec/utils: Align YUV411 by as much as the other YUV variants
Michael Niedermayer [Sat, 7 Mar 2015 13:30:34 +0000 (14:30 +0100)]
avcodec/utils: Align YUV411 by as much as the other YUV variants

Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash2.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Tested-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e3201c38d53d2b8b24d0bc95d726b2cb1752dc12)

Conflicts:

libavcodec/utils.c

4 years agoFix buffer_size argument to init_put_bits() in multiple encoders.
Dyami Caliri [Thu, 26 Feb 2015 18:17:01 +0000 (10:17 -0800)]
Fix buffer_size argument to init_put_bits() in multiple encoders.

Several encoders were multiplying the buffer size by 8, in order to get
a bit size. However, the buffer_size argument is for the byte size of
the buffer. We had experienced crashes encoding prores (Anatoliy) at
size 4096x4096.
(cherry picked from commit 50833c9f7b4e1922197a8955669f8ab3589c8cef)

Conflicts:

libavcodec/proresenc_kostya.c

Conflicts:

libavcodec/faxcompr.c
libavcodec/s302menc.c

Conflicts:

libavcodec/adpcmenc.c

Conflicts:

libavcodec/adpcmenc.c
libavcodec/proresenc.c

4 years agoavcodec/a64multienc: fix use of uninitialized values in to_meta_with_crop
Andreas Cadhalpun [Sun, 22 Feb 2015 19:48:38 +0000 (20:48 +0100)]
avcodec/a64multienc: fix use of uninitialized values in to_meta_with_crop

Averaging over 2 pixels doesn't work correctly for the last pixel, because the
rest of the buffer is not initialized.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 87513d654546a99f8ddb045ca4fa5d33778a617e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/x86/mlpdsp_init: Simplify mlp_filter_channel_x86()
Michael Niedermayer [Thu, 19 Feb 2015 15:25:29 +0000 (16:25 +0100)]
avcodec/x86/mlpdsp_init: Simplify mlp_filter_channel_x86()

Based on patch by Francisco Blas Izquierdo Riera
Commit message partly taken from carl

fixes a compilation
error in mlpdsp_init.c with -fstack-check and some gcc compilers (I
reproduced the issue with gcc 4.7.3) by simplifying the code.

See also https://bugs.gentoo.org/show_bug.cgi?id=471756

$ make libavcodec/x86/mlpdsp_init.o
libavcodec/x86/mlpdsp_init.c: In function ‘mlp_filter_channel_x86’:
libavcodec/x86/mlpdsp_init.c:142:5: error: can’t find a register in
class ‘GENERAL_REGS’ while reloading ‘asm’
libavcodec/x86/mlpdsp_init.c:142:5: error: ‘asm’ operand has impossible
constraints

4551 -> 4509 dezicycles

Reviewed-by: Ramiro Polla <ramiro.polla@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03f39fbb2a558153a3c464edec1378d637a755fe)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/vqf: Use 64bit for ret to avoid overflow
Michael Niedermayer [Fri, 20 Feb 2015 20:00:57 +0000 (21:00 +0100)]
avformat/vqf: Use 64bit for ret to avoid overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb08687180683a755d0fe9d425280d0e4d1e6db2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/gxf: Use 64bit for res to avoid overflow
Michael Niedermayer [Fri, 20 Feb 2015 19:14:56 +0000 (20:14 +0100)]
avformat/gxf: Use 64bit for res to avoid overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 12987f89007ee82b9d3a6090085dfaef8461ab8b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mjpegdec: Skip blocks which are outside the visible area
Michael Niedermayer [Wed, 11 Feb 2015 02:33:53 +0000 (03:33 +0100)]
avcodec/mjpegdec: Skip blocks which are outside the visible area

Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08509c8f86626815a3e9e68d600d1aacbb8df4bf)

Conflicts:

libavcodec/mjpegdec.c
(cherry picked from commit 5553947db2af443778f781a107d9fe9ad6ec5d17)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264_ps: More completely check the bit depths
Michael Niedermayer [Fri, 6 Feb 2015 03:11:56 +0000 (04:11 +0100)]
avcodec/h264_ps: More completely check the bit depths

Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69aa79365c1e8e1cb597d33e77bf1062c2ef47d4)

Conflicts:

libavcodec/h264_ps.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/thp: Check av_get_packet() for failure not only for partial output
Michael Niedermayer [Thu, 5 Feb 2015 02:45:21 +0000 (03:45 +0100)]
avformat/thp: Check av_get_packet() for failure not only for partial output

Fixes null pointer dereference
Fixes: signal_sigsegv_db2c1f_3108_cov_163322880_pikmin2_opening1_partial.thp

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f2579dbb4b31e6ae731e7f5555680528ef3020ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mjpegdec: Check number of components for JPEG-LS
Michael Niedermayer [Wed, 4 Feb 2015 19:48:30 +0000 (20:48 +0100)]
avcodec/mjpegdec: Check number of components for JPEG-LS

Fixes out of array accesses
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fabbfaa095660982cc0bc63242c459561fa37037)

Conflicts:

libavcodec/mjpegdec.c

4 years agoavformat/mpc8: Use uint64_t in *_get_v() to avoid undefined behavior
Michael Niedermayer [Wed, 4 Feb 2015 13:47:41 +0000 (14:47 +0100)]
avformat/mpc8: Use uint64_t in *_get_v() to avoid undefined behavior

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 05e161952954acf247e0fd1fdef00559675c4d4d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mpc8: fix broken pointer math
wm4 [Tue, 3 Feb 2015 18:04:11 +0000 (19:04 +0100)]
avformat/mpc8: fix broken pointer math

This could overflow and crash at least on 32 bit systems.

Reviewed-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b737a2c52857b214be246ff615c6293730033cfa)

Conflicts:

libavformat/mpc8.c
(cherry picked from commit 49dd89f9027f3def12e170bb7d986d37812eedba)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mpc8: fix hang with fuzzed file
wm4 [Tue, 3 Feb 2015 18:04:12 +0000 (19:04 +0100)]
avformat/mpc8: fix hang with fuzzed file

This can lead to an endless loop by seeking back a few bytes after each
attempted chunk read. Assuming negative sizes are always invalid, this
is easy to fix. Other code in this demuxer treats negative sizes as
invalid as well.

Fixes ticket #4262.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 56cc024220886927350cfc26ee695062ca7ecaf4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/smacker: Fix number suffix
Michael Niedermayer [Sun, 1 Feb 2015 18:36:13 +0000 (19:36 +0100)]
avformat/smacker: Fix number suffix

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 465f3705b1ef832fd6904750d018f81f9044f3ab)

Conflicts:

libavformat/smacker.c
(cherry picked from commit ef3687998fb46d7d936bb6023f25c1dc324a4640)

4 years agosmackerdemuxer: check some values before instead of just after malloc()
Michael Niedermayer [Fri, 16 Dec 2011 01:57:22 +0000 (02:57 +0100)]
smackerdemuxer: check some values before instead of just after malloc()
Fixes Ticket777
Bug Found by: Diana Elena Muscalu

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c402c1c976dc5bd63908d1aaff5b60521cbbee92)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/utils: Fix number suffixes in tb_unreliable()
Michael Niedermayer [Sun, 1 Feb 2015 18:19:25 +0000 (19:19 +0100)]
avformat/utils: Fix number suffixes in tb_unreliable()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b15bba2aec93776bfdc69a1bca42a4795a7d191)

Conflicts:

libavformat/utils.c
(cherry picked from commit e651a2f88c219e74c9851563e74100f7652a6005)

4 years agoavcodec/flac_parser: fix handling EOF if no headers are found
Michael Niedermayer [Sat, 17 Jan 2015 00:56:03 +0000 (01:56 +0100)]
avcodec/flac_parser: fix handling EOF if no headers are found

Fixes assertion failure
Fixes Ticket4269

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c4d85fc23c100f7a27d9bad710eb153214868e27)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/vmdvideo: Check len before using it in method 3
Michael Niedermayer [Tue, 16 Dec 2014 15:24:55 +0000 (16:24 +0100)]
avcodec/vmdvideo: Check len before using it in method 3

Fixes out of array access
Fixes: asan_heap-oob_4d23ba_91_cov_3853393937_128.vmd

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3030fb7e0d41836f8add6399e9a7c7b740b48bfd)

Conflicts:

libavcodec/vmdvideo.c

4 years agoavformat/aviobuf: Check that avio_seek() target is non negative
Michael Niedermayer [Sun, 14 Dec 2014 16:26:11 +0000 (17:26 +0100)]
avformat/aviobuf: Check that avio_seek() target is non negative

Fixes out of array access

Suggested-by: Andrew Scherkus <scherkus@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ed86dbd05d61363dc1c0d33f3267e2177c985fdd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale/x86/rgb2rgb_template: fix crash with tiny size and nv12 output
Michael Niedermayer [Wed, 3 Dec 2014 19:21:56 +0000 (20:21 +0100)]
swscale/x86/rgb2rgb_template: fix crash with tiny size and nv12 output

Fixes Ticket4151

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8524558858b7e14bc50afa10233e0194f591ab9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/pngdec: Check IHDR/IDAT order
Michael Niedermayer [Wed, 26 Nov 2014 14:45:47 +0000 (15:45 +0100)]
avcodec/pngdec: Check IHDR/IDAT order

Fixes out of array access
Fixes: asan_heap-oob_20a6c26_2690_cov_3434532168_mail.png
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 79ceaf827be0b070675d4cd0a55c3386542defd8)

Conflicts:

libavcodec/pngdec.c

4 years agoavcodec/mjpegdec: Fix context fields becoming inconsistent
Michael Niedermayer [Tue, 25 Nov 2014 12:53:06 +0000 (13:53 +0100)]
avcodec/mjpegdec: Fix context fields becoming inconsistent

Fixes out of array access
Fixes: asan_heap-oob_1ca4f85_2760_cov_144449187_miss_congeniality_pegasus_ljpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0eecf40935b22644e6cd74c586057237ecfd6844)

Conflicts:

libavcodec/mjpegdec.c
(cherry picked from commit 32d3acac727f3f4a6489ca129a5ea4ccdfcb34a5)

Conflicts:

libavcodec/mjpegdec.c
(cherry picked from commit 8d8ac60d70aee50d44a3e1d7de276598de041640)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/wmaprodec: Fix integer overflow in sfb_offsets initialization
Michael Niedermayer [Mon, 10 Nov 2014 22:07:50 +0000 (23:07 +0100)]
avcodec/wmaprodec: Fix integer overflow in sfb_offsets initialization

Fixes out of array read
Fixes: asan_heap-oob_2aec5b0_1828_classical_22_16_2_16000_v3c_0_exclusive_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5dcb99033df16eccc4dbbc4a099ad64457f9f090)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dxa: check dimensions
Michael Niedermayer [Tue, 28 Oct 2014 14:26:42 +0000 (15:26 +0100)]
avcodec/dxa: check dimensions

Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc22c4e54d5716f28f28db8f99c74cc90)

Conflicts:

libavcodec/dxa.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mpegts: Check desc_len / get8() return code
Michael Niedermayer [Sat, 4 Oct 2014 02:29:40 +0000 (04:29 +0200)]
avformat/mpegts: Check desc_len / get8() return code

Fixes out of array read
Fixes: signal_sigsegv_844d59_10_signal_sigsegv_a17bb7_366_mpegts_mpeg2video_mp2_dvbsub_topfield.rec

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c3d7f00ee3e09801f56f25db8b5961f25e842bd2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/smc: fix off by 1 error
Michael Niedermayer [Fri, 3 Oct 2014 20:50:45 +0000 (22:50 +0200)]
avcodec/smc: fix off by 1 error

Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9d62335e89d118a5b4e202edf39d905)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/qpeg: fix off by 1 error in MV bounds check
Michael Niedermayer [Fri, 3 Oct 2014 19:08:52 +0000 (21:08 +0200)]
avcodec/qpeg: fix off by 1 error in MV bounds check

Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1ca26d0fff3a3baf61a40207032143f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/gifdec: factorize interleave end handling out
Michael Niedermayer [Fri, 3 Oct 2014 18:15:52 +0000 (20:15 +0200)]
avcodec/gifdec: factorize interleave end handling out

also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864be8fb9653643519dea1c6492f1dde57)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/pngdec: Calculate MPNG bytewidth more defensively
Michael Niedermayer [Fri, 3 Oct 2014 15:54:21 +0000 (17:54 +0200)]
avcodec/pngdec: Calculate MPNG bytewidth more defensively

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934a29df05c7af65aef2a480b15f572c4)

Conflicts:

libavcodec/pngdec.c

4 years agoavcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Michael Niedermayer [Fri, 3 Oct 2014 15:35:58 +0000 (17:35 +0200)]
avcodec/pngdec: Check bits per pixel before setting monoblack pixel format

Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020c2dbf0201fe7df3dad9e7e0b2e1bb6)

Conflicts:

libavcodec/pngdec.c

4 years agoavcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Michael Niedermayer [Fri, 3 Oct 2014 12:45:04 +0000 (14:45 +0200)]
avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks

Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f21b761ca15dbb470cd619a1ebf86c3e)

Conflicts:

libavcodec/mmvideo.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/utils: Add case for jv to avcodec_align_dimensions2()
Michael Niedermayer [Fri, 3 Oct 2014 02:30:58 +0000 (04:30 +0200)]
avcodec/utils: Add case for jv to avcodec_align_dimensions2()

Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376a736d243aef4a1d121abebce912e6b)

Conflicts:

libavcodec/utils.c

4 years agoavcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Michael Niedermayer [Thu, 2 Oct 2014 23:50:27 +0000 (01:50 +0200)]
avcodec/mjpegdec: check bits per pixel for changes similar to dimensions

Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6df8243f06c87962b873bd563e58cd39)

Conflicts:

libavcodec/mjpegdec.c
(cherry picked from commit 94371a404c663c3dae3d542fa43951567ab67f82)

Conflicts:

libavcodec/mjpegdec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/jpeglsdec: Check run value more completely in ls_decode_line()
Michael Niedermayer [Thu, 2 Oct 2014 21:17:21 +0000 (23:17 +0200)]
avcodec/jpeglsdec: Check run value more completely in ls_decode_line()

previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410a17dc72c30ee7f3145fcacc425f4f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoconfigure: add noexecstack to linker options if supported.
Reimar Döffinger [Sun, 21 Sep 2014 08:58:10 +0000 (09:58 +0100)]
configure: add noexecstack to linker options if supported.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit b7082d953fda93f7841ffffe7d15a6c3cd15bdee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavutil/lzo: Fix integer overflow
Michael Niedermayer [Fri, 20 Jun 2014 01:15:28 +0000 (03:15 +0200)]
avutil/lzo: Fix integer overflow

Embargoed-till: 2014-06-27 requested by researcher, but embargo broken by libav today (git and mailing list)

Fixes: LMS-2014-06-16-4
Found-by: "Don A. Bailey" <donb@securitymouse.com>
See: ccda51b14c0fcae2fad73a24872dce75a7964996
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d6af26c55c1ea30f85a7d9edbc373f53be1743ee)

Conflicts:

libavutil/lzo.c
(cherry picked from commit 7b5c706494a775b2b0d0e0a38448610802eef8f4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agomatroska: Fix use after free
Dale Curtis [Thu, 10 Jan 2013 19:05:29 +0000 (11:05 -0800)]
matroska: Fix use after free

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit ae3d41636942cbc0236bad21ad06c65f4eb0f096)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agodnxhdenc: fix mb_rc size
Michael Niedermayer [Fri, 17 Jan 2014 19:09:48 +0000 (20:09 +0100)]
dnxhdenc: fix mb_rc size

Fixes out of array access with RC_VARIANCE set to 0

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61310beba705957e6366f0392a0b005b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoarm: Don't clobber callee saved registers in scalarproduct
Martin Storsjö [Fri, 20 Dec 2013 13:02:35 +0000 (15:02 +0200)]
arm: Don't clobber callee saved registers in scalarproduct

q4-q7/d8-d15 are supposed to not be clobbered by the callee.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit d307e408d4a9ada22df443cc38be77cc5e492694)

5 years agoavformat/utils: do not override pts in h264 when they are provided from the demuxer
Michael Niedermayer [Sat, 26 Oct 2013 23:03:19 +0000 (01:03 +0200)]
avformat/utils: do not override pts in h264 when they are provided from the demuxer

Fixes Ticket2143

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1e5271a9fd6ddcceb083f2185a4bbd8d44c9a813)

5 years agoupdate for 0.7.16 n0.7.16
Michael Niedermayer [Sun, 6 Oct 2013 17:07:56 +0000 (19:07 +0200)]
update for 0.7.16

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge tag 'n0.8.15' into release/0.7
Michael Niedermayer [Sun, 6 Oct 2013 16:49:35 +0000 (18:49 +0200)]
Merge tag 'n0.8.15' into release/0.7

FFmpeg 0.8.15 release

* tag 'n0.8.15': (49 commits)
  update for 0.8.15
  avcodec/ffv1enc: update buffer check for 16bps
  avcodec/dsputil: fix signedness in sizeof() comparissions
  avcodec/pngdsp: fix (un)signed type in end comparission
  matroska_read_seek: Fix used streams for subtitle index compensation
  jpeg2000: check log2_cblk dimensions
  avcodec/rpza: Perform pointer advance and checks before using the pointers
  update all trac links to use the trac subdomain
  doc/APIchanges: List merge commit hashes and version numbers
  apichanges: fix 2 wrong hashes
  avcodec/parser: reset indexes on realloc failure
  mpeg12dec: avoid reinitialization on PS changes when possible.
  mpegts: only reopen pmt_cb filter if its different from the previous.
  Autodetect idcin only if audio properties allow decoding.
  alacenc: Fix missing sign_extend()
  h264_cavlc: fix reading skip run
  Update changelog for 0.7.8 release
  aac: check the maximum number of channels
  oggdec: fix faulty cleanup prototype
  qdm2: check that the FFT size is a power of 2
  ...

Conflicts:
Doxyfile
RELEASE
VERSION
libavformat/matroskadec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoupdate for 0.8.15 n0.8.15
Michael Niedermayer [Sun, 6 Oct 2013 15:48:25 +0000 (17:48 +0200)]
update for 0.8.15

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoavcodec/ffv1enc: update buffer check for 16bps
Michael Niedermayer [Mon, 9 Sep 2013 15:58:18 +0000 (17:58 +0200)]
avcodec/ffv1enc: update buffer check for 16bps

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f1854b5c79d1a64dd3b41b80640ef1e7f)

Conflicts:

libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)

5 years agoavcodec/dsputil: fix signedness in sizeof() comparissions
Michael Niedermayer [Fri, 30 Aug 2013 21:40:47 +0000 (23:40 +0200)]
avcodec/dsputil: fix signedness in sizeof() comparissions

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9c686c78aa97954306fb63453299760)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoavcodec/pngdsp: fix (un)signed type in end comparission
Michael Niedermayer [Fri, 30 Aug 2013 21:14:32 +0000 (23:14 +0200)]
avcodec/pngdsp: fix (un)signed type in end comparission

Fixes out of array accesses
Fixes Ticket2919

Found_by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86736f59d6a527d8bc807d09b93f971c0fe0bb07)

Conflicts:

libavcodec/pngdsp.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agomatroska_read_seek: Fix used streams for subtitle index compensation
Michael Niedermayer [Mon, 20 May 2013 02:00:30 +0000 (04:00 +0200)]
matroska_read_seek: Fix used streams for subtitle index compensation

Might fix Ticket1907 (I have no testcase so i cant test)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4758e32a6c48044f77102a49110c79b4f338f648)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agojpeg2000: check log2_cblk dimensions
Michael Niedermayer [Sat, 24 Aug 2013 01:19:40 +0000 (03:19 +0200)]
jpeg2000: check log2_cblk dimensions

Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368eaabf99e6c2046103acb33957e63b7)

Conflicts:

libavcodec/jpeg2000dec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:

libavcodec/j2kdec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoavcodec/rpza: Perform pointer advance and checks before using the pointers
Michael Niedermayer [Wed, 21 Aug 2013 23:07:32 +0000 (01:07 +0200)]
avcodec/rpza: Perform pointer advance and checks before using the pointers

Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745da2ac7fb3faacb116788c32f4753f34)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoupdate all trac links to use the trac subdomain
Michael Niedermayer [Sat, 29 Jun 2013 17:48:27 +0000 (19:48 +0200)]
update all trac links to use the trac subdomain

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agodoc/APIchanges: List merge commit hashes and version numbers
Michael Niedermayer [Wed, 13 Feb 2013 02:32:23 +0000 (03:32 +0100)]
doc/APIchanges: List merge commit hashes and version numbers

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoapichanges: fix 2 wrong hashes
Michael Niedermayer [Thu, 14 Feb 2013 20:13:32 +0000 (21:13 +0100)]
apichanges: fix 2 wrong hashes

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2f3bc5122822687dc388f7352c92cf6db456cf7c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoavcodec/parser: reset indexes on realloc failure
Michael Niedermayer [Thu, 26 Sep 2013 19:03:48 +0000 (21:03 +0200)]
avcodec/parser: reset indexes on realloc failure

Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9abfb2ae75bb32bc44e2c34194c8dc40a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge remote-tracking branch 'qatar/release/0.7' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:43:33 +0000 (17:43 +0200)]
Merge remote-tracking branch 'qatar/release/0.7' into release/0.8

* qatar/release/0.7:
  Update changelog for 0.7.8 release
  aac: check the maximum number of channels
  oggdec: fix faulty cleanup prototype
  qdm2: check that the FFT size is a power of 2
  rv10: check that extradata is large enough
  lavf: make sure stream probe data gets freed.
  dfa: check for invalid access in decode_wdlt().
  avfiltergraph: check for sws opts being non-NULL before using them.

Conflicts:
Changelog
libavformat/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit 'f844cb9bced3148fca2db5bbb092929526108005' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:36:39 +0000 (17:36 +0200)]
Merge commit 'f844cb9bced3148fca2db5bbb092929526108005' into release/0.8

* commit 'f844cb9bced3148fca2db5bbb092929526108005':
  iff: validate CMAP palette size
  wmaprodec: require block_align to be set.
  lzo: fix overflow checking in copy_backptr()
  flacdec: simplify bounds checking in flac_probe()
  atrac3: avoid oversized shifting in decode_bytes()
  lavf: fix arithmetic overflows in avformat_seek_file()

Conflicts:
libavformat/iff.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit '9c713f30e4913a28d93eb37ea5db7f62be4c0ef6' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:31:56 +0000 (17:31 +0200)]
Merge commit '9c713f30e4913a28d93eb37ea5db7f62be4c0ef6' into release/0.8

* commit '9c713f30e4913a28d93eb37ea5db7f62be4c0ef6':
  parser: fix large overreads
  dsputil: fix invalid array indexing
  shorten: use the unsigned type where needed

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit '5ebb5a32bdd910a8afb316c51ed0b322f5600ae5' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:25:47 +0000 (17:25 +0200)]
Merge commit '5ebb5a32bdd910a8afb316c51ed0b322f5600ae5' into release/0.8

* commit '5ebb5a32bdd910a8afb316c51ed0b322f5600ae5':
  shorten: report meaningful errors
  shorten: set invalid channels count to 0

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit 'd785f6940144eb6ce4c24309ed034056b81395bc' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:22:35 +0000 (17:22 +0200)]
Merge commit 'd785f6940144eb6ce4c24309ed034056b81395bc' into release/0.8

* commit 'd785f6940144eb6ce4c24309ed034056b81395bc':
  shorten: validate that the channel count in the header is not <= 0
  matroskadec: request a read buffer for the wav header
  h264: check for luma and chroma bit depth being equal
  xxan: fix invalid memory access in xan_decode_frame_type0()
  wmadec: require block_align to be set.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit '5025dbc577c9a9e0109cb363ac630a9eeda6dc1d' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:17:10 +0000 (17:17 +0200)]
Merge commit '5025dbc577c9a9e0109cb363ac630a9eeda6dc1d' into release/0.8

* commit '5025dbc577c9a9e0109cb363ac630a9eeda6dc1d':
  wmaprodec: return an error, not 0, when the input is too small.
  vorbisdec: Error on bark_map_size equal to 0.
  Update RELEASE file for 0.7.8
  update year to 2013
  oggdec: make sure the private parse data is cleaned up
  indeo5: update AVCodecContext width/height on size change
  doc: filters: Correct BNF FILTER description

Conflicts:
RELEASE

Merged-by: Michael Niedermayer <michaelni@gmx.at>
6 years agompeg12dec: avoid reinitialization on PS changes when possible.
Michael Niedermayer [Mon, 8 Jul 2013 19:46:20 +0000 (21:46 +0200)]
mpeg12dec: avoid reinitialization on PS changes when possible.

Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df73528659925819dec31c4c8c0887f0321)

Conflicts:
libavcodec/mpeg12.c

6 years agompeg12dec: avoid reinitialization on PS changes when possible.
Michael Niedermayer [Mon, 8 Jul 2013 19:46:20 +0000 (21:46 +0200)]
mpeg12dec: avoid reinitialization on PS changes when possible.

Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df73528659925819dec31c4c8c0887f0321)

Conflicts:
libavcodec/mpeg12.c

6 years agompegts: only reopen pmt_cb filter if its different from the previous.
Michael Niedermayer [Fri, 5 Jul 2013 01:27:07 +0000 (03:27 +0200)]
mpegts: only reopen pmt_cb filter if its different from the previous.

Fixes Ticket2632

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b009267910df10c004b5f340a090d45da29089a0)

6 years agompegts: only reopen pmt_cb filter if its different from the previous.
Michael Niedermayer [Fri, 5 Jul 2013 01:27:07 +0000 (03:27 +0200)]
mpegts: only reopen pmt_cb filter if its different from the previous.

Fixes Ticket2632

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b009267910df10c004b5f340a090d45da29089a0)

6 years agoAutodetect idcin only if audio properties allow decoding.
Carl Eugen Hoyos [Wed, 19 Jun 2013 14:31:10 +0000 (16:31 +0200)]
Autodetect idcin only if audio properties allow decoding.

Fixes ticket #2688.
(cherry picked from commit 06bede95fcea47d2e51e8ff248c15311f335b898)

6 years agoAutodetect idcin only if audio properties allow decoding.
Carl Eugen Hoyos [Wed, 19 Jun 2013 14:31:10 +0000 (16:31 +0200)]
Autodetect idcin only if audio properties allow decoding.

Fixes ticket #2688.
(cherry picked from commit 06bede95fcea47d2e51e8ff248c15311f335b898)

6 years agoalacenc: Fix missing sign_extend()
Michael Niedermayer [Wed, 12 Jun 2013 22:01:13 +0000 (00:01 +0200)]
alacenc: Fix missing sign_extend()

Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc56f7e7d60767dd27ba8e846a05e8ae)

Conflicts:
libavcodec/alacenc.c

6 years agoalacenc: Fix missing sign_extend()
Michael Niedermayer [Wed, 12 Jun 2013 22:01:13 +0000 (00:01 +0200)]
alacenc: Fix missing sign_extend()

Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc56f7e7d60767dd27ba8e846a05e8ae)

Conflicts:
libavcodec/alacenc.c

6 years agoh264_cavlc: fix reading skip run
Michael Niedermayer [Thu, 30 May 2013 16:30:42 +0000 (18:30 +0200)]
h264_cavlc: fix reading skip run

Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd295c03720e00d3de83e1abcbedd4b9)

Conflicts:
libavcodec/h264_cavlc.c

6 years agoh264_cavlc: fix reading skip run
Michael Niedermayer [Thu, 30 May 2013 16:30:42 +0000 (18:30 +0200)]
h264_cavlc: fix reading skip run

Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd295c03720e00d3de83e1abcbedd4b9)

Conflicts:
libavcodec/h264_cavlc.c

6 years agoUpdate changelog for 0.7.8 release
Reinhard Tartler [Sat, 11 May 2013 10:08:35 +0000 (12:08 +0200)]
Update changelog for 0.7.8 release

6 years agoaac: check the maximum number of channels
Reinhard Tartler [Tue, 7 May 2013 05:13:50 +0000 (07:13 +0200)]
aac: check the maximum number of channels

Broken bitstreams could report a larger than specified number of
channels and cause outbound writes.

CC:libav-stable@libav.org
(cherry picked from commit a943a132f36f4df8fe2f749744677b71984abce7)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/aacdec.c

6 years agooggdec: fix faulty cleanup prototype
Luca Barbato [Wed, 9 Jan 2013 19:49:34 +0000 (20:49 +0100)]
oggdec: fix faulty cleanup prototype

(cherry picked from commit fba8e5b608577fc660989d0057a55818254a3744)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoqdm2: check that the FFT size is a power of 2
Anton Khirnov [Tue, 9 Apr 2013 13:25:20 +0000 (15:25 +0200)]
qdm2: check that the FFT size is a power of 2

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 34f87a58532ed652a6e0283c1d044ee5df0aef0b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agorv10: check that extradata is large enough
Anton Khirnov [Tue, 9 Apr 2013 18:33:25 +0000 (20:33 +0200)]
rv10: check that extradata is large enough

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

(cherry picked from commit 01d376f598fe95478036f5d1e3e5e14ffe32d4bf)

Conflicts:

libavcodec/rv10.c

6 years agoiff: validate CMAP palette size
Kostya Shishkov [Sun, 17 Mar 2013 19:22:19 +0000 (20:22 +0100)]
iff: validate CMAP palette size

Fixes CVE-2013-2495

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
CC: libav-stable@libav.org
(cherry picked from commit 50c449ac24fbb4c03c15d2e2026cef2204b80385)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 31a77177ff323ef83944c60a8654891213ab6691)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agolavf: make sure stream probe data gets freed.
Anton Khirnov [Wed, 27 Mar 2013 16:56:59 +0000 (17:56 +0100)]
lavf: make sure stream probe data gets freed.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit dbb1425811a672eddf4acf0513237cdf20f83756)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agowmaprodec: require block_align to be set.
Anton Khirnov [Wed, 6 Mar 2013 08:58:00 +0000 (09:58 +0100)]
wmaprodec: require block_align to be set.

Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit cacad1c058f66558ec727faac3b277d2dee264d4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 20373a66ec68d958c266f643a7d0e5ec254c0fcc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agodfa: check for invalid access in decode_wdlt().
Anton Khirnov [Wed, 27 Mar 2013 17:18:38 +0000 (18:18 +0100)]
dfa: check for invalid access in decode_wdlt().

This can happen when the number of skipped lines is not consistent with
the number of coded lines.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 3623589edc7b1257bb45aa9e52c9631e133f22b6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoavfiltergraph: check for sws opts being non-NULL before using them.
Anton Khirnov [Sun, 17 Mar 2013 15:14:58 +0000 (16:14 +0100)]
avfiltergraph: check for sws opts being non-NULL before using them.

Avoid snprintfing a NULL pointer.

CC: libav-stable@libav.org
(cherry picked from commit 6e3c13a559e9ff300b5ca60e1d503e594d7f055c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoparser: fix large overreads
Michael Niedermayer [Wed, 3 Oct 2012 14:06:23 +0000 (16:06 +0200)]
parser: fix large overreads

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 096abfa15052977eed93f0b5e01afd2d47c53c1f)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
6 years agolzo: fix overflow checking in copy_backptr()
Xi Wang [Fri, 15 Mar 2013 10:59:22 +0000 (06:59 -0400)]
lzo: fix overflow checking in copy_backptr()

The check `src > dst' in the form `&c->out[-back] > c->out' invokes
pointer overflow, which is undefined behavior in C.

Remove the check.  Also replace `&c->out[-back] < c->out_start' with
a safe form `c->out - c->out_start < back' to avoid overflow.

CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit ca6c3f2c53be70aa3c38e8f1292809db89ea1ba6)

Conflicts:
libavutil/lzo.c

6 years agodsputil: fix invalid array indexing
Mans Rullgard [Thu, 26 Apr 2012 13:00:43 +0000 (14:00 +0100)]
dsputil: fix invalid array indexing

Indexing outside an array is invalid and causes errors with
gcc 4.8.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 0a07f2b346433a9a2677c69c6b29a1a827e39109)

Signed-off-by: Diego Biurrun <diego@biurrun.de>
6 years agoflacdec: simplify bounds checking in flac_probe()
Xi Wang [Fri, 15 Mar 2013 11:11:47 +0000 (07:11 -0400)]
flacdec: simplify bounds checking in flac_probe()

Simplify `p->buf > p->buf + p->buf_size - 4' as `p->buf_size < 4'.
Avoid a possible out-of-bounds pointer, which is undefined behavior
in C.

CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 8425d693eefbedbb41f91735614d41067695aa37)

6 years agoatrac3: avoid oversized shifting in decode_bytes()
Xi Wang [Fri, 15 Mar 2013 10:31:21 +0000 (06:31 -0400)]
atrac3: avoid oversized shifting in decode_bytes()

When `off' is 0, `0x537F6103 << 32' in the following expression invokes
undefined behavior, the result of which is not necessarily 0.

    (0x537F6103 >> (off * 8)) | (0x537F6103 << (32 - (off * 8)))

Avoid oversized shifting.

CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit eba1ff31304e407db3cefd7532108408f364367b)

Conflicts:
libavcodec/atrac3.c

6 years agolavf: fix arithmetic overflows in avformat_seek_file()
Mans Rullgard [Fri, 7 Dec 2012 13:53:56 +0000 (13:53 +0000)]
lavf: fix arithmetic overflows in avformat_seek_file()

The values compared here can be more than INT64_MAX apart.  Since the
difference is always positive, converting to uint64_t before subtracting
gives the correct result without overflows.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 91ac403b1316d59b4f43c4ea0f237e24cec2819a)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
6 years agoshorten: use the unsigned type where needed
Luca Barbato [Tue, 5 Mar 2013 16:12:35 +0000 (17:12 +0100)]
shorten: use the unsigned type where needed

get_uint returns an unsigned value, use an unsigned to store
blocksize to make sure the comparison logic is correct and report
correctly the error for the channel count not supported.

CC: libav-stable@libav.org
(cherry picked from commit 5cf7c72757779a740e897a97710aac044fe5258c)
(cherry picked from commit 88089eecfd7e604d40d078b4f4206c647cb2e2b4)
(cherry picked from commit f42d03746afe491dd02bb6372961e85e78299864)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/shorten.c

6 years agoshorten: report meaningful errors
Reinhard Tartler [Tue, 7 May 2013 05:29:06 +0000 (07:29 +0200)]
shorten: report meaningful errors

(cherry picked from commit 4c364eb2b856fc33cf7b42f7c7b979e69fde5f3a)
(cherry picked from commit 0daf1428e82926dc5a8c72a0ff4c93aaa8a84ed9)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/shorten.c

6 years agoshorten: set invalid channels count to 0
Reinhard Tartler [Tue, 7 May 2013 05:26:19 +0000 (07:26 +0200)]
shorten: set invalid channels count to 0

Prevent the loop shorten_decode_close from writing and freeing out of
the array boundary.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
CC: libav-stable@libav.org
(cherry picked from commit c10da30d8426a1f681d99a780b6e311f7fb4e5c5)
(cherry picked from commit 21d568be179c54a1596d1377b4da7fbe755bfe7f)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/shorten.c

6 years agoshorten: validate that the channel count in the header is not <= 0
Justin Ruggles [Tue, 23 Oct 2012 04:40:51 +0000 (00:40 -0400)]
shorten: validate that the channel count in the header is not <= 0

(cherry picked from commit 4c53f4aed3edfa58360c7a2a468782eae31d3176)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/shorten.c

6 years agowmaprodec: return an error, not 0, when the input is too small.
Anton Khirnov [Wed, 6 Mar 2013 09:02:50 +0000 (10:02 +0100)]
wmaprodec: return an error, not 0, when the input is too small.

Returning 0 may result in an infinite loop in valid calling programs. A
decoder should never return 0 without producing any output.

CC:libav-stable@libav.org
(cherry picked from commit 4c0080b7e7d501e2720d2a61f5186a18377f9d63)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 60dd8b5733f9ec4919fbc732ace1be8184dde880)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agomatroskadec: request a read buffer for the wav header
Luca Barbato [Tue, 12 Mar 2013 17:56:28 +0000 (18:56 +0100)]
matroskadec: request a read buffer for the wav header

Solve an infiniloop.

CC: libav-stable@libav.org
(cherry picked from commit 37cb3b180a1dc3d6f123f68e0806585ebc2578b6)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
6 years agovorbisdec: Error on bark_map_size equal to 0.
Michael Niedermayer [Thu, 10 Jan 2013 23:54:12 +0000 (00:54 +0100)]
vorbisdec: Error on bark_map_size equal to 0.

The value is used to calculate output LSP curve and a division by zero
and out of array accesses would occur.

CVE-2013-0894

CC: libav-stable@libav.org
Reported-by: Dale Curtis <dalecurtis@chromium.org>
Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 11dcecfcca0eca1a571792c4fa3c21fb2cfddddc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 494ddd377ada76ed555f7a3f49391455daa099c9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoh264: check for luma and chroma bit depth being equal
Reinhard Tartler [Tue, 7 May 2013 05:25:10 +0000 (07:25 +0200)]
h264: check for luma and chroma bit depth being equal

The decoder assumes a single bit depth for all the planes while
the specification allows different bit depths for luma and chroma.

Avoid the possible problems described in CVE-2013-2277

Conflicts:
libavcodec/h264.c

6 years agoUpdate RELEASE file for 0.7.8
Reinhard Tartler [Sun, 17 Feb 2013 08:10:52 +0000 (09:10 +0100)]
Update RELEASE file for 0.7.8

6 years agoxxan: fix invalid memory access in xan_decode_frame_type0()
Reinhard Tartler [Tue, 7 May 2013 05:24:16 +0000 (07:24 +0200)]
xxan: fix invalid memory access in xan_decode_frame_type0()

The loop a few lines below the xan_unpack() call accesses up to
dec_size * 2 bytes into y_buffer, so dec_size must be limited to
buffer_size / 2.

CC:libav-stable@libav.org
(cherry picked from commit 8a49d2bcbe7573bb4b765728b2578fac0d19763f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 62a657de168cf501acb23d48cc1aa00793dc83f3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/xxan.c

6 years agowmadec: require block_align to be set.
Anton Khirnov [Wed, 6 Mar 2013 08:58:00 +0000 (09:58 +0100)]
wmadec: require block_align to be set.

Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit ea1136baafb1fe271cb56c3f4d7bff0267e3c70f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit c1f479e8df24284237c80ad959619fc85e29a26d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>