ffmpeg.git
4 years agoUpdate for FFmpeg 1.2.12 n1.2.12
Michael Niedermayer [Thu, 12 Feb 2015 13:28:03 +0000 (14:28 +0100)]
Update for FFmpeg 1.2.12

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mjpegdec: Skip blocks which are outside the visible area
Michael Niedermayer [Wed, 11 Feb 2015 02:33:53 +0000 (03:33 +0100)]
avcodec/mjpegdec: Skip blocks which are outside the visible area

Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08509c8f86626815a3e9e68d600d1aacbb8df4bf)

Conflicts:

libavcodec/mjpegdec.c
(cherry picked from commit b881a97b9977b79dfe3ce02d61542c630fe78c14)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264_slice: ignore SAR changes in slices after the first
Michael Niedermayer [Sat, 7 Feb 2015 02:34:48 +0000 (03:34 +0100)]
avcodec/h264_slice: ignore SAR changes in slices after the first

Fixes race condition and null pointer dereference
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 38d5241b7f36c1571a88517a0650caade16dd5f4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:

libavcodec/h264_slice.c

4 years agoavcodec/h264_slice: Check picture structure before setting the related fields
Michael Niedermayer [Sat, 7 Feb 2015 01:22:44 +0000 (02:22 +0100)]
avcodec/h264_slice: Check picture structure before setting the related fields

This might fix a hypothetical race condition

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f111831ed61103f9fa8fdda41473a23da016bdaa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:

libavcodec/h264_slice.c

Conflicts:

libavcodec/h264.c

4 years agoavcodec/h264_slice: Do not change frame_num after the first slice
Michael Niedermayer [Sat, 7 Feb 2015 01:06:20 +0000 (02:06 +0100)]
avcodec/h264_slice: Do not change frame_num after the first slice

Fixes potential race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f906982c9411f3062e3ce68013309b37c213c4dd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:

libavcodec/h264_slice.c

4 years agoavutil/opt: Fix type used to access AV_OPT_TYPE_SAMPLE_FMT
Michael Niedermayer [Fri, 6 Feb 2015 21:16:08 +0000 (22:16 +0100)]
avutil/opt: Fix type used to access AV_OPT_TYPE_SAMPLE_FMT

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1750b45cdf7498d0a05bea29cafcb26aa576d595)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavutil/opt: Fix types used to access AV_OPT_TYPE_PIXEL_FMT
Michael Niedermayer [Fri, 6 Feb 2015 21:14:15 +0000 (22:14 +0100)]
avutil/opt: Fix types used to access AV_OPT_TYPE_PIXEL_FMT

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a0640e63463e6428b80422c89e1bfc96147ecfc6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: Be more strict on rejecting pps/sps changes
Michael Niedermayer [Fri, 6 Feb 2015 14:09:54 +0000 (15:09 +0100)]
avcodec/h264: Be more strict on rejecting pps/sps changes

Fixes race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6fafc62b0bd0e206deb77a7aabbf3a370ad80789)

Conflicts:

libavcodec/h264.c

4 years agoavcodec/h264_ps: More completely check the bit depths
Michael Niedermayer [Fri, 6 Feb 2015 03:11:56 +0000 (04:11 +0100)]
avcodec/h264_ps: More completely check the bit depths

Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69aa79365c1e8e1cb597d33e77bf1062c2ef47d4)

Conflicts:

libavcodec/h264_ps.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/thp: Check av_get_packet() for failure not only for partial output
Michael Niedermayer [Thu, 5 Feb 2015 02:45:21 +0000 (03:45 +0100)]
avformat/thp: Check av_get_packet() for failure not only for partial output

Fixes null pointer dereference
Fixes: signal_sigsegv_db2c1f_3108_cov_163322880_pikmin2_opening1_partial.thp

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f2579dbb4b31e6ae731e7f5555680528ef3020ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale/utils: Limit filter shifting so as not to read from prior the array
Michael Niedermayer [Wed, 4 Feb 2015 23:12:08 +0000 (00:12 +0100)]
swscale/utils: Limit filter shifting so as not to read from prior the array

Fixes out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 692b22626ec9a9585f667c124a186b1a9796e432)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mjpegdec: Check number of components for JPEG-LS
Michael Niedermayer [Wed, 4 Feb 2015 19:48:30 +0000 (20:48 +0100)]
avcodec/mjpegdec: Check number of components for JPEG-LS

Fixes out of array accesses
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fabbfaa095660982cc0bc63242c459561fa37037)

Conflicts:

libavcodec/mjpegdec.c

4 years agoavcodec/mjpegdec: Check escape sequence validity
Michael Niedermayer [Wed, 4 Feb 2015 19:13:18 +0000 (20:13 +0100)]
avcodec/mjpegdec: Check escape sequence validity

Fixes assertion failure
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mpc8: Use uint64_t in *_get_v() to avoid undefined behavior
Michael Niedermayer [Wed, 4 Feb 2015 13:47:41 +0000 (14:47 +0100)]
avformat/mpc8: Use uint64_t in *_get_v() to avoid undefined behavior

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 05e161952954acf247e0fd1fdef00559675c4d4d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mpc8: fix broken pointer math
wm4 [Tue, 3 Feb 2015 18:04:11 +0000 (19:04 +0100)]
avformat/mpc8: fix broken pointer math

This could overflow and crash at least on 32 bit systems.

Reviewed-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b737a2c52857b214be246ff615c6293730033cfa)

Conflicts:

libavformat/mpc8.c

4 years agoavformat/mpc8: fix hang with fuzzed file
wm4 [Tue, 3 Feb 2015 18:04:12 +0000 (19:04 +0100)]
avformat/mpc8: fix hang with fuzzed file

This can lead to an endless loop by seeking back a few bytes after each
attempted chunk read. Assuming negative sizes are always invalid, this
is easy to fix. Other code in this demuxer treats negative sizes as
invalid as well.

Fixes ticket #4262.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 56cc024220886927350cfc26ee695062ca7ecaf4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/tta: fix crash with corrupted files
wm4 [Tue, 3 Feb 2015 13:41:10 +0000 (14:41 +0100)]
avformat/tta: fix crash with corrupted files

av_add_index_entry() can fail, for example because the parameters are
invalid, or because memory allocation fails. Check this; it can actually
happen with corrupted files.

The second hunk is just for robustness. Just in case functions like
ff_reduce_index() remove entries. (Not sure if this can actually
happen.)

Fixes ticket #4294.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6a0cd529a35190d9374b0b26504e71857cd67b83)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mpegvideo_enc: Fix number suffixes in rc_buffer_size calculation
Michael Niedermayer [Sun, 1 Feb 2015 18:40:13 +0000 (19:40 +0100)]
avcodec/mpegvideo_enc: Fix number suffixes in rc_buffer_size calculation

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4531e2c489d279bfc90d54ca26ed898c5b265a7f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264_cabac: use int instead of long for mbb_xy
Michael Niedermayer [Sun, 1 Feb 2015 18:39:22 +0000 (19:39 +0100)]
avcodec/h264_cabac: use int instead of long for mbb_xy

The mb address fits in int

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 592ba6ec106206f97133c9345313010c76361e12)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/smacker: Fix number suffix
Michael Niedermayer [Sun, 1 Feb 2015 18:36:13 +0000 (19:36 +0100)]
avformat/smacker: Fix number suffix

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 465f3705b1ef832fd6904750d018f81f9044f3ab)

Conflicts:

libavformat/smacker.c

4 years agoavformat/matroskadec: Fix number suffixes
Michael Niedermayer [Sun, 1 Feb 2015 18:34:52 +0000 (19:34 +0100)]
avformat/matroskadec: Fix number suffixes

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc3cdb00d084222a107e61e7168903bf3d3d0b47)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswresample/dither: Cleanup number suffixes
Michael Niedermayer [Sun, 1 Feb 2015 18:27:00 +0000 (19:27 +0100)]
swresample/dither: Cleanup number suffixes

The <<31 case needs LL

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c77cc2c1766666cdb5f14daee0f75e397bf7a194)

Conflicts:

libswresample/dither.c

4 years agoavformat/utils: Fix number suffixes in tb_unreliable()
Michael Niedermayer [Sun, 1 Feb 2015 18:19:25 +0000 (19:19 +0100)]
avformat/utils: Fix number suffixes in tb_unreliable()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b15bba2aec93776bfdc69a1bca42a4795a7d191)

Conflicts:

libavformat/utils.c
(cherry picked from commit e651a2f88c219e74c9851563e74100f7652a6005)

4 years agoavformat/rmdec: Check for overflow in ff_rm_read_mdpr_codecdata()
Michael Niedermayer [Fri, 5 Dec 2014 23:18:29 +0000 (00:18 +0100)]
avformat/rmdec: Check for overflow in ff_rm_read_mdpr_codecdata()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 03abf55f252945c70f4a79eaf4d609cee4d98710)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agortpdec_h263_rfc2190: Clear the stored bits if discarding buffered data
Martin Storsjö [Wed, 17 Dec 2014 11:50:06 +0000 (13:50 +0200)]
rtpdec_h263_rfc2190: Clear the stored bits if discarding buffered data

If we throw away the buffered incomplete frame, make sure to also
throw away the buffered bits of an incomplete byte at the same
time.

(cherry picked from commit df07c07b3de0a5e8890078944de1eb5cb8372ef8)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale: check memory allocations
Vittorio Giovara [Wed, 17 Dec 2014 15:02:07 +0000 (16:02 +0100)]
swscale: check memory allocations

Bug-Id: CID 1257779
(cherry picked from commit 1dd797e3c9f179f957316a0becbec048b42df8aa)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoopt: check memory allocation
Vittorio Giovara [Thu, 18 Dec 2014 19:26:57 +0000 (20:26 +0100)]
opt: check memory allocation

Bug-Id: CID 1257771
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/flac_parser: fix handling EOF if no headers are found
Michael Niedermayer [Sat, 17 Jan 2015 00:56:03 +0000 (01:56 +0100)]
avcodec/flac_parser: fix handling EOF if no headers are found

Fixes assertion failure
Fixes Ticket4269

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c4d85fc23c100f7a27d9bad710eb153214868e27)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/movenc: workaround bug in "PathScale EKOPath(tm) Compiler Suite Version...
Michael Niedermayer [Sat, 10 Jan 2015 02:43:54 +0000 (03:43 +0100)]
avformat/movenc: workaround bug in "PathScale EKOPath(tm) Compiler Suite Version 4.0.12.1"

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7824dc5150c0ea44ffa7cd4d57803f9a9697e7d7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoffmpeg: Clear error message array at init.
Michael Niedermayer [Thu, 8 Jan 2015 22:02:30 +0000 (23:02 +0100)]
ffmpeg: Clear error message array at init.

This avoids printing uninitialized bytes if no error message is set

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6d1a2efb8ac399a003ea7d3b6f8c641d192567ee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dvdsubdec: error on bitmaps with size 0
wm4 [Wed, 7 Jan 2015 22:57:50 +0000 (23:57 +0100)]
avcodec/dvdsubdec: error on bitmaps with size 0

Attemtping to decode them could lead to invalid writes with some fuzzed
samples.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcaa9099b3648b47060e1724a97dc98b63c83702)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agocmdutils: update copyright year to 2015.
Johan Andersson [Sat, 3 Jan 2015 16:31:36 +0000 (17:31 +0100)]
cmdutils: update copyright year to 2015.

(cherry picked from commit 3e160652219ff4da433f5672ae1e5f4956abb815)

Conflicts:

cmdutils.c

4 years agoavformat/mov: Fix mixed declaration and statement warning
Michael Niedermayer [Tue, 6 Jan 2015 18:51:38 +0000 (19:51 +0100)]
avformat/mov: Fix mixed declaration and statement warning

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit db27f50e0658e91758e8a17fdcf390e6bc93c1d2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/matroskadec: Use av_freep() to avoid leaving stale pointers in memory
Michael Niedermayer [Tue, 6 Jan 2015 11:48:38 +0000 (12:48 +0100)]
avformat/matroskadec: Use av_freep() to avoid leaving stale pointers in memory

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e70e4aca50696040cc9256ec96e5c31d9641432)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agomov: Fix negative size calculation in mov_read_default().
Dale Curtis [Tue, 6 Jan 2015 00:34:17 +0000 (16:34 -0800)]
mov: Fix negative size calculation in mov_read_default().

The previous code assumed if an atom was marked with a 64-bit
size extension, it actually had that data available. The new
code verfies there's enough data in the atom for this to be
done.

Failure to verify causes total_size > atom.size which will
result in negative size calculations later on.

Found-by: Paul Mehta <paul@paulmehta.com>
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ebd76a9c57558e284e94da367dd23b435e6a6d0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mov: fix integer overflow in mov_read_udta_string()
Michael Niedermayer [Tue, 6 Jan 2015 03:29:10 +0000 (04:29 +0100)]
avformat/mov: fix integer overflow in mov_read_udta_string()

Found-by: Paul Mehta <paul@paulmehta.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3859868c75313e318ebc5d0d33baada62d45dd75)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agomov: Avoid overflow with mov_metadata_raw()
Dale Curtis [Tue, 6 Jan 2015 00:19:09 +0000 (16:19 -0800)]
mov: Avoid overflow with mov_metadata_raw()

The code previously added 1 to len without checking its size,
resulting in an overflow which can corrupt value[-1] -- which
may be used to store unaligned ptr information for certain
allocators.

Found-by: Paul Mehta <paul@paulmehta.com>
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dvdsubdec: fix out of bounds accesses
wm4 [Mon, 5 Jan 2015 03:45:26 +0000 (04:45 +0100)]
avcodec/dvdsubdec: fix out of bounds accesses

The code blindly trusted buffer offsets read from the file in the RLE
decoder. Explicitly check the offset. Also error out on other RLE
decoding errors.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c9151de7c42553bb145be608df8513c1287f1f24)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/flvdec: Increase string array size
Michael Niedermayer [Thu, 1 Jan 2015 17:15:16 +0000 (18:15 +0100)]
avformat/flvdec: Increase string array size

Fixes parsing httphostheader of Scarlatti\,\ Pieter-Jan\ Belder\ -\ Sonata\ K113\ in\ A\ major\ -\ Alle.flv

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit eb767a276bfdb9a0493bdb0b38203638230b7ccb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/cdxl: Fix integer overflow of image_size
Michael Niedermayer [Wed, 31 Dec 2014 20:41:46 +0000 (21:41 +0100)]
avformat/cdxl: Fix integer overflow of image_size

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3eb5cbe0c50d0a0bbe10bcabbd6b16d73d93c128)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agolavf/segment: remove duplicated and inconsistent cleanup code in seg_write_packet()
Stefano Sabatini [Tue, 21 Jan 2014 18:58:41 +0000 (19:58 +0100)]
lavf/segment: remove duplicated and inconsistent cleanup code in seg_write_packet()

In particular, avoid to leave around the seg->avf pointer to freed
structure, and fix crash with:
ffmpeg -f lavfi -i testsrc -c:v h264 -map 0 -f segment foo-%d.ts
(cherry picked from commit 169065fbfb3da1ab776379c333aebc54bb1f1bc4)

Found-by: Qinghao Tang
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mov: Fix memleaks for duplicate STCO/CO64/STSC atoms
Michael Niedermayer [Wed, 26 Nov 2014 17:16:15 +0000 (18:16 +0100)]
avformat/mov: Fix memleaks for duplicate STCO/CO64/STSC atoms

Also see [FFmpeg-devel] [PATCH] avformat/mov: strengthen some table allocations
which contains more fixes but is unfinished

Fixes: signal_sigabrt_7ffff6ac7bb9_3484_cov_1830000177_starfox2.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1b5d11240692025f036e945bc37968735679320a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/indeo3: ensure offsets are non negative n1.2.11
Michael Niedermayer [Thu, 18 Dec 2014 17:57:27 +0000 (18:57 +0100)]
avcodec/indeo3: ensure offsets are non negative

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 368642361f3a589d7b0c23ea327d988edb434e3f)

Conflicts:

libavcodec/indeo3.c

4 years agoavcodec/h264: Check *log2_weight_denom
Michael Niedermayer [Thu, 18 Dec 2014 02:16:39 +0000 (03:16 +0100)]
avcodec/h264: Check *log2_weight_denom

Fixes undefined behavior
Fixes: signal_sigsegv_14768d2_2248_cov_3629497219_h264_h264___pi_20070614T182942.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61296d41e2de3b41304339e4631dd44c2e15f805)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: Clear delayed_pic on deallocation
Michael Niedermayer [Wed, 17 Dec 2014 20:27:37 +0000 (21:27 +0100)]
avcodec/h264: Clear delayed_pic on deallocation

Fixes use of freed memory

Fixes: case5_av_frame_copy_props.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e8714f6f93d1a32f4e4655209960afcf4c185214)

Conflicts:

libavcodec/h264.c

4 years agoavcodec/indeo3: use signed variables to avoid underflow
Michael Niedermayer [Wed, 17 Dec 2014 02:14:21 +0000 (03:14 +0100)]
avcodec/indeo3: use signed variables to avoid underflow

Fixes out of array read
Fixes: signal_sigsegv_1b0a4da_1865_cov_2167818389_computer_anger.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3305acdc92fa37869f160a11a87741c8a0de0454)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: make the first field of H264Context an AVClass
Michael Niedermayer [Wed, 17 Dec 2014 00:31:48 +0000 (01:31 +0100)]
avcodec/h264: make the first field of H264Context an AVClass

Fixes use of freed memory
Fixes: asan_heap-uaf_3660f67_757_cov_1257014655_Hi422FR1_SONY_A.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3b5b139ad853b6f69c6a0b036815a60e7b3f261)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale: increase yuv2rgb table headroom
Michael Niedermayer [Tue, 16 Dec 2014 21:21:21 +0000 (22:21 +0100)]
swscale: increase yuv2rgb table headroom

Fixes out of array access
Fixes: case2_bad_read_yuv2rgbx32.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mov: check atom nesting depth
Michael Niedermayer [Tue, 16 Dec 2014 20:14:40 +0000 (21:14 +0100)]
avformat/mov: check atom nesting depth

Fixes call stack overflow
Fixes: case1_call_stack_overflow.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit caa7a3914f499f74b3ee346f26d598ebdc0ec210)

Conflicts:

libavformat/isom.h

4 years agoavcodec/utvideodec: Fix handling of slice_height=0
Michael Niedermayer [Tue, 16 Dec 2014 19:45:31 +0000 (20:45 +0100)]
avcodec/utvideodec: Fix handling of slice_height=0

Fixes out of array accesses
Fixes: asan_heap-oob_25bcd7e_3783_cov_3553517262_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3881606240953b9275a247a1c98a567f3c44890f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/vmdvideo: Check len before using it in method 3
Michael Niedermayer [Tue, 16 Dec 2014 15:24:55 +0000 (16:24 +0100)]
avcodec/vmdvideo: Check len before using it in method 3

Fixes out of array access
Fixes: asan_heap-oob_4d23ba_91_cov_3853393937_128.vmd

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3030fb7e0d41836f8add6399e9a7c7b740b48bfd)

Conflicts:

libavcodec/vmdvideo.c

4 years agoUpdate for 1.2.11
Michael Niedermayer [Mon, 15 Dec 2014 23:10:02 +0000 (00:10 +0100)]
Update for 1.2.11

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/aviobuf: Check that avio_seek() target is non negative
Michael Niedermayer [Sun, 14 Dec 2014 16:26:11 +0000 (17:26 +0100)]
avformat/aviobuf: Check that avio_seek() target is non negative

Fixes out of array access

Suggested-by: Andrew Scherkus <scherkus@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ed86dbd05d61363dc1c0d33f3267e2177c985fdd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswresample/soxr_resample: fix error handling
Rob Sykes [Sat, 13 Dec 2014 20:12:56 +0000 (21:12 +0100)]
swresample/soxr_resample: fix error handling

Fixes CID1257659

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b6f2253741f3023928e61ae5105ccd4b1c515fb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/matroskadec: fix handling of recursive SeekHead elements
wm4 [Sat, 6 Dec 2014 15:53:30 +0000 (16:53 +0100)]
avformat/matroskadec: fix handling of recursive SeekHead elements

When matroska_execute_seekhead() is called, it goes through the list of
seekhead entries and attempts to read elements not read yet. When doing
this, the parser can find further SeekHead elements, and will extend the
matroska->seekhead list. This can lead to a (practically) infinite loop
with certain broken files. (Maybe it can happen even with valid files.
The demuxer doesn't seem to check correctly whether an element has
already been read.)

Fix this by ignoring elements that were added to the seekhead field
during executing seekhead entries.

This does not fix the possible situation when multiple SeekHead elements
after the file header (i.e. occur after the "before_pos" file position)
point to the same elements. These elements will probably be parsed
multiple times, likely leading to bugs.

Fixes ticket #4162.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6551acab6877addae815decd02aeca33ba4990c8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale/x86/rgb2rgb_template: fix crash with tiny size and nv12 output
Michael Niedermayer [Wed, 3 Dec 2014 19:21:56 +0000 (20:21 +0100)]
swscale/x86/rgb2rgb_template: fix crash with tiny size and nv12 output

Fixes Ticket4151

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8524558858b7e14bc50afa10233e0194f591ab9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/rmdec: Check codec_data_size
Michael Niedermayer [Wed, 3 Dec 2014 19:01:18 +0000 (20:01 +0100)]
avformat/rmdec: Check codec_data_size

Fixes infinite loop
Fixes Ticket4154

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a6f730730b82645a9d31aad0968487cb77d6946c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale/x86/rgb2rgb_template: handle the first 2 lines with C in rgb24toyv12_*()
Michael Niedermayer [Wed, 19 Nov 2014 23:43:45 +0000 (00:43 +0100)]
swscale/x86/rgb2rgb_template: handle the first 2 lines with C in rgb24toyv12_*()

This avoids out of array accesses
Should fix Ticket3451

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4388e78a0f022c8572996f9ab568a39b5f716f9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/motion_est: use 2x8x8 for interlaced qpel
Michael Niedermayer [Mon, 1 Dec 2014 12:23:24 +0000 (13:23 +0100)]
avcodec/motion_est: use 2x8x8 for interlaced qpel

Fixes out of array read
Fixes Ticket4121

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b50e003e1cb6a215df44ffa3354603bf600b4aa3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/pngdec: Check IHDR/IDAT order
Michael Niedermayer [Wed, 26 Nov 2014 14:45:47 +0000 (15:45 +0100)]
avcodec/pngdec: Check IHDR/IDAT order

Fixes out of array access
Fixes: asan_heap-oob_20a6c26_2690_cov_3434532168_mail.png
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 79ceaf827be0b070675d4cd0a55c3386542defd8)

Conflicts:

libavcodec/pngdec.c

4 years agoavcodec/flacdec: Call ff_flacdsp_init() unconditionally
Michael Niedermayer [Wed, 26 Nov 2014 02:29:03 +0000 (03:29 +0100)]
avcodec/flacdec: Call ff_flacdsp_init() unconditionally

Fixes out of array access
Fixes: signal_sigsegv_324b135_3398_cov_246853371_short.flac
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5c01ccdf5a9a330d4c51a9b9ea721fd8f1fb70b)

Conflicts:

libavcodec/flacdec.c

4 years agoavcodec/utils: Check that the data is complete in avpriv_bprint_to_extradata()
Michael Niedermayer [Tue, 25 Nov 2014 13:45:30 +0000 (14:45 +0100)]
avcodec/utils: Check that the data is complete in avpriv_bprint_to_extradata()

Fixes out of array read
Fixes: asan_heap-oob_4d2250_814_cov_2745172097_JACOsub_capability_tester.jss
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3d5d95db3f5d8e2093e9e19d0c46e86f54ed2a5d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mjpegdec: Fix context fields becoming inconsistent
Michael Niedermayer [Tue, 25 Nov 2014 12:53:06 +0000 (13:53 +0100)]
avcodec/mjpegdec: Fix context fields becoming inconsistent

Fixes out of array access
Fixes: asan_heap-oob_1ca4f85_2760_cov_144449187_miss_congeniality_pegasus_ljpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0eecf40935b22644e6cd74c586057237ecfd6844)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/hlsenc: Free context after hls_append_segment
Michael Niedermayer [Sun, 16 Nov 2014 03:02:56 +0000 (04:02 +0100)]
avformat/hlsenc: Free context after hls_append_segment

Fixes reading uninitialized memory

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 530eb6acf8ee867bf00728bf7efaf505da107e17)

Conflicts:

libavformat/hlsenc.c
(cherry picked from commit 0ac22f043bee2f1c4daf5e1044b014326325d929)

Conflicts:

libavformat/hlsenc.c
(cherry picked from commit 134d3e1c0331462ea94c78a5e13a63b20d283653)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/wmaprodec: Fix integer overflow in sfb_offsets initialization
Michael Niedermayer [Mon, 10 Nov 2014 22:07:50 +0000 (23:07 +0100)]
avcodec/wmaprodec: Fix integer overflow in sfb_offsets initialization

Fixes out of array read
Fixes: asan_heap-oob_2aec5b0_1828_classical_22_16_2_16000_v3c_0_exclusive_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5dcb99033df16eccc4dbbc4a099ad64457f9f090)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/utvideodec: fix assumtation that slice_height >= 1
Michael Niedermayer [Mon, 10 Nov 2014 18:44:20 +0000 (19:44 +0100)]
avcodec/utvideodec: fix assumtation that slice_height >= 1

Fixes out of array read
Fixes: asan_heap-oob_2573085_3783_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7656c4c6e66f8a787d384f027ad824cc1677fda1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/options_table fix min of audio channels and sample rate
Michael Niedermayer [Mon, 3 Nov 2014 12:20:24 +0000 (13:20 +0100)]
avcodec/options_table fix min of audio channels and sample rate

Found-by: Lukasz Marek <lukasz.m.luki2@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 206c98f303e833c9e94427c9e3f9867f85265f78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoupdate for 1.2.10 n1.2.10
Michael Niedermayer [Sun, 2 Nov 2014 10:34:41 +0000 (11:34 +0100)]
update for 1.2.10

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/svq1dec: zero terminate embedded message before printing
Michael Niedermayer [Thu, 30 Oct 2014 17:16:25 +0000 (18:16 +0100)]
avcodec/svq1dec: zero terminate embedded message before printing

Fixes out of array access
Fixes: asan_stack-oob_49b1e5_10_009.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91ba2efa949470e9157b652535d207a101f91e0)

Conflicts:

libavcodec/svq1dec.c

4 years agoavcodec/utils: Align dimensions by at least their chroma sub-sampling factors.
Michael Niedermayer [Wed, 29 Oct 2014 13:15:29 +0000 (14:15 +0100)]
avcodec/utils: Align dimensions by at least their chroma sub-sampling factors.

Fixes: out of array accesses
Fixes: asan_heap-oob_112c6b3_13_012.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df74811cd53e45fcbbd3b77a1c42416816687c5c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dnxhddec: treat pix_fmt like width/height
Michael Niedermayer [Tue, 28 Oct 2014 23:57:07 +0000 (00:57 +0100)]
avcodec/dnxhddec: treat pix_fmt like width/height

Fixes out of array accesses
Fixes: asan_heap-oob_22c9a39_16_015.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3c0e0bf6f53df0977f3878d4f5cec99dff8de9e)

Conflicts:

libavcodec/dnxhddec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dxa: check dimensions
Michael Niedermayer [Tue, 28 Oct 2014 14:26:42 +0000 (15:26 +0100)]
avcodec/dxa: check dimensions

Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc22c4e54d5716f28f28db8f99c74cc90)

Conflicts:

libavcodec/dxa.c

4 years agoavcodec/dirac_arith: fix integer overflow
Michael Niedermayer [Tue, 28 Oct 2014 01:14:41 +0000 (02:14 +0100)]
avcodec/dirac_arith: fix integer overflow

Fixes: asan_heap-oob_1078676_9_008.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39680caceebfc6abf09b17032048752c014e57a8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
Michael Niedermayer [Tue, 28 Oct 2014 00:23:40 +0000 (01:23 +0100)]
avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y

Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5145d22b88b9835db81c4d286b931a78e08ab76a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Michael Niedermayer [Tue, 28 Oct 2014 00:23:40 +0000 (01:23 +0100)]
avcodec/diracdec: Use 64bit in calculation of codeblock coordinates

Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e6069636a918c8c04db17e864e3d8151c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agopostproc: fix qp count
Michael Niedermayer [Mon, 13 Oct 2014 14:02:42 +0000 (16:02 +0200)]
postproc: fix qp count

Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0b7e5d0d75e7d8762dd04d35f8c0821736164372)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agopostproc/postprocess: fix quant store for fq mode
Michael Niedermayer [Sun, 12 Oct 2014 18:26:27 +0000 (20:26 +0200)]
postproc/postprocess: fix quant store for fq mode

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 941aaa39e8cd78ba4d16dfcec767290aec9a0136)

Conflicts:

tests/ref/fate/filter-pp3
(cherry picked from commit 705748caf3f6a4a3e74ad3d2fc547a5a0213a521)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264_slice: Clear table pointers to avoid stale pointers
Michael Niedermayer [Sun, 2 Nov 2014 00:55:40 +0000 (01:55 +0100)]
avcodec/h264_slice: Clear table pointers to avoid stale pointers

Might fix Ticket3889

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 547fce95858ef83f8c25ae347e3ae3b8ba437fd9)

Conflicts:

libavcodec/h264_slice.c

Conflicts:

libavcodec/h264.c

4 years agoFix compilation after aa9d7058.
Carl Eugen Hoyos [Sat, 1 Nov 2014 23:42:48 +0000 (00:42 +0100)]
Fix compilation after aa9d7058.

4 years agoavcodec/h264_parser: rewrite the parse_nal_units() loop logic based on h264.c
Michael Niedermayer [Mon, 27 Oct 2014 03:30:11 +0000 (04:30 +0100)]
avcodec/h264_parser: rewrite the parse_nal_units() loop logic based on h264.c

Fixes Ticket4011

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69a9a90d2ef795162074be24e3ad2182a8676af2)

Conflicts:

libavcodec/h264_parser.c

4 years agoavcodec/h264: simplify find_start_code()
Michael Niedermayer [Mon, 27 Oct 2014 03:24:20 +0000 (04:24 +0100)]
avcodec/h264: simplify find_start_code()

this also uses avpriv_find_start_code(), though no speed change is expected as
the area searched is generally small

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b678da5e386c138316954e867d595f946666051)

Conflicts:

libavcodec/h264.h

4 years agoMove get_avc_nalsize() and find_start_code() to h264.h
Michael Niedermayer [Mon, 27 Oct 2014 03:22:42 +0000 (04:22 +0100)]
Move get_avc_nalsize() and find_start_code() to h264.h

This allows sharing them with the h264 parser

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4898440f6bd19152373969159fff057b532c6374)

Conflicts:

libavcodec/h264.c
libavcodec/h264.h

4 years agoavformat/matroskadec: do not trust the default duration to be the real 1/timebase...
Michael Niedermayer [Wed, 8 Oct 2014 18:23:40 +0000 (20:23 +0200)]
avformat/matroskadec: do not trust the default duration to be the real 1/timebase if its less than 5fps

Fixes Ticket3980

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit be695ee389724d713e1b8a61ef899fe1795193ce)

4 years agoavformat/mpeg: increase score for short mpeg-ps by 1
Michael Niedermayer [Sun, 7 Sep 2014 17:18:21 +0000 (19:18 +0200)]
avformat/mpeg: increase score for short mpeg-ps by 1

Fixes Ticket 3855

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5109ce2017c15202275b33bce58ec9d88044eeb5)

Conflicts:
libavformat/mpeg.c

4 years agoUpdate for 1.2.9 n1.2.9
Michael Niedermayer [Tue, 7 Oct 2014 17:40:13 +0000 (19:40 +0200)]
Update for 1.2.9

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswresample/swresample: fix sample drop loop end condition
Michael Niedermayer [Sun, 5 Oct 2014 23:08:20 +0000 (01:08 +0200)]
swresample/swresample: fix sample drop loop end condition

Fixes Ticket3985

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f9fefa499f0af48f47ea73c8ce0b25df0976c315)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: Check mode before considering mixed mode intra prediction
Michael Niedermayer [Sat, 4 Oct 2014 12:51:46 +0000 (14:51 +0200)]
avcodec/h264: Check mode before considering mixed mode intra prediction

Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de3043f012ad0f1ef11027d9488067e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mpegts: Check desc_len / get8() return code
Michael Niedermayer [Sat, 4 Oct 2014 02:29:40 +0000 (04:29 +0200)]
avformat/mpegts: Check desc_len / get8() return code

Fixes out of array read
Fixes: signal_sigsegv_844d59_10_signal_sigsegv_a17bb7_366_mpegts_mpeg2video_mp2_dvbsub_topfield.rec

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c3d7f00ee3e09801f56f25db8b5961f25e842bd2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/smc: fix off by 1 error
Michael Niedermayer [Fri, 3 Oct 2014 20:50:45 +0000 (22:50 +0200)]
avcodec/smc: fix off by 1 error

Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9d62335e89d118a5b4e202edf39d905)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/qpeg: fix off by 1 error in MV bounds check
Michael Niedermayer [Fri, 3 Oct 2014 19:08:52 +0000 (21:08 +0200)]
avcodec/qpeg: fix off by 1 error in MV bounds check

Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1ca26d0fff3a3baf61a40207032143f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/gifdec: factorize interleave end handling out
Michael Niedermayer [Fri, 3 Oct 2014 18:15:52 +0000 (20:15 +0200)]
avcodec/gifdec: factorize interleave end handling out

also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864be8fb9653643519dea1c6492f1dde57)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/cinepak: fix integer underflow
Michael Niedermayer [Fri, 3 Oct 2014 17:33:01 +0000 (19:33 +0200)]
avcodec/cinepak: fix integer underflow

Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c506957f40aafd794e06de1a7e341e9d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/pngdec: Calculate MPNG bytewidth more defensively
Michael Niedermayer [Fri, 3 Oct 2014 15:54:21 +0000 (17:54 +0200)]
avcodec/pngdec: Calculate MPNG bytewidth more defensively

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934a29df05c7af65aef2a480b15f572c4)

Conflicts:

libavcodec/pngdec.c

4 years agoavcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Michael Niedermayer [Fri, 3 Oct 2014 15:35:58 +0000 (17:35 +0200)]
avcodec/pngdec: Check bits per pixel before setting monoblack pixel format

Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020c2dbf0201fe7df3dad9e7e0b2e1bb6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/tiff: more completely check bpp/bppcount
Michael Niedermayer [Fri, 3 Oct 2014 14:08:32 +0000 (16:08 +0200)]
avcodec/tiff: more completely check bpp/bppcount

Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa419aa5d320540d5a1b3f8fd9b82ab7e5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Michael Niedermayer [Fri, 3 Oct 2014 12:45:04 +0000 (14:45 +0200)]
avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks

Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f21b761ca15dbb470cd619a1ebf86c3e)

Conflicts:

libavcodec/mmvideo.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/utils: Add case for jv to avcodec_align_dimensions2()
Michael Niedermayer [Fri, 3 Oct 2014 02:30:58 +0000 (04:30 +0200)]
avcodec/utils: Add case for jv to avcodec_align_dimensions2()

Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376a736d243aef4a1d121abebce912e6b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Michael Niedermayer [Thu, 2 Oct 2014 23:50:27 +0000 (01:50 +0200)]
avcodec/mjpegdec: check bits per pixel for changes similar to dimensions

Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6df8243f06c87962b873bd563e58cd39)

Conflicts:

libavcodec/mjpegdec.c
(cherry picked from commit 94371a404c663c3dae3d542fa43951567ab67f82)

Conflicts:

libavcodec/mjpegdec.c

4 years agoavcodec/jpeglsdec: Check run value more completely in ls_decode_line()
Michael Niedermayer [Thu, 2 Oct 2014 21:17:21 +0000 (23:17 +0200)]
avcodec/jpeglsdec: Check run value more completely in ls_decode_line()

previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410a17dc72c30ee7f3145fcacc425f4f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoconfigure: add noexecstack to linker options if supported.
Reimar Döffinger [Sun, 21 Sep 2014 08:58:10 +0000 (09:58 +0100)]
configure: add noexecstack to linker options if supported.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit b7082d953fda93f7841ffffe7d15a6c3cd15bdee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>