ffmpeg.git
4 years agoavcodec/dvdsubdec: fix accessing dangling pointers
wm4 [Thu, 8 Jan 2015 16:19:17 +0000 (17:19 +0100)]
avcodec/dvdsubdec: fix accessing dangling pointers

dvdsub_decode() can call append_to_cached_buf() 2 times, the second time
with ctx->buf as argument. If the second append_to_cached_buf() reallocs
ctx->buf, the argument will be a pointer to the previous, freed block.
This can cause invalid reads at least with some fuzzed files - and
possibly with valid files.

Since packets can apparently not be larger than 64K (even if packets are
combined), just use a fixed size buffer. It will be allocated as part of
the DVDSubContext, and although some memory is "wasted", it's relatively
minimal by modern standards and should be acceptable.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 816577716bc6170bccfea3b9e865618b69a4b426)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dvdsubdec: error on bitmaps with size 0
wm4 [Wed, 7 Jan 2015 22:57:50 +0000 (23:57 +0100)]
avcodec/dvdsubdec: error on bitmaps with size 0

Attemtping to decode them could lead to invalid writes with some fuzzed
samples.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcaa9099b3648b47060e1724a97dc98b63c83702)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agocmdutils: update copyright year to 2015.
Johan Andersson [Sat, 3 Jan 2015 16:31:36 +0000 (17:31 +0100)]
cmdutils: update copyright year to 2015.

(cherry picked from commit 3e160652219ff4da433f5672ae1e5f4956abb815)

Conflicts:

cmdutils.c

4 years agoavformat/mov: Fix mixed declaration and statement warning
Michael Niedermayer [Tue, 6 Jan 2015 18:51:38 +0000 (19:51 +0100)]
avformat/mov: Fix mixed declaration and statement warning

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit db27f50e0658e91758e8a17fdcf390e6bc93c1d2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/utils: Clear pointer in ff_alloc_extradata() to avoid leaving a stale pointe...
Michael Niedermayer [Tue, 6 Jan 2015 11:53:53 +0000 (12:53 +0100)]
avformat/utils: Clear pointer in ff_alloc_extradata() to avoid leaving a stale pointer in memory

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bbfca8e84b0e69abba523d665536c0135fc1c00e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/matroskadec: Use av_freep() to avoid leaving stale pointers in memory
Michael Niedermayer [Tue, 6 Jan 2015 11:48:38 +0000 (12:48 +0100)]
avformat/matroskadec: Use av_freep() to avoid leaving stale pointers in memory

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e70e4aca50696040cc9256ec96e5c31d9641432)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agolavfi: check av_strdup() return value
Paul B Mahol [Tue, 6 Jan 2015 09:42:59 +0000 (09:42 +0000)]
lavfi: check av_strdup() return value

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 145a84717b62e086cdb5f26649ad9f1b51ef38d0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agomov: Fix negative size calculation in mov_read_default().
Dale Curtis [Tue, 6 Jan 2015 00:34:17 +0000 (16:34 -0800)]
mov: Fix negative size calculation in mov_read_default().

The previous code assumed if an atom was marked with a 64-bit
size extension, it actually had that data available. The new
code verfies there's enough data in the atom for this to be
done.

Failure to verify causes total_size > atom.size which will
result in negative size calculations later on.

Found-by: Paul Mehta <paul@paulmehta.com>
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ebd76a9c57558e284e94da367dd23b435e6a6d0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mov: fix integer overflow in mov_read_udta_string()
Michael Niedermayer [Tue, 6 Jan 2015 03:29:10 +0000 (04:29 +0100)]
avformat/mov: fix integer overflow in mov_read_udta_string()

Found-by: Paul Mehta <paul@paulmehta.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3859868c75313e318ebc5d0d33baada62d45dd75)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agomov: Avoid overflow with mov_metadata_raw()
Dale Curtis [Tue, 6 Jan 2015 00:19:09 +0000 (16:19 -0800)]
mov: Avoid overflow with mov_metadata_raw()

The code previously added 1 to len without checking its size,
resulting in an overflow which can corrupt value[-1] -- which
may be used to store unaligned ptr information for certain
allocators.

Found-by: Paul Mehta <paul@paulmehta.com>
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dvdsubdec: fix out of bounds accesses
wm4 [Mon, 5 Jan 2015 03:45:26 +0000 (04:45 +0100)]
avcodec/dvdsubdec: fix out of bounds accesses

The code blindly trusted buffer offsets read from the file in the RLE
decoder. Explicitly check the offset. Also error out on other RLE
decoding errors.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c9151de7c42553bb145be608df8513c1287f1f24)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavfilter/vf_sab: fix filtering tiny images
Michael Niedermayer [Sun, 4 Jan 2015 00:03:26 +0000 (01:03 +0100)]
avfilter/vf_sab: fix filtering tiny images

Fixes out of array reads

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9bff052b51f27f6cce04e8d7d8b405c710d7ad67)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/flvdec: Increase string array size
Michael Niedermayer [Thu, 1 Jan 2015 17:15:16 +0000 (18:15 +0100)]
avformat/flvdec: Increase string array size

Fixes parsing httphostheader of Scarlatti\,\ Pieter-Jan\ Belder\ -\ Sonata\ K113\ in\ A\ major\ -\ Alle.flv

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit eb767a276bfdb9a0493bdb0b38203638230b7ccb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/flvdec: do not inject dts=0 metadata packets which failed to be parsed into...
Michael Niedermayer [Thu, 1 Jan 2015 17:07:24 +0000 (18:07 +0100)]
avformat/flvdec: do not inject dts=0 metadata packets which failed to be parsed into a new data stream

Such data streams (which then contain no other packets except the faulty one)
confuse some user applications, like VLC
Works around vlcticket 12389

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 322f0f5960a743cac47252d90a0f1ea7a025feff)

Conflicts:

libavformat/flvdec.c

4 years agoavformat/cdxl: Fix integer overflow of image_size n2.1.7
Michael Niedermayer [Wed, 31 Dec 2014 20:41:46 +0000 (21:41 +0100)]
avformat/cdxl: Fix integer overflow of image_size

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3eb5cbe0c50d0a0bbe10bcabbd6b16d73d93c128)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoUpdate for 2.1.7
Michael Niedermayer [Tue, 30 Dec 2014 15:16:23 +0000 (16:16 +0100)]
Update for 2.1.7

4 years agolavf/segment: remove duplicated and inconsistent cleanup code in seg_write_packet()
Stefano Sabatini [Tue, 21 Jan 2014 18:58:41 +0000 (19:58 +0100)]
lavf/segment: remove duplicated and inconsistent cleanup code in seg_write_packet()

In particular, avoid to leave around the seg->avf pointer to freed
structure, and fix crash with:
ffmpeg -f lavfi -i testsrc -c:v h264 -map 0 -f segment foo-%d.ts
(cherry picked from commit 169065fbfb3da1ab776379c333aebc54bb1f1bc4)

Found-by: Qinghao Tang
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mov: Fix memleaks for duplicate STCO/CO64/STSC atoms
Michael Niedermayer [Wed, 26 Nov 2014 17:16:15 +0000 (18:16 +0100)]
avformat/mov: Fix memleaks for duplicate STCO/CO64/STSC atoms

Also see [FFmpeg-devel] [PATCH] avformat/mov: strengthen some table allocations
which contains more fixes but is unfinished

Fixes: signal_sigabrt_7ffff6ac7bb9_3484_cov_1830000177_starfox2.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1b5d11240692025f036e945bc37968735679320a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agommvideo: check frame dimensions
Anton Khirnov [Sun, 14 Dec 2014 20:01:59 +0000 (21:01 +0100)]
mmvideo: check frame dimensions

The frame size must be set by the caller and each dimension must be a
multiple of 2.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
See: 8b0e96e1f21b761ca15dbb470cd619a1ebf86c3e
These should be redundant, but are backported for saftey anyway
(cherry picked from commit b0273232d8fffdc8a977ccdad460b8071a0e353c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agojvdec: check frame dimensions
Anton Khirnov [Sun, 14 Dec 2014 20:01:59 +0000 (21:01 +0100)]
jvdec: check frame dimensions

The frame size must be set by the caller and each dimension must be a
multiple of 8.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
See: 105654e376a736d243aef4a1d121abebce912e6b
These should be redundant, but are backported for saftey anyway
(cherry picked from commit e012cb8dea7969c7b3927dbf846ef2742cd4a7ab)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/indeo3: ensure offsets are non negative
Michael Niedermayer [Thu, 18 Dec 2014 17:57:27 +0000 (18:57 +0100)]
avcodec/indeo3: ensure offsets are non negative

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 368642361f3a589d7b0c23ea327d988edb434e3f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: Check *log2_weight_denom
Michael Niedermayer [Thu, 18 Dec 2014 02:16:39 +0000 (03:16 +0100)]
avcodec/h264: Check *log2_weight_denom

Fixes undefined behavior
Fixes: signal_sigsegv_14768d2_2248_cov_3629497219_h264_h264___pi_20070614T182942.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61296d41e2de3b41304339e4631dd44c2e15f805)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/hevc_ps: Check diff_cu_qp_delta_depth
Michael Niedermayer [Thu, 18 Dec 2014 01:09:23 +0000 (02:09 +0100)]
avcodec/hevc_ps: Check diff_cu_qp_delta_depth

Fixes undefined behavior
Fixes: asan_static-oob_17aa046_582_cov_1577759978_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3281fa892599d71b4dc298a426af8296419cd90e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: Clear delayed_pic on deallocation
Michael Niedermayer [Wed, 17 Dec 2014 20:27:37 +0000 (21:27 +0100)]
avcodec/h264: Clear delayed_pic on deallocation

Fixes use of freed memory

Fixes: case5_av_frame_copy_props.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e8714f6f93d1a32f4e4655209960afcf4c185214)

Conflicts:

libavcodec/h264.c

4 years agoavcodec/hevc: clear filter_slice_edges() on allocation
Michael Niedermayer [Wed, 17 Dec 2014 18:42:57 +0000 (19:42 +0100)]
avcodec/hevc: clear filter_slice_edges() on allocation

This avoids use of uninitialized memory
Fixes: asan_static-oob_17aa046_582_cov_212287884_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aa8d12554868c32436750f881954193087219c8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/indeo3: use signed variables to avoid underflow
Michael Niedermayer [Wed, 17 Dec 2014 02:14:21 +0000 (03:14 +0100)]
avcodec/indeo3: use signed variables to avoid underflow

Fixes out of array read
Fixes: signal_sigsegv_1b0a4da_1865_cov_2167818389_computer_anger.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3305acdc92fa37869f160a11a87741c8a0de0454)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: make the first field of H264Context an AVClass
Michael Niedermayer [Wed, 17 Dec 2014 00:31:48 +0000 (01:31 +0100)]
avcodec/h264: make the first field of H264Context an AVClass

Fixes use of freed memory
Fixes: asan_heap-uaf_3660f67_757_cov_1257014655_Hi422FR1_SONY_A.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3b5b139ad853b6f69c6a0b036815a60e7b3f261)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale: increase yuv2rgb table headroom
Michael Niedermayer [Tue, 16 Dec 2014 21:21:21 +0000 (22:21 +0100)]
swscale: increase yuv2rgb table headroom

Fixes out of array access
Fixes: case2_bad_read_yuv2rgbx32.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mov: check atom nesting depth
Michael Niedermayer [Tue, 16 Dec 2014 20:14:40 +0000 (21:14 +0100)]
avformat/mov: check atom nesting depth

Fixes call stack overflow
Fixes: case1_call_stack_overflow.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit caa7a3914f499f74b3ee346f26d598ebdc0ec210)

Conflicts:

libavformat/isom.h

Conflicts:

libavformat/isom.h

4 years agoavcodec/utvideodec: Fix handling of slice_height=0
Michael Niedermayer [Tue, 16 Dec 2014 19:45:31 +0000 (20:45 +0100)]
avcodec/utvideodec: Fix handling of slice_height=0

Fixes out of array accesses
Fixes: asan_heap-oob_25bcd7e_3783_cov_3553517262_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3881606240953b9275a247a1c98a567f3c44890f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/vmdvideo: Check len before using it in method 3
Michael Niedermayer [Tue, 16 Dec 2014 15:24:55 +0000 (16:24 +0100)]
avcodec/vmdvideo: Check len before using it in method 3

Fixes out of array access
Fixes: asan_heap-oob_4d23ba_91_cov_3853393937_128.vmd

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3030fb7e0d41836f8add6399e9a7c7b740b48bfd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoconfigure: create the tests directory like the doc directory
Michael Niedermayer [Mon, 15 Dec 2014 03:32:23 +0000 (04:32 +0100)]
configure: create the tests directory like the doc directory

This fixes an issue where the tests directory is not created for out of tree
builds before its needed

Tested-by: Dave Yeo <daveryeo@telus.net>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e631872f13b6be0583603d45a11e53319754bc8d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agolavu/frame: fix malloc error path in av_frame_copy_props()
wm4 [Mon, 15 Dec 2014 03:32:58 +0000 (04:32 +0100)]
lavu/frame: fix malloc error path in av_frame_copy_props()

The error path frees all side data, but forgets to reset the side data
count. This can blow up later in av_frame_unref() and free_side_data().

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a400edbb6d00c0211de38e4f1b4f593681db91d8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/aviobuf: Check that avio_seek() target is non negative
Michael Niedermayer [Sun, 14 Dec 2014 16:26:11 +0000 (17:26 +0100)]
avformat/aviobuf: Check that avio_seek() target is non negative

Fixes out of array access

Suggested-by: Andrew Scherkus <scherkus@google.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ed86dbd05d61363dc1c0d33f3267e2177c985fdd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswresample/soxr_resample: fix error handling
Rob Sykes [Sat, 13 Dec 2014 20:12:56 +0000 (21:12 +0100)]
swresample/soxr_resample: fix error handling

Fixes CID1257659

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b6f2253741f3023928e61ae5105ccd4b1c515fb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/matroskadec: fix handling of recursive SeekHead elements
wm4 [Sat, 6 Dec 2014 15:53:30 +0000 (16:53 +0100)]
avformat/matroskadec: fix handling of recursive SeekHead elements

When matroska_execute_seekhead() is called, it goes through the list of
seekhead entries and attempts to read elements not read yet. When doing
this, the parser can find further SeekHead elements, and will extend the
matroska->seekhead list. This can lead to a (practically) infinite loop
with certain broken files. (Maybe it can happen even with valid files.
The demuxer doesn't seem to check correctly whether an element has
already been read.)

Fix this by ignoring elements that were added to the seekhead field
during executing seekhead entries.

This does not fix the possible situation when multiple SeekHead elements
after the file header (i.e. occur after the "before_pos" file position)
point to the same elements. These elements will probably be parsed
multiple times, likely leading to bugs.

Fixes ticket #4162.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6551acab6877addae815decd02aeca33ba4990c8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale/x86/rgb2rgb_template: fix crash with tiny size and nv12 output
Michael Niedermayer [Wed, 3 Dec 2014 19:21:56 +0000 (20:21 +0100)]
swscale/x86/rgb2rgb_template: fix crash with tiny size and nv12 output

Fixes Ticket4151

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8524558858b7e14bc50afa10233e0194f591ab9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/rmdec: Check codec_data_size
Michael Niedermayer [Wed, 3 Dec 2014 19:01:18 +0000 (20:01 +0100)]
avformat/rmdec: Check codec_data_size

Fixes infinite loop
Fixes Ticket4154

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a6f730730b82645a9d31aad0968487cb77d6946c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/motion_est: use 2x8x8 for interlaced qpel
Michael Niedermayer [Mon, 1 Dec 2014 12:23:24 +0000 (13:23 +0100)]
avcodec/motion_est: use 2x8x8 for interlaced qpel

Fixes out of array read
Fixes Ticket4121

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b50e003e1cb6a215df44ffa3354603bf600b4aa3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agofix Makefile objects for pulseaudio support
Michael Stypa [Fri, 28 Nov 2014 14:54:50 +0000 (15:54 +0100)]
fix Makefile objects for pulseaudio support

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb58c771ade66afcc623250e1c7ac8191381d991)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/rsd: make tag_buf string larger
Clément Bœsch [Fri, 19 Dec 2014 23:17:43 +0000 (00:17 +0100)]
avformat/rsd: make tag_buf string larger

av_get_codec_tag_string() uses more that 1 char for unprintable characters.

(cherry picked from commit edbbb11488e1fce9b9703535936d2e1731e2e318)

4 years agoUpdate for 2.1.6 n2.1.6
Michael Niedermayer [Fri, 28 Nov 2014 19:04:28 +0000 (20:04 +0100)]
Update for 2.1.6

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/svq3: Dont memcpy AVFrame
Michael Niedermayer [Fri, 3 Oct 2014 22:13:26 +0000 (00:13 +0200)]
avcodec/svq3: Dont memcpy AVFrame

This avoids out of array accesses

Fixes: asan_heap-uaf_21f42e4_9_asan_heap-uaf_21f42e4_278_gl2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 075a165d2715837d125a9cc714fb430ccf6c9d6b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/hevc_ps: Check num_long_term_ref_pics_sps
Michael Niedermayer [Fri, 28 Nov 2014 02:46:56 +0000 (03:46 +0100)]
avcodec/hevc_ps: Check num_long_term_ref_pics_sps

Fixes out of array access
Fixes: signal_sigsegv_35bd0f0_1182_cov_791726764_STRUCT_B_Samsung_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ea38e5a6b75706477898eb1e6582d667dbb9946c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/rawdec: Check the return code of avpicture_get_size()
Michael Niedermayer [Wed, 26 Nov 2014 17:56:39 +0000 (18:56 +0100)]
avcodec/rawdec: Check the return code of avpicture_get_size()

Fixes out of array access
Fixes: asan_heap-oob_22388d0_3435_cov_3297128910_small_roll5_FlashCine1.cine
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1d3a3b9f8907625b361420d48fe05716859620ff)

Conflicts:

libavcodec/rawdec.c

4 years agoavcodec/pngdec: Check IHDR/IDAT order
Michael Niedermayer [Wed, 26 Nov 2014 14:45:47 +0000 (15:45 +0100)]
avcodec/pngdec: Check IHDR/IDAT order

Fixes out of array access
Fixes: asan_heap-oob_20a6c26_2690_cov_3434532168_mail.png
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 79ceaf827be0b070675d4cd0a55c3386542defd8)

Conflicts:

libavcodec/pngdec.c

4 years agoavcodec/flacdec: Call ff_flacdsp_init() unconditionally
Michael Niedermayer [Wed, 26 Nov 2014 02:29:03 +0000 (03:29 +0100)]
avcodec/flacdec: Call ff_flacdsp_init() unconditionally

Fixes out of array access
Fixes: signal_sigsegv_324b135_3398_cov_246853371_short.flac
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5c01ccdf5a9a330d4c51a9b9ea721fd8f1fb70b)

Conflicts:

libavcodec/flacdec.c

4 years agoavcodec/utils: Check that the data is complete in avpriv_bprint_to_extradata()
Michael Niedermayer [Tue, 25 Nov 2014 13:45:30 +0000 (14:45 +0100)]
avcodec/utils: Check that the data is complete in avpriv_bprint_to_extradata()

Fixes out of array read
Fixes: asan_heap-oob_4d2250_814_cov_2745172097_JACOsub_capability_tester.jss
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3d5d95db3f5d8e2093e9e19d0c46e86f54ed2a5d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mjpegdec: Fix context fields becoming inconsistent
Michael Niedermayer [Tue, 25 Nov 2014 12:53:06 +0000 (13:53 +0100)]
avcodec/mjpegdec: Fix context fields becoming inconsistent

Fixes out of array access
Fixes: asan_heap-oob_1ca4f85_2760_cov_144449187_miss_congeniality_pegasus_ljpg.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0eecf40935b22644e6cd74c586057237ecfd6844)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswscale/x86/rgb2rgb_template: handle the first 2 lines with C in rgb24toyv12_*()
Michael Niedermayer [Wed, 19 Nov 2014 23:43:45 +0000 (00:43 +0100)]
swscale/x86/rgb2rgb_template: handle the first 2 lines with C in rgb24toyv12_*()

This avoids out of array accesses
Should fix Ticket3451

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4388e78a0f022c8572996f9ab568a39b5f716f9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/hlsenc: Free context after hls_append_segment
Michael Niedermayer [Sun, 16 Nov 2014 03:02:56 +0000 (04:02 +0100)]
avformat/hlsenc: Free context after hls_append_segment

Fixes reading uninitialized memory

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 530eb6acf8ee867bf00728bf7efaf505da107e17)

Conflicts:

libavformat/hlsenc.c
(cherry picked from commit 0ac22f043bee2f1c4daf5e1044b014326325d929)

Conflicts:

libavformat/hlsenc.c
(cherry picked from commit 134d3e1c0331462ea94c78a5e13a63b20d283653)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/wmaprodec: Fix integer overflow in sfb_offsets initialization
Michael Niedermayer [Mon, 10 Nov 2014 22:07:50 +0000 (23:07 +0100)]
avcodec/wmaprodec: Fix integer overflow in sfb_offsets initialization

Fixes out of array read
Fixes: asan_heap-oob_2aec5b0_1828_classical_22_16_2_16000_v3c_0_exclusive_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5dcb99033df16eccc4dbbc4a099ad64457f9f090)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/utvideodec: fix assumtation that slice_height >= 1
Michael Niedermayer [Mon, 10 Nov 2014 18:44:20 +0000 (19:44 +0100)]
avcodec/utvideodec: fix assumtation that slice_height >= 1

Fixes out of array read
Fixes: asan_heap-oob_2573085_3783_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7656c4c6e66f8a787d384f027ad824cc1677fda1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/options_table fix min of audio channels and sample rate
Michael Niedermayer [Mon, 3 Nov 2014 12:20:24 +0000 (13:20 +0100)]
avcodec/options_table fix min of audio channels and sample rate

Found-by: Lukasz Marek <lukasz.m.luki2@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 206c98f303e833c9e94427c9e3f9867f85265f78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agolavc/utils: Make pix_fmt desc pointer const.
Carl Eugen Hoyos [Wed, 29 Oct 2014 23:27:04 +0000 (00:27 +0100)]
lavc/utils: Make pix_fmt desc pointer const.

Fixes an "initialization discards qualifiers from pointer target type" warning.
(cherry picked from commit f05855414ed4cce97c06ba2a31f4987af47e6d4e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264_slice: Clear table pointers to avoid stale pointers
Michael Niedermayer [Sun, 2 Nov 2014 00:55:40 +0000 (01:55 +0100)]
avcodec/h264_slice: Clear table pointers to avoid stale pointers

Might fix Ticket3889

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 547fce95858ef83f8c25ae347e3ae3b8ba437fd9)

Conflicts:

libavcodec/h264_slice.c

4 years agoavcodec/svq1dec: zero terminate embedded message before printing
Michael Niedermayer [Thu, 30 Oct 2014 17:16:25 +0000 (18:16 +0100)]
avcodec/svq1dec: zero terminate embedded message before printing

Fixes out of array access
Fixes: asan_stack-oob_49b1e5_10_009.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91ba2efa949470e9157b652535d207a101f91e0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/cook: check that the subpacket sizes fit in block_align
Michael Niedermayer [Thu, 30 Oct 2014 15:53:09 +0000 (16:53 +0100)]
avcodec/cook: check that the subpacket sizes fit in block_align

Fixes out of array read
Fixes: asan_heap-oob_fb5c50_19_018.rmvb
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 10e32618acce9c3fc64c061eb7907e8a8d2749ae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/g2meet: check tile dimensions to avoid integer overflow
Michael Niedermayer [Thu, 30 Oct 2014 00:19:17 +0000 (01:19 +0100)]
avcodec/g2meet: check tile dimensions to avoid integer overflow

Fixes out of array access
Fixes: asan_heap-oob_12a55d3_30_029.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 32e666c354e4a3160d8cf1d303cb51990b095c87)

Conflicts:

libavcodec/g2meet.c

4 years agoavcodec/utils: Align dimensions by at least their chroma sub-sampling factors.
Michael Niedermayer [Wed, 29 Oct 2014 13:15:29 +0000 (14:15 +0100)]
avcodec/utils: Align dimensions by at least their chroma sub-sampling factors.

Fixes: out of array accesses
Fixes: asan_heap-oob_112c6b3_13_012.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df74811cd53e45fcbbd3b77a1c42416816687c5c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dnxhddec: treat pix_fmt like width/height
Michael Niedermayer [Tue, 28 Oct 2014 23:57:07 +0000 (00:57 +0100)]
avcodec/dnxhddec: treat pix_fmt like width/height

Fixes out of array accesses
Fixes: asan_heap-oob_22c9a39_16_015.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3c0e0bf6f53df0977f3878d4f5cec99dff8de9e)

Conflicts:

libavcodec/dnxhddec.c

4 years agoavcodec/dxa: check dimensions
Michael Niedermayer [Tue, 28 Oct 2014 14:26:42 +0000 (15:26 +0100)]
avcodec/dxa: check dimensions

Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc22c4e54d5716f28f28db8f99c74cc90)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/dirac_arith: fix integer overflow
Michael Niedermayer [Tue, 28 Oct 2014 01:14:41 +0000 (02:14 +0100)]
avcodec/dirac_arith: fix integer overflow

Fixes: asan_heap-oob_1078676_9_008.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39680caceebfc6abf09b17032048752c014e57a8)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
Michael Niedermayer [Tue, 28 Oct 2014 00:23:40 +0000 (01:23 +0100)]
avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y

Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5145d22b88b9835db81c4d286b931a78e08ab76a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Michael Niedermayer [Tue, 28 Oct 2014 00:23:40 +0000 (01:23 +0100)]
avcodec/diracdec: Use 64bit in calculation of codeblock coordinates

Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e6069636a918c8c04db17e864e3d8151c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agopostproc: fix qp count
Michael Niedermayer [Mon, 13 Oct 2014 14:02:42 +0000 (16:02 +0200)]
postproc: fix qp count

Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0b7e5d0d75e7d8762dd04d35f8c0821736164372)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agopostproc/postprocess: fix quant store for fq mode
Michael Niedermayer [Sun, 12 Oct 2014 18:26:27 +0000 (20:26 +0200)]
postproc/postprocess: fix quant store for fq mode

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 941aaa39e8cd78ba4d16dfcec767290aec9a0136)

Conflicts:

tests/ref/fate/filter-pp3
(cherry picked from commit 705748caf3f6a4a3e74ad3d2fc547a5a0213a521)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoutvideoenc: properly set slice height/last line
Christophe Gisquet [Thu, 9 Oct 2014 21:27:38 +0000 (23:27 +0200)]
utvideoenc: properly set slice height/last line

Mimic decoder and obey sampling.

Does not affect fate tests for utvideo.
Fixes ticket #3949.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb530dda7d76790b08ee3b7f67e251f3ce48c359)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoswresample/swresample: fix sample drop loop end condition
Michael Niedermayer [Sun, 5 Oct 2014 23:08:20 +0000 (01:08 +0200)]
swresample/swresample: fix sample drop loop end condition

Fixes Ticket3985

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f9fefa499f0af48f47ea73c8ce0b25df0976c315)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: Check mode before considering mixed mode intra prediction
Michael Niedermayer [Sat, 4 Oct 2014 12:51:46 +0000 (14:51 +0200)]
avcodec/h264: Check mode before considering mixed mode intra prediction

Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de3043f012ad0f1ef11027d9488067e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/mpegts: Check desc_len / get8() return code
Michael Niedermayer [Sat, 4 Oct 2014 02:29:40 +0000 (04:29 +0200)]
avformat/mpegts: Check desc_len / get8() return code

Fixes out of array read
Fixes: signal_sigsegv_844d59_10_signal_sigsegv_a17bb7_366_mpegts_mpeg2video_mp2_dvbsub_topfield.rec

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c3d7f00ee3e09801f56f25db8b5961f25e842bd2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/vorbisdec: Fix off by 1 error in ptns_to_read
Michael Niedermayer [Sat, 4 Oct 2014 01:12:34 +0000 (03:12 +0200)]
avcodec/vorbisdec: Fix off by 1 error in ptns_to_read

Fixes read of uninitialized memory
Fixes: asan_heap-uaf_18dac2b_9_asan_heap-uaf_22eb375_208_beta3_test_small.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c50704ebf1777bee76772c4835d9760b3721057)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/smc: fix off by 1 error
Michael Niedermayer [Fri, 3 Oct 2014 20:50:45 +0000 (22:50 +0200)]
avcodec/smc: fix off by 1 error

Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9d62335e89d118a5b4e202edf39d905)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/qpeg: fix off by 1 error in MV bounds check
Michael Niedermayer [Fri, 3 Oct 2014 19:08:52 +0000 (21:08 +0200)]
avcodec/qpeg: fix off by 1 error in MV bounds check

Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1ca26d0fff3a3baf61a40207032143f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/gifdec: factorize interleave end handling out
Michael Niedermayer [Fri, 3 Oct 2014 18:15:52 +0000 (20:15 +0200)]
avcodec/gifdec: factorize interleave end handling out

also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864be8fb9653643519dea1c6492f1dde57)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/cinepak: fix integer underflow
Michael Niedermayer [Fri, 3 Oct 2014 17:33:01 +0000 (19:33 +0200)]
avcodec/cinepak: fix integer underflow

Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c506957f40aafd794e06de1a7e341e9d5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/pngdec: Calculate MPNG bytewidth more defensively
Michael Niedermayer [Fri, 3 Oct 2014 15:54:21 +0000 (17:54 +0200)]
avcodec/pngdec: Calculate MPNG bytewidth more defensively

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934a29df05c7af65aef2a480b15f572c4)

Conflicts:

libavcodec/pngdec.c

4 years agoavcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Michael Niedermayer [Fri, 3 Oct 2014 15:35:58 +0000 (17:35 +0200)]
avcodec/pngdec: Check bits per pixel before setting monoblack pixel format

Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020c2dbf0201fe7df3dad9e7e0b2e1bb6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/tiff: more completely check bpp/bppcount
Michael Niedermayer [Fri, 3 Oct 2014 14:08:32 +0000 (16:08 +0200)]
avcodec/tiff: more completely check bpp/bppcount

Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa419aa5d320540d5a1b3f8fd9b82ab7e5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Michael Niedermayer [Fri, 3 Oct 2014 12:45:04 +0000 (14:45 +0200)]
avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks

Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f21b761ca15dbb470cd619a1ebf86c3e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/utils: Add case for jv to avcodec_align_dimensions2()
Michael Niedermayer [Fri, 3 Oct 2014 02:30:58 +0000 (04:30 +0200)]
avcodec/utils: Add case for jv to avcodec_align_dimensions2()

Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376a736d243aef4a1d121abebce912e6b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Michael Niedermayer [Thu, 2 Oct 2014 23:50:27 +0000 (01:50 +0200)]
avcodec/mjpegdec: check bits per pixel for changes similar to dimensions

Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6df8243f06c87962b873bd563e58cd39)

Conflicts:

libavcodec/mjpegdec.c

4 years agoavcodec/jpeglsdec: Check run value more completely in ls_decode_line()
Michael Niedermayer [Thu, 2 Oct 2014 21:17:21 +0000 (23:17 +0200)]
avcodec/jpeglsdec: Check run value more completely in ls_decode_line()

previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410a17dc72c30ee7f3145fcacc425f4f2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoconfigure: add noexecstack to linker options if supported.
Reimar Döffinger [Sun, 21 Sep 2014 08:58:10 +0000 (09:58 +0100)]
configure: add noexecstack to linker options if supported.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit b7082d953fda93f7841ffffe7d15a6c3cd15bdee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/ac3enc_template: fix out of array read
Michael Niedermayer [Sat, 27 Sep 2014 18:34:44 +0000 (20:34 +0200)]
avcodec/ac3enc_template: fix out of array read

Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3b68ebccfe308fa839fc30fa634e4de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavutil/x86/cpu: fix cpuid sub-leaf selection
lvqcl [Sat, 27 Sep 2014 11:21:31 +0000 (13:21 +0200)]
avutil/x86/cpu: fix cpuid sub-leaf selection

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e58fc44649d07d523fcd17aa10d9eb0d3a5ef3f4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agolibavutil/opt: fix av_opt_set_channel_layout() to access correct memory address
Philip DeCamp [Wed, 24 Sep 2014 20:15:18 +0000 (16:15 -0400)]
libavutil/opt: fix av_opt_set_channel_layout() to access correct memory address

Signed-off-by: Philip DeCamp <decamp@mit.edu>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 857fc0a71f1b52fbba3281ba64b5a35195458622)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/riffenc: Filter out "BottomUp" in ff_put_bmp_header()
Benoit Fouet [Tue, 23 Sep 2014 08:07:10 +0000 (10:07 +0200)]
avformat/riffenc: Filter out "BottomUp" in ff_put_bmp_header()

Fixes Ticket1304

Commit message and extradata size bugfix by commiter
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6843b9dc78bc966bb30121828ef4f6b6755cf877)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/webp: fix default palette color 0xff000000 -> 0x00000000
Pascal Massimino [Mon, 22 Sep 2014 21:48:57 +0000 (14:48 -0700)]
avcodec/webp: fix default palette color 0xff000000 -> 0x00000000

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e5b3112996c3da45aa03b39c5ade375d40d4407d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/libilbc: support for latest git of libilbc
Gianluigi Tiesi [Fri, 19 Sep 2014 02:49:36 +0000 (04:49 +0200)]
avcodec/libilbc: support for latest git of libilbc

in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c18c8cf3fe2a4b5cc1ebf2f3300bdfe5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agox86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
James Almer [Wed, 5 Mar 2014 22:44:36 +0000 (19:44 -0300)]
x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()

Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067430de1655a7b28536b5afed48bded5)

Conflicts:

libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agolibavcodec/webp: treat out-of-bound palette index as translucent black
Pascal Massimino [Tue, 16 Sep 2014 15:01:07 +0000 (17:01 +0200)]
libavcodec/webp: treat out-of-bound palette index as translucent black

See https://code.google.com/p/webp/issues/detail?id=206
for a description of the problem/fix.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This patch makes the decoder follow the recommendation of the spec.
There is some disagreement (see "[FFmpeg-devel] [PATCH]: libavcodec/webp")
about what would be best to be written in the spec, so in case the spec
is changed again, this potentially would need to be amended or reverted
(cherry picked from commit 4fd21d58a72c38ab63c3a4483b420db260fa7b8d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoapetag: Fix APE tag size check
Katerina Barone-Adesi [Mon, 15 Sep 2014 23:40:24 +0000 (01:40 +0200)]
apetag: Fix APE tag size check

The size variable is (correctly) unsigned, but is passed to several functions
which take signed parameters, such as avio_read, sometimes after having
numbers added to it. So ensure that size remains within the bounds that
these functions can handle.

CC: libav-stable@libav.org
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit c5560e72d0bb69f8a1ac9536570398f84388f396)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agotools/crypto_bench: fix build when AV_READ_TIME is unavailable
Michael Niedermayer [Tue, 16 Sep 2014 16:04:51 +0000 (18:04 +0200)]
tools/crypto_bench: fix build when AV_READ_TIME is unavailable

Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a99134f1a71994a0dc4542a0d6bee8e36146b60)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/m4vdec: Check for non startcode 00 00 00 sequences in probe
Michael Niedermayer [Sun, 7 Sep 2014 14:39:39 +0000 (16:39 +0200)]
avformat/m4vdec: Check for non startcode 00 00 00 sequences in probe

Fixes miss detection of PCM as m4v
Fixes Ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c1835c52a4be2e4e996f83c91a8d5a147b01100)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Michael Niedermayer [Sun, 7 Sep 2014 12:14:52 +0000 (14:14 +0200)]
avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()

Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd28130bf0dd409b2367ca0f8c9b2417)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_...
Michael Niedermayer [Sun, 7 Sep 2014 11:00:47 +0000 (13:00 +0200)]
avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()

The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba43f64312ae4a05d97afecf1b7b1330c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_ch...
Michael Niedermayer [Sun, 7 Sep 2014 10:52:24 +0000 (12:52 +0200)]
avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37511fbbc98b164c07620b9ebc59ec68)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavcodec/h264: Allow partial escaping
Michael Niedermayer [Sat, 6 Sep 2014 23:42:28 +0000 (01:42 +0200)]
avcodec/h264: Allow partial escaping

Fixes Ticket3923

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 033a5334badd8af48f13c6fd1e6827f8e3f2c2f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
4 years agoavformat/swfdec: Do not change the pixel format
Michael Niedermayer [Tue, 2 Sep 2014 14:42:33 +0000 (16:42 +0200)]
avformat/swfdec: Do not change the pixel format

This is currently not supported
Fixes part of Ticket 3539

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c2430304dfb3cc0e3a59ce6d1b59ebdcc934a0c2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>