ffmpeg.git
2 years agoavcodec/hevc_ps: Fix undefined shift in pcm code
Michael Niedermayer [Sun, 27 Aug 2017 21:59:09 +0000 (23:59 +0200)]
avcodec/hevc_ps: Fix undefined shift in pcm code

Fixes: runtime error: shift exponent -1 is negative
Fixes: 3091/clusterfuzz-testcase-minimized-6229767969832960

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2a83866c9f9531eb096c9b9fe0550e742b931ad1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/sbrdsp_fixed: Fix undefined overflows in autocorrelate()
Michael Niedermayer [Sat, 26 Aug 2017 12:00:55 +0000 (14:00 +0200)]
avcodec/sbrdsp_fixed: Fix undefined overflows in autocorrelate()

Fixes: runtime error: signed integer overflow: 8903997421129740175 + 354481484684609529 cannot be represented in type 'long'
Fixes: 2045/clusterfuzz-testcase-minimized-6751255865065472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eefb68c9c335dda423c9115ba11dc4bb3e73e3f9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/mvdec: Fix DoS due to lack of eof check
Michael Niedermayer [Thu, 24 Aug 2017 23:15:30 +0000 (01:15 +0200)]
avformat/mvdec: Fix DoS due to lack of eof check

Fixes: loop.mv

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f05e2e2dc1a89f38cd9f0960a6561083d714f1e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rl2: Fix DoS due to lack of eof check
孙浩 and 张洪亮(望初) [Thu, 24 Aug 2017 23:15:29 +0000 (01:15 +0200)]
avformat/rl2: Fix DoS due to lack of eof check

Fixes: loop.rl2

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 96f24d1bee7fe7bac08e2b7c74db1a046c9dc0de)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rmdec: Fix DoS due to lack of eof check
孙浩 and 张洪亮(望初) [Thu, 24 Aug 2017 23:15:28 +0000 (01:15 +0200)]
avformat/rmdec: Fix DoS due to lack of eof check

Fixes: loop.ivr

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 124eb202e70678539544f6268efc98131f19fa49)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/cinedec: Fix DoS due to lack of eof check
孙浩 and 张洪亮(望初) [Thu, 24 Aug 2017 23:15:27 +0000 (01:15 +0200)]
avformat/cinedec: Fix DoS due to lack of eof check

Fixes: loop.cine

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7e80b63ecd259d69d383623e75b318bf2bd491f6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/asfdec: Fix DoS due to lack of eof check
孙浩 and 张洪亮(望初) [Fri, 25 Aug 2017 10:37:25 +0000 (12:37 +0200)]
avformat/asfdec: Fix DoS due to lack of eof check

Fixes: loop.asf

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7f9ec5593e04827249e7aeb466da06a98a0d7329)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/hls: Fix DoS due to infinite loop
Michael Niedermayer [Fri, 25 Aug 2017 23:26:58 +0000 (01:26 +0200)]
avformat/hls: Fix DoS due to infinite loop

Fixes: loop.m3u

The default max iteration count of 1000 is arbitrary and ideas for a better solution are welcome

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Previous version reviewed-by: Steven Liu <lingjiujianke@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ec414892ddcad88313848494b6fc5f437c9ca4a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoffprobe: Fix NULL pointer handling in color parameter printing
Michael Niedermayer [Tue, 22 Aug 2017 15:27:17 +0000 (17:27 +0200)]
ffprobe: Fix NULL pointer handling in color parameter printing

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 351e28f9a799d9bbbb33dd10c964dca7219fa13b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoffprobe: Fix null pointer dereference with color primaries
Michael Niedermayer [Tue, 22 Aug 2017 09:02:38 +0000 (11:02 +0200)]
ffprobe: Fix null pointer dereference with color primaries

Found-by: AD-lab of venustech
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 837cb4325b712ff1aab531bf41668933f61d75d2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: Check delta_pocs in ff_hevc_decode_short_term_rps()
Michael Niedermayer [Sun, 20 Aug 2017 22:18:48 +0000 (00:18 +0200)]
avcodec/hevc_ps: Check delta_pocs in ff_hevc_decode_short_term_rps()

Fixes: integer overflow
Fixes: 2893/clusterfuzz-testcase-minimized-5809330567774208

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2b44dcbc44e99daf9515753e9fd4c2e1ea53a2fa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rtpdec_h264: Fix heap-buffer-overflow
Michael Niedermayer [Wed, 23 Aug 2017 19:30:37 +0000 (21:30 +0200)]
avformat/rtpdec_h264: Fix heap-buffer-overflow

Fixes: rtp_sdp/poc.sdp

Found-by: Bingchang <l.bing.chang.bc@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c42a1388a6d1bfd8001bf6a4241d8ca27e49326d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/aviobuf: Fix signed integer overflow in avio_seek()
Vitaly Buka [Sun, 20 Aug 2017 18:56:47 +0000 (11:56 -0700)]
avformat/aviobuf: Fix signed integer overflow in avio_seek()

Signed integer overflow is undefined behavior.
Detected with clang and -fsanitize=signed-integer-overflow

Signed-off-by: Vitaly Buka <vitalybuka@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eca2a49716ae1f42804dd3545da2f740edf03250)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/mov: Fix signed integer overflows with total_size
Vitaly Buka [Sun, 20 Aug 2017 18:56:47 +0000 (11:56 -0700)]
avformat/mov: Fix signed integer overflows with total_size

Signed integer overflow is undefined behavior.
Detected with clang and -fsanitize=signed-integer-overflow

Signed-off-by: Vitaly Buka <vitalybuka@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a404cb5b90b878cbe1bb528fac65cf508668cc5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/utils: Fix signed integer overflow in rc_initial_buffer_occupancy initialization
Vitaly Buka [Sun, 20 Aug 2017 18:56:47 +0000 (11:56 -0700)]
avcodec/utils: Fix signed integer overflow in rc_initial_buffer_occupancy initialization

Signed integer overflow is undefined behavior.
Detected with clang and -fsanitize=signed-integer-overflow

Signed-off-by: Vitaly Buka <vitalybuka@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8c2bb10ddfef1f151b9455d152c9aca91140a4b0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_template: Fix running cleanup in decode_ics_info()
Michael Niedermayer [Mon, 21 Aug 2017 00:15:49 +0000 (02:15 +0200)]
avcodec/aacdec_template: Fix running cleanup in decode_ics_info()

Fixes: out of array read
Fixes: 2873/clusterfuzz-testcase-minimized-5924145713905664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Previous version reviewed-by: Alex Converse <alex.converse@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6f03ffb47d51368a4bbc87702df8446e4660845d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/me_cmp: Fix crashes on ARM due to misalignment
Michael Niedermayer [Sat, 19 Aug 2017 21:38:58 +0000 (23:38 +0200)]
avcodec/me_cmp: Fix crashes on ARM due to misalignment

Adds a diff_pixels_unaligned()

Fixes: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=872503

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc488ec28aec4bc91ba47283c49c9f7f25696eaa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/pixlet: Fixes: undefined shift in av_mod_uintp2()
Michael Niedermayer [Fri, 18 Aug 2017 14:42:59 +0000 (16:42 +0200)]
avcodec/pixlet: Fixes: undefined shift in av_mod_uintp2()

Fixes: runtime error: shift exponent 4294967289 is too large for 32-bit type 'int'
Fixes: 3030/clusterfuzz-testcase-minimized-4649809254285312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8754ccd3b319fdf4e2beed5657a3e327999c64ce)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_dwt_template: Fix integer overflow in vertical_compose53iL0()
Michael Niedermayer [Fri, 18 Aug 2017 14:42:58 +0000 (16:42 +0200)]
avcodec/dirac_dwt_template: Fix integer overflow in vertical_compose53iL0()

Fixes: runtime error: signed integer overflow: 2147483646 + 2 cannot be represented in type 'int'
Fixes: 3013/clusterfuzz-testcase-minimized-4644084197097472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a165b53daa8a3a526d2328ca72c4aa9e7f163045)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/fic: Fixes signed integer overflow
Michael Niedermayer [Thu, 17 Aug 2017 16:24:37 +0000 (18:24 +0200)]
avcodec/fic: Fixes signed integer overflow

Fixes: runtime error: signed integer overflow: 1037142357 + 1227025305 cannot be represented in type 'int'
Fixes: 3024/clusterfuzz-testcase-minimized-5885660323905536

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c9d5b015c2022e8deebb93367f8ee8a8eb779e8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/snowdec: Fix off by 1 error
Michael Niedermayer [Thu, 17 Aug 2017 18:32:03 +0000 (20:32 +0200)]
avcodec/snowdec: Fix off by 1 error

Fixes: runtime error: index 4 out of bounds for type 'int8_t [4]'
Fixes: 3023/clusterfuzz-testcase-minimized-6421736130084864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d132683ddd4050d3fe103ca88c73258c3442dc34)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/pixlet: fixes integer overflow in read_highpass()
Michael Niedermayer [Thu, 17 Aug 2017 01:54:56 +0000 (03:54 +0200)]
avcodec/pixlet: fixes integer overflow in read_highpass()

Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int32_t' (aka 'int'); cast to an unsigned type to negate this value to itself
Fixes: 2879/clusterfuzz-testcase-minimized-6317542639403008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cadab5a2a74d715fc16325bd89f8b8091def1083)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/zmbv: Check decomp_size
Michael Niedermayer [Wed, 16 Aug 2017 14:03:23 +0000 (16:03 +0200)]
avcodec/zmbv: Check decomp_size

Fixes: OOM
Fixes: 2710/clusterfuzz-testcase-minimized-4750001420894208

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Tomas Härdin <tjoppen@acc.umu.se>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 931c0ac95cebe62f2bdd53a81bf40e3916be6476)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Fixes integer overflow
Michael Niedermayer [Tue, 15 Aug 2017 01:32:44 +0000 (03:32 +0200)]
avcodec/diracdec: Fixes integer overflow

Fixes: runtime error: signed integer overflow: 340018243 * 27 cannot be represented in type 'int'
Fixes: 2861/clusterfuzz-testcase-minimized-5361070510178304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 92da23093c784b1d9f0db4db51d28ea80a59e759)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Check perspective_exp and zrs_exp.
Michael Niedermayer [Tue, 15 Aug 2017 01:32:43 +0000 (03:32 +0200)]
avcodec/diracdec: Check perspective_exp and zrs_exp.

Fixes: undefined shift
Fixes: runtime error: shift exponent 264 is too large for 32-bit type 'int'
Fixes: 2860/clusterfuzz-testcase-minimized-4672811689836544

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1e6cab874512070b36267a5a53fd053f90072fa2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/ffv1dec_template: Fix undefined shift
Michael Niedermayer [Fri, 11 Aug 2017 16:20:03 +0000 (18:20 +0200)]
avcodec/ffv1dec_template: Fix undefined shift

Fixes: runtime error: left shift of negative value -127
Fixes: 2834/clusterfuzz-testcase-minimized-5988039123795968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 62702eebded6c6341d214405812a981f80e46ea2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpeg4videodec: Clear mcsel before decoding an image
Michael Niedermayer [Sun, 6 Aug 2017 11:32:54 +0000 (13:32 +0200)]
avcodec/mpeg4videodec: Clear mcsel before decoding an image

Fixes: runtime error: signed integer overflow: 2146467840 + 1032192 cannot be represented in type 'int'
Fixes: 2826/clusterfuzz-testcase-minimized-5901511613743104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7735ed29741d985e1e670249ca56e7a1ce18b729)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_dwt: Fixes integer overflows in COMPOSE_DAUB97*
Michael Niedermayer [Sun, 6 Aug 2017 03:01:45 +0000 (05:01 +0200)]
avcodec/dirac_dwt: Fixes integer overflows in COMPOSE_DAUB97*

Fix multiple: runtime error: signed integer overflow: 6497 * 3409630 cannot be represented in type 'int'
Fixes: 2819/clusterfuzz-testcase-minimized-4743700301217792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a5380f9c1c460acccb2edaa8609e4a57c0456088)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_fixed: fix invalid shift in predict()
Michael Niedermayer [Fri, 4 Aug 2017 01:26:30 +0000 (03:26 +0200)]
avcodec/aacdec_fixed: fix invalid shift in predict()

Fixes: runtime error: shift exponent -2 is negative
Fixes: 2818/clusterfuzz-testcase-minimized-5062943676825600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1e443051b277f73b94a2f660d3fd31a1a7beab52)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264_slice: Fix overflow in slice offset
Michael Niedermayer [Fri, 4 Aug 2017 00:41:05 +0000 (02:41 +0200)]
avcodec/h264_slice: Fix overflow in slice offset

Fixes: runtime error: signed integer overflow: 1610612736 * 2 cannot be represented in type 'int'
Fixes: 2817/clusterfuzz-testcase-minimized-5289691240726528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1f53bde6d817ae13a47748f321adbdfa79e15982)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/utils: fix memory leak in avformat_free_context
Steven Siloti [Tue, 18 Jul 2017 18:26:39 +0000 (11:26 -0700)]
avformat/utils: fix memory leak in avformat_free_context

The pointer to the packet queue is stored in the internal structure
so the queue needs to be flushed before internal is freed.

Signed-off-by: Steven Siloti <ssiloti@bittorrent.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 949debd1d1df3a96315b3a3083831162845c1188)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoswscale: fix gbrap16 alpha channel issues
James Cowgill [Thu, 3 Aug 2017 15:21:54 +0000 (16:21 +0100)]
swscale: fix gbrap16 alpha channel issues

Fixes filter-pixfmts-scale test failing on big-endian systems due to
alpSrc not being cast to (const int32_t**).

Also fixes distortions in the output alpha channel values by copying the
alpha channel code from the rgba64 case found elsewhere in output.c.

Fixes ticket 6555.

Signed-off-by: James Cowgill <James.Cowgill@imgtec.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 013ec23cbe5d78a04b1b6c00c43f45773e45e7e5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264idct_template: Fix integer overflow in ff_h264_idct_add()
Michael Niedermayer [Tue, 1 Aug 2017 17:56:07 +0000 (19:56 +0200)]
avcodec/h264idct_template: Fix integer overflow in ff_h264_idct_add()

Fixes: runtime error: signed integer overflow: 26215360 + 2121330944 cannot be represented in type 'int'
Fixes: 2809/clusterfuzz-testcase-minimized-4785181833560064

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
(cherry picked from commit d1bfa80ec464d475a0de3f513bbb62bcd356099a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdsp: fix integer overflow
Michael Niedermayer [Sat, 29 Jul 2017 13:55:36 +0000 (15:55 +0200)]
avcodec/diracdsp: fix integer overflow

Fixes: runtime error: signed integer overflow: 11 * 225726413 cannot be represented in type 'int'
Fixes: 2764/clusterfuzz-testcase-minimized-5382561922547712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b2d9d7226943d6229a17e31714ce5162bdf88b33)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Check weight_log2denom
Michael Niedermayer [Sat, 29 Jul 2017 13:46:50 +0000 (15:46 +0200)]
avcodec/diracdec: Check weight_log2denom

Fixes: runtime error: shift exponent -1 is negative
Fixes: 2742/clusterfuzz-testcase-minimized-5724322402402304
Fixes: 2744/clusterfuzz-testcase-minimized-4672435653705728
Fixes: 2749/clusterfuzz-testcase-minimized-5298741273690112

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 880f5c59139e1d85d3a0b3433103f3fea17ff2d3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/nvenc: only push cuda context on encoder close if encoder exists
Timo Rothenpieler [Wed, 30 Aug 2017 19:12:23 +0000 (21:12 +0200)]
avcodec/nvenc: only push cuda context on encoder close if encoder exists

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
2 years agoavfilter/vf_ssim: fix temp size calculation
Muhammad Faiz [Thu, 3 Aug 2017 00:59:09 +0000 (07:59 +0700)]
avfilter/vf_ssim: fix temp size calculation

Also use av_mallocz_array.
Fix Ticket6519.

Reviewed-by: Tobias Rapp <t.rapp@noa-archive.com>
Signed-off-by: Muhammad Faiz <mfcc64@gmail.com>
(cherry picked from commit f2d23ec03f28c6233059687c65a9124f65f8c312)

2 years agoChangelog:update n3.3.3
Michael Niedermayer [Sat, 29 Jul 2017 17:17:56 +0000 (19:17 +0200)]
Changelog:update

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_dwt: Fix multiple integer overflows in COMPOSE_DD97iH0()
Michael Niedermayer [Fri, 28 Jul 2017 01:22:40 +0000 (03:22 +0200)]
avcodec/dirac_dwt: Fix multiple integer overflows in COMPOSE_DD97iH0()

Fixes: runtime error: signed integer overflow: 9 * 335544320 cannot be represented in type 'int'
Fixes: 2739/clusterfuzz-testcase-minimized-6737297955356672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bf8ab72ae95bb11f2c281d464594c2f6ba70326b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Fix integer overflow in divide3()
Michael Niedermayer [Thu, 27 Jul 2017 21:49:27 +0000 (23:49 +0200)]
avcodec/diracdec: Fix integer overflow in divide3()

Fixes: runtime error: signed integer overflow: -1073746548 * 21845 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c0220c768c7fc933a76c863ebbb0abdf68a88533)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/takdec: Fix integer overflow in decode_subframe()
Michael Niedermayer [Thu, 27 Jul 2017 21:49:26 +0000 (23:49 +0200)]
avcodec/takdec: Fix integer overflow in decode_subframe()

Fixes: runtime error: signed integer overflow: -536870912 - 1972191120 cannot be represented in type 'int'
Fixes: 2711/clusterfuzz-testcase-minimized-4975142398590976

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2c630d159ffe8a9822e81f9c041652762b37e068)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rtmppkt: Convert ff_amf_get_field_value() to bytestream2
Michael Niedermayer [Fri, 28 Jul 2017 12:37:26 +0000 (14:37 +0200)]
avformat/rtmppkt: Convert ff_amf_get_field_value() to bytestream2

Fixes: out of array accesses

Found-by: JunDong Xie of Ant-financial Light-Year Security Lab
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ffcc82219cef0928bed2d558b19ef6ea35634130)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rtmppkt: Convert ff_amf_tag_size() to bytestream2
Michael Niedermayer [Fri, 28 Jul 2017 11:41:59 +0000 (13:41 +0200)]
avformat/rtmppkt: Convert ff_amf_tag_size() to bytestream2

Fixes: out of array accesses
Fixes: crash-9238fa9e8d4fde3beda1f279626f53812cb001cb-SEGV

Found-by: JunDong Xie of Ant-financial Light-Year Security Lab
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 08c073434e25cba8c43aae5ed9554fdd594adfb0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Fix integer overflow in signed multiplication in UNPACK_ARITH()
Michael Niedermayer [Wed, 26 Jul 2017 18:26:43 +0000 (20:26 +0200)]
avcodec/diracdec: Fix integer overflow in signed multiplication in UNPACK_ARITH()

Fixes: runtime error: signed integer overflow: 1073741823 * 4 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e275a74b09cc87f4334ed572f919b7647d4bea1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/pixlet: Simplify nbits computation
Michael Niedermayer [Wed, 26 Jul 2017 18:10:28 +0000 (20:10 +0200)]
avcodec/pixlet: Simplify nbits computation

Fixes multiple integer overflows
Fixes: runtime error: signed integer overflow: 1 + 2147483647 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aeddb3607be94b1d6fef41b602b07f08223ea565)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dnxhddec: Move mb height check out of non hr branch
Michael Niedermayer [Wed, 26 Jul 2017 01:26:59 +0000 (03:26 +0200)]
avcodec/dnxhddec: Move mb height check out of non hr branch

Fixes: out of array access
Fixes: poc.dnxhd

Found-by: Bingchang, Liu@VARAS of IIE
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 296debd213bd6dce7647cedd34eb64e5b94cdc92)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: fix integer overflow in log2_parallel_merge_level_minus2
Michael Niedermayer [Mon, 24 Jul 2017 13:48:37 +0000 (15:48 +0200)]
avcodec/hevc_ps: fix integer overflow in log2_parallel_merge_level_minus2

Fixes: runtime error: signed integer overflow: -2147483647 - 2 cannot be represented in type 'int'
Fixes: 2702/clusterfuzz-testcase-minimized-4511932591636480

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 74c1c22d7f0d25f527ed2ebf62493be5ad52c972)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/oggparsecelt: Do not re-allocate os->private
Michael Niedermayer [Tue, 25 Jul 2017 01:19:07 +0000 (03:19 +0200)]
avformat/oggparsecelt: Do not re-allocate os->private

Fixes: double free
Fixes: clusterfuzz-testcase-minimized-5080550145785856

Found-by: ClusterFuzz
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7140761481e4296723a592019a0244ebe6c1a8cf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/ylc: Fix shift overflow
Michael Niedermayer [Sat, 22 Jul 2017 00:57:12 +0000 (02:57 +0200)]
avcodec/ylc: Fix shift overflow

Fixes: runtime error: shift exponent 32 is too large for 32-bit type 'unsigned int'
Fixes: 2698/clusterfuzz-testcase-minimized-4713541443518464

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03a9e6ff303ad82e75b734edbe4917ca5fd60159)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacps: Fix multiple integer overflow in map_val_34_to_20()
Michael Niedermayer [Fri, 21 Jul 2017 22:44:14 +0000 (00:44 +0200)]
avcodec/aacps: Fix multiple integer overflow in map_val_34_to_20()

Fixes: avcodec/aacps.c:511:40: runtime error: signed integer overflow: 1509077651 + 758068176 cannot be represented in type 'int'
Fixes: 2678/clusterfuzz-testcase-minimized-4702787684270080

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0764fe1d09833ae4dcf9e427df09378d0d6a3386)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_fixed: fix: left shift of negative value -1
Michael Niedermayer [Sun, 23 Jul 2017 14:52:47 +0000 (16:52 +0200)]
avcodec/aacdec_fixed: fix: left shift of negative value -1

Fixes: 2699/clusterfuzz-testcase-minimized-5631303862976512

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2dfb8c417891e0cc3670f8e0791ea0c7071314fe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_vlc: Fix undefined shift
Michael Niedermayer [Tue, 18 Jul 2017 23:43:24 +0000 (01:43 +0200)]
avcodec/dirac_vlc: Fix undefined shift

Fixes: runtime error: shift exponent 64 is too large for 64-bit type 'residual' (aka 'unsigned long')
Fixes: 2674/clusterfuzz-testcase-minimized-4999700518273024

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 69e7daf6ce2a5893936ba18572c58180b29d67f9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agodoc/filters: typo in frei0r
Brice Waegeneire [Fri, 21 Jul 2017 22:09:29 +0000 (00:09 +0200)]
doc/filters: typo in frei0r

Signed-off-by: Brice Waegeneire <brice.wge@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6a6eec485d23b0c47a7cfeb94995db1be91c0e1a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cfhd: Fix decoding regression due to height check
Vodyannikov Aleksandr [Fri, 21 Jul 2017 09:49:45 +0000 (11:49 +0200)]
avcodec/cfhd: Fix decoding regression due to height check

Fixes: Ticket6546

Regression since: 54aaadf648073149f1ac34f56cbde4e6c5aa22ef

Reviewed-by: Muhammad Faiz <mfcc64@gmail.com>
Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47c93657249f1a4bc8a7aaf2f9f3a33510bee38c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoUpdate for 3.3.3
Michael Niedermayer [Wed, 19 Jul 2017 13:28:08 +0000 (15:28 +0200)]
Update for 3.3.3

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_template (fixed point): Check gain in decode_cce() to avoid undefined...
Michael Niedermayer [Sat, 15 Jul 2017 20:22:52 +0000 (22:22 +0200)]
avcodec/aacdec_template (fixed point): Check gain in decode_cce() to avoid undefined shifts later

Fixes: runtime error: shift exponent 47 is too large for 32-bit type 'int'
Fixes: 2581/clusterfuzz-testcase-minimized-4681474395602944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2886142e0c3b5f4304c6e2a2bd282770a8a47f93)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/ffv1dec_template: Fix signed integer overflow
Michael Niedermayer [Sun, 16 Jul 2017 00:08:50 +0000 (02:08 +0200)]
avcodec/ffv1dec_template: Fix signed integer overflow

Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 2634/clusterfuzz-testcase-minimized-4540890636877824

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4de4308d2aa3bfaa286ab566caf087d523cf9a85)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_template: Fix undefined integer overflow in apply_tns()
Michael Niedermayer [Thu, 13 Jul 2017 22:45:29 +0000 (00:45 +0200)]
avcodec/aacdec_template: Fix undefined integer overflow in apply_tns()

Fixes: runtime error: signed integer overflow: -2147483648 - 1202286525 cannot be represented in type 'int'
Fixes: 2071/clusterfuzz-testcase-minimized-6036414271586304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0ef8f03133a0bd83c74200a8cf30982c0f574016)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/magicyuv: Check that vlc len is not too large
Michael Niedermayer [Wed, 12 Jul 2017 15:25:16 +0000 (17:25 +0200)]
avcodec/magicyuv: Check that vlc len is not too large

Fixes: runtime error: shift exponent -95 is negative
Fixes: 2568/clusterfuzz-testcase-minimized-4926115716005888

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 341f01290c2353669ed2263f56e1a9f4c67cc597)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mjpegdec: Clip DC also on the negative side.
Michael Niedermayer [Wed, 12 Jul 2017 14:24:18 +0000 (16:24 +0200)]
avcodec/mjpegdec: Clip DC also on the negative side.

Fixes: runtime error: signed integer overflow: -16711425 + -2130772346 cannot be represented in type 'int'
Fixes: 2533/clusterfuzz-testcase-minimized-5372857678823424

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c28f648b19dd36ff9bc869ad527a1569a0b623e2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacps (fixed point): Fix multiple signed integer overflows
Michael Niedermayer [Sun, 9 Jul 2017 13:19:18 +0000 (15:19 +0200)]
avcodec/aacps (fixed point): Fix multiple signed integer overflows

Fixes: runtime error: signed integer overflow: 1421978265 - -1810326882 cannot be represented in type 'int'
Fixes: 2527/clusterfuzz-testcase-minimized-5260915396050944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 80b9e40b6f1e15db9f36c195e7375e65f6b4924f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/ylc: Fix vlc of 31 bits
Michael Niedermayer [Sat, 8 Jul 2017 20:51:57 +0000 (22:51 +0200)]
avcodec/ylc: Fix vlc of 31 bits

Fixes: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 2515/clusterfuzz-testcase-minimized-6197200012967936

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fe9242204d33db070b8a9d907d93c9ead8a6f3ee)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/sbrdsp_fixed: Fix integer overflow in sbr_hf_apply_noise()
Michael Niedermayer [Sun, 2 Jul 2017 01:30:54 +0000 (03:30 +0200)]
avcodec/sbrdsp_fixed: Fix integer overflow in sbr_hf_apply_noise()

Fixes: runtime error: signed integer overflow: -2049425300 + -117591631 cannot be represented in type 'int'
Fixes: part of 2096/clusterfuzz-testcase-minimized-4901566068817920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2061de8a3f73f14806e5f6ccaf9a635f740a54e6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcdec: do not let updated extradata corrupt state
Michael Niedermayer [Tue, 4 Jul 2017 20:33:52 +0000 (22:33 +0200)]
avcodec/hevcdec: do not let updated extradata corrupt state

Fixes: out of array access
Fixes: 2451/clusterfuzz-testcase-minimized-4781613957251072

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Hendrik Leppkes <h.leppkes@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8cfbc6629c1fe5755b59a3bcfd95ad08b843a07)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/wavpack: Fix invalid shift
Michael Niedermayer [Tue, 27 Jun 2017 11:47:32 +0000 (13:47 +0200)]
avcodec/wavpack: Fix invalid shift

Fixes: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 2377/clusterfuzz-testcase-minimized-6108505935183872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c07af720984acaafaa273369080b458d73975775)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264_slice: Fix signed integer overflow
Michael Niedermayer [Tue, 4 Jul 2017 22:05:11 +0000 (00:05 +0200)]
avcodec/h264_slice: Fix signed integer overflow

Fixes: runtime error: signed integer overflow: 26 + 2147483644 cannot be represented in type 'int'
Fixes: 2456/clusterfuzz-testcase-minimized-4822695051001856

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7592d97f10134422d4509ab1287796af70e003ba)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: Fix integer overflow with beta/tc offsets
Michael Niedermayer [Fri, 30 Jun 2017 14:23:32 +0000 (16:23 +0200)]
avcodec/hevc_ps: Fix integer overflow with beta/tc offsets

Fixes: runtime error: signed integer overflow: 2113929216 * 2 cannot be represented in type 'int'
Fixes: 2422/clusterfuzz-testcase-minimized-5242114713583616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit de54a37c1dfa2817b5838720fac44e82312ccbfd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cfhd: Fix invalid left shift of negative value
Michael Niedermayer [Wed, 28 Jun 2017 18:47:59 +0000 (20:47 +0200)]
avcodec/cfhd: Fix invalid left shift of negative value

Fixes: runtime error: left shift of negative value -1
Fixes: 2395/clusterfuzz-testcase-minimized-6540529313513472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c709f009dad20d99b28918f4f8d7cd394b838def)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/vb: Check vertical GMC component before multiply
Michael Niedermayer [Wed, 28 Jun 2017 18:29:02 +0000 (20:29 +0200)]
avcodec/vb: Check vertical GMC component before multiply

Fixes: runtime error: signed integer overflow: 8224 * 663584 cannot be represented in type 'int'
Fixes: 2393/clusterfuzz-testcase-minimized-6128334993883136

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc6ab72bc7af27189e7b524b97e45c6fcadab5cf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcdec: do basic validity check on delta_chroma_weight and offset
Michael Niedermayer [Tue, 27 Jun 2017 12:11:00 +0000 (14:11 +0200)]
avcodec/hevcdec: do basic validity check on delta_chroma_weight and offset

Fixes: runtime error: signed integer overflow: 2147483520 + 128 cannot be represented in type 'int'
Fixes: 2385/clusterfuzz-testcase-minimized-6594333576790016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2c874548d663225a61b9c25a8b2ce490d26b65fa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000dwt: Fix integer overflow in dwt_decode97_int()
Michael Niedermayer [Mon, 26 Jun 2017 13:05:08 +0000 (15:05 +0200)]
avcodec/jpeg2000dwt: Fix integer overflow in dwt_decode97_int()

Fixes: runtime error: signed integer overflow: -163654656 * 256 cannot be represented in type 'int'
Fixes: 2367/clusterfuzz-testcase-minimized-4648678897745920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ea5366670e26b2c6c396e6a5f49827a2b71e6dd6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/apedec: Fix integer overflow
Michael Niedermayer [Sun, 16 Jul 2017 12:57:20 +0000 (14:57 +0200)]
avcodec/apedec: Fix integer overflow

Fixes: out of array access
Fixes: PoC.ape and others

Found-by: Bingchang, Liu@VARAS of IIE
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ba4beaf6149f7241c8bd85fe853318c2f6837ad0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/wavpack: Fix integer overflow in wv_unpack_stereo()
Michael Niedermayer [Sat, 24 Jun 2017 22:13:53 +0000 (00:13 +0200)]
avcodec/wavpack: Fix integer overflow in wv_unpack_stereo()

Fixes: runtime error: signed integer overflow: 2080374785 + 2080374784 cannot be represented in type 'int'
Fixes: 2351/clusterfuzz-testcase-minimized-5359403240783872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 73ea2a028e12a7d779834f78dc496c8c4b08361f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: Fix max_dec_buffer check
Michael Niedermayer [Sat, 24 Jun 2017 22:00:13 +0000 (00:00 +0200)]
avcodec/hevc_ps: Fix max_dec_buffer check

Fixes: runtime error: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int'
Fixes: 2339/clusterfuzz-testcase-minimized-6663164320022528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 63e7bfe78e6d764097e845248f6d77b28b2b235c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpeg4videodec: Fix GMC with videos of dimension 1
Michael Niedermayer [Sat, 24 Jun 2017 11:45:35 +0000 (13:45 +0200)]
avcodec/mpeg4videodec: Fix GMC with videos of dimension 1

Fixes: runtime error: shift exponent -1 is negative
Fixes: 2338/clusterfuzz-testcase-minimized-5153426541379584

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4976a3411f71518d17a57e373b62517f066648fd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/wavpack: Fix integer overflow
Michael Niedermayer [Thu, 22 Jun 2017 23:58:48 +0000 (01:58 +0200)]
avcodec/wavpack: Fix integer overflow

Fixes: runtime error: signed integer overflow: 227511904 + 1964113935 cannot be represented in type 'int'
Fixes: 2331/clusterfuzz-testcase-minimized-6182185830711296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 24e95f9d4de012f51fdd5767dff0b3142e13ec3a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/takdec: Fix integer overflow
Michael Niedermayer [Thu, 22 Jun 2017 19:21:56 +0000 (21:21 +0200)]
avcodec/takdec: Fix integer overflow

Fixes: runtime error: signed integer overflow: 512 + 2147483146 cannot be represented in type 'int'
Fixes: 2314/clusterfuzz-testcase-minimized-4519333877252096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c2ef4f6b4d52a7b7184c747ffea3576926ea1b1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/tiff: Update pointer only when the result is used
Michael Niedermayer [Thu, 22 Jun 2017 18:21:05 +0000 (20:21 +0200)]
avcodec/tiff: Update pointer only when the result is used

Fixes: runtime error: signed integer overflow: 538976288 * 32 cannot be represented in type 'int'
Fixes: 2310/clusterfuzz-testcase-minimized-4534784887881728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 27f80ab0160d2e64007e1c9799ffd4504cc13eb5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cfhd: Check bpc before setting bpc in context
Michael Niedermayer [Wed, 21 Jun 2017 17:34:31 +0000 (19:34 +0200)]
avcodec/cfhd: Check bpc before setting bpc in context

Fixes: runtime error: shift exponent 32 is too large for 32-bit type 'int'
Fixes: 2306/clusterfuzz-testcase-minimized-5002997392211968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6f1d2355a7e4d681bea82b4cf4280272d9fe8af3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cfhd: Fix undefined shift
Michael Niedermayer [Wed, 21 Jun 2017 15:56:34 +0000 (17:56 +0200)]
avcodec/cfhd: Fix undefined shift

Fixes: runtime error: left shift of negative value -1
Fixes: 2303/clusterfuzz-testcase-minimized-5529675273076736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5a950f4e32a9756391f81987246d96b6549dd447)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_filter: Fix invalid shift
Michael Niedermayer [Tue, 20 Jun 2017 12:38:34 +0000 (14:38 +0200)]
avcodec/hevc_filter: Fix invalid shift

Fixes: runtime error: left shift of negative value -1

Fixes: 2299/clusterfuzz-testcase-minimized-4843509351710720

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d7b3d5c3f2e2ff1994762b5e09c05fbc33790b5b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpeg4videodec: Fix overflow in virtual_ref computation
Michael Niedermayer [Tue, 20 Jun 2017 11:52:06 +0000 (13:52 +0200)]
avcodec/mpeg4videodec: Fix overflow in virtual_ref computation

Fixes: runtime error: signed integer overflow: 262144 * -16120 cannot be represented in type 'int'
Fixes: 2292/clusterfuzz-testcase-minimized-6156080415506432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5443c4bdf4828ac5b7b19cf54feb496c2da40079)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/lpc: signed integer overflow in compute_lpc_coefs() (aacdec_fixed)
Michael Niedermayer [Wed, 14 Jun 2017 23:35:49 +0000 (01:35 +0200)]
avcodec/lpc: signed integer overflow in compute_lpc_coefs() (aacdec_fixed)

Fixes: runtime error: signed integer overflow: -1575818955 + -915383657 cannot be represented in type 'int'
Fixes: 2224/clusterfuzz-testcase-minimized-6208559949807616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e95fcfe8fb28fdfdaecec465c60aad79bc340a3d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/wavpack: Fix undefined integer negation
Michael Niedermayer [Mon, 19 Jun 2017 12:08:58 +0000 (14:08 +0200)]
avcodec/wavpack: Fix undefined integer negation

Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 2291/clusterfuzz-testcase-minimized-5538453481586688

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5f89747086af741ddc34e2378cde8519b8faee78)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_fixed: Check s for being too small
Michael Niedermayer [Mon, 19 Jun 2017 12:04:32 +0000 (14:04 +0200)]
avcodec/aacdec_fixed: Check s for being too small

Fixes: runtime error: shift exponent -8 is negative
Fixes: 2286/clusterfuzz-testcase-minimized-5711764169687040

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cf7edbd6c5d48d7302877352f7b60092d5b65243)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/htmlsubtitles: Replace very slow redundant sscanf() calls by cleaner and...
Michael Niedermayer [Sun, 11 Jun 2017 15:58:45 +0000 (17:58 +0200)]
avcodec/htmlsubtitles: Replace very slow redundant sscanf() calls by cleaner and faster code

This reduces the worst case from O(n²) to O(n) time

Fixes Timeout
Fixes: 2127/clusterfuzz-testcase-minimized-6595787859427328

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4132218b87cd6fb13abd162e3037ef4563286baa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264: Fix mix of lossless and lossy MBs decoding
Anton Mitrofanov [Wed, 14 Jun 2017 00:01:56 +0000 (03:01 +0300)]
avcodec/h264: Fix mix of lossless and lossy MBs decoding

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit cf231b68da1150c100114f2c5671b7ed740f917a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264_mb: Fix 8x8dct in lossless for new versions of x264
Anton Mitrofanov [Tue, 13 Jun 2017 20:37:29 +0000 (23:37 +0300)]
avcodec/h264_mb: Fix 8x8dct in lossless for new versions of x264

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 06dda70f1e7c69a3b1684af5e6930431c62c527a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264_cabac: Fix CABAC+8x8dct in 4:4:4
Anton Mitrofanov [Tue, 30 May 2017 23:37:41 +0000 (02:37 +0300)]
avcodec/h264_cabac: Fix CABAC+8x8dct in 4:4:4

Use the correct ctxIdxInc calculation for coded_block_flag.
Keep old behavior for old versions of x264 for backward compatibility.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 840b41b2a643fc8f0617c0370125a19c02c6b586)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/takdec: Fixes: integer overflow in AV_SAMPLE_FMT_U8P output
Michael Niedermayer [Sun, 18 Jun 2017 12:37:19 +0000 (14:37 +0200)]
avcodec/takdec: Fixes: integer overflow in AV_SAMPLE_FMT_U8P output

Fixes: runtime error: signed integer overflow: 2147483543 + 128 cannot be represented in type 'int'
Fixes: 2234/clusterfuzz-testcase-minimized-6266896041115648

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 27c20068054d8c6786833234f7b6db19f1e98362)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000dsp: Reorder operations in ict_int() to avoid 2 integer overflows
Michael Niedermayer [Sat, 17 Jun 2017 13:06:21 +0000 (15:06 +0200)]
avcodec/jpeg2000dsp: Reorder operations in ict_int() to avoid 2 integer overflows

Fixes: runtime error: signed integer overflow: 58065 * 51981 cannot be represented in type 'int'
Fixes: 2271/clusterfuzz-testcase-minimized-5778297776504832

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c746f92a8e03d5a062359fba836eba4b3530687e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcpred_template: Fix left shift of negative value
Michael Niedermayer [Sat, 17 Jun 2017 12:54:19 +0000 (14:54 +0200)]
avcodec/hevcpred_template: Fix left shift of negative value

Fixes: runtime error: left shift of negative value -1
Fixes: 2250/clusterfuzz-testcase-minimized-5693382112313344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c94326c1fc2fb5719c6f28fe1b95c0c74417998b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcdec: Fix signed integer overflow in decode_lt_rps()
Michael Niedermayer [Fri, 16 Jun 2017 22:34:08 +0000 (00:34 +0200)]
avcodec/hevcdec: Fix signed integer overflow in decode_lt_rps()

Fixes: runtime error: signed integer overflow: 2147483647 + 6 cannot be represented in type 'int'
Fixes: 2263/clusterfuzz-testcase-minimized-4800359627227136

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edbf5e20c75f06d6987bc823e63aa4e649ccddd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000dec: Check nonzerobits more completely
Michael Niedermayer [Fri, 16 Jun 2017 17:57:08 +0000 (19:57 +0200)]
avcodec/jpeg2000dec: Check nonzerobits more completely

Fixes: runtime error: shift exponent 36 is too large for 32-bit type 'int'
Fixes: 2239/clusterfuzz-testcase-minimized-5639766592716800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dfb61ea2630029b7aec7911aade769bf1a914eea)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/shorten: Sanity check maxnlpc
Michael Niedermayer [Fri, 9 Jun 2017 00:16:54 +0000 (02:16 +0200)]
avcodec/shorten: Sanity check maxnlpc

Fixes OOM
Fixes: 2131/clusterfuzz-testcase-minimized-4718045157130240

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e77ddd31a8e14bcf5eccd6008d866ae90b4b0d4c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/truemotion2: Move skip computation after checks
Michael Niedermayer [Thu, 15 Jun 2017 21:41:46 +0000 (23:41 +0200)]
avcodec/truemotion2: Move skip computation after checks

Fixes: runtime error: signed integer overflow: 630067357 * 4 cannot be represented in type 'int'
Fixes: 2233/clusterfuzz-testcase-minimized-5943031318446080

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3c716682a8b69e6644a385a663aaf0e5dc808ae8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000: Fixes integer overflow in ff_jpeg2000_ceildivpow2()
Michael Niedermayer [Thu, 15 Jun 2017 21:26:18 +0000 (23:26 +0200)]
avcodec/jpeg2000: Fixes integer overflow in ff_jpeg2000_ceildivpow2()

Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 2231/clusterfuzz-testcase-minimized-4565181982048256

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3fadc57c5c170f31455abacbcbd67115d7321d7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dnxhd_parser: Do not return invalid value from dnxhd_find_frame_end() on...
Michael Niedermayer [Wed, 14 Jun 2017 14:58:20 +0000 (16:58 +0200)]
avcodec/dnxhd_parser: Do not return invalid value from dnxhd_find_frame_end() on error

Fixes: Null pointer dereference

Fixes: CVE-2017-9608
Found-by: Yihan Lian
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 611b35627488a8d0763e75c25ee0875c5b7987dd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcdec: Check nb_sps
Michael Niedermayer [Wed, 14 Jun 2017 23:28:28 +0000 (01:28 +0200)]
avcodec/hevcdec: Check nb_sps

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc406744620710911de9157eafa3e61d0246566f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_refs: Check nb_refs in add_candidate_ref()
Michael Niedermayer [Wed, 14 Jun 2017 23:26:01 +0000 (01:26 +0200)]
avcodec/hevc_refs: Check nb_refs in add_candidate_ref()

Fixes: runtime error: index 16 out of bounds for type 'int [16]'
Fixes: 2209/clusterfuzz-testcase-minimized-5012343912136704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1cb4ef526dd1e5f547d0354efb0831d07e967919)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>