ffmpeg.git
2 years agoavcodec/dirac_dwt: Fix integer overflow in COMPOSE_53iL0()
Michael Niedermayer [Sun, 8 Oct 2017 23:46:28 +0000 (01:46 +0200)]
avcodec/dirac_dwt: Fix integer overflow in COMPOSE_53iL0()

Fixes: runtime error: signed integer overflow: 2147483646 + 2 cannot be represented in type 'int'
Fixes: 3485/clusterfuzz-testcase-minimized-4940429332054016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bdee75a4e750735ab3039f004275ac8479072048)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpeg_er: Clear mcsel in mpeg_er_decode_mb()
Michael Niedermayer [Sun, 8 Oct 2017 22:32:30 +0000 (00:32 +0200)]
avcodec/mpeg_er: Clear mcsel in mpeg_er_decode_mb()

Fixes out of array read
Should fix: 3516/clusterfuzz-testcase-minimized-4608518562775040 (not reprodoceable)

Found-by: Insu Yun, Georgia Tech.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 127a362630e11fe724e2e63fc871791fdcbcfa64)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpeg4videodec: Use 64 bit intermediates for sprite delta
Michael Niedermayer [Sun, 8 Oct 2017 19:41:54 +0000 (21:41 +0200)]
avcodec/mpeg4videodec: Use 64 bit intermediates for sprite delta

Fixes: runtime error: signed integer overflow: -104713 * 65536 cannot be represented in type 'int'
Fixes: 3453/clusterfuzz-testcase-minimized-5555554657239040
Fixes: 3528/clusterfuzz-testcase-minimized-6283628420005888

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e38f280fece38e270a6462a02cc034f4116a7912)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/x86/lossless_videoencdsp: Fix handling of small widths
Michael Niedermayer [Fri, 29 Sep 2017 22:20:09 +0000 (00:20 +0200)]
avcodec/x86/lossless_videoencdsp: Fix handling of small widths

Fixes out of array access
Fixes: crash-huf.avi

Regression since: 6b41b4414934cc930468ccd5db598dd6ef643987

This could also be fixed by adding checks in the C code that calls the dsp

Found-by: Zhibin Hu and 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit df62b70de8aaa285168e72fe8f6e740843ca91fa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/truemotion2: Fix integer overflows in tm2_high_chroma()
Michael Niedermayer [Sat, 30 Sep 2017 16:54:06 +0000 (18:54 +0200)]
avcodec/truemotion2: Fix integer overflows in tm2_high_chroma()

Fixes: runtime error: signed integer overflow: -1408475220 + -1408475220 cannot be represented in type 'int'
Fixes: 3336/clusterfuzz-testcase-minimized-5656839179993088

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 44874b4f5ec2c605c70393573b9d85540ebc2d81)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_template: Clear tns present flag on error
Michael Niedermayer [Sat, 30 Sep 2017 16:54:05 +0000 (18:54 +0200)]
avcodec/aacdec_template: Clear tns present flag on error

Fixes: 3444/clusterfuzz-testcase-minimized-6270352105668608

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dcf9bae4a93f54cb5767bc97db4a809efd396f8b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/proresdec2: SKIP_BITS() does not work with len=32
Michael Niedermayer [Mon, 2 Oct 2017 02:18:22 +0000 (04:18 +0200)]
avcodec/proresdec2: SKIP_BITS() does not work with len=32

Fixes: invalid shift
Fixes: 3482/clusterfuzz-testcase-minimized-5446915875405824

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c37138e01a93da2f9dd2cc5d4b77e5a38581d130)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcdsp_template: Fix undefined shift
Michael Niedermayer [Mon, 2 Oct 2017 02:18:21 +0000 (04:18 +0200)]
avcodec/hevcdsp_template: Fix undefined shift

Fixes: runtime error: left shift of negative value -255
Fixes: 3373/clusterfuzz-testcase-minimized-5604083912146944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fbdab6eca7874fbeba6aa79c269f345e4d43f5d4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000: Check that codsty->log2_prec_widths/heights has been initialized
Michael Niedermayer [Mon, 4 Sep 2017 20:23:26 +0000 (22:23 +0200)]
avcodec/jpeg2000: Check that codsty->log2_prec_widths/heights has been initialized

Fixes: OOM
Fixes: 2225/clusterfuzz-testcase-minimized-5505632079708160

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 64e034da954125ef98fb8f9153f9706cdb8a96fe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/takdec: Fix integer overflow in decode_lpc()
Michael Niedermayer [Fri, 22 Sep 2017 18:45:27 +0000 (20:45 +0200)]
avcodec/takdec: Fix integer overflow in decode_lpc()

Fixes: runtime error: signed integer overflow: 16748560 + 2143729712 cannot be represented in type 'int'
Fixes: 3202/clusterfuzz-testcase-minimized-4988291642294272

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5d31f03a0264cac24434c8108daef4ccba6d28f9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/proresdec2: Check bits in DECODE_CODEWORD(), fixes invalid shift
Michael Niedermayer [Fri, 22 Sep 2017 18:45:28 +0000 (20:45 +0200)]
avcodec/proresdec2: Check bits in DECODE_CODEWORD(), fixes invalid shift

Fixes: runtime error: shift exponent 42 is too large for 32-bit type 'unsigned int'
Fixes: 3410/clusterfuzz-testcase-minimized-5313377960198144

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f5eaf0b5956e492ee5023929669b1d09aaf6299)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/takdec: Fix integer overflows in decode_subframe()
Michael Niedermayer [Fri, 22 Sep 2017 18:45:26 +0000 (20:45 +0200)]
avcodec/takdec: Fix integer overflows in decode_subframe()

Fixes: runtime error: signed integer overflow: -1562477869 + -691460395 cannot be represented in type 'int'
Fixes: 3196/clusterfuzz-testcase-minimized-4528307146063872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3dabb9c69db114b1f30c30e0a2788cffc50bac40)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_dwt: Fix integer overflow in COMPOSE_FIDELITYi*()
Michael Niedermayer [Mon, 18 Sep 2017 00:53:25 +0000 (02:53 +0200)]
avcodec/dirac_dwt: Fix integer overflow in COMPOSE_FIDELITYi*()

Fixes: runtime error: signed integer overflow: 161 * 13872281 cannot be represented in type 'int'

Fixes: 3295/clusterfuzz-testcase-minimized-4738998142500864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 67da2685e03805230207daab83ab43a390fbb887)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/ffv1dec: Fix integer overflow in read_quant_table()
Michael Niedermayer [Mon, 18 Sep 2017 15:26:09 +0000 (17:26 +0200)]
avcodec/ffv1dec: Fix integer overflow in read_quant_table()

Fixes: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 3361/clusterfuzz-testcase-minimized-5065842955911168

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d00fc952b6c261dd8eb0f7552b9ccf985dbc2b20)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/svq3: Fix overflow in svq3_add_idct_c()
Michael Niedermayer [Mon, 18 Sep 2017 15:03:55 +0000 (17:03 +0200)]
avcodec/svq3: Fix overflow in svq3_add_idct_c()

Fixes: runtime error: signed integer overflow: 2147392585 + 524288 cannot be represented in type 'int'
Fixes: 3348/clusterfuzz-testcase-minimized-4809500517203968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2c933c51687db958d8045d25ed87848342e869f6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/pngdec: Clean up on av_frame_ref() failure
Michael Niedermayer [Sun, 17 Sep 2017 00:42:11 +0000 (02:42 +0200)]
avcodec/pngdec: Clean up on av_frame_ref() failure

Fixes: memleak
Fixes: 3203/clusterfuzz-testcase-minimized-4514553595428864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5480e82d77770e81e897a8c217f3c7f0c13a6de1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: Fix c?_qp_offset_list size
Michael Niedermayer [Sun, 10 Sep 2017 19:10:17 +0000 (21:10 +0200)]
avcodec/hevc_ps: Fix c?_qp_offset_list size

Fixes: runtime error: index 5 out of bounds for type 'int8_t const[5]'
Fixes:3175/clusterfuzz-testcase-minimized-4736774054084608

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abf3f9fa232409c00b60041464604a91fa5612c0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000dsp: Fix multiple integer overflows in ict_int()
Michael Niedermayer [Fri, 8 Sep 2017 21:29:12 +0000 (23:29 +0200)]
avcodec/jpeg2000dsp: Fix multiple integer overflows in ict_int()

Fixes: runtime error: signed integer overflow: 22553 * -188962 cannot be represented in type 'int'
Fixes: 3042/clusterfuzz-testcase-minimized-5174210131394560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d025e742843ca3532bd49ebbfebeacd51337347)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcdsp_template: Fix undefined shift in put_hevc_pel_bi_w_pixels
Michael Niedermayer [Sat, 9 Sep 2017 23:32:51 +0000 (01:32 +0200)]
avcodec/hevcdsp_template: Fix undefined shift in put_hevc_pel_bi_w_pixels

Fixes: runtime error: left shift of negative value -95
Fixes: 3077/clusterfuzz-testcase-minimized-4684917524922368

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c225da68cffbea11270a758ff42859194c980863)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Fix overflow in DC computation
Michael Niedermayer [Sat, 9 Sep 2017 23:32:50 +0000 (01:32 +0200)]
avcodec/diracdec: Fix overflow in DC computation

Fixes: runtime error: signed integer overflow: 11896 + 2147483646 cannot be represented in type 'int'
Fixes: 3053/clusterfuzz-testcase-minimized-6355082062856192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b5995856a4236c27f231210bb08d70688e045192)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/asfdec: Fix DoS in asf_build_simple_index()
Michael Niedermayer [Mon, 4 Sep 2017 22:16:29 +0000 (00:16 +0200)]
avformat/asfdec: Fix DoS in asf_build_simple_index()

Fixes: Missing EOF check in loop
No testcase

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit afc9c683ed9db01edb357bc8c19edad4282b3a97)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/mov: Fix DoS in read_tfra()
Michael Niedermayer [Mon, 4 Sep 2017 22:16:29 +0000 (00:16 +0200)]
avformat/mov: Fix DoS in read_tfra()

Fixes: Missing EOF check in loop
No testcase

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9cb4eb772839c5e1de2855d126bf74ff16d13382)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_dwt: Fix multiple overflows in 9/7 lifting
Michael Niedermayer [Fri, 1 Sep 2017 17:56:11 +0000 (19:56 +0200)]
avcodec/dirac_dwt: Fix multiple overflows in 9/7 lifting

Fixes: runtime error: signed integer overflow: 1073901567 + 1073901567 cannot be represented in type 'int'
Fixes: 3124/clusterfuzz-testcase-minimized-454643435752652

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f71cd44147e7a914f80fcfacca46c9e7b0374362)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Fix integer overflow in INTRA_DC_PRED()
Michael Niedermayer [Fri, 1 Sep 2017 17:56:10 +0000 (19:56 +0200)]
avcodec/diracdec: Fix integer overflow in INTRA_DC_PRED()

Fixes: runtime error: signed integer overflow: 1168175789 + 1168178473 cannot be represented in type 'int'
Fixes: 3081/clusterfuzz-testcase-minimized-4807564879462400
Fixes: 2844/clusterfuzz-testcase-minimized-5561715838156800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2a0823ae966be3ad40e5dba6ec4c4dc1e8c6bcad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/mxfdec: Fix Sign error in mxf_read_primer_pack()
孙浩(晓黑) [Tue, 29 Aug 2017 21:59:21 +0000 (23:59 +0200)]
avformat/mxfdec: Fix Sign error in mxf_read_primer_pack()

Fixes: 20170829B.mxf

Co-Author: 张洪亮(望初)" <wangchu.zhl@alibaba-inc.com>
Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9d00fb9d70ee8c0cc7002b89318c5be00f1bbdad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/mxfdec: Fix DoS issues in mxf_read_index_entry_array()
孙浩(晓黑) [Tue, 29 Aug 2017 21:59:21 +0000 (23:59 +0200)]
avformat/mxfdec: Fix DoS issues in mxf_read_index_entry_array()

Fixes: 20170829A.mxf

Co-Author: 张洪亮(望初)" <wangchu.zhl@alibaba-inc.com>
Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 900f39692ca0337a98a7cf047e4e2611071810c2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/nsvdec: Fix DoS due to lack of eof check in nsvs_file_offset loop.
孙浩(晓黑) [Tue, 29 Aug 2017 21:59:21 +0000 (23:59 +0200)]
avformat/nsvdec: Fix DoS due to lack of eof check in nsvs_file_offset loop.

Fixes: 20170829.nsv

Co-Author: 张洪亮(望初)" <wangchu.zhl@alibaba-inc.com>
Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c24bcb553650b91e9eff15ef6e54ca73de2453b7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/snowdec: Fix integer overflow in decode_subband_slice_buffered()
Michael Niedermayer [Sun, 27 Aug 2017 22:30:33 +0000 (00:30 +0200)]
avcodec/snowdec: Fix integer overflow in decode_subband_slice_buffered()

Fixes: runtime error: signed integer overflow: 267 * 8388608 cannot be represented in type 'int'
Fixes: 2743/clusterfuzz-testcase-minimized-5820652076400640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 732f9764561558a388c05483ed6a722a5c67b05c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: Fix undefined shift in pcm code
Michael Niedermayer [Sun, 27 Aug 2017 21:59:09 +0000 (23:59 +0200)]
avcodec/hevc_ps: Fix undefined shift in pcm code

Fixes: runtime error: shift exponent -1 is negative
Fixes: 3091/clusterfuzz-testcase-minimized-6229767969832960

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2a83866c9f9531eb096c9b9fe0550e742b931ad1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/sbrdsp_fixed: Fix undefined overflows in autocorrelate()
Michael Niedermayer [Sat, 26 Aug 2017 12:00:55 +0000 (14:00 +0200)]
avcodec/sbrdsp_fixed: Fix undefined overflows in autocorrelate()

Fixes: runtime error: signed integer overflow: 8903997421129740175 + 354481484684609529 cannot be represented in type 'long'
Fixes: 2045/clusterfuzz-testcase-minimized-6751255865065472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eefb68c9c335dda423c9115ba11dc4bb3e73e3f9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/mvdec: Fix DoS due to lack of eof check
Michael Niedermayer [Thu, 24 Aug 2017 23:15:30 +0000 (01:15 +0200)]
avformat/mvdec: Fix DoS due to lack of eof check

Fixes: loop.mv

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f05e2e2dc1a89f38cd9f0960a6561083d714f1e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rl2: Fix DoS due to lack of eof check
孙浩 and 张洪亮(望初) [Thu, 24 Aug 2017 23:15:29 +0000 (01:15 +0200)]
avformat/rl2: Fix DoS due to lack of eof check

Fixes: loop.rl2

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 96f24d1bee7fe7bac08e2b7c74db1a046c9dc0de)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rmdec: Fix DoS due to lack of eof check
孙浩 and 张洪亮(望初) [Thu, 24 Aug 2017 23:15:28 +0000 (01:15 +0200)]
avformat/rmdec: Fix DoS due to lack of eof check

Fixes: loop.ivr

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 124eb202e70678539544f6268efc98131f19fa49)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/cinedec: Fix DoS due to lack of eof check
孙浩 and 张洪亮(望初) [Thu, 24 Aug 2017 23:15:27 +0000 (01:15 +0200)]
avformat/cinedec: Fix DoS due to lack of eof check

Fixes: loop.cine

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7e80b63ecd259d69d383623e75b318bf2bd491f6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/asfdec: Fix DoS due to lack of eof check
孙浩 and 张洪亮(望初) [Fri, 25 Aug 2017 10:37:25 +0000 (12:37 +0200)]
avformat/asfdec: Fix DoS due to lack of eof check

Fixes: loop.asf

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7f9ec5593e04827249e7aeb466da06a98a0d7329)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/hls: Fix DoS due to infinite loop
Michael Niedermayer [Fri, 25 Aug 2017 23:26:58 +0000 (01:26 +0200)]
avformat/hls: Fix DoS due to infinite loop

Fixes: loop.m3u

The default max iteration count of 1000 is arbitrary and ideas for a better solution are welcome

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Previous version reviewed-by: Steven Liu <lingjiujianke@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ec414892ddcad88313848494b6fc5f437c9ca4a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoffprobe: Fix NULL pointer handling in color parameter printing
Michael Niedermayer [Tue, 22 Aug 2017 15:27:17 +0000 (17:27 +0200)]
ffprobe: Fix NULL pointer handling in color parameter printing

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 351e28f9a799d9bbbb33dd10c964dca7219fa13b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoffprobe: Fix null pointer dereference with color primaries
Michael Niedermayer [Tue, 22 Aug 2017 09:02:38 +0000 (11:02 +0200)]
ffprobe: Fix null pointer dereference with color primaries

Found-by: AD-lab of venustech
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 837cb4325b712ff1aab531bf41668933f61d75d2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b2c39fcc3c0749490dc93bca80f56724878b55fe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: Check delta_pocs in ff_hevc_decode_short_term_rps()
Michael Niedermayer [Sun, 20 Aug 2017 22:18:48 +0000 (00:18 +0200)]
avcodec/hevc_ps: Check delta_pocs in ff_hevc_decode_short_term_rps()

Fixes: integer overflow
Fixes: 2893/clusterfuzz-testcase-minimized-5809330567774208

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2b44dcbc44e99daf9515753e9fd4c2e1ea53a2fa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/aviobuf: Fix signed integer overflow in avio_seek()
Vitaly Buka [Sun, 20 Aug 2017 18:56:47 +0000 (11:56 -0700)]
avformat/aviobuf: Fix signed integer overflow in avio_seek()

Signed integer overflow is undefined behavior.
Detected with clang and -fsanitize=signed-integer-overflow

Signed-off-by: Vitaly Buka <vitalybuka@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eca2a49716ae1f42804dd3545da2f740edf03250)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/mov: Fix signed integer overflows with total_size
Vitaly Buka [Sun, 20 Aug 2017 18:56:47 +0000 (11:56 -0700)]
avformat/mov: Fix signed integer overflows with total_size

Signed integer overflow is undefined behavior.
Detected with clang and -fsanitize=signed-integer-overflow

Signed-off-by: Vitaly Buka <vitalybuka@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a404cb5b90b878cbe1bb528fac65cf508668cc5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/utils: Fix signed integer overflow in rc_initial_buffer_occupancy initialization
Vitaly Buka [Sun, 20 Aug 2017 18:56:47 +0000 (11:56 -0700)]
avcodec/utils: Fix signed integer overflow in rc_initial_buffer_occupancy initialization

Signed integer overflow is undefined behavior.
Detected with clang and -fsanitize=signed-integer-overflow

Signed-off-by: Vitaly Buka <vitalybuka@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8c2bb10ddfef1f151b9455d152c9aca91140a4b0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_template: Fix running cleanup in decode_ics_info()
Michael Niedermayer [Mon, 21 Aug 2017 00:15:49 +0000 (02:15 +0200)]
avcodec/aacdec_template: Fix running cleanup in decode_ics_info()

Fixes: out of array read
Fixes: 2873/clusterfuzz-testcase-minimized-5924145713905664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Previous version reviewed-by: Alex Converse <alex.converse@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6f03ffb47d51368a4bbc87702df8446e4660845d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/me_cmp: Fix crashes on ARM due to misalignment
Michael Niedermayer [Sat, 19 Aug 2017 21:38:58 +0000 (23:38 +0200)]
avcodec/me_cmp: Fix crashes on ARM due to misalignment

Adds a diff_pixels_unaligned()

Fixes: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=872503

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc488ec28aec4bc91ba47283c49c9f7f25696eaa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_dwt_template: Fix integer overflow in vertical_compose53iL0()
Michael Niedermayer [Fri, 18 Aug 2017 14:42:58 +0000 (16:42 +0200)]
avcodec/dirac_dwt_template: Fix integer overflow in vertical_compose53iL0()

Fixes: runtime error: signed integer overflow: 2147483646 + 2 cannot be represented in type 'int'
Fixes: 3013/clusterfuzz-testcase-minimized-4644084197097472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a165b53daa8a3a526d2328ca72c4aa9e7f163045)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/fic: Fixes signed integer overflow
Michael Niedermayer [Thu, 17 Aug 2017 16:24:37 +0000 (18:24 +0200)]
avcodec/fic: Fixes signed integer overflow

Fixes: runtime error: signed integer overflow: 1037142357 + 1227025305 cannot be represented in type 'int'
Fixes: 3024/clusterfuzz-testcase-minimized-5885660323905536

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c9d5b015c2022e8deebb93367f8ee8a8eb779e8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/snowdec: Fix off by 1 error
Michael Niedermayer [Thu, 17 Aug 2017 18:32:03 +0000 (20:32 +0200)]
avcodec/snowdec: Fix off by 1 error

Fixes: runtime error: index 4 out of bounds for type 'int8_t [4]'
Fixes: 3023/clusterfuzz-testcase-minimized-6421736130084864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d132683ddd4050d3fe103ca88c73258c3442dc34)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Check perspective_exp and zrs_exp.
Michael Niedermayer [Tue, 15 Aug 2017 01:32:43 +0000 (03:32 +0200)]
avcodec/diracdec: Check perspective_exp and zrs_exp.

Fixes: undefined shift
Fixes: runtime error: shift exponent 264 is too large for 32-bit type 'int'
Fixes: 2860/clusterfuzz-testcase-minimized-4672811689836544

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1e6cab874512070b36267a5a53fd053f90072fa2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpeg4videodec: Clear mcsel before decoding an image
Michael Niedermayer [Sun, 6 Aug 2017 11:32:54 +0000 (13:32 +0200)]
avcodec/mpeg4videodec: Clear mcsel before decoding an image

Fixes: runtime error: signed integer overflow: 2146467840 + 1032192 cannot be represented in type 'int'
Fixes: 2826/clusterfuzz-testcase-minimized-5901511613743104

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7735ed29741d985e1e670249ca56e7a1ce18b729)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_dwt: Fixes integer overflows in COMPOSE_DAUB97*
Michael Niedermayer [Sun, 6 Aug 2017 03:01:45 +0000 (05:01 +0200)]
avcodec/dirac_dwt: Fixes integer overflows in COMPOSE_DAUB97*

Fix multiple: runtime error: signed integer overflow: 6497 * 3409630 cannot be represented in type 'int'
Fixes: 2819/clusterfuzz-testcase-minimized-4743700301217792

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a5380f9c1c460acccb2edaa8609e4a57c0456088)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_fixed: fix invalid shift in predict()
Michael Niedermayer [Fri, 4 Aug 2017 01:26:30 +0000 (03:26 +0200)]
avcodec/aacdec_fixed: fix invalid shift in predict()

Fixes: runtime error: shift exponent -2 is negative
Fixes: 2818/clusterfuzz-testcase-minimized-5062943676825600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1e443051b277f73b94a2f660d3fd31a1a7beab52)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264_slice: Fix overflow in slice offset
Michael Niedermayer [Fri, 4 Aug 2017 00:41:05 +0000 (02:41 +0200)]
avcodec/h264_slice: Fix overflow in slice offset

Fixes: runtime error: signed integer overflow: 1610612736 * 2 cannot be represented in type 'int'
Fixes: 2817/clusterfuzz-testcase-minimized-5289691240726528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1f53bde6d817ae13a47748f321adbdfa79e15982)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/utils: fix memory leak in avformat_free_context
Steven Siloti [Tue, 18 Jul 2017 18:26:39 +0000 (11:26 -0700)]
avformat/utils: fix memory leak in avformat_free_context

The pointer to the packet queue is stored in the internal structure
so the queue needs to be flushed before internal is freed.

Signed-off-by: Steven Siloti <ssiloti@bittorrent.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 949debd1d1df3a96315b3a3083831162845c1188)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dirac_dwt: Fix multiple integer overflows in COMPOSE_DD97iH0()
Michael Niedermayer [Fri, 28 Jul 2017 01:22:40 +0000 (03:22 +0200)]
avcodec/dirac_dwt: Fix multiple integer overflows in COMPOSE_DD97iH0()

Fixes: runtime error: signed integer overflow: 9 * 335544320 cannot be represented in type 'int'
Fixes: 2739/clusterfuzz-testcase-minimized-6737297955356672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bf8ab72ae95bb11f2c281d464594c2f6ba70326b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Fix integer overflow in divide3()
Michael Niedermayer [Thu, 27 Jul 2017 21:49:27 +0000 (23:49 +0200)]
avcodec/diracdec: Fix integer overflow in divide3()

Fixes: runtime error: signed integer overflow: -1073746548 * 21845 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c0220c768c7fc933a76c863ebbb0abdf68a88533)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/takdec: Fix integer overflow in decode_subframe()
Michael Niedermayer [Thu, 27 Jul 2017 21:49:26 +0000 (23:49 +0200)]
avcodec/takdec: Fix integer overflow in decode_subframe()

Fixes: runtime error: signed integer overflow: -536870912 - 1972191120 cannot be represented in type 'int'
Fixes: 2711/clusterfuzz-testcase-minimized-4975142398590976

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2c630d159ffe8a9822e81f9c041652762b37e068)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rtmppkt: Convert ff_amf_get_field_value() to bytestream2
Michael Niedermayer [Fri, 28 Jul 2017 12:37:26 +0000 (14:37 +0200)]
avformat/rtmppkt: Convert ff_amf_get_field_value() to bytestream2

Fixes: out of array accesses

Found-by: JunDong Xie of Ant-financial Light-Year Security Lab
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ffcc82219cef0928bed2d558b19ef6ea35634130)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/rtmppkt: Convert ff_amf_tag_size() to bytestream2
Michael Niedermayer [Fri, 28 Jul 2017 11:41:59 +0000 (13:41 +0200)]
avformat/rtmppkt: Convert ff_amf_tag_size() to bytestream2

Fixes: out of array accesses
Fixes: crash-9238fa9e8d4fde3beda1f279626f53812cb001cb-SEGV

Found-by: JunDong Xie of Ant-financial Light-Year Security Lab
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 08c073434e25cba8c43aae5ed9554fdd594adfb0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Fix integer overflow in signed multiplication in UNPACK_ARITH()
Michael Niedermayer [Wed, 26 Jul 2017 18:26:43 +0000 (20:26 +0200)]
avcodec/diracdec: Fix integer overflow in signed multiplication in UNPACK_ARITH()

Fixes: runtime error: signed integer overflow: 1073741823 * 4 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e275a74b09cc87f4334ed572f919b7647d4bea1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dnxhddec: Move mb height check out of non hr branch
Michael Niedermayer [Wed, 26 Jul 2017 01:26:59 +0000 (03:26 +0200)]
avcodec/dnxhddec: Move mb height check out of non hr branch

Fixes: out of array access
Fixes: poc.dnxhd

Found-by: Bingchang, Liu@VARAS of IIE
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 296debd213bd6dce7647cedd34eb64e5b94cdc92)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: fix integer overflow in log2_parallel_merge_level_minus2
Michael Niedermayer [Mon, 24 Jul 2017 13:48:37 +0000 (15:48 +0200)]
avcodec/hevc_ps: fix integer overflow in log2_parallel_merge_level_minus2

Fixes: runtime error: signed integer overflow: -2147483647 - 2 cannot be represented in type 'int'
Fixes: 2702/clusterfuzz-testcase-minimized-4511932591636480

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 74c1c22d7f0d25f527ed2ebf62493be5ad52c972)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/oggparsecelt: Do not re-allocate os->private
Michael Niedermayer [Tue, 25 Jul 2017 01:19:07 +0000 (03:19 +0200)]
avformat/oggparsecelt: Do not re-allocate os->private

Fixes: double free
Fixes: clusterfuzz-testcase-minimized-5080550145785856

Found-by: ClusterFuzz
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7140761481e4296723a592019a0244ebe6c1a8cf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacps: Fix multiple integer overflow in map_val_34_to_20()
Michael Niedermayer [Fri, 21 Jul 2017 22:44:14 +0000 (00:44 +0200)]
avcodec/aacps: Fix multiple integer overflow in map_val_34_to_20()

Fixes: avcodec/aacps.c:511:40: runtime error: signed integer overflow: 1509077651 + 758068176 cannot be represented in type 'int'
Fixes: 2678/clusterfuzz-testcase-minimized-4702787684270080

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0764fe1d09833ae4dcf9e427df09378d0d6a3386)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_fixed: fix: left shift of negative value -1
Michael Niedermayer [Sun, 23 Jul 2017 14:52:47 +0000 (16:52 +0200)]
avcodec/aacdec_fixed: fix: left shift of negative value -1

Fixes: 2699/clusterfuzz-testcase-minimized-5631303862976512

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2dfb8c417891e0cc3670f8e0791ea0c7071314fe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agodoc/filters: typo in frei0r
Brice Waegeneire [Fri, 21 Jul 2017 22:09:29 +0000 (00:09 +0200)]
doc/filters: typo in frei0r

Signed-off-by: Brice Waegeneire <brice.wge@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6a6eec485d23b0c47a7cfeb94995db1be91c0e1a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cfhd: Fix decoding regression due to height check
Vodyannikov Aleksandr [Fri, 21 Jul 2017 09:49:45 +0000 (11:49 +0200)]
avcodec/cfhd: Fix decoding regression due to height check

Fixes: Ticket6546

Regression since: 54aaadf648073149f1ac34f56cbde4e6c5aa22ef

Reviewed-by: Muhammad Faiz <mfcc64@gmail.com>
Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47c93657249f1a4bc8a7aaf2f9f3a33510bee38c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoChangelog: update n3.0.9
Michael Niedermayer [Wed, 19 Jul 2017 11:48:31 +0000 (13:48 +0200)]
Changelog: update

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_template (fixed point): Check gain in decode_cce() to avoid undefined...
Michael Niedermayer [Sat, 15 Jul 2017 20:22:52 +0000 (22:22 +0200)]
avcodec/aacdec_template (fixed point): Check gain in decode_cce() to avoid undefined shifts later

Fixes: runtime error: shift exponent 47 is too large for 32-bit type 'int'
Fixes: 2581/clusterfuzz-testcase-minimized-4681474395602944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2886142e0c3b5f4304c6e2a2bd282770a8a47f93)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_template: Fix undefined integer overflow in apply_tns()
Michael Niedermayer [Thu, 13 Jul 2017 22:45:29 +0000 (00:45 +0200)]
avcodec/aacdec_template: Fix undefined integer overflow in apply_tns()

Fixes: runtime error: signed integer overflow: -2147483648 - 1202286525 cannot be represented in type 'int'
Fixes: 2071/clusterfuzz-testcase-minimized-6036414271586304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0ef8f03133a0bd83c74200a8cf30982c0f574016)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mjpegdec: Clip DC also on the negative side.
Michael Niedermayer [Wed, 12 Jul 2017 14:24:18 +0000 (16:24 +0200)]
avcodec/mjpegdec: Clip DC also on the negative side.

Fixes: runtime error: signed integer overflow: -16711425 + -2130772346 cannot be represented in type 'int'
Fixes: 2533/clusterfuzz-testcase-minimized-5372857678823424

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c28f648b19dd36ff9bc869ad527a1569a0b623e2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacps (fixed point): Fix multiple signed integer overflows
Michael Niedermayer [Sun, 9 Jul 2017 13:19:18 +0000 (15:19 +0200)]
avcodec/aacps (fixed point): Fix multiple signed integer overflows

Fixes: runtime error: signed integer overflow: 1421978265 - -1810326882 cannot be represented in type 'int'
Fixes: 2527/clusterfuzz-testcase-minimized-5260915396050944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 80b9e40b6f1e15db9f36c195e7375e65f6b4924f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/sbrdsp_fixed: Fix integer overflow in sbr_hf_apply_noise()
Michael Niedermayer [Sun, 2 Jul 2017 01:30:54 +0000 (03:30 +0200)]
avcodec/sbrdsp_fixed: Fix integer overflow in sbr_hf_apply_noise()

Fixes: runtime error: signed integer overflow: -2049425300 + -117591631 cannot be represented in type 'int'
Fixes: part of 2096/clusterfuzz-testcase-minimized-4901566068817920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2061de8a3f73f14806e5f6ccaf9a635f740a54e6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/wavpack: Fix invalid shift
Michael Niedermayer [Tue, 27 Jun 2017 11:47:32 +0000 (13:47 +0200)]
avcodec/wavpack: Fix invalid shift

Fixes: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 2377/clusterfuzz-testcase-minimized-6108505935183872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c07af720984acaafaa273369080b458d73975775)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: Fix integer overflow with beta/tc offsets
Michael Niedermayer [Fri, 30 Jun 2017 14:23:32 +0000 (16:23 +0200)]
avcodec/hevc_ps: Fix integer overflow with beta/tc offsets

Fixes: runtime error: signed integer overflow: 2113929216 * 2 cannot be represented in type 'int'
Fixes: 2422/clusterfuzz-testcase-minimized-5242114713583616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit de54a37c1dfa2817b5838720fac44e82312ccbfd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cfhd: Fix invalid left shift of negative value
Michael Niedermayer [Wed, 28 Jun 2017 18:47:59 +0000 (20:47 +0200)]
avcodec/cfhd: Fix invalid left shift of negative value

Fixes: runtime error: left shift of negative value -1
Fixes: 2395/clusterfuzz-testcase-minimized-6540529313513472

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c709f009dad20d99b28918f4f8d7cd394b838def)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/vb: Check vertical GMC component before multiply
Michael Niedermayer [Wed, 28 Jun 2017 18:29:02 +0000 (20:29 +0200)]
avcodec/vb: Check vertical GMC component before multiply

Fixes: runtime error: signed integer overflow: 8224 * 663584 cannot be represented in type 'int'
Fixes: 2393/clusterfuzz-testcase-minimized-6128334993883136

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc6ab72bc7af27189e7b524b97e45c6fcadab5cf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000dwt: Fix integer overflow in dwt_decode97_int()
Michael Niedermayer [Mon, 26 Jun 2017 13:05:08 +0000 (15:05 +0200)]
avcodec/jpeg2000dwt: Fix integer overflow in dwt_decode97_int()

Fixes: runtime error: signed integer overflow: -163654656 * 256 cannot be represented in type 'int'
Fixes: 2367/clusterfuzz-testcase-minimized-4648678897745920

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ea5366670e26b2c6c396e6a5f49827a2b71e6dd6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/apedec: Fix integer overflow
Michael Niedermayer [Sun, 16 Jul 2017 12:57:20 +0000 (14:57 +0200)]
avcodec/apedec: Fix integer overflow

Fixes: out of array access
Fixes: PoC.ape and others

Found-by: Bingchang, Liu@VARAS of IIE
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ba4beaf6149f7241c8bd85fe853318c2f6837ad0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoUpdate for 3.0.9
Michael Niedermayer [Sun, 25 Jun 2017 00:55:08 +0000 (02:55 +0200)]
Update for 3.0.9

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/wavpack: Fix integer overflow in wv_unpack_stereo()
Michael Niedermayer [Sat, 24 Jun 2017 22:13:53 +0000 (00:13 +0200)]
avcodec/wavpack: Fix integer overflow in wv_unpack_stereo()

Fixes: runtime error: signed integer overflow: 2080374785 + 2080374784 cannot be represented in type 'int'
Fixes: 2351/clusterfuzz-testcase-minimized-5359403240783872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 73ea2a028e12a7d779834f78dc496c8c4b08361f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpeg4videodec: Fix GMC with videos of dimension 1
Michael Niedermayer [Sat, 24 Jun 2017 11:45:35 +0000 (13:45 +0200)]
avcodec/mpeg4videodec: Fix GMC with videos of dimension 1

Fixes: runtime error: shift exponent -1 is negative
Fixes: 2338/clusterfuzz-testcase-minimized-5153426541379584

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4976a3411f71518d17a57e373b62517f066648fd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/wavpack: Fix integer overflow
Michael Niedermayer [Thu, 22 Jun 2017 23:58:48 +0000 (01:58 +0200)]
avcodec/wavpack: Fix integer overflow

Fixes: runtime error: signed integer overflow: 227511904 + 1964113935 cannot be represented in type 'int'
Fixes: 2331/clusterfuzz-testcase-minimized-6182185830711296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 24e95f9d4de012f51fdd5767dff0b3142e13ec3a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/takdec: Fix integer overflow
Michael Niedermayer [Thu, 22 Jun 2017 19:21:56 +0000 (21:21 +0200)]
avcodec/takdec: Fix integer overflow

Fixes: runtime error: signed integer overflow: 512 + 2147483146 cannot be represented in type 'int'
Fixes: 2314/clusterfuzz-testcase-minimized-4519333877252096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c2ef4f6b4d52a7b7184c747ffea3576926ea1b1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/tiff: Update pointer only when the result is used
Michael Niedermayer [Thu, 22 Jun 2017 18:21:05 +0000 (20:21 +0200)]
avcodec/tiff: Update pointer only when the result is used

Fixes: runtime error: signed integer overflow: 538976288 * 32 cannot be represented in type 'int'
Fixes: 2310/clusterfuzz-testcase-minimized-4534784887881728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 27f80ab0160d2e64007e1c9799ffd4504cc13eb5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cfhd: Check bpc before setting bpc in context
Michael Niedermayer [Wed, 21 Jun 2017 17:34:31 +0000 (19:34 +0200)]
avcodec/cfhd: Check bpc before setting bpc in context

Fixes: runtime error: shift exponent 32 is too large for 32-bit type 'int'
Fixes: 2306/clusterfuzz-testcase-minimized-5002997392211968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6f1d2355a7e4d681bea82b4cf4280272d9fe8af3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cfhd: Fix undefined shift
Michael Niedermayer [Wed, 21 Jun 2017 15:56:34 +0000 (17:56 +0200)]
avcodec/cfhd: Fix undefined shift

Fixes: runtime error: left shift of negative value -1
Fixes: 2303/clusterfuzz-testcase-minimized-5529675273076736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5a950f4e32a9756391f81987246d96b6549dd447)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_filter: Fix invalid shift
Michael Niedermayer [Tue, 20 Jun 2017 12:38:34 +0000 (14:38 +0200)]
avcodec/hevc_filter: Fix invalid shift

Fixes: runtime error: left shift of negative value -1

Fixes: 2299/clusterfuzz-testcase-minimized-4843509351710720

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d7b3d5c3f2e2ff1994762b5e09c05fbc33790b5b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpeg4videodec: Fix overflow in virtual_ref computation
Michael Niedermayer [Tue, 20 Jun 2017 11:52:06 +0000 (13:52 +0200)]
avcodec/mpeg4videodec: Fix overflow in virtual_ref computation

Fixes: runtime error: signed integer overflow: 262144 * -16120 cannot be represented in type 'int'
Fixes: 2292/clusterfuzz-testcase-minimized-6156080415506432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5443c4bdf4828ac5b7b19cf54feb496c2da40079)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/lpc: signed integer overflow in compute_lpc_coefs() (aacdec_fixed)
Michael Niedermayer [Wed, 14 Jun 2017 23:35:49 +0000 (01:35 +0200)]
avcodec/lpc: signed integer overflow in compute_lpc_coefs() (aacdec_fixed)

Fixes: runtime error: signed integer overflow: -1575818955 + -915383657 cannot be represented in type 'int'
Fixes: 2224/clusterfuzz-testcase-minimized-6208559949807616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e95fcfe8fb28fdfdaecec465c60aad79bc340a3d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/wavpack: Fix undefined integer negation
Michael Niedermayer [Mon, 19 Jun 2017 12:08:58 +0000 (14:08 +0200)]
avcodec/wavpack: Fix undefined integer negation

Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 2291/clusterfuzz-testcase-minimized-5538453481586688

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5f89747086af741ddc34e2378cde8519b8faee78)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/aacdec_fixed: Check s for being too small
Michael Niedermayer [Mon, 19 Jun 2017 12:04:32 +0000 (14:04 +0200)]
avcodec/aacdec_fixed: Check s for being too small

Fixes: runtime error: shift exponent -8 is negative
Fixes: 2286/clusterfuzz-testcase-minimized-5711764169687040

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cf7edbd6c5d48d7302877352f7b60092d5b65243)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/htmlsubtitles: Replace very slow redundant sscanf() calls by cleaner and...
Michael Niedermayer [Sun, 11 Jun 2017 15:58:45 +0000 (17:58 +0200)]
avcodec/htmlsubtitles: Replace very slow redundant sscanf() calls by cleaner and faster code

This reduces the worst case from O(n²) to O(n) time

Fixes Timeout
Fixes: 2127/clusterfuzz-testcase-minimized-6595787859427328

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4132218b87cd6fb13abd162e3037ef4563286baa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264: Fix mix of lossless and lossy MBs decoding
Anton Mitrofanov [Wed, 14 Jun 2017 00:01:56 +0000 (03:01 +0300)]
avcodec/h264: Fix mix of lossless and lossy MBs decoding

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit cf231b68da1150c100114f2c5671b7ed740f917a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264_mb: Fix 8x8dct in lossless for new versions of x264
Anton Mitrofanov [Tue, 13 Jun 2017 20:37:29 +0000 (23:37 +0300)]
avcodec/h264_mb: Fix 8x8dct in lossless for new versions of x264

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 06dda70f1e7c69a3b1684af5e6930431c62c527a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/h264_cabac: Fix CABAC+8x8dct in 4:4:4
Anton Mitrofanov [Tue, 30 May 2017 23:37:41 +0000 (02:37 +0300)]
avcodec/h264_cabac: Fix CABAC+8x8dct in 4:4:4

Use the correct ctxIdxInc calculation for coded_block_flag.
Keep old behavior for old versions of x264 for backward compatibility.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 840b41b2a643fc8f0617c0370125a19c02c6b586)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/takdec: Fixes: integer overflow in AV_SAMPLE_FMT_U8P output
Michael Niedermayer [Sun, 18 Jun 2017 12:37:19 +0000 (14:37 +0200)]
avcodec/takdec: Fixes: integer overflow in AV_SAMPLE_FMT_U8P output

Fixes: runtime error: signed integer overflow: 2147483543 + 128 cannot be represented in type 'int'
Fixes: 2234/clusterfuzz-testcase-minimized-6266896041115648

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 27c20068054d8c6786833234f7b6db19f1e98362)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000dsp: Reorder operations in ict_int() to avoid 2 integer overflows
Michael Niedermayer [Sat, 17 Jun 2017 13:06:21 +0000 (15:06 +0200)]
avcodec/jpeg2000dsp: Reorder operations in ict_int() to avoid 2 integer overflows

Fixes: runtime error: signed integer overflow: 58065 * 51981 cannot be represented in type 'int'
Fixes: 2271/clusterfuzz-testcase-minimized-5778297776504832

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c746f92a8e03d5a062359fba836eba4b3530687e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcpred_template: Fix left shift of negative value
Michael Niedermayer [Sat, 17 Jun 2017 12:54:19 +0000 (14:54 +0200)]
avcodec/hevcpred_template: Fix left shift of negative value

Fixes: runtime error: left shift of negative value -1
Fixes: 2250/clusterfuzz-testcase-minimized-5693382112313344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c94326c1fc2fb5719c6f28fe1b95c0c74417998b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcdec: Fix signed integer overflow in decode_lt_rps()
Michael Niedermayer [Fri, 16 Jun 2017 22:34:08 +0000 (00:34 +0200)]
avcodec/hevcdec: Fix signed integer overflow in decode_lt_rps()

Fixes: runtime error: signed integer overflow: 2147483647 + 6 cannot be represented in type 'int'
Fixes: 2263/clusterfuzz-testcase-minimized-4800359627227136

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edbf5e20c75f06d6987bc823e63aa4e649ccddd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000dec: Check nonzerobits more completely
Michael Niedermayer [Fri, 16 Jun 2017 17:57:08 +0000 (19:57 +0200)]
avcodec/jpeg2000dec: Check nonzerobits more completely

Fixes: runtime error: shift exponent 36 is too large for 32-bit type 'int'
Fixes: 2239/clusterfuzz-testcase-minimized-5639766592716800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dfb61ea2630029b7aec7911aade769bf1a914eea)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>