ffmpeg.git
18 months agoavcodec/hevc_cabac: Move prefix check in coeff_abs_level_remaining_decode() down
Michael Niedermayer [Mon, 15 Jan 2018 22:42:57 +0000 (23:42 +0100)]
avcodec/hevc_cabac: Move prefix check in coeff_abs_level_remaining_decode() down

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 94d4237a7a294ce80e1e577b38e9c93e8882aff9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/truemotion2: Fix integer overflow in TM2_RECALC_BLOCK()
Michael Niedermayer [Sat, 20 Jan 2018 03:10:50 +0000 (04:10 +0100)]
avcodec/truemotion2: Fix integer overflow in TM2_RECALC_BLOCK()

Fixes: signed integer overflow: 1477974040 - -1877995504 cannot be represented in type 'int'
Fixes: 4861/clusterfuzz-testcase-minimized-4570316383715328

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 56a53340ed4cc55898e49c07081311ebb2816630)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/snowdec: Fix integer overflow before htaps check
Michael Niedermayer [Mon, 15 Jan 2018 02:03:36 +0000 (03:03 +0100)]
avcodec/snowdec: Fix integer overflow before htaps check

Fixes: runtime error: signed integer overflow: -1094995529 * 2 cannot be represented in type 'int'
Fixes: 4828/clusterfuzz-testcase-minimized-5100849937252352

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2eecf3cf8eeae67697934df326e98df2149881e5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/ulti: Check number of blocks at init
Michael Niedermayer [Mon, 15 Jan 2018 18:03:48 +0000 (19:03 +0100)]
avcodec/ulti: Check number of blocks at init

Fixes: Timeout
Fixes: 4832/clusterfuzz-testcase-4699096590843904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 725353525e73bbe5b6b4d01528252675f2417a02)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/ac3dec_fixed: Fix integer overflow in scale_coefs()
Michael Niedermayer [Sat, 13 Jan 2018 23:39:39 +0000 (00:39 +0100)]
avcodec/ac3dec_fixed: Fix integer overflow in scale_coefs()

Fixes: runtime error: signed integer overflow: 2147483520 + 128 cannot be represented in type 'int'
Fixes: 4800/clusterfuzz-testcase-minimized-6110372403609600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f38c75893c852cf19dcf3e4553549ba1e70950)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavformat/lrcdec: Fix memory leak in lrc_read_header()
Nikolas Bowe [Fri, 19 Jan 2018 21:17:07 +0000 (13:17 -0800)]
avformat/lrcdec: Fix memory leak in lrc_read_header()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef5994e09d07ace62a672fcdc84761231288edad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavformat/matroskadec: Fix float-cast-overflow undefined behavior in matroska_parse_tr...
Nikolas Bowe [Thu, 18 Jan 2018 23:21:56 +0000 (15:21 -0800)]
avformat/matroskadec: Fix float-cast-overflow undefined behavior in matroska_parse_tracks()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e07649e618caedc07eaf2f4d09253de7f77d14f0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoconfigure: bump year
Carl Eugen Hoyos [Mon, 1 Jan 2018 17:05:55 +0000 (18:05 +0100)]
configure: bump year

Happy new year!

(cherry picked from commit bddf31ba7570325dd2c8d033eae3d0dd74127f96)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/utils: Avoid hardcoding duplicated types in sizeof()
Michael Niedermayer [Sat, 3 Jun 2017 23:53:58 +0000 (01:53 +0200)]
avcodec/utils: Avoid hardcoding duplicated types in sizeof()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 860d991fcd715233b5b9eb1f6c7bf0aadefb6061)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/arm/sbrdsp_neon: Use a free register instead of putting 2 things in one
Michael Niedermayer [Thu, 11 Jan 2018 21:47:10 +0000 (22:47 +0100)]
avcodec/arm/sbrdsp_neon: Use a free register instead of putting 2 things in one

Fixes high pitched shriek
Fixes: 25420848_1478428308873746_4255813235963330560_n.mp4

Reported-by: Dale Curtis <dalecurtis@google.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7dbbb75ee32f87108ca9e15f5551dbbe69fe2641)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/h264addpx_template: Fixes integer overflows
Michael Niedermayer [Sun, 7 Jan 2018 02:48:43 +0000 (03:48 +0100)]
avcodec/h264addpx_template: Fixes integer overflows

Fixes: signed integer overflow: 512 + 2147483491 cannot be represented in type 'int'
Fixes: 4780/clusterfuzz-testcase-minimized-4709066174627840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d6945aeee419a8417b8019c7c92227e12e45b7ad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/dirac_dwt: Fix overflows in COMPOSE_HAARiH0/COMPOSE_HAARiL0
Michael Niedermayer [Sun, 7 Jan 2018 19:58:49 +0000 (20:58 +0100)]
avcodec/dirac_dwt: Fix overflows in COMPOSE_HAARiH0/COMPOSE_HAARiL0

Fixes: 4830/clusterfuzz-testcase-minimized-5255392054476800
Fixes: signed integer overflow: 2147483646 - -7 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0e62a2373475f58c72c0faf5568be00b26909585)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/diracdec: Fix integer overflow with quant
Michael Niedermayer [Sun, 7 Jan 2018 19:43:24 +0000 (20:43 +0100)]
avcodec/diracdec: Fix integer overflow with quant

Fixes: signed integer overflow: 2 + 2147483646 cannot be represented in type 'int'
Fixes: 4792/clusterfuzz-testcase-minimized-6322450775146496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eaa93175895568ef6c2542b13104874907d9c4ef)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/opus_parser: Check payload_len in parse_opus_ts_header()
Michael Niedermayer [Fri, 5 Jan 2018 21:12:07 +0000 (22:12 +0100)]
avcodec/opus_parser: Check payload_len in parse_opus_ts_header()

Fixes: clusterfuzz-testcase-minimized-6134545979277312
Fixes: crbug 797469

Reported-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1bcd7fefcb3c1ec47978fdc64a9e8dfb9512ae62)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/jpeg2000dsp: Fix integer overflows in ict_int()
Michael Niedermayer [Sun, 7 Jan 2018 03:12:57 +0000 (04:12 +0100)]
avcodec/jpeg2000dsp: Fix integer overflows in ict_int()

Fixes: signed integer overflow: 46802 * -71230 cannot be represented in type 'int'
Fixes: 4756/clusterfuzz-testcase-minimized-4812495563784192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b3192c64b5bdcb0474cda437d2d5f9421d68811e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/h264_slice: Do not attempt to render into frames already output
Michael Niedermayer [Wed, 3 Jan 2018 22:42:01 +0000 (23:42 +0100)]
avcodec/h264_slice: Do not attempt to render into frames already output

Fixes: null pointer dereference
Fixes: 4698/clusterfuzz-testcase-minimized-5096956322906112

This testcase does not reproduce the issue before 03b82b3ab9883cef017e513c7d0b3b986b3b3e7b

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 476665d4de989dba48ec1195215ccc8db54538f4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/dnxhddec: Check dc vlc
Michael Niedermayer [Wed, 3 Jan 2018 22:42:00 +0000 (23:42 +0100)]
avcodec/dnxhddec: Check dc vlc

Fixes: signed integer overflow: 1024 + 2147483640 cannot be represented in type 'int'
Fixes: 4671/clusterfuzz-testcase-minimized-6027464343027712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b2be76c0a472b729756ed7a91225c209d0dd1d2e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/exr: Check buf_size more completely
Michael Niedermayer [Fri, 29 Dec 2017 02:00:19 +0000 (03:00 +0100)]
avcodec/exr: Check buf_size more completely

Fixes: Out of heap array read
Fixes: 4683/clusterfuzz-testcase-minimized-6152313673613312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 903be5e4f66268273dc6e3c42a7fdeaab32066ef)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/flacdec: Fix overflow in multiplication in decode_subframe_fixed()
Michael Niedermayer [Tue, 26 Dec 2017 22:24:44 +0000 (23:24 +0100)]
avcodec/flacdec: Fix overflow in multiplication in decode_subframe_fixed()

Fixes: signed integer overflow: 2 * 1629495328 cannot be represented in type 'int'
Fixes: 4716/clusterfuzz-testcase-minimized-5835915940331520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3d23f7a0969bf76ad6dcdc2c4a5cd3ae884745a8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/hevcdsp_template: Fix Invalid shifts in put_hevc_qpel_bi_w_h() and put_hevc_q...
Michael Niedermayer [Tue, 26 Dec 2017 22:24:45 +0000 (23:24 +0100)]
avcodec/hevcdsp_template: Fix Invalid shifts in put_hevc_qpel_bi_w_h() and put_hevc_qpel_bi_w_w()

Fixes: left shift of negative value -1
Fixes: 4690/clusterfuzz-testcase-minimized-6117482428366848

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d135f3c514ac1723256c8e0f5cdd466fe98a2578)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/flacdec: avoid undefined shift
Michael Niedermayer [Tue, 26 Dec 2017 22:24:43 +0000 (23:24 +0100)]
avcodec/flacdec: avoid undefined shift

Fixes: shift exponent 32 is too large for 32-bit type 'unsigned int'
Fixes: 4688/clusterfuzz-testcase-minimized-6572210748653568

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 560daf88913b0de59a4d845bcd19254b406388dd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/hevcdsp_template.c: Fix undefined shift in FUNC(dequant)
Michael Niedermayer [Fri, 22 Dec 2017 02:12:03 +0000 (03:12 +0100)]
avcodec/hevcdsp_template.c: Fix undefined shift in FUNC(dequant)

Fixes: runtime error: left shift of negative value -180
Fixes: 4626/clusterfuzz-testcase-minimized-5647837887987712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c9ab5ef9c1ee852c80c859c9e07efe8730b57ed)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/dirac_dwt: Fix integer overflow in COMPOSE_DD97iH0() and COMPOSE_DD137iL0()
Michael Niedermayer [Fri, 22 Dec 2017 02:06:14 +0000 (03:06 +0100)]
avcodec/dirac_dwt: Fix integer overflow in COMPOSE_DD97iH0() and COMPOSE_DD137iL0()

Fixes: runtime error: signed integer overflow: 2147483646 + 33554433 cannot be represented in type 'int'
Fixes: 4563/clusterfuzz-testcase-minimized-5438979567517696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4d70fbeec8cbab072b3a9b9f760b8deaaef240f2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/hevc_cabac: Fix integer overflow in ff_hevc_cu_qp_delta_abs()
Michael Niedermayer [Fri, 15 Dec 2017 17:17:13 +0000 (18:17 +0100)]
avcodec/hevc_cabac: Fix integer overflow in ff_hevc_cu_qp_delta_abs()

Fixes: signed integer overflow: 2147483647 + 1073741824 cannot be represented in type 'int'
Fixes: 4555/clusterfuzz-testcase-minimized-4505532481142784

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0ee143558d55b590774dba69cff5a16eda089a4d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/hevc_sei: Fix integer overflows in decode_nal_sei_message()
Michael Niedermayer [Fri, 15 Dec 2017 16:50:12 +0000 (17:50 +0100)]
avcodec/hevc_sei: Fix integer overflows in decode_nal_sei_message()

Fixes: signed integer overflow: 2147483520 + 255 cannot be represented in type 'int'
Fixes: 4554/clusterfuzz-testcase-minimized-4843714515042304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 991ef6e5b9a6a9d95e274ff6bff52db1c82b3808)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/hevcdsp_template: Fix undefined shift in put_hevc_qpel_bi_w_hv()
Michael Niedermayer [Fri, 15 Dec 2017 12:06:30 +0000 (13:06 +0100)]
avcodec/hevcdsp_template: Fix undefined shift in put_hevc_qpel_bi_w_hv()

Fixes: runtime error: left shift of negative value -3
Fixes: 4524/clusterfuzz-testcase-minimized-6055590120914944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 439fbb9c8b2a90e97c44c7c57245e01ca84c865d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agolibavfilter/af_dcshift.c: Fixed repeated spelling error
Kelly Ledford [Tue, 12 Dec 2017 19:31:23 +0000 (11:31 -0800)]
libavfilter/af_dcshift.c: Fixed repeated spelling error

'threshhold' should be 'threshold'

Signed-off-by: Kelly Ledford <kelly.ledford@intel.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc219082bb04b9a4725bfe7e78ce0950244e6e84)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavfilter/formats: fix wrong function name in error message
Jun Zhao [Mon, 4 Dec 2017 04:50:34 +0000 (12:50 +0800)]
avfilter/formats: fix wrong function name in error message

Use perdefined micro __FUNCTION__ rather than hard coding function name
to fix wrong function name in error message.

Signed-off-by: Jun Zhao <jun.zhao@intel.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4280948702bc256e21c375790b889c735d233b0d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/amrwbdec: Fix division by 0 in voice_factor()
Michael Niedermayer [Thu, 7 Dec 2017 14:32:54 +0000 (15:32 +0100)]
avcodec/amrwbdec: Fix division by 0 in voice_factor()

The added value matches "Digital cellular telecommunications system (Phase 2+) (GSM); Universal Mobile Telecommunications System (UMTS); LTE; Extended Adaptive Multi-Rate - Wideband (AMR-WB+) codec; Floating-point ANSI-C code (3GPP TS 26.304 version 14.0.0 Release 14)
Extended Adaptive Multi-Rate - Wideband (AMR-WB+) codec; Floating-point ANSI-C code"

Fixes: runtime error: division by zero
Fixes: 4415/clusterfuzz-testcase-minimized-4677752314658816

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1d0817d56b66797118880358ea7d7a2acfdca429)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/diracdsp: Fix integer overflow in PUT_SIGNED_RECT_CLAMPED()
Michael Niedermayer [Sat, 2 Dec 2017 20:53:22 +0000 (21:53 +0100)]
avcodec/diracdsp: Fix integer overflow in PUT_SIGNED_RECT_CLAMPED()

Fixes: runtime error: signed integer overflow: 2147483646 + 2048 cannot be represented in type 'int'
Fixes: 4479/clusterfuzz-testcase-minimized-6529894147162112

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 610dd74502a58e8bb0f1d8fcbc7015f86b78d70e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavcodec/dirac_dwt: Fix integer overflows in COMPOSE_DAUB97*
Michael Niedermayer [Sat, 2 Dec 2017 20:48:04 +0000 (21:48 +0100)]
avcodec/dirac_dwt: Fix integer overflows in COMPOSE_DAUB97*

Fixes: 4478/clusterfuzz-testcase-minimized-4752113767809024
Fixes: runtime error: signed integer overflow: -2147483626 + -319489 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e9a13a5a33bf7566591216e335f2529612100bb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavformat/libssh: check the user provided a password before trying to use it
James Almer [Sun, 11 Jun 2017 17:17:30 +0000 (14:17 -0300)]
avformat/libssh: check the user provided a password before trying to use it

Fixes ticket #6413

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 8ddb6820bd52df6ed616abc3d8be200b126aa8c1)

21 months agoChangelog: update n3.0.10
Michael Niedermayer [Fri, 1 Dec 2017 23:30:18 +0000 (00:30 +0100)]
Changelog: update

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/vorbis: Fix another 1 << 31 > int32_t::max() with 1u.
Dale Curtis [Thu, 30 Nov 2017 20:20:36 +0000 (12:20 -0800)]
avcodec/vorbis: Fix another 1 << 31 > int32_t::max() with 1u.

Didn't notice this one when 9648cc6d was landed.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 95bacb521af8cd28f146f045437c9f75717a493a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoDon't manipulate duration when it's AV_NOPTS_VALUE.
Dale Curtis [Tue, 28 Nov 2017 22:26:55 +0000 (14:26 -0800)]
Don't manipulate duration when it's AV_NOPTS_VALUE.

This leads to signed integer overflow.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit c5fd57f483d2ad8e34551b78509f1e14136f73c0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/vorbis: 1 << 31 > int32_t::max(), so use 1u << 31 instead.
Dale Curtis [Wed, 22 Nov 2017 18:58:39 +0000 (10:58 -0800)]
avcodec/vorbis: 1 << 31 > int32_t::max(), so use 1u << 31 instead.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9648cc6d7fdbb0a260bed1e3e23300569cff9579)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavformat/utils: Prevent undefined shift with wrap_bits > 64.
Dale Curtis [Fri, 17 Nov 2017 21:35:56 +0000 (13:35 -0800)]
avformat/utils: Prevent undefined shift with wrap_bits > 64.

2LL << (wrap_bits=64 - 1) does not fit in int64_t; change the
code to use a uint64_t (2ULL) and add an av_assert2() to
ensure wrap_bits <= 64.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03fbc0daa7e37af024f8b017a28105c32bbe25ca)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/j2kenc: Fix out of array access in encode_cblk()
Michael Niedermayer [Thu, 30 Nov 2017 22:42:04 +0000 (23:42 +0100)]
avcodec/j2kenc: Fix out of array access in encode_cblk()

Fixes: 4427/clusterfuzz-testcase-minimized-5106919271301120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0674087004538599797688785f6ac82358abc23b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/hevcdsp_template: Fix undefined shift in put_hevc_epel_bi_w_h()
Michael Niedermayer [Thu, 30 Nov 2017 20:27:37 +0000 (21:27 +0100)]
avcodec/hevcdsp_template: Fix undefined shift in put_hevc_epel_bi_w_h()

Fixes: runtime error: left shift of negative value -127
Fixes: 4397/clusterfuzz-testcase-minimized-4779061080489984

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0409d333115e623b5ccdbb364d64ca2a52fd8467)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/mlpdsp: Fix signed integer overflow, 2nd try
Michael Niedermayer [Mon, 20 Nov 2017 17:45:45 +0000 (18:45 +0100)]
avcodec/mlpdsp: Fix signed integer overflow, 2nd try

The outputted bits should match what is used in the lossless check

Fixes: runtime error: signed integer overflow: -538697856 * 256 cannot be represented in type 'int'
Fixes: 4326/clusterfuzz-testcase-minimized-5689449645080576

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 97c00edaa043043c29d985653e7e1687b56dfa23)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/kgv1dec: Check that there is enough input for maximum RLE compression
Michael Niedermayer [Wed, 22 Nov 2017 19:14:54 +0000 (20:14 +0100)]
avcodec/kgv1dec: Check that there is enough input for maximum RLE compression

Fixes: Timeout
Fixes: 4271/clusterfuzz-testcase-4676667768307712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3aad94bf2b140cfba8ae69d018da05d4948ef37f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/dirac_dwt: Fix integer overflow in COMPOSE_FIDELITYi*
Michael Niedermayer [Sat, 25 Nov 2017 02:15:16 +0000 (03:15 +0100)]
avcodec/dirac_dwt: Fix integer overflow in COMPOSE_FIDELITYi*

Fixes: runtime error: signed integer overflow: -2143827186 - 7404944 cannot be represented in type 'int'
Fixes: 4354/clusterfuzz-testcase-minimized-4671122764201984

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2b6964f764382742bb052a1ee3b7167cac35332f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/mpeg4videodec: Check also for negative versions in the validity check
Michael Niedermayer [Tue, 21 Nov 2017 02:15:53 +0000 (03:15 +0100)]
avcodec/mpeg4videodec: Check also for negative versions in the validity check

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0e7865ce4152f8b04cda6a698bbee4fd4a94009d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoClose ogg stream upon error when using AV_EF_EXPLODE.
Dale Curtis [Mon, 20 Nov 2017 20:07:57 +0000 (12:07 -0800)]
Close ogg stream upon error when using AV_EF_EXPLODE.

Without this there can be multiple memory leaks for unrecognized
ogg streams.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bce8fc0754c4b31f574a4372c6d7996ed29f7c2a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoFix undefined shift on assumed 8-bit input.
Dale Curtis [Sat, 18 Nov 2017 00:05:30 +0000 (16:05 -0800)]
Fix undefined shift on assumed 8-bit input.

decode_user_data() attempts to create an integer |build|
value with 8 bits of spacing for 3 components. However
each component is an int32_t, so shifting each component
is undefined for values outside of the 8 bit range.

This patch simply clamps input to 8-bits per component
and prints out a warning that the values were clamped.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7010dd98b575d2e39fca947e609b85be7490b269)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoUse ff_thread_once for fixed, float table init.
Dale Curtis [Fri, 17 Nov 2017 22:51:09 +0000 (14:51 -0800)]
Use ff_thread_once for fixed, float table init.

These tables are static so they should only be initialized once
instead of on every call to ff_mpadsp_init().

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5eaaffaf64d1854493f0fe9ec822eed1b3cd9fe1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavformat/mov: Propagate errors in mov_switch_root.
Jacob Trimble [Mon, 20 Nov 2017 20:05:02 +0000 (12:05 -0800)]
avformat/mov: Propagate errors in mov_switch_root.

Signed-off-by: Jacob Trimble <modmaker@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d9cf3bf16b94cd9db10dabad695c69c5cff4f58)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/hevcdsp_template: Fix invalid shift in put_hevc_epel_bi_w_v()
Michael Niedermayer [Fri, 17 Nov 2017 21:01:29 +0000 (22:01 +0100)]
avcodec/hevcdsp_template: Fix invalid shift in put_hevc_epel_bi_w_v()

Fixes: runtime error: left shift of negative value -255
Fixes: 4037/clusterfuzz-testcase-minimized-5290998163832832

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7d88586e4728e97349f98e07ff782bb168ab96c3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/mlpdsp: Fix undefined shift ff_mlp_pack_output()
Michael Niedermayer [Wed, 15 Nov 2017 02:38:37 +0000 (03:38 +0100)]
avcodec/mlpdsp: Fix undefined shift ff_mlp_pack_output()

Fixes: runtime error: left shift of negative value -7862264
Fixes: 4074/clusterfuzz-testcase-minimized-4516104123711488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f7f70738e8dd77a698a5e28bba552ea7064af21)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/zmbv: Check that the buffer is large enough for mvec
Michael Niedermayer [Wed, 15 Nov 2017 16:11:12 +0000 (17:11 +0100)]
avcodec/zmbv: Check that the buffer is large enough for mvec

Fixes: Timeout
Fixes: 4143/clusterfuzz-testcase-4736864637419520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2ab9568a2c3349039eec29fb960fe39de354b514)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/dirac_dwt: Fix integer overflow in COMPOSE_DD137iL0()
Michael Niedermayer [Tue, 14 Nov 2017 02:40:07 +0000 (03:40 +0100)]
avcodec/dirac_dwt: Fix integer overflow in COMPOSE_DD137iL0()

Fixes: 4035/clusterfuzz-testcase-minimized-6479308925173760
Fixes: runtime error: signed integer overflow: 9 * 402653183 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 73964680d7bce6d81ddc553a24d73e9a1c9156f9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/wmv2dec: Check end of bitstream in parse_mb_skip() and ff_wmv2_decode_mb()
Michael Niedermayer [Sat, 16 Sep 2017 23:28:07 +0000 (01:28 +0200)]
avcodec/wmv2dec: Check end of bitstream in parse_mb_skip() and ff_wmv2_decode_mb()

Fixes: Timeout
Fixes: 3200/clusterfuzz-testcase-5750022136135680

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 65e0a7c473f23f1833538ffecf53c81fe500b5e4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/snowdec: Check for remaining bitstream in decode_blocks()
Michael Niedermayer [Wed, 15 Nov 2017 20:17:16 +0000 (21:17 +0100)]
avcodec/snowdec: Check for remaining bitstream in decode_blocks()

Fixes: Timeout
Fixes: 3142/clusterfuzz-testcase-5007853163118592

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4527ec2216109867498edc3ac8a17fd879b5d017)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/snowdec: Check intra block dc differences.
Michael Niedermayer [Wed, 15 Nov 2017 20:17:15 +0000 (21:17 +0100)]
avcodec/snowdec: Check intra block dc differences.

Fixes: Timeout
Fixes: 3142/clusterfuzz-testcase-5007853163118592

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c3b9bbcc6edf2d83fe4857484cfa0839872188c6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavformat/mov: Check size of STSC allocation
Fredrik Hubinette [Thu, 16 Nov 2017 01:24:30 +0000 (17:24 -0800)]
avformat/mov: Check size of STSC allocation

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a6fdd75fe6440d2f4150cb456a9078aa68b00fdb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/vc2enc: Clear coef_buf on allocation
Michael Niedermayer [Wed, 15 Nov 2017 15:53:34 +0000 (16:53 +0100)]
avcodec/vc2enc: Clear coef_buf on allocation

Fixes: Use of uninitialized memory
Fixes: assertion failure

Reviewed-by: <atomnuker>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6d00905f8134a2932e5c00dd1ec8b2a1f0a38035)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/h264dec: Fix potential array overread
Michael Niedermayer [Sat, 21 Oct 2017 16:04:44 +0000 (18:04 +0200)]
avcodec/h264dec: Fix potential array overread

add padding before scantable arrays

See: 522d850e68ec4b77d3477b3c8f55b1ba00a9d69a

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 380b48fb9fdc7b0c40d67e026f9b3accb12794eb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/x86/mpegvideodsp: Fix signedness bug in need_emu
Michael Niedermayer [Mon, 13 Nov 2017 19:47:48 +0000 (20:47 +0100)]
avcodec/x86/mpegvideodsp: Fix signedness bug in need_emu

Fixes: out of array read
Fixes: 3516/attachment-311488.dat

Found-by: Insu Yun, Georgia Tech.
Tested-by: wuninsu@gmail.com
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 58cf31cee7a456057f337b3102a03206d833d5e8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/aacpsdsp_template: Fix integer overflows in ps_decorrelate_c()
Michael Niedermayer [Sun, 5 Nov 2017 20:20:08 +0000 (21:20 +0100)]
avcodec/aacpsdsp_template: Fix integer overflows in ps_decorrelate_c()

Fixes: runtime error: signed integer overflow: 1939661764 - -454942263 cannot be represented in type 'int'
Fixes: 3191/clusterfuzz-testcase-minimized-5688798451073024

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2afe05402f05d485f0c356b04dc562f0510d317d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/aacdec_fixed: Fix undefined shift
Michael Niedermayer [Sun, 5 Nov 2017 20:20:07 +0000 (21:20 +0100)]
avcodec/aacdec_fixed: Fix undefined shift

Fixes: runtime error: left shift of negative value -801112064
Fixes: 3492/clusterfuzz-testcase-minimized-5784775283441664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fca198fb5bf42ba6b765b3f75b11738e4b4fc2a9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/mdct_*: Fix integer overflow in addition in RESCALE()
Michael Niedermayer [Sun, 5 Nov 2017 20:20:06 +0000 (21:20 +0100)]
avcodec/mdct_*: Fix integer overflow in addition in RESCALE()

Fixes: runtime error: signed integer overflow: 1219998458 - -1469874012 cannot be represented in type 'int'
Fixes: 3443/clusterfuzz-testcase-minimized-5369987105554432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 770c934fa1635f4fadf5db4fc5cc5ad15d82455a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/snowdec: Fix integer overflow in header parsing
Michael Niedermayer [Sun, 5 Nov 2017 20:20:05 +0000 (21:20 +0100)]
avcodec/snowdec: Fix integer overflow in header parsing

Fixes: 3984/clusterfuzz-testcase-minimized-5265759929368576
Fixes: runtime error: signed integer overflow: -1085585801 + -1094995529 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c897a9285846b6a072b9650976afd4f091b7a71f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/cngdec: Fix integer clipping
Michael Niedermayer [Thu, 2 Nov 2017 17:34:09 +0000 (18:34 +0100)]
avcodec/cngdec: Fix integer clipping

Fixes: runtime error: value -36211.7 is outside the range of representable values of type 'short'
Fixes: 2992/clusterfuzz-testcase-6649611793989632

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 51090133b31bc719ea868db15d3ee38e9dbe90f1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/sbrdsp_fixed: Fix integer overflow in shift in sbr_hf_g_filt_c()
Michael Niedermayer [Wed, 1 Nov 2017 13:00:20 +0000 (14:00 +0100)]
avcodec/sbrdsp_fixed: Fix integer overflow in shift in sbr_hf_g_filt_c()

Fixes: runtime error: shift exponent 66 is too large for 64-bit type 'long long'
Fixes: 3642/clusterfuzz-testcase-minimized-5443853801750528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 981e99ab99986935affad7c164ebdfe28e8ea7f8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavutil/softfloat: Add FLOAT_MIN
Michael Niedermayer [Wed, 1 Nov 2017 13:00:18 +0000 (14:00 +0100)]
avutil/softfloat: Add FLOAT_MIN

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/aacsbr_fixed: Fix division by zero in sbr_gain_calc()
Michael Niedermayer [Wed, 1 Nov 2017 13:00:19 +0000 (14:00 +0100)]
avcodec/aacsbr_fixed: Fix division by zero in sbr_gain_calc()

Fixes: 3642/clusterfuzz-testcase-minimized-5443853801750528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7d1dec466895eed12f2c79b7ab5447f5390fe869)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/h264idct_template: Fix integer overflows in ff_h264_idct8_add()
Michael Niedermayer [Sat, 4 Nov 2017 00:19:20 +0000 (01:19 +0100)]
avcodec/h264idct_template: Fix integer overflows in ff_h264_idct8_add()

Fixes: runtime error: signed integer overflow: -503316480 + -2013265038 cannot be represented in type 'int'
Fixes: 3805/clusterfuzz-testcase-minimized-6578427831255040

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e131b8cedb00043dcc97cc05ca04749ec8ff57de)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/xan: Check for bitstream end in xan_huffman_decode()
Michael Niedermayer [Fri, 3 Nov 2017 16:48:29 +0000 (17:48 +0100)]
avcodec/xan: Check for bitstream end in xan_huffman_decode()

Fixes: Timeout
Fixes: 3707/clusterfuzz-testcase-6465922706440192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4b51437dccd62fc5491280db44e3c21b44aeeb3f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavformat: Free the internal codec context at the end
Luca Barbato [Tue, 11 Apr 2017 23:46:30 +0000 (01:46 +0200)]
avformat: Free the internal codec context at the end

Avoid a use after free in avformat_find_stream_info.

(cherry picked from commit 9e4a5eb51b9f3b2bff0ef08e0074b7fe4893075d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/xan: Improve overlapping check
Michael Niedermayer [Mon, 30 Oct 2017 22:21:40 +0000 (23:21 +0100)]
avcodec/xan: Improve overlapping check

Fixes: memcpy-param-overlap
Fixes: 3612/clusterfuzz-testcase-minimized-6393461273001984

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e8fafef1db43ead4eae5a6301ccc300e73aa47da)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/aacdec_fixed: Fix integer overflow in apply_dependent_coupling_fixed()
Michael Niedermayer [Fri, 27 Oct 2017 00:23:21 +0000 (02:23 +0200)]
avcodec/aacdec_fixed: Fix integer overflow in apply_dependent_coupling_fixed()

Fixes: runtime error: signed integer overflow: 623487 * 536870912 cannot be represented in type 'int'
Fixes: 3594/clusterfuzz-testcase-minimized-4650622935629824

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 41d96af2a74cb5df50346b160067facd43149667)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/aacdec_fixed: Fix integer overflow in predict()
Michael Niedermayer [Fri, 27 Oct 2017 00:23:20 +0000 (02:23 +0200)]
avcodec/aacdec_fixed: Fix integer overflow in predict()

Fixes: runtime error: signed integer overflow: -2110708110 + -82837504 cannot be represented in type 'int'
Fixes: 3547/clusterfuzz-testcase-minimized-6009386439802880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0976752420706c0a8b3cb8fd61497a47c7d7270f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/jpeglsdec: Check for end of bitstream in ls_decode_line()
Michael Niedermayer [Wed, 25 Oct 2017 22:02:57 +0000 (00:02 +0200)]
avcodec/jpeglsdec: Check for end of bitstream in ls_decode_line()

Fixes: 1773/clusterfuzz-testcase-minimized-4832523987189760

Fixes: Timeout

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f80224ed19a4c012549fd460d529c7c04e68cf21)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
21 months agoavcodec/jpeglsdec: Check ilv for being a supported value
Michael Niedermayer [Wed, 25 Oct 2017 22:02:56 +0000 (00:02 +0200)]
avcodec/jpeglsdec: Check ilv for being a supported value

Fixes: 1773/clusterfuzz-testcase-minimized-4832523987189760

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fe533628b9604e2f8e5179d5c5dd17c3cb764265)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
22 months agovc2enc_dwt: pad the temporary buffer by the slice size
Rostislav Pehlivanov [Wed, 8 Nov 2017 23:50:04 +0000 (23:50 +0000)]
vc2enc_dwt: pad the temporary buffer by the slice size

Since non-Haar wavelets need to look into pixels outside the frame, we
need to pad the buffer. The old factor of two seemed to be a workaround
that fact and only padded to the left and bottom. This correctly pads
by the slice size and as such reduces memory usage and potential
exploits.
Reported by Liu Bingchang.

Ideally, there should be no temporary buffer but the encoder is designed
to deinterleave the coefficients into the classical wavelet structure
with the lower frequency values in the top left corner.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
(cherry picked from commit 3228ac730c11eca49d5680d5550128e397061c85)

22 months agoUpdate for 3.0.10
Michael Niedermayer [Thu, 26 Oct 2017 16:48:41 +0000 (18:48 +0200)]
Update for 3.0.10

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/snowdec: Check mv_scale
Michael Niedermayer [Fri, 13 Oct 2017 01:06:54 +0000 (03:06 +0200)]
avcodec/snowdec: Check mv_scale

Fixes: runtime error: signed integer overflow: 2 * -1094995530 cannot be represented in type 'int'
Fixes: 3512/clusterfuzz-testcase-minimized-4812747210489856

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 393d6fc7395611a38792e3c271b2be42ac45e672)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/pafvideo: Check for bitstream end in decode_0()
Michael Niedermayer [Fri, 13 Oct 2017 01:06:53 +0000 (03:06 +0200)]
avcodec/pafvideo: Check for bitstream end in decode_0()

Fixes: Timeout
Fixes: 3529/clusterfuzz-testcase-5057068371279872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9c85329cd02e9284892bf263ce6133b2fc479792)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/ffv1dec: Fix out of array read in slice counting
Michael Niedermayer [Mon, 9 Oct 2017 09:49:28 +0000 (11:49 +0200)]
avcodec/ffv1dec: Fix out of array read in slice counting

Fixes: test-201710.mp4

Found-by: 连一汉 <lianyihan@360.cn> and Zhibin Hu
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c20f4fcb74da2d0432c7b54499bb98f48236b904)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/dirac_dwt: Fix integer overflow in COMPOSE_53iL0()
Michael Niedermayer [Sun, 8 Oct 2017 23:46:28 +0000 (01:46 +0200)]
avcodec/dirac_dwt: Fix integer overflow in COMPOSE_53iL0()

Fixes: runtime error: signed integer overflow: 2147483646 + 2 cannot be represented in type 'int'
Fixes: 3485/clusterfuzz-testcase-minimized-4940429332054016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bdee75a4e750735ab3039f004275ac8479072048)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/mpeg_er: Clear mcsel in mpeg_er_decode_mb()
Michael Niedermayer [Sun, 8 Oct 2017 22:32:30 +0000 (00:32 +0200)]
avcodec/mpeg_er: Clear mcsel in mpeg_er_decode_mb()

Fixes out of array read
Should fix: 3516/clusterfuzz-testcase-minimized-4608518562775040 (not reprodoceable)

Found-by: Insu Yun, Georgia Tech.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 127a362630e11fe724e2e63fc871791fdcbcfa64)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/mpeg4videodec: Use 64 bit intermediates for sprite delta
Michael Niedermayer [Sun, 8 Oct 2017 19:41:54 +0000 (21:41 +0200)]
avcodec/mpeg4videodec: Use 64 bit intermediates for sprite delta

Fixes: runtime error: signed integer overflow: -104713 * 65536 cannot be represented in type 'int'
Fixes: 3453/clusterfuzz-testcase-minimized-5555554657239040
Fixes: 3528/clusterfuzz-testcase-minimized-6283628420005888

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e38f280fece38e270a6462a02cc034f4116a7912)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/x86/lossless_videoencdsp: Fix handling of small widths
Michael Niedermayer [Fri, 29 Sep 2017 22:20:09 +0000 (00:20 +0200)]
avcodec/x86/lossless_videoencdsp: Fix handling of small widths

Fixes out of array access
Fixes: crash-huf.avi

Regression since: 6b41b4414934cc930468ccd5db598dd6ef643987

This could also be fixed by adding checks in the C code that calls the dsp

Found-by: Zhibin Hu and 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit df62b70de8aaa285168e72fe8f6e740843ca91fa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/truemotion2: Fix integer overflows in tm2_high_chroma()
Michael Niedermayer [Sat, 30 Sep 2017 16:54:06 +0000 (18:54 +0200)]
avcodec/truemotion2: Fix integer overflows in tm2_high_chroma()

Fixes: runtime error: signed integer overflow: -1408475220 + -1408475220 cannot be represented in type 'int'
Fixes: 3336/clusterfuzz-testcase-minimized-5656839179993088

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 44874b4f5ec2c605c70393573b9d85540ebc2d81)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/aacdec_template: Clear tns present flag on error
Michael Niedermayer [Sat, 30 Sep 2017 16:54:05 +0000 (18:54 +0200)]
avcodec/aacdec_template: Clear tns present flag on error

Fixes: 3444/clusterfuzz-testcase-minimized-6270352105668608

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dcf9bae4a93f54cb5767bc97db4a809efd396f8b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/proresdec2: SKIP_BITS() does not work with len=32
Michael Niedermayer [Mon, 2 Oct 2017 02:18:22 +0000 (04:18 +0200)]
avcodec/proresdec2: SKIP_BITS() does not work with len=32

Fixes: invalid shift
Fixes: 3482/clusterfuzz-testcase-minimized-5446915875405824

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c37138e01a93da2f9dd2cc5d4b77e5a38581d130)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/hevcdsp_template: Fix undefined shift
Michael Niedermayer [Mon, 2 Oct 2017 02:18:21 +0000 (04:18 +0200)]
avcodec/hevcdsp_template: Fix undefined shift

Fixes: runtime error: left shift of negative value -255
Fixes: 3373/clusterfuzz-testcase-minimized-5604083912146944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fbdab6eca7874fbeba6aa79c269f345e4d43f5d4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/jpeg2000: Check that codsty->log2_prec_widths/heights has been initialized
Michael Niedermayer [Mon, 4 Sep 2017 20:23:26 +0000 (22:23 +0200)]
avcodec/jpeg2000: Check that codsty->log2_prec_widths/heights has been initialized

Fixes: OOM
Fixes: 2225/clusterfuzz-testcase-minimized-5505632079708160

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 64e034da954125ef98fb8f9153f9706cdb8a96fe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/takdec: Fix integer overflow in decode_lpc()
Michael Niedermayer [Fri, 22 Sep 2017 18:45:27 +0000 (20:45 +0200)]
avcodec/takdec: Fix integer overflow in decode_lpc()

Fixes: runtime error: signed integer overflow: 16748560 + 2143729712 cannot be represented in type 'int'
Fixes: 3202/clusterfuzz-testcase-minimized-4988291642294272

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5d31f03a0264cac24434c8108daef4ccba6d28f9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/proresdec2: Check bits in DECODE_CODEWORD(), fixes invalid shift
Michael Niedermayer [Fri, 22 Sep 2017 18:45:28 +0000 (20:45 +0200)]
avcodec/proresdec2: Check bits in DECODE_CODEWORD(), fixes invalid shift

Fixes: runtime error: shift exponent 42 is too large for 32-bit type 'unsigned int'
Fixes: 3410/clusterfuzz-testcase-minimized-5313377960198144

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f5eaf0b5956e492ee5023929669b1d09aaf6299)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/takdec: Fix integer overflows in decode_subframe()
Michael Niedermayer [Fri, 22 Sep 2017 18:45:26 +0000 (20:45 +0200)]
avcodec/takdec: Fix integer overflows in decode_subframe()

Fixes: runtime error: signed integer overflow: -1562477869 + -691460395 cannot be represented in type 'int'
Fixes: 3196/clusterfuzz-testcase-minimized-4528307146063872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3dabb9c69db114b1f30c30e0a2788cffc50bac40)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/dirac_dwt: Fix integer overflow in COMPOSE_FIDELITYi*()
Michael Niedermayer [Mon, 18 Sep 2017 00:53:25 +0000 (02:53 +0200)]
avcodec/dirac_dwt: Fix integer overflow in COMPOSE_FIDELITYi*()

Fixes: runtime error: signed integer overflow: 161 * 13872281 cannot be represented in type 'int'

Fixes: 3295/clusterfuzz-testcase-minimized-4738998142500864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 67da2685e03805230207daab83ab43a390fbb887)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/ffv1dec: Fix integer overflow in read_quant_table()
Michael Niedermayer [Mon, 18 Sep 2017 15:26:09 +0000 (17:26 +0200)]
avcodec/ffv1dec: Fix integer overflow in read_quant_table()

Fixes: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 3361/clusterfuzz-testcase-minimized-5065842955911168

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d00fc952b6c261dd8eb0f7552b9ccf985dbc2b20)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/svq3: Fix overflow in svq3_add_idct_c()
Michael Niedermayer [Mon, 18 Sep 2017 15:03:55 +0000 (17:03 +0200)]
avcodec/svq3: Fix overflow in svq3_add_idct_c()

Fixes: runtime error: signed integer overflow: 2147392585 + 524288 cannot be represented in type 'int'
Fixes: 3348/clusterfuzz-testcase-minimized-4809500517203968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2c933c51687db958d8045d25ed87848342e869f6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
23 months agoavcodec/pngdec: Clean up on av_frame_ref() failure
Michael Niedermayer [Sun, 17 Sep 2017 00:42:11 +0000 (02:42 +0200)]
avcodec/pngdec: Clean up on av_frame_ref() failure

Fixes: memleak
Fixes: 3203/clusterfuzz-testcase-minimized-4514553595428864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5480e82d77770e81e897a8c217f3c7f0c13a6de1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevc_ps: Fix c?_qp_offset_list size
Michael Niedermayer [Sun, 10 Sep 2017 19:10:17 +0000 (21:10 +0200)]
avcodec/hevc_ps: Fix c?_qp_offset_list size

Fixes: runtime error: index 5 out of bounds for type 'int8_t const[5]'
Fixes:3175/clusterfuzz-testcase-minimized-4736774054084608

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abf3f9fa232409c00b60041464604a91fa5612c0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/jpeg2000dsp: Fix multiple integer overflows in ict_int()
Michael Niedermayer [Fri, 8 Sep 2017 21:29:12 +0000 (23:29 +0200)]
avcodec/jpeg2000dsp: Fix multiple integer overflows in ict_int()

Fixes: runtime error: signed integer overflow: 22553 * -188962 cannot be represented in type 'int'
Fixes: 3042/clusterfuzz-testcase-minimized-5174210131394560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d025e742843ca3532bd49ebbfebeacd51337347)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/hevcdsp_template: Fix undefined shift in put_hevc_pel_bi_w_pixels
Michael Niedermayer [Sat, 9 Sep 2017 23:32:51 +0000 (01:32 +0200)]
avcodec/hevcdsp_template: Fix undefined shift in put_hevc_pel_bi_w_pixels

Fixes: runtime error: left shift of negative value -95
Fixes: 3077/clusterfuzz-testcase-minimized-4684917524922368

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c225da68cffbea11270a758ff42859194c980863)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/diracdec: Fix overflow in DC computation
Michael Niedermayer [Sat, 9 Sep 2017 23:32:50 +0000 (01:32 +0200)]
avcodec/diracdec: Fix overflow in DC computation

Fixes: runtime error: signed integer overflow: 11896 + 2147483646 cannot be represented in type 'int'
Fixes: 3053/clusterfuzz-testcase-minimized-6355082062856192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b5995856a4236c27f231210bb08d70688e045192)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/asfdec: Fix DoS in asf_build_simple_index()
Michael Niedermayer [Mon, 4 Sep 2017 22:16:29 +0000 (00:16 +0200)]
avformat/asfdec: Fix DoS in asf_build_simple_index()

Fixes: Missing EOF check in loop
No testcase

Found-by: Xiaohei and Wangchu from Alibaba Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit afc9c683ed9db01edb357bc8c19edad4282b3a97)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>