ffmpeg.git
17 months agoavformat/oggparseogm: Check lb against psize
Michael Niedermayer [Fri, 9 Mar 2018 00:05:20 +0000 (01:05 +0100)]
avformat/oggparseogm: Check lb against psize

No testcase, this was found during code review

Found-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3e7c847aaf5a298b62afae12b4ecfb8e12385998)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/oggparseogm: Fix undefined shift in ogm_packet()
Michael Niedermayer [Thu, 8 Mar 2018 22:14:04 +0000 (23:14 +0100)]
avformat/oggparseogm: Fix undefined shift in ogm_packet()

Fixes: shift exponent 48 is too large for 32-bit type 'int'
Fixes: Chromium bug 786793
Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 010b7b30b721b90993e05e9ee6338e88bb8debb3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/avidec: Fix integer overflow in cum_len check
Michael Niedermayer [Thu, 8 Mar 2018 21:40:50 +0000 (22:40 +0100)]
avformat/avidec: Fix integer overflow in cum_len check

Fixes: signed integer overflow: 3775922176 * 4278190080 cannot be represented in type 'long'
Fixes: Chromium bug 791237

Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 06e092e7819b9437da32925200e7c369f93d82e7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/oggparsetheora: Do not adjust AV_NOPTS_VALUE
Michael Niedermayer [Thu, 8 Mar 2018 16:28:36 +0000 (17:28 +0100)]
avformat/oggparsetheora: Do not adjust AV_NOPTS_VALUE

Fixes: Chromium bug 795653
Fixes: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long'
Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 02ecda4aba69670ca744ccc640391b7621f01fb0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/utils: Fix integer overflow of fps_first/last_dts
Michael Niedermayer [Tue, 6 Mar 2018 23:10:11 +0000 (00:10 +0100)]
avformat/utils: Fix integer overflow of fps_first/last_dts

Fixes: runtime error: signed integer overflow: 7738135736989908991 - -7898362169240453118 cannot be represented in type 'long'
Fixes: Chromium bug 796778
Reported-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1b1362e408cd6acb63fef126b814b0d16562aa8e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/oggdec: Fix metadata memleak on multiple headers
Michael Niedermayer [Tue, 6 Mar 2018 17:14:12 +0000 (18:14 +0100)]
avformat/oggdec: Fix metadata memleak on multiple headers

Fixes: Chromium bug 800123
Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit da069e9c68ec1a54e618940dcb9ebae9bf179a32)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agolibavformat/oggparsevorbis: Fix memleak on multiple headers
Michael Niedermayer [Tue, 6 Mar 2018 17:14:12 +0000 (18:14 +0100)]
libavformat/oggparsevorbis: Fix memleak on multiple headers

Fixes: Chromium bug 800123
Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3934aa495d786845d9f541c84ee405c096938f76)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/truemotion2rt: Check input buffer size
Michael Niedermayer [Thu, 22 Feb 2018 02:04:40 +0000 (03:04 +0100)]
avcodec/truemotion2rt: Check input buffer size

Fixes: Timeout
Fixes: 6250/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2RT_fuzzer-5479814011027456

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b5c29b6c2ab00f8fb545475238a99f575b5d81d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/g2meet: Check tile dimensions with av_image_check_size2()
Michael Niedermayer [Thu, 22 Feb 2018 01:34:05 +0000 (02:34 +0100)]
avcodec/g2meet: Check tile dimensions with av_image_check_size2()

Fixes: OOM
Fixes: 6216/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G2M_fuzzer-4983807968018432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3981fb8d2a03cdb3399590da8621a7bcc22e2964)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/exr: fix invalid shift in unpack_14()
Michael Niedermayer [Wed, 21 Feb 2018 03:29:44 +0000 (04:29 +0100)]
avcodec/exr: fix invalid shift in unpack_14()

Fixes: 6154/clusterfuzz-testcase-minimized-5762231061970944
Fixes: runtime error: shift exponent 63 is too large for 32-bit type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 49062a90174b6e4104876c0257dc673a0da854ca)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/bintext: sanity check dimensions
Michael Niedermayer [Mon, 26 Feb 2018 20:17:08 +0000 (21:17 +0100)]
avcodec/bintext: sanity check dimensions

Fixes: Timeout
Fixes: 6277/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XBIN_fuzzer-6047202288861184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 090c0abff9c8b27304614f15d9464dbf4ea59833)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/utvideodec: Check subsample factors
Michael Niedermayer [Mon, 26 Feb 2018 02:02:48 +0000 (03:02 +0100)]
avcodec/utvideodec: Check subsample factors

Fixes: Out of array read
Fixes: heap_poc

Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7414d0bda7763f9bd69c26c068e482ab297c1c96)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/smc: Check input packet size
Michael Niedermayer [Fri, 23 Feb 2018 02:40:02 +0000 (03:40 +0100)]
avcodec/smc: Check input packet size

Fixes: Timeout
Fixes: 6261/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMC_fuzzer-5811309653262336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0293663483ab5dbfff23602a62800d84e021b33c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cavsdec: Check alpha/beta offset
Michael Niedermayer [Tue, 20 Feb 2018 22:11:01 +0000 (23:11 +0100)]
avcodec/cavsdec: Check alpha/beta offset

Fixes: Integer overflow
Fixes: 6183/clusterfuzz-testcase-minimized-6269224436629504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ae2eb04648839bfc6c61c32cb0f124e91bb7ff8e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/diracdec: Fix integer overflow in mv computation
Michael Niedermayer [Sun, 18 Feb 2018 20:51:38 +0000 (21:51 +0100)]
avcodec/diracdec: Fix integer overflow in mv computation

Fixes: signed integer overflow: -2072 + -2147483646 cannot be represented in type 'int'
Fixes: 6097/clusterfuzz-testcase-minimized-5034145253163008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47e65ad63b3d067445c4de41a7718b83fc07767c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264_parse: Clear invalid chroma weights in ff_h264_pred_weight_table()
Michael Niedermayer [Sun, 18 Feb 2018 16:12:28 +0000 (17:12 +0100)]
avcodec/h264_parse: Clear invalid chroma weights in ff_h264_pred_weight_table()

Fixes: 6037/clusterfuzz-testcase-minimized-5030249784934400
Fixes: signed integer overflow: 256 * 16992036 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 85c85fffff3f9c75301db3eba1bd5f2fb1e6285d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/aacdec_templat: Fix integer overflow in apply_ltp()
Michael Niedermayer [Sun, 18 Feb 2018 15:55:52 +0000 (16:55 +0100)]
avcodec/aacdec_templat: Fix integer overflow in apply_ltp()

Fixes: signed integer overflow: -1625276744 + -1041893960 cannot be represented in type 'int'
Fixes: 5948/clusterfuzz-testcase-minimized-5791479856365568

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 33fe17bdc88d51a8e0c87aa1e8011aaaf38a7a90)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/jpeg2000dwt: Fix integer overflows in sr_1d53()
Michael Niedermayer [Sat, 17 Feb 2018 23:11:33 +0000 (00:11 +0100)]
avcodec/jpeg2000dwt: Fix integer overflows in sr_1d53()

Fixes: 5918/clusterfuzz-testcase-minimized-5120505435652096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 793347a54579ee954b58d336b82eed4a1786de21)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/diracdec: Use int64 in global mv to prevent overflow
Michael Niedermayer [Sat, 17 Feb 2018 22:54:44 +0000 (23:54 +0100)]
avcodec/diracdec: Use int64 in global mv to prevent overflow

Fixes: runtime error: signed integer overflow: 361 * -6295541 cannot be represented in type 'int'
Fixes: 5911/clusterfuzz-testcase-minimized-6450382197751808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cbcbefdc3b4cbc917d2f8b2dd216fb12121a838b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dxtory: Remove code that corrupts dimensions
Michael Niedermayer [Sat, 17 Feb 2018 20:27:16 +0000 (21:27 +0100)]
avcodec/dxtory: Remove code that corrupts dimensions

Fixes: Timeout
Fixes: 5796/clusterfuzz-testcase-minimized-5206729085157376

Does someone have a valid sample that triggers this path ?

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3748746a4d6988484d34516f7a3c6febf7bdf488)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dirac_dwt_template: Fix Integer overflow in horizontal_compose_dd137i()
Michael Niedermayer [Sat, 17 Feb 2018 20:47:09 +0000 (21:47 +0100)]
avcodec/dirac_dwt_template: Fix Integer overflow in horizontal_compose_dd137i()

Fixes: 5894/clusterfuzz-testcase-minimized-5315325420634112
Fixes: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 647fa49495c39a48b7ccb92acd8fb975b1575456)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevcdec: Check luma/chroma_log2_weight_denom
Michael Niedermayer [Sat, 17 Feb 2018 20:42:34 +0000 (21:42 +0100)]
avcodec/hevcdec: Check luma/chroma_log2_weight_denom

Fixes: signed integer overflow: 3 + 2147483647 cannot be represented in type 'int'
Fixes: 5888/clusterfuzz-testcase-minimized-5634701067812864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f82dd4c09b2decb033f1e339d4be81efd38554f1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/jpeg2000dec: Use av_image_check_size2()
Michael Niedermayer [Sat, 17 Feb 2018 03:20:53 +0000 (04:20 +0100)]
avcodec/jpeg2000dec: Use av_image_check_size2()

Fixes: OOM
Fixes: 5733/clusterfuzz-testcase-minimized-4906757966004224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 01370b31aced784593f2bc0836f4ba6fd8e7f6b3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vp8: Check for bitstream end before vp7_fade_frame()
Michael Niedermayer [Sat, 17 Feb 2018 03:20:52 +0000 (04:20 +0100)]
avcodec/vp8: Check for bitstream end before vp7_fade_frame()

Fixes: Timeout
Fixes: 5653/clusterfuzz-testcase-5497680018014208

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit de675648cef7e451ca82fabaee0d8ec1fe653311)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/exr: Check remaining bits in last get code loop
Michael Niedermayer [Wed, 14 Feb 2018 12:01:46 +0000 (13:01 +0100)]
avcodec/exr: Check remaining bits in last get code loop

Fixes: runtime error: shift exponent -7 is negative
Fixes: 3902/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-6081926122176512

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dd8351b1184b8054925c28ecc5fcb6dbbc177fad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavutil/common: Fix integer overflow in av_clip_uint8_c() and av_clip_uint16_c()
Michael Niedermayer [Wed, 14 Feb 2018 02:54:13 +0000 (03:54 +0100)]
avutil/common: Fix integer overflow in av_clip_uint8_c() and av_clip_uint16_c()

Fixes: 5567/clusterfuzz-testcase-minimized-5769966247739392
Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ab6f571ef71967da7c7c1cfba483d3597c7357d5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264_cabac: Tighten allowed coeff_abs range
Michael Niedermayer [Tue, 13 Feb 2018 23:32:30 +0000 (00:32 +0100)]
avcodec/h264_cabac: Tighten allowed coeff_abs range

Fixes: integer overflows
Reported-by: "Xiaohan Wang (王消寒)" <xhwang@chromium.org>
Based on limits in "8.5 Transform coefficient decoding process and picture
construction process prior to deblocking  filter process"

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f26a63c4ee1bdbe21d7ab462cd66f8ba20b14244)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264_cavlc: Set valid qscale value in ff_h264_decode_mb_cavlc()
Xiaohan Wang [Sat, 3 Feb 2018 09:43:35 +0000 (01:43 -0800)]
avcodec/h264_cavlc: Set valid qscale value in ff_h264_decode_mb_cavlc()

When ff_h264_decode_mb_cavlc() failed due to wrong sl->qscale values,
e.g. dquant out of range, set the qscale to be a valid value before
returning -1 and exiting the function. The qscale value can be used
later e.g. in loop filter.

BUG=806122

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 71f39de2a57efc8db1d607b09c162c3b806cd45d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vp3: Error out on invalid num_coeffs in unpack_vlcs()
Michael Niedermayer [Sun, 11 Feb 2018 02:38:54 +0000 (03:38 +0100)]
avcodec/vp3: Error out on invalid num_coeffs in unpack_vlcs()

This fixes a hypothetical integer overflow

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f2318aee8ca8df1c84092f7d6691a2d0df02c474)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mpeg4videodec: Ignore multiple VOL headers
Michael Niedermayer [Fri, 9 Feb 2018 21:24:58 +0000 (22:24 +0100)]
avcodec/mpeg4videodec: Ignore multiple VOL headers

Fixes: Ticket7005

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 63a4bdbf3b732504e54cc2b9ec0886e6242a90bc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vp3: Check eob_run
Michael Niedermayer [Fri, 9 Feb 2018 03:17:16 +0000 (04:17 +0100)]
avcodec/vp3: Check eob_run

Fixes: out of array access
Fixes: 5919/clusterfuzz-testcase-minimized-5859311382167552
Fixes: special case for theora (untested due to lack of sample)

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 570023eab3e2962b4ad8345a157c1e18ca1a6eca)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/pafvideo: Check allocated frame size
Michael Niedermayer [Sun, 4 Feb 2018 01:14:49 +0000 (02:14 +0100)]
avcodec/pafvideo: Check allocated frame size

Fixes: OOM
Fixes: 5549/clusterfuzz-testcase-minimized-5390553567985664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 66acb630286cf1bf03bfbdab6c7c784ff20bde61)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/scpr: Fix reading a pixel before the first
Michael Niedermayer [Sat, 3 Feb 2018 17:49:07 +0000 (18:49 +0100)]
avcodec/scpr: Fix reading a pixel before the first

Fixes: 5540/clusterfuzz-testcase-minimized-6122458273808384

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0fb33a82890753233225c61863fff1fcc9d970d4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mpeg2dec: Fix field selection for skipped macroblocks
Nekopanda [Sat, 10 Feb 2018 09:36:32 +0000 (18:36 +0900)]
avcodec/mpeg2dec: Fix field selection for skipped macroblocks

For B field pictures, the spec says,

> The prediction shall be made from the field of the same parity as the field being predicted.

I did it.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b154cb3e90a3e599cadf477d815a9854b7bb4e1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/huffyuvdec: Check input buffer size
Michael Niedermayer [Wed, 31 Jan 2018 18:20:10 +0000 (19:20 +0100)]
avcodec/huffyuvdec: Check input buffer size

Fixes: Timeout
Fixes: 5487/clusterfuzz-testcase-4696837035393024

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 08c220d26cff51ca2f6896b65aebfa3accc67290)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/utvideodec: Fix bytes left check in decode_frame()
Michael Niedermayer [Fri, 2 Feb 2018 20:44:57 +0000 (21:44 +0100)]
avcodec/utvideodec: Fix bytes left check in decode_frame()

Fixes: out of array read
Fixes: poc-2017.avi

Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 118e1b0b3370dd1c0da442901b486689efd1654b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wavpack: Fix integer overflow in FFABS
Michael Niedermayer [Wed, 31 Jan 2018 01:50:18 +0000 (02:50 +0100)]
avcodec/wavpack: Fix integer overflow in FFABS

Fixes: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 5396/clusterfuzz-testcase-minimized-6558555529281536

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e50bd61e4ff97bd7fc6cbd7ec4ca514e17a70c4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/aacsbr_fixed: Fix overflows in rounding in sbr_hf_assemble()
Michael Niedermayer [Wed, 31 Jan 2018 17:13:07 +0000 (18:13 +0100)]
avcodec/aacsbr_fixed: Fix overflows in rounding in sbr_hf_assemble()

Fixes: runtime error: signed integer overflow: 2052929346 + 204817098 cannot be represented in type 'int'
Fixes: 5275/clusterfuzz-testcase-minimized-5367635958038528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b1bef755f617af9685b592d866b3eb7f3c4b02b1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/exr: Fix memleaks in decode_header()
Michael Niedermayer [Wed, 31 Jan 2018 16:50:21 +0000 (17:50 +0100)]
avcodec/exr: Fix memleaks in decode_header()

Fixes: 4793/clusterfuzz-testcase-minimized-5707366629638144

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0a2560a9775be7c5df09c85c9908b05e711a54a3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dirac_dwt: Fix several integer overflows
Michael Niedermayer [Thu, 25 Jan 2018 22:14:37 +0000 (23:14 +0100)]
avcodec/dirac_dwt: Fix several integer overflows

Fixes: runtime error: signed integer overflow: -2146071175 + -268479557 cannot be represented in type 'int'
Fixes: 5237/clusterfuzz-testcase-minimized-4569895275593728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fe1e6c06d03432c3e9208f019533c1d701f485d0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/indeo5: Do not leave frame_type set to an invalid value
Michael Niedermayer [Thu, 25 Jan 2018 23:24:49 +0000 (00:24 +0100)]
avcodec/indeo5: Do not leave frame_type set to an invalid value

Fixes: null pointer dereference
Fixes: 5264/clusterfuzz-testcase-minimized-4621956621008896

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2ff9f178519b68d4d1d606eb5451ad81da948efc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevc_ps: Check log2_sao_offset_scale_*
Michael Niedermayer [Wed, 24 Jan 2018 02:15:23 +0000 (03:15 +0100)]
avcodec/hevc_ps: Check log2_sao_offset_scale_*

Fixes: 4868/clusterfuzz-testcase-minimized-6236542906400768
Fixes: runtime error: shift exponent 126 is too large for 32-bit type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a75a75c62efc645ec28444e4675c325b8f2bb1a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevc_ps: extract SPS fields required for hvcC construction
Aman Gupta [Wed, 27 Sep 2017 01:04:12 +0000 (18:04 -0700)]
avcodec/hevc_ps: extract SPS fields required for hvcC construction

Signed-off-by: Aman Gupta <aman@tmm1.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mpeg4videodec: Avoid possibly aliasing violating casts
Michael Niedermayer [Sun, 28 Jan 2018 01:29:02 +0000 (02:29 +0100)]
avcodec/mpeg4videodec: Avoid possibly aliasing violating casts

Found-by: kierank
Reviewed-by: Kieran Kunhya <kieran618@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d4967c04e040b3b2f937cad88599af825147ec94)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/get_bits: Document the return code of get_vlc2()
Michael Niedermayer [Sun, 28 Jan 2018 01:29:01 +0000 (02:29 +0100)]
avcodec/get_bits: Document the return code of get_vlc2()

Found-by: kierank
Reviewed-by: Kieran Kunhya <kieran618@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a94ff4ccd4f2329c599e37cabe4152dae60359e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mpeg4videodec: Check mb_num also against 0
Michael Niedermayer [Sun, 28 Jan 2018 01:29:00 +0000 (02:29 +0100)]
avcodec/mpeg4videodec: Check mb_num also against 0

The spec implies that 0 is invalid in addition to the existing checks

Found-by: <kierank>
Reviewed-by: Kieran Kunhya <kieran618@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 05f4703a168a336363750e32bcfdd6f303fbdbc3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavfilter/vf_transpose: Fix used plane count.
Michael Niedermayer [Wed, 24 Jan 2018 18:38:05 +0000 (19:38 +0100)]
avfilter/vf_transpose: Fix used plane count.

Fixes out of array access
Fixes: poc.mp4

Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c6939f65a116b1ffed345d29d8621ee4ffb32235)
(cherry picked from commit 3f621455d62e46745453568d915badd5b1e5bcd5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevc_cabac: Check prefix so as to avoid invalid shifts in coeff_abs_level_rem...
Michael Niedermayer [Mon, 15 Jan 2018 22:46:44 +0000 (23:46 +0100)]
avcodec/hevc_cabac: Check prefix so as to avoid invalid shifts in coeff_abs_level_remaining_decode()

I suspect that this can be limited tighter, but i failed to find anything
in the spec that would confirm that.

Fixes: 4833/clusterfuzz-testcase-minimized-5302840101699584
Fixes: runtime error: left shift of 134217730 by 4 places cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a026a3efaeb9c2026668dccbbda339a21ab3206b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mjpegdec: Fix integer overflow in DC dequantization
Michael Niedermayer [Wed, 24 Jan 2018 02:28:49 +0000 (03:28 +0100)]
avcodec/mjpegdec: Fix integer overflow in DC dequantization

Fixes: runtime error: signed integer overflow: -65535 * 65312 cannot be represented in type 'int'
Fixes: 4900/clusterfuzz-testcase-minimized-5769019744321536

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1bfc1aa004950c5ad527d823a08b8a19eef34eb0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dxtory: Fix bits left checks
Michael Niedermayer [Mon, 22 Jan 2018 13:02:59 +0000 (14:02 +0100)]
avcodec/dxtory: Fix bits left checks

Fixes: Timeout
Fixes: 4863/clusterfuzz-testcase-6347354178322432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6e1a167c5564085385488b4f579e9efb987d4bfa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevc_cabac: Move prefix check in coeff_abs_level_remaining_decode() down
Michael Niedermayer [Mon, 15 Jan 2018 22:42:57 +0000 (23:42 +0100)]
avcodec/hevc_cabac: Move prefix check in coeff_abs_level_remaining_decode() down

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 94d4237a7a294ce80e1e577b38e9c93e8882aff9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/truemotion2: Fix integer overflow in TM2_RECALC_BLOCK()
Michael Niedermayer [Sat, 20 Jan 2018 03:10:50 +0000 (04:10 +0100)]
avcodec/truemotion2: Fix integer overflow in TM2_RECALC_BLOCK()

Fixes: signed integer overflow: 1477974040 - -1877995504 cannot be represented in type 'int'
Fixes: 4861/clusterfuzz-testcase-minimized-4570316383715328

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 56a53340ed4cc55898e49c07081311ebb2816630)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/snowdec: Fix integer overflow before htaps check
Michael Niedermayer [Mon, 15 Jan 2018 02:03:36 +0000 (03:03 +0100)]
avcodec/snowdec: Fix integer overflow before htaps check

Fixes: runtime error: signed integer overflow: -1094995529 * 2 cannot be represented in type 'int'
Fixes: 4828/clusterfuzz-testcase-minimized-5100849937252352

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2eecf3cf8eeae67697934df326e98df2149881e5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/ulti: Check number of blocks at init
Michael Niedermayer [Mon, 15 Jan 2018 18:03:48 +0000 (19:03 +0100)]
avcodec/ulti: Check number of blocks at init

Fixes: Timeout
Fixes: 4832/clusterfuzz-testcase-4699096590843904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 725353525e73bbe5b6b4d01528252675f2417a02)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/jpeg2000: Check sum of sizes of band->prec before allocating
Michael Niedermayer [Sat, 13 Jan 2018 23:39:40 +0000 (00:39 +0100)]
avcodec/jpeg2000: Check sum of sizes of band->prec before allocating

Fixes: OOM
Fixes: 4810/clusterfuzz-testcase-minimized-6034253235093504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6887e412434776eb260ad3904f565be491dd5726)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/ac3dec_fixed: Fix integer overflow in scale_coefs()
Michael Niedermayer [Sat, 13 Jan 2018 23:39:39 +0000 (00:39 +0100)]
avcodec/ac3dec_fixed: Fix integer overflow in scale_coefs()

Fixes: runtime error: signed integer overflow: 2147483520 + 128 cannot be represented in type 'int'
Fixes: 4800/clusterfuzz-testcase-minimized-6110372403609600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f38c75893c852cf19dcf3e4553549ba1e70950)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/lrcdec: Fix memory leak in lrc_read_header()
Nikolas Bowe [Fri, 19 Jan 2018 21:17:07 +0000 (13:17 -0800)]
avformat/lrcdec: Fix memory leak in lrc_read_header()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef5994e09d07ace62a672fcdc84761231288edad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/matroskadec: Fix float-cast-overflow undefined behavior in matroska_parse_tr...
Nikolas Bowe [Thu, 18 Jan 2018 23:21:56 +0000 (15:21 -0800)]
avformat/matroskadec: Fix float-cast-overflow undefined behavior in matroska_parse_tracks()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e07649e618caedc07eaf2f4d09253de7f77d14f0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoconfigure: bump year
Carl Eugen Hoyos [Mon, 1 Jan 2018 17:05:55 +0000 (18:05 +0100)]
configure: bump year

Happy new year!

(cherry picked from commit bddf31ba7570325dd2c8d033eae3d0dd74127f96)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/utils: Avoid hardcoding duplicated types in sizeof()
Michael Niedermayer [Sat, 3 Jun 2017 23:53:58 +0000 (01:53 +0200)]
avcodec/utils: Avoid hardcoding duplicated types in sizeof()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 860d991fcd715233b5b9eb1f6c7bf0aadefb6061)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/arm/sbrdsp_neon: Use a free register instead of putting 2 things in one
Michael Niedermayer [Thu, 11 Jan 2018 21:47:10 +0000 (22:47 +0100)]
avcodec/arm/sbrdsp_neon: Use a free register instead of putting 2 things in one

Fixes high pitched shriek
Fixes: 25420848_1478428308873746_4255813235963330560_n.mp4

Reported-by: Dale Curtis <dalecurtis@google.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7dbbb75ee32f87108ca9e15f5551dbbe69fe2641)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264addpx_template: Fixes integer overflows
Michael Niedermayer [Sun, 7 Jan 2018 02:48:43 +0000 (03:48 +0100)]
avcodec/h264addpx_template: Fixes integer overflows

Fixes: signed integer overflow: 512 + 2147483491 cannot be represented in type 'int'
Fixes: 4780/clusterfuzz-testcase-minimized-4709066174627840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d6945aeee419a8417b8019c7c92227e12e45b7ad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dirac_dwt: Fix overflows in COMPOSE_HAARiH0/COMPOSE_HAARiL0
Michael Niedermayer [Sun, 7 Jan 2018 19:58:49 +0000 (20:58 +0100)]
avcodec/dirac_dwt: Fix overflows in COMPOSE_HAARiH0/COMPOSE_HAARiL0

Fixes: 4830/clusterfuzz-testcase-minimized-5255392054476800
Fixes: signed integer overflow: 2147483646 - -7 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0e62a2373475f58c72c0faf5568be00b26909585)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/diracdec: Fix integer overflow with quant
Michael Niedermayer [Sun, 7 Jan 2018 19:43:24 +0000 (20:43 +0100)]
avcodec/diracdec: Fix integer overflow with quant

Fixes: signed integer overflow: 2 + 2147483646 cannot be represented in type 'int'
Fixes: 4792/clusterfuzz-testcase-minimized-6322450775146496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eaa93175895568ef6c2542b13104874907d9c4ef)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/opus_parser: Check payload_len in parse_opus_ts_header()
Michael Niedermayer [Fri, 5 Jan 2018 21:12:07 +0000 (22:12 +0100)]
avcodec/opus_parser: Check payload_len in parse_opus_ts_header()

Fixes: clusterfuzz-testcase-minimized-6134545979277312
Fixes: crbug 797469

Reported-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1bcd7fefcb3c1ec47978fdc64a9e8dfb9512ae62)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/jpeg2000dsp: Fix integer overflows in ict_int()
Michael Niedermayer [Sun, 7 Jan 2018 03:12:57 +0000 (04:12 +0100)]
avcodec/jpeg2000dsp: Fix integer overflows in ict_int()

Fixes: signed integer overflow: 46802 * -71230 cannot be represented in type 'int'
Fixes: 4756/clusterfuzz-testcase-minimized-4812495563784192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b3192c64b5bdcb0474cda437d2d5f9421d68811e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264_slice: Do not attempt to render into frames already output
Michael Niedermayer [Wed, 3 Jan 2018 22:42:01 +0000 (23:42 +0100)]
avcodec/h264_slice: Do not attempt to render into frames already output

Fixes: null pointer dereference
Fixes: 4698/clusterfuzz-testcase-minimized-5096956322906112

This testcase does not reproduce the issue before 03b82b3ab9883cef017e513c7d0b3b986b3b3e7b

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 476665d4de989dba48ec1195215ccc8db54538f4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dnxhddec: Check dc vlc
Michael Niedermayer [Wed, 3 Jan 2018 22:42:00 +0000 (23:42 +0100)]
avcodec/dnxhddec: Check dc vlc

Fixes: signed integer overflow: 1024 + 2147483640 cannot be represented in type 'int'
Fixes: 4671/clusterfuzz-testcase-minimized-6027464343027712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b2be76c0a472b729756ed7a91225c209d0dd1d2e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavformat/hvcc: zero initialize the nal buffers past the last written byte
James Almer [Fri, 23 Feb 2018 03:03:15 +0000 (00:03 -0300)]
avformat/hvcc: zero initialize the nal buffers past the last written byte

Prevents use of uninitialized values.

Fixes ticket #7038.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 9482ec1b203e4cf51d7f60c85d261cc13f9a9d2f)

19 months agoswresample/rematrix: fix update of channel matrix if input or output layout is undefined
Tobias Rapp [Wed, 14 Feb 2018 16:01:08 +0000 (17:01 +0100)]
swresample/rematrix: fix update of channel matrix if input or output layout is undefined

Prefer direct in/out channel count values over channel layout, when
available. Fixes a pan filter bug (ticket #6790).

Signed-off-by: Tobias Rapp <t.rapp@noa-archive.com>
(cherry picked from commit 6325bd3717348615adafb52e4da2fd01a3007d0a)

19 months agoconfigure: add support for libnpp* from cuda sdk 9
Timo Rothenpieler [Tue, 29 Aug 2017 11:30:29 +0000 (13:30 +0200)]
configure: add support for libnpp* from cuda sdk 9

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
19 months agoavcodec/nvenc: also clear data pointer after unregistering a resource
Timo Rothenpieler [Sun, 28 Jan 2018 12:05:09 +0000 (13:05 +0100)]
avcodec/nvenc: also clear data pointer after unregistering a resource

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
19 months agoavcodec/nvenc: add some more error case checks
Timo Rothenpieler [Sun, 28 Jan 2018 11:51:20 +0000 (12:51 +0100)]
avcodec/nvenc: add some more error case checks

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
19 months agoavcodec/nvenc: unregister input resource when unmapping
Timo Rothenpieler [Sun, 28 Jan 2018 11:39:03 +0000 (12:39 +0100)]
avcodec/nvenc: unregister input resource when unmapping

Currently the resource is only ever unregistered when the
registered_frames array is fully in use and an unmapped entry is re-used
and cleaned up.
I'm pretty sure the frame will have been cleaned up before that happens,
so I'm kinda surprised this never blew up.

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
19 months agoavcodec/nvenc: refcount input frame mappings
Timo Rothenpieler [Fri, 26 Jan 2018 19:16:53 +0000 (20:16 +0100)]
avcodec/nvenc: refcount input frame mappings

If some logic like vsync in ffmpeg.c duplicates frames, it might pass
the same frame twice, which will result in a crash due it being
effectively mapped and unmapped twice.

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
20 months agoavformat/libssh: check the user provided a password before trying to use it
James Almer [Sun, 11 Jun 2017 17:17:30 +0000 (14:17 -0300)]
avformat/libssh: check the user provided a password before trying to use it

Fixes ticket #6413

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 8ddb6820bd52df6ed616abc3d8be200b126aa8c1)

20 months agochangelog: update with previous commit n3.3.6
James Almer [Sat, 30 Dec 2017 22:38:23 +0000 (19:38 -0300)]
changelog: update with previous commit

Signed-off-by: James Almer <jamrial@gmail.com>
20 months agox264: Support version 153
Luca Barbato [Tue, 26 Dec 2017 11:32:42 +0000 (12:32 +0100)]
x264: Support version 153

It has native simultaneus 8 and 10 bit support.

(cherry picked from commit c6558e8840fbb2386bf8742e4d68dd6e067d262e)

20 months agoUpdate for 3.3.6
Michael Niedermayer [Sat, 30 Dec 2017 20:13:19 +0000 (21:13 +0100)]
Update for 3.3.6

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/exr: Check buf_size more completely
Michael Niedermayer [Fri, 29 Dec 2017 02:00:19 +0000 (03:00 +0100)]
avcodec/exr: Check buf_size more completely

Fixes: Out of heap array read
Fixes: 4683/clusterfuzz-testcase-minimized-6152313673613312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 903be5e4f66268273dc6e3c42a7fdeaab32066ef)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/flacdec: Fix overflow in multiplication in decode_subframe_fixed()
Michael Niedermayer [Tue, 26 Dec 2017 22:24:44 +0000 (23:24 +0100)]
avcodec/flacdec: Fix overflow in multiplication in decode_subframe_fixed()

Fixes: signed integer overflow: 2 * 1629495328 cannot be represented in type 'int'
Fixes: 4716/clusterfuzz-testcase-minimized-5835915940331520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3d23f7a0969bf76ad6dcdc2c4a5cd3ae884745a8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/hevcdsp_template: Fix Invalid shifts in put_hevc_qpel_bi_w_h() and put_hevc_q...
Michael Niedermayer [Tue, 26 Dec 2017 22:24:45 +0000 (23:24 +0100)]
avcodec/hevcdsp_template: Fix Invalid shifts in put_hevc_qpel_bi_w_h() and put_hevc_qpel_bi_w_w()

Fixes: left shift of negative value -1
Fixes: 4690/clusterfuzz-testcase-minimized-6117482428366848

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d135f3c514ac1723256c8e0f5cdd466fe98a2578)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/flacdec: avoid undefined shift
Michael Niedermayer [Tue, 26 Dec 2017 22:24:43 +0000 (23:24 +0100)]
avcodec/flacdec: avoid undefined shift

Fixes: shift exponent 32 is too large for 32-bit type 'unsigned int'
Fixes: 4688/clusterfuzz-testcase-minimized-6572210748653568

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 560daf88913b0de59a4d845bcd19254b406388dd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/hevcdsp_template.c: Fix undefined shift in FUNC(dequant)
Michael Niedermayer [Fri, 22 Dec 2017 02:12:03 +0000 (03:12 +0100)]
avcodec/hevcdsp_template.c: Fix undefined shift in FUNC(dequant)

Fixes: runtime error: left shift of negative value -180
Fixes: 4626/clusterfuzz-testcase-minimized-5647837887987712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c9ab5ef9c1ee852c80c859c9e07efe8730b57ed)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/dirac_dwt: Fix integer overflow in COMPOSE_DD97iH0() and COMPOSE_DD137iL0()
Michael Niedermayer [Fri, 22 Dec 2017 02:06:14 +0000 (03:06 +0100)]
avcodec/dirac_dwt: Fix integer overflow in COMPOSE_DD97iH0() and COMPOSE_DD137iL0()

Fixes: runtime error: signed integer overflow: 2147483646 + 33554433 cannot be represented in type 'int'
Fixes: 4563/clusterfuzz-testcase-minimized-5438979567517696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4d70fbeec8cbab072b3a9b9f760b8deaaef240f2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/hevc_cabac: Fix integer overflow in ff_hevc_cu_qp_delta_abs()
Michael Niedermayer [Fri, 15 Dec 2017 17:17:13 +0000 (18:17 +0100)]
avcodec/hevc_cabac: Fix integer overflow in ff_hevc_cu_qp_delta_abs()

Fixes: signed integer overflow: 2147483647 + 1073741824 cannot be represented in type 'int'
Fixes: 4555/clusterfuzz-testcase-minimized-4505532481142784

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0ee143558d55b590774dba69cff5a16eda089a4d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agotests/audiomatch: Add missing return code at the end of main()
Michael Niedermayer [Tue, 19 Dec 2017 20:05:40 +0000 (21:05 +0100)]
tests/audiomatch: Add missing return code at the end of main()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 65da5c56e661a839e017db4c51c73d6f3d8a8fcb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/hevc_sei: Fix integer overflows in decode_nal_sei_message()
Michael Niedermayer [Fri, 15 Dec 2017 16:50:12 +0000 (17:50 +0100)]
avcodec/hevc_sei: Fix integer overflows in decode_nal_sei_message()

Fixes: signed integer overflow: 2147483520 + 255 cannot be represented in type 'int'
Fixes: 4554/clusterfuzz-testcase-minimized-4843714515042304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 991ef6e5b9a6a9d95e274ff6bff52db1c82b3808)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/hevcdsp_template: Fix undefined shift in put_hevc_qpel_bi_w_hv()
Michael Niedermayer [Fri, 15 Dec 2017 12:06:30 +0000 (13:06 +0100)]
avcodec/hevcdsp_template: Fix undefined shift in put_hevc_qpel_bi_w_hv()

Fixes: runtime error: left shift of negative value -3
Fixes: 4524/clusterfuzz-testcase-minimized-6055590120914944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 439fbb9c8b2a90e97c44c7c57245e01ca84c865d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agolibavfilter/af_dcshift.c: Fixed repeated spelling error
Kelly Ledford [Tue, 12 Dec 2017 19:31:23 +0000 (11:31 -0800)]
libavfilter/af_dcshift.c: Fixed repeated spelling error

'threshhold' should be 'threshold'

Signed-off-by: Kelly Ledford <kelly.ledford@intel.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc219082bb04b9a4725bfe7e78ce0950244e6e84)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavfilter/formats: fix wrong function name in error message
Jun Zhao [Mon, 4 Dec 2017 04:50:34 +0000 (12:50 +0800)]
avfilter/formats: fix wrong function name in error message

Use perdefined micro __FUNCTION__ rather than hard coding function name
to fix wrong function name in error message.

Signed-off-by: Jun Zhao <jun.zhao@intel.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4280948702bc256e21c375790b889c735d233b0d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/amrwbdec: Fix division by 0 in voice_factor()
Michael Niedermayer [Thu, 7 Dec 2017 14:32:54 +0000 (15:32 +0100)]
avcodec/amrwbdec: Fix division by 0 in voice_factor()

The added value matches "Digital cellular telecommunications system (Phase 2+) (GSM); Universal Mobile Telecommunications System (UMTS); LTE; Extended Adaptive Multi-Rate - Wideband (AMR-WB+) codec; Floating-point ANSI-C code (3GPP TS 26.304 version 14.0.0 Release 14)
Extended Adaptive Multi-Rate - Wideband (AMR-WB+) codec; Floating-point ANSI-C code"

Fixes: runtime error: division by zero
Fixes: 4415/clusterfuzz-testcase-minimized-4677752314658816

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1d0817d56b66797118880358ea7d7a2acfdca429)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/diracdsp: Fix integer overflow in PUT_SIGNED_RECT_CLAMPED()
Michael Niedermayer [Sat, 2 Dec 2017 20:53:22 +0000 (21:53 +0100)]
avcodec/diracdsp: Fix integer overflow in PUT_SIGNED_RECT_CLAMPED()

Fixes: runtime error: signed integer overflow: 2147483646 + 2048 cannot be represented in type 'int'
Fixes: 4479/clusterfuzz-testcase-minimized-6529894147162112

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 610dd74502a58e8bb0f1d8fcbc7015f86b78d70e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/dirac_dwt: Fix integer overflows in COMPOSE_DAUB97*
Michael Niedermayer [Sat, 2 Dec 2017 20:48:04 +0000 (21:48 +0100)]
avcodec/dirac_dwt: Fix integer overflows in COMPOSE_DAUB97*

Fixes: 4478/clusterfuzz-testcase-minimized-4752113767809024
Fixes: runtime error: signed integer overflow: -2147483626 + -319489 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e9a13a5a33bf7566591216e335f2529612100bb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/extract_extradata_bsf: Fix leak discovered via fuzzing
Nikolas Bowe [Tue, 5 Dec 2017 23:11:26 +0000 (15:11 -0800)]
avcodec/extract_extradata_bsf: Fix leak discovered via fuzzing

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5a412a5c3cc216ae1d15e6b884bda7214b73a5b0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/vorbis: Fix another 1 << 31 > int32_t::max() with 1u.
Dale Curtis [Thu, 30 Nov 2017 20:20:36 +0000 (12:20 -0800)]
avcodec/vorbis: Fix another 1 << 31 > int32_t::max() with 1u.

Didn't notice this one when 9648cc6d was landed.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 95bacb521af8cd28f146f045437c9f75717a493a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoDon't manipulate duration when it's AV_NOPTS_VALUE.
Dale Curtis [Tue, 28 Nov 2017 22:26:55 +0000 (14:26 -0800)]
Don't manipulate duration when it's AV_NOPTS_VALUE.

This leads to signed integer overflow.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit c5fd57f483d2ad8e34551b78509f1e14136f73c0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/vorbis: 1 << 31 > int32_t::max(), so use 1u << 31 instead.
Dale Curtis [Wed, 22 Nov 2017 18:58:39 +0000 (10:58 -0800)]
avcodec/vorbis: 1 << 31 > int32_t::max(), so use 1u << 31 instead.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9648cc6d7fdbb0a260bed1e3e23300569cff9579)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavformat/utils: Prevent undefined shift with wrap_bits > 64.
Dale Curtis [Fri, 17 Nov 2017 21:35:56 +0000 (13:35 -0800)]
avformat/utils: Prevent undefined shift with wrap_bits > 64.

2LL << (wrap_bits=64 - 1) does not fit in int64_t; change the
code to use a uint64_t (2ULL) and add an av_assert2() to
ensure wrap_bits <= 64.

Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03fbc0daa7e37af024f8b017a28105c32bbe25ca)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/j2kenc: Fix out of array access in encode_cblk()
Michael Niedermayer [Thu, 30 Nov 2017 22:42:04 +0000 (23:42 +0100)]
avcodec/j2kenc: Fix out of array access in encode_cblk()

Fixes: 4427/clusterfuzz-testcase-minimized-5106919271301120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0674087004538599797688785f6ac82358abc23b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>