ffmpeg.git
17 months agolavc/libopusdec: Allow avcodec_open2 to call .close
Matt Wolenetz [Tue, 10 Apr 2018 20:59:25 +0000 (13:59 -0700)]
lavc/libopusdec: Allow avcodec_open2 to call .close

If there is a decoder initialization failure detected in avcodec_open2
after .init is called, allow graceful decoder .close to prevent leaking
libopus decoder allocations.

BUG=828526

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e43e97f0e0f0596b56ceb2f887fe7414f202f081)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/movtextdec: Check style_start/end
Michael Niedermayer [Sun, 8 Apr 2018 01:29:44 +0000 (03:29 +0200)]
avcodec/movtextdec: Check style_start/end

Limits based on 3GPP TS 26.245 V14.0.0
Fixes: Timeout
Fixes: 6377/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOVTEXT_fuzzer-5175929115508736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 249aca8f98ff7fb09c12ea68e23c862c62203b95)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/aacsbr_fixed: Fix integer overflow in sbr_hf_assemble()
Michael Niedermayer [Sat, 7 Apr 2018 19:55:06 +0000 (21:55 +0200)]
avcodec/aacsbr_fixed: Fix integer overflow in sbr_hf_assemble()

Fixes: runtime error: signed integer overflow: 2052929346 + 204817098 cannot be represented in type 'int'

This was missed in b1bef755f617af9685b592d866b3eb7f3c4b02b1
Fixes: 5275/clusterfuzz-testcase-minimized-5367635958038528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c837918f50a7bbd6150afd340857ea43fe4717c7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agolibavcodec/rv34: error out earlier on missing references
Michael Niedermayer [Mon, 2 Apr 2018 18:01:07 +0000 (20:01 +0200)]
libavcodec/rv34: error out earlier on missing references

Fixes visual corruption on seeking

Fixes: downloadTest_clip_24M.rmvb

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6cd81d68c5e4b0ff00288970c4151ff4031c0ea9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoswresample/swresample: Fix for seg fault in swr_convert_internal() -> sum2_float...
Hendrik Schreiber [Thu, 5 Apr 2018 11:58:37 +0000 (13:58 +0200)]
swresample/swresample: Fix for seg fault in swr_convert_internal() -> sum2_float during dithering.

Removed +len1 in call to s->mix_2_1_f() as I found no logical explanation for it. After removal, problem was gone.

Signed-off-by: Hendrik Schreiber <hs@tagtraum.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 647fd4b8292e3bfae30b1086aa842a5ee47ee868)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/aacdec_fixed: Fix integer overflow in apply_independent_coupling_fixed()
Michael Niedermayer [Sat, 31 Mar 2018 19:19:19 +0000 (21:19 +0200)]
avcodec/aacdec_fixed: Fix integer overflow in apply_independent_coupling_fixed()

I was not able to reproduce this, this fix is based on just the fuzzer log.
Fixes: 4959/clusterfuzz-testcase-minimized-6035350934781952

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 197a4e8feed45b2e5868760240e83636818f32a9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cscd: Error out when LZ* decompression fails
Michael Niedermayer [Sun, 11 Mar 2018 23:05:04 +0000 (00:05 +0100)]
avcodec/cscd: Error out when LZ* decompression fails

Fixes: Timeout
Fixes: 6304/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CSCD_fuzzer-5754772461191168

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d52be5d4e91871a22dac70af3e0ab429e95a2d10)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/imgconvert: Fix loss mask bug in avcodec_find_best_pix_fmt_of_list()
heimdallr [Sat, 31 Mar 2018 12:37:23 +0000 (19:37 +0700)]
avcodec/imgconvert: Fix loss mask bug in avcodec_find_best_pix_fmt_of_list()

example:

AVPixelFormat pixFmts[] = { AV_PIX_FMT_RGB24, AV_PIX_FMT_RGBA };
int loss = 0;
AVPixelFormat best = avcodec_find_best_pix_fmt_of_list(pixFmts, AV_PIX_FMT_BGRA, 1, &loss);

best is AV_PIX_FMT_RGB24. But AV_PIX_FMT_RGBA is better.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 354b26a3945eadd4ed8fcd801dfefad2566241de)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavfilter/vf_signature: use av_strlcpy()
Michael Niedermayer [Fri, 30 Mar 2018 00:16:31 +0000 (02:16 +0200)]
avfilter/vf_signature: use av_strlcpy()

Fixes: out of array access

Found-by: Kira <kira_cxy@foxmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 35eeff30caf34df835206f1c12bcf4b7c2bd6758)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/utvideodec: Set pro flag based on fourcc
Michael Niedermayer [Sat, 31 Mar 2018 01:10:43 +0000 (03:10 +0200)]
avcodec/utvideodec: Set pro flag based on fourcc

This avoids mixing 8bit variants with pro and 10bit with non pro mode.
Fixes: out of array read
Fixes: poc_03_30.avi

Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47b7c68ae54560e2308bdb6be4fb076c73b93081)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmalosslessdec: Fix null pointer dereference in decode_frame()
Michael Niedermayer [Sun, 25 Mar 2018 00:51:28 +0000 (01:51 +0100)]
avcodec/wmalosslessdec: Fix null pointer dereference in decode_frame()

Fixes: 2018_03_23_poc.wav
Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ea15915b2dc5aaa80c91879fbd183475a7e66e54)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/tableprint_vlc: Fix build failure with --enable-hardcoded-tables
Michael Niedermayer [Wed, 28 Mar 2018 23:07:24 +0000 (01:07 +0200)]
avcodec/tableprint_vlc: Fix build failure with --enable-hardcoded-tables

Found-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5c75438b893539dd17998c489fb4c540fc5a6e48)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/mov: Move +1 in check to avoid hypothetical overflow in add_ctts_entry()
Michael Niedermayer [Sat, 3 Feb 2018 20:36:22 +0000 (21:36 +0100)]
avformat/mov: Move +1 in check to avoid hypothetical overflow in add_ctts_entry()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eb60b9d3aaaa42265fb1960be6fff6383cfdbf37)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/get_bits: Make sure the input bitstream with padding can be addressed
Michael Niedermayer [Sat, 24 Mar 2018 00:38:53 +0000 (01:38 +0100)]
avcodec/get_bits: Make sure the input bitstream with padding can be addressed

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e529fe7633762cb26a665fb6dee3be29b15285cc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/mov: Check STSC and remove invalid entries
Michael Niedermayer [Fri, 16 Mar 2018 18:53:36 +0000 (19:53 +0100)]
avformat/mov: Check STSC and remove invalid entries

Fixes assertion failure
Fixes: crbug 822547, crbug 822666 and crbug 823009

Affects: aark15sd_9A62E2FA.mp4

Found-by: ClusterFuzz
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9e67447a4ffacf28af8bace33faf3ea432ddc43e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/nuv: rtjpeg with dimensions less than 16 would result in no decoded pixels...
Michael Niedermayer [Tue, 27 Feb 2018 14:17:12 +0000 (15:17 +0100)]
avcodec/nuv: rtjpeg with dimensions less than 16 would result in no decoded pixels thus reject it

Fixes: Timeout
Fixes: 6297/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NUV_fuzzer-4882404863901696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 939440ad1aa820bed51f54d273b4fa6c5016d9f9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/nuv: Check for minimum input size for uncomprssed and rtjpeg
Michael Niedermayer [Tue, 27 Feb 2018 14:17:12 +0000 (15:17 +0100)]
avcodec/nuv: Check for minimum input size for uncomprssed and rtjpeg

Fixes: Timeout
Fixes: 6297/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NUV_fuzzer-4882404863901696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8ee3265dbe2e85537affe3b3055b00ba8646aa70)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmalosslessdec: Reset num_saved_bits on error path
Michael Niedermayer [Sat, 10 Mar 2018 23:13:57 +0000 (00:13 +0100)]
avcodec/wmalosslessdec: Reset num_saved_bits on error path

Fixes: NULL pointer dereference
Fixes: poc-201803.wav
Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 64c9ce0abc0fd8774b523afda3ddb17c86caa86a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/mov: Fix integer overflows related to sample_duration
Michael Niedermayer [Fri, 9 Mar 2018 15:43:29 +0000 (16:43 +0100)]
avformat/mov: Fix integer overflows related to sample_duration

Fixes: runtime error: signed integer overflow: -9166684017437101870 + -2495066639299164439 cannot be represented in type

Fixes: Chromium bug 791349

Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2f37082827a405430c40408ee2db19ea2866ce64)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/oggparsedaala: Do not adjust AV_NOPTS_VALUE
Michael Niedermayer [Thu, 8 Mar 2018 16:28:36 +0000 (17:28 +0100)]
avformat/oggparsedaala: Do not adjust AV_NOPTS_VALUE

Fixes: potential signed integer overflow

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f655ddfb47e8484b205b14c7f871c643ad24d701)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/oggparseogm: Check lb against psize
Michael Niedermayer [Fri, 9 Mar 2018 00:05:20 +0000 (01:05 +0100)]
avformat/oggparseogm: Check lb against psize

No testcase, this was found during code review

Found-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3e7c847aaf5a298b62afae12b4ecfb8e12385998)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/oggparseogm: Fix undefined shift in ogm_packet()
Michael Niedermayer [Thu, 8 Mar 2018 22:14:04 +0000 (23:14 +0100)]
avformat/oggparseogm: Fix undefined shift in ogm_packet()

Fixes: shift exponent 48 is too large for 32-bit type 'int'
Fixes: Chromium bug 786793
Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 010b7b30b721b90993e05e9ee6338e88bb8debb3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/avidec: Fix integer overflow in cum_len check
Michael Niedermayer [Thu, 8 Mar 2018 21:40:50 +0000 (22:40 +0100)]
avformat/avidec: Fix integer overflow in cum_len check

Fixes: signed integer overflow: 3775922176 * 4278190080 cannot be represented in type 'long'
Fixes: Chromium bug 791237

Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 06e092e7819b9437da32925200e7c369f93d82e7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/oggparsetheora: Do not adjust AV_NOPTS_VALUE
Michael Niedermayer [Thu, 8 Mar 2018 16:28:36 +0000 (17:28 +0100)]
avformat/oggparsetheora: Do not adjust AV_NOPTS_VALUE

Fixes: Chromium bug 795653
Fixes: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long'
Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 02ecda4aba69670ca744ccc640391b7621f01fb0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/utils: Fix integer overflow of fps_first/last_dts
Michael Niedermayer [Tue, 6 Mar 2018 23:10:11 +0000 (00:10 +0100)]
avformat/utils: Fix integer overflow of fps_first/last_dts

Fixes: runtime error: signed integer overflow: 7738135736989908991 - -7898362169240453118 cannot be represented in type 'long'
Fixes: Chromium bug 796778
Reported-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1b1362e408cd6acb63fef126b814b0d16562aa8e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/oggdec: Fix metadata memleak on multiple headers
Michael Niedermayer [Tue, 6 Mar 2018 17:14:12 +0000 (18:14 +0100)]
avformat/oggdec: Fix metadata memleak on multiple headers

Fixes: Chromium bug 800123
Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit da069e9c68ec1a54e618940dcb9ebae9bf179a32)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agolibavformat/oggparsevorbis: Fix memleak on multiple headers
Michael Niedermayer [Tue, 6 Mar 2018 17:14:12 +0000 (18:14 +0100)]
libavformat/oggparsevorbis: Fix memleak on multiple headers

Fixes: Chromium bug 800123
Reported-by: Matt Wolenetz <wolenetz@google.com>
Reviewed-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3934aa495d786845d9f541c84ee405c096938f76)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/truemotion2rt: Check input buffer size
Michael Niedermayer [Thu, 22 Feb 2018 02:04:40 +0000 (03:04 +0100)]
avcodec/truemotion2rt: Check input buffer size

Fixes: Timeout
Fixes: 6250/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2RT_fuzzer-5479814011027456

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b5c29b6c2ab00f8fb545475238a99f575b5d81d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/g2meet: Check tile dimensions with av_image_check_size2()
Michael Niedermayer [Thu, 22 Feb 2018 01:34:05 +0000 (02:34 +0100)]
avcodec/g2meet: Check tile dimensions with av_image_check_size2()

Fixes: OOM
Fixes: 6216/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G2M_fuzzer-4983807968018432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3981fb8d2a03cdb3399590da8621a7bcc22e2964)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/exr: fix invalid shift in unpack_14()
Michael Niedermayer [Wed, 21 Feb 2018 03:29:44 +0000 (04:29 +0100)]
avcodec/exr: fix invalid shift in unpack_14()

Fixes: 6154/clusterfuzz-testcase-minimized-5762231061970944
Fixes: runtime error: shift exponent 63 is too large for 32-bit type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 49062a90174b6e4104876c0257dc673a0da854ca)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/bintext: sanity check dimensions
Michael Niedermayer [Mon, 26 Feb 2018 20:17:08 +0000 (21:17 +0100)]
avcodec/bintext: sanity check dimensions

Fixes: Timeout
Fixes: 6277/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XBIN_fuzzer-6047202288861184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 090c0abff9c8b27304614f15d9464dbf4ea59833)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/utvideodec: Check subsample factors
Michael Niedermayer [Mon, 26 Feb 2018 02:02:48 +0000 (03:02 +0100)]
avcodec/utvideodec: Check subsample factors

Fixes: Out of array read
Fixes: heap_poc

Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7414d0bda7763f9bd69c26c068e482ab297c1c96)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/smc: Check input packet size
Michael Niedermayer [Fri, 23 Feb 2018 02:40:02 +0000 (03:40 +0100)]
avcodec/smc: Check input packet size

Fixes: Timeout
Fixes: 6261/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SMC_fuzzer-5811309653262336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0293663483ab5dbfff23602a62800d84e021b33c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cavsdec: Check alpha/beta offset
Michael Niedermayer [Tue, 20 Feb 2018 22:11:01 +0000 (23:11 +0100)]
avcodec/cavsdec: Check alpha/beta offset

Fixes: Integer overflow
Fixes: 6183/clusterfuzz-testcase-minimized-6269224436629504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ae2eb04648839bfc6c61c32cb0f124e91bb7ff8e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/diracdec: Fix integer overflow in mv computation
Michael Niedermayer [Sun, 18 Feb 2018 20:51:38 +0000 (21:51 +0100)]
avcodec/diracdec: Fix integer overflow in mv computation

Fixes: signed integer overflow: -2072 + -2147483646 cannot be represented in type 'int'
Fixes: 6097/clusterfuzz-testcase-minimized-5034145253163008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47e65ad63b3d067445c4de41a7718b83fc07767c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264_parse: Clear invalid chroma weights in ff_h264_pred_weight_table()
Michael Niedermayer [Sun, 18 Feb 2018 16:12:28 +0000 (17:12 +0100)]
avcodec/h264_parse: Clear invalid chroma weights in ff_h264_pred_weight_table()

Fixes: 6037/clusterfuzz-testcase-minimized-5030249784934400
Fixes: signed integer overflow: 256 * 16992036 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 85c85fffff3f9c75301db3eba1bd5f2fb1e6285d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/aacdec_templat: Fix integer overflow in apply_ltp()
Michael Niedermayer [Sun, 18 Feb 2018 15:55:52 +0000 (16:55 +0100)]
avcodec/aacdec_templat: Fix integer overflow in apply_ltp()

Fixes: signed integer overflow: -1625276744 + -1041893960 cannot be represented in type 'int'
Fixes: 5948/clusterfuzz-testcase-minimized-5791479856365568

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 33fe17bdc88d51a8e0c87aa1e8011aaaf38a7a90)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/jpeg2000dwt: Fix integer overflows in sr_1d53()
Michael Niedermayer [Sat, 17 Feb 2018 23:11:33 +0000 (00:11 +0100)]
avcodec/jpeg2000dwt: Fix integer overflows in sr_1d53()

Fixes: 5918/clusterfuzz-testcase-minimized-5120505435652096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 793347a54579ee954b58d336b82eed4a1786de21)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/diracdec: Use int64 in global mv to prevent overflow
Michael Niedermayer [Sat, 17 Feb 2018 22:54:44 +0000 (23:54 +0100)]
avcodec/diracdec: Use int64 in global mv to prevent overflow

Fixes: runtime error: signed integer overflow: 361 * -6295541 cannot be represented in type 'int'
Fixes: 5911/clusterfuzz-testcase-minimized-6450382197751808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cbcbefdc3b4cbc917d2f8b2dd216fb12121a838b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dxtory: Remove code that corrupts dimensions
Michael Niedermayer [Sat, 17 Feb 2018 20:27:16 +0000 (21:27 +0100)]
avcodec/dxtory: Remove code that corrupts dimensions

Fixes: Timeout
Fixes: 5796/clusterfuzz-testcase-minimized-5206729085157376

Does someone have a valid sample that triggers this path ?

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3748746a4d6988484d34516f7a3c6febf7bdf488)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dirac_dwt_template: Fix Integer overflow in horizontal_compose_dd137i()
Michael Niedermayer [Sat, 17 Feb 2018 20:47:09 +0000 (21:47 +0100)]
avcodec/dirac_dwt_template: Fix Integer overflow in horizontal_compose_dd137i()

Fixes: 5894/clusterfuzz-testcase-minimized-5315325420634112
Fixes: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 647fa49495c39a48b7ccb92acd8fb975b1575456)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevcdec: Check luma/chroma_log2_weight_denom
Michael Niedermayer [Sat, 17 Feb 2018 20:42:34 +0000 (21:42 +0100)]
avcodec/hevcdec: Check luma/chroma_log2_weight_denom

Fixes: signed integer overflow: 3 + 2147483647 cannot be represented in type 'int'
Fixes: 5888/clusterfuzz-testcase-minimized-5634701067812864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f82dd4c09b2decb033f1e339d4be81efd38554f1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/jpeg2000dec: Use av_image_check_size2()
Michael Niedermayer [Sat, 17 Feb 2018 03:20:53 +0000 (04:20 +0100)]
avcodec/jpeg2000dec: Use av_image_check_size2()

Fixes: OOM
Fixes: 5733/clusterfuzz-testcase-minimized-4906757966004224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 01370b31aced784593f2bc0836f4ba6fd8e7f6b3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vp8: Check for bitstream end before vp7_fade_frame()
Michael Niedermayer [Sat, 17 Feb 2018 03:20:52 +0000 (04:20 +0100)]
avcodec/vp8: Check for bitstream end before vp7_fade_frame()

Fixes: Timeout
Fixes: 5653/clusterfuzz-testcase-5497680018014208

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit de675648cef7e451ca82fabaee0d8ec1fe653311)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/exr: Check remaining bits in last get code loop
Michael Niedermayer [Wed, 14 Feb 2018 12:01:46 +0000 (13:01 +0100)]
avcodec/exr: Check remaining bits in last get code loop

Fixes: runtime error: shift exponent -7 is negative
Fixes: 3902/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-6081926122176512

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dd8351b1184b8054925c28ecc5fcb6dbbc177fad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavutil/common: Fix integer overflow in av_clip_uint8_c() and av_clip_uint16_c()
Michael Niedermayer [Wed, 14 Feb 2018 02:54:13 +0000 (03:54 +0100)]
avutil/common: Fix integer overflow in av_clip_uint8_c() and av_clip_uint16_c()

Fixes: 5567/clusterfuzz-testcase-minimized-5769966247739392
Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ab6f571ef71967da7c7c1cfba483d3597c7357d5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264_cabac: Tighten allowed coeff_abs range
Michael Niedermayer [Tue, 13 Feb 2018 23:32:30 +0000 (00:32 +0100)]
avcodec/h264_cabac: Tighten allowed coeff_abs range

Fixes: integer overflows
Reported-by: "Xiaohan Wang (王消寒)" <xhwang@chromium.org>
Based on limits in "8.5 Transform coefficient decoding process and picture
construction process prior to deblocking  filter process"

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f26a63c4ee1bdbe21d7ab462cd66f8ba20b14244)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264_cavlc: Set valid qscale value in ff_h264_decode_mb_cavlc()
Xiaohan Wang [Sat, 3 Feb 2018 09:43:35 +0000 (01:43 -0800)]
avcodec/h264_cavlc: Set valid qscale value in ff_h264_decode_mb_cavlc()

When ff_h264_decode_mb_cavlc() failed due to wrong sl->qscale values,
e.g. dquant out of range, set the qscale to be a valid value before
returning -1 and exiting the function. The qscale value can be used
later e.g. in loop filter.

BUG=806122

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 71f39de2a57efc8db1d607b09c162c3b806cd45d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vp3: Error out on invalid num_coeffs in unpack_vlcs()
Michael Niedermayer [Sun, 11 Feb 2018 02:38:54 +0000 (03:38 +0100)]
avcodec/vp3: Error out on invalid num_coeffs in unpack_vlcs()

This fixes a hypothetical integer overflow

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f2318aee8ca8df1c84092f7d6691a2d0df02c474)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mpeg4videodec: Ignore multiple VOL headers
Michael Niedermayer [Fri, 9 Feb 2018 21:24:58 +0000 (22:24 +0100)]
avcodec/mpeg4videodec: Ignore multiple VOL headers

Fixes: Ticket7005

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 63a4bdbf3b732504e54cc2b9ec0886e6242a90bc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vp3: Check eob_run
Michael Niedermayer [Fri, 9 Feb 2018 03:17:16 +0000 (04:17 +0100)]
avcodec/vp3: Check eob_run

Fixes: out of array access
Fixes: 5919/clusterfuzz-testcase-minimized-5859311382167552
Fixes: special case for theora (untested due to lack of sample)

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 570023eab3e2962b4ad8345a157c1e18ca1a6eca)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/pafvideo: Check allocated frame size
Michael Niedermayer [Sun, 4 Feb 2018 01:14:49 +0000 (02:14 +0100)]
avcodec/pafvideo: Check allocated frame size

Fixes: OOM
Fixes: 5549/clusterfuzz-testcase-minimized-5390553567985664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 66acb630286cf1bf03bfbdab6c7c784ff20bde61)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/scpr: Fix reading a pixel before the first
Michael Niedermayer [Sat, 3 Feb 2018 17:49:07 +0000 (18:49 +0100)]
avcodec/scpr: Fix reading a pixel before the first

Fixes: 5540/clusterfuzz-testcase-minimized-6122458273808384

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0fb33a82890753233225c61863fff1fcc9d970d4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mpeg2dec: Fix field selection for skipped macroblocks
Nekopanda [Sat, 10 Feb 2018 09:36:32 +0000 (18:36 +0900)]
avcodec/mpeg2dec: Fix field selection for skipped macroblocks

For B field pictures, the spec says,

> The prediction shall be made from the field of the same parity as the field being predicted.

I did it.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b154cb3e90a3e599cadf477d815a9854b7bb4e1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/huffyuvdec: Check input buffer size
Michael Niedermayer [Wed, 31 Jan 2018 18:20:10 +0000 (19:20 +0100)]
avcodec/huffyuvdec: Check input buffer size

Fixes: Timeout
Fixes: 5487/clusterfuzz-testcase-4696837035393024

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 08c220d26cff51ca2f6896b65aebfa3accc67290)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/utvideodec: Fix bytes left check in decode_frame()
Michael Niedermayer [Fri, 2 Feb 2018 20:44:57 +0000 (21:44 +0100)]
avcodec/utvideodec: Fix bytes left check in decode_frame()

Fixes: out of array read
Fixes: poc-2017.avi

Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 118e1b0b3370dd1c0da442901b486689efd1654b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wavpack: Fix integer overflow in FFABS
Michael Niedermayer [Wed, 31 Jan 2018 01:50:18 +0000 (02:50 +0100)]
avcodec/wavpack: Fix integer overflow in FFABS

Fixes: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 5396/clusterfuzz-testcase-minimized-6558555529281536

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e50bd61e4ff97bd7fc6cbd7ec4ca514e17a70c4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/aacsbr_fixed: Fix overflows in rounding in sbr_hf_assemble()
Michael Niedermayer [Wed, 31 Jan 2018 17:13:07 +0000 (18:13 +0100)]
avcodec/aacsbr_fixed: Fix overflows in rounding in sbr_hf_assemble()

Fixes: runtime error: signed integer overflow: 2052929346 + 204817098 cannot be represented in type 'int'
Fixes: 5275/clusterfuzz-testcase-minimized-5367635958038528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b1bef755f617af9685b592d866b3eb7f3c4b02b1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/exr: Fix memleaks in decode_header()
Michael Niedermayer [Wed, 31 Jan 2018 16:50:21 +0000 (17:50 +0100)]
avcodec/exr: Fix memleaks in decode_header()

Fixes: 4793/clusterfuzz-testcase-minimized-5707366629638144

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0a2560a9775be7c5df09c85c9908b05e711a54a3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dirac_dwt: Fix several integer overflows
Michael Niedermayer [Thu, 25 Jan 2018 22:14:37 +0000 (23:14 +0100)]
avcodec/dirac_dwt: Fix several integer overflows

Fixes: runtime error: signed integer overflow: -2146071175 + -268479557 cannot be represented in type 'int'
Fixes: 5237/clusterfuzz-testcase-minimized-4569895275593728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fe1e6c06d03432c3e9208f019533c1d701f485d0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/indeo5: Do not leave frame_type set to an invalid value
Michael Niedermayer [Thu, 25 Jan 2018 23:24:49 +0000 (00:24 +0100)]
avcodec/indeo5: Do not leave frame_type set to an invalid value

Fixes: null pointer dereference
Fixes: 5264/clusterfuzz-testcase-minimized-4621956621008896

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2ff9f178519b68d4d1d606eb5451ad81da948efc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevc_ps: Check log2_sao_offset_scale_*
Michael Niedermayer [Wed, 24 Jan 2018 02:15:23 +0000 (03:15 +0100)]
avcodec/hevc_ps: Check log2_sao_offset_scale_*

Fixes: 4868/clusterfuzz-testcase-minimized-6236542906400768
Fixes: runtime error: shift exponent 126 is too large for 32-bit type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a75a75c62efc645ec28444e4675c325b8f2bb1a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevc_ps: extract SPS fields required for hvcC construction
Aman Gupta [Wed, 27 Sep 2017 01:04:12 +0000 (18:04 -0700)]
avcodec/hevc_ps: extract SPS fields required for hvcC construction

Signed-off-by: Aman Gupta <aman@tmm1.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mpeg4videodec: Avoid possibly aliasing violating casts
Michael Niedermayer [Sun, 28 Jan 2018 01:29:02 +0000 (02:29 +0100)]
avcodec/mpeg4videodec: Avoid possibly aliasing violating casts

Found-by: kierank
Reviewed-by: Kieran Kunhya <kieran618@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d4967c04e040b3b2f937cad88599af825147ec94)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/get_bits: Document the return code of get_vlc2()
Michael Niedermayer [Sun, 28 Jan 2018 01:29:01 +0000 (02:29 +0100)]
avcodec/get_bits: Document the return code of get_vlc2()

Found-by: kierank
Reviewed-by: Kieran Kunhya <kieran618@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a94ff4ccd4f2329c599e37cabe4152dae60359e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mpeg4videodec: Check mb_num also against 0
Michael Niedermayer [Sun, 28 Jan 2018 01:29:00 +0000 (02:29 +0100)]
avcodec/mpeg4videodec: Check mb_num also against 0

The spec implies that 0 is invalid in addition to the existing checks

Found-by: <kierank>
Reviewed-by: Kieran Kunhya <kieran618@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 05f4703a168a336363750e32bcfdd6f303fbdbc3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavfilter/vf_transpose: Fix used plane count.
Michael Niedermayer [Wed, 24 Jan 2018 18:38:05 +0000 (19:38 +0100)]
avfilter/vf_transpose: Fix used plane count.

Fixes out of array access
Fixes: poc.mp4

Found-by: GwanYeong Kim <gy741.kim@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c6939f65a116b1ffed345d29d8621ee4ffb32235)
(cherry picked from commit 3f621455d62e46745453568d915badd5b1e5bcd5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevc_cabac: Check prefix so as to avoid invalid shifts in coeff_abs_level_rem...
Michael Niedermayer [Mon, 15 Jan 2018 22:46:44 +0000 (23:46 +0100)]
avcodec/hevc_cabac: Check prefix so as to avoid invalid shifts in coeff_abs_level_remaining_decode()

I suspect that this can be limited tighter, but i failed to find anything
in the spec that would confirm that.

Fixes: 4833/clusterfuzz-testcase-minimized-5302840101699584
Fixes: runtime error: left shift of 134217730 by 4 places cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a026a3efaeb9c2026668dccbbda339a21ab3206b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mjpegdec: Fix integer overflow in DC dequantization
Michael Niedermayer [Wed, 24 Jan 2018 02:28:49 +0000 (03:28 +0100)]
avcodec/mjpegdec: Fix integer overflow in DC dequantization

Fixes: runtime error: signed integer overflow: -65535 * 65312 cannot be represented in type 'int'
Fixes: 4900/clusterfuzz-testcase-minimized-5769019744321536

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1bfc1aa004950c5ad527d823a08b8a19eef34eb0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dxtory: Fix bits left checks
Michael Niedermayer [Mon, 22 Jan 2018 13:02:59 +0000 (14:02 +0100)]
avcodec/dxtory: Fix bits left checks

Fixes: Timeout
Fixes: 4863/clusterfuzz-testcase-6347354178322432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6e1a167c5564085385488b4f579e9efb987d4bfa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/hevc_cabac: Move prefix check in coeff_abs_level_remaining_decode() down
Michael Niedermayer [Mon, 15 Jan 2018 22:42:57 +0000 (23:42 +0100)]
avcodec/hevc_cabac: Move prefix check in coeff_abs_level_remaining_decode() down

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 94d4237a7a294ce80e1e577b38e9c93e8882aff9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/truemotion2: Fix integer overflow in TM2_RECALC_BLOCK()
Michael Niedermayer [Sat, 20 Jan 2018 03:10:50 +0000 (04:10 +0100)]
avcodec/truemotion2: Fix integer overflow in TM2_RECALC_BLOCK()

Fixes: signed integer overflow: 1477974040 - -1877995504 cannot be represented in type 'int'
Fixes: 4861/clusterfuzz-testcase-minimized-4570316383715328

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 56a53340ed4cc55898e49c07081311ebb2816630)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/snowdec: Fix integer overflow before htaps check
Michael Niedermayer [Mon, 15 Jan 2018 02:03:36 +0000 (03:03 +0100)]
avcodec/snowdec: Fix integer overflow before htaps check

Fixes: runtime error: signed integer overflow: -1094995529 * 2 cannot be represented in type 'int'
Fixes: 4828/clusterfuzz-testcase-minimized-5100849937252352

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2eecf3cf8eeae67697934df326e98df2149881e5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/ulti: Check number of blocks at init
Michael Niedermayer [Mon, 15 Jan 2018 18:03:48 +0000 (19:03 +0100)]
avcodec/ulti: Check number of blocks at init

Fixes: Timeout
Fixes: 4832/clusterfuzz-testcase-4699096590843904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 725353525e73bbe5b6b4d01528252675f2417a02)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/jpeg2000: Check sum of sizes of band->prec before allocating
Michael Niedermayer [Sat, 13 Jan 2018 23:39:40 +0000 (00:39 +0100)]
avcodec/jpeg2000: Check sum of sizes of band->prec before allocating

Fixes: OOM
Fixes: 4810/clusterfuzz-testcase-minimized-6034253235093504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6887e412434776eb260ad3904f565be491dd5726)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/ac3dec_fixed: Fix integer overflow in scale_coefs()
Michael Niedermayer [Sat, 13 Jan 2018 23:39:39 +0000 (00:39 +0100)]
avcodec/ac3dec_fixed: Fix integer overflow in scale_coefs()

Fixes: runtime error: signed integer overflow: 2147483520 + 128 cannot be represented in type 'int'
Fixes: 4800/clusterfuzz-testcase-minimized-6110372403609600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f38c75893c852cf19dcf3e4553549ba1e70950)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/lrcdec: Fix memory leak in lrc_read_header()
Nikolas Bowe [Fri, 19 Jan 2018 21:17:07 +0000 (13:17 -0800)]
avformat/lrcdec: Fix memory leak in lrc_read_header()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef5994e09d07ace62a672fcdc84761231288edad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/matroskadec: Fix float-cast-overflow undefined behavior in matroska_parse_tr...
Nikolas Bowe [Thu, 18 Jan 2018 23:21:56 +0000 (15:21 -0800)]
avformat/matroskadec: Fix float-cast-overflow undefined behavior in matroska_parse_tracks()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e07649e618caedc07eaf2f4d09253de7f77d14f0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoconfigure: bump year
Carl Eugen Hoyos [Mon, 1 Jan 2018 17:05:55 +0000 (18:05 +0100)]
configure: bump year

Happy new year!

(cherry picked from commit bddf31ba7570325dd2c8d033eae3d0dd74127f96)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/utils: Avoid hardcoding duplicated types in sizeof()
Michael Niedermayer [Sat, 3 Jun 2017 23:53:58 +0000 (01:53 +0200)]
avcodec/utils: Avoid hardcoding duplicated types in sizeof()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 860d991fcd715233b5b9eb1f6c7bf0aadefb6061)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/arm/sbrdsp_neon: Use a free register instead of putting 2 things in one
Michael Niedermayer [Thu, 11 Jan 2018 21:47:10 +0000 (22:47 +0100)]
avcodec/arm/sbrdsp_neon: Use a free register instead of putting 2 things in one

Fixes high pitched shriek
Fixes: 25420848_1478428308873746_4255813235963330560_n.mp4

Reported-by: Dale Curtis <dalecurtis@google.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7dbbb75ee32f87108ca9e15f5551dbbe69fe2641)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264addpx_template: Fixes integer overflows
Michael Niedermayer [Sun, 7 Jan 2018 02:48:43 +0000 (03:48 +0100)]
avcodec/h264addpx_template: Fixes integer overflows

Fixes: signed integer overflow: 512 + 2147483491 cannot be represented in type 'int'
Fixes: 4780/clusterfuzz-testcase-minimized-4709066174627840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d6945aeee419a8417b8019c7c92227e12e45b7ad)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dirac_dwt: Fix overflows in COMPOSE_HAARiH0/COMPOSE_HAARiL0
Michael Niedermayer [Sun, 7 Jan 2018 19:58:49 +0000 (20:58 +0100)]
avcodec/dirac_dwt: Fix overflows in COMPOSE_HAARiH0/COMPOSE_HAARiL0

Fixes: 4830/clusterfuzz-testcase-minimized-5255392054476800
Fixes: signed integer overflow: 2147483646 - -7 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0e62a2373475f58c72c0faf5568be00b26909585)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/diracdec: Fix integer overflow with quant
Michael Niedermayer [Sun, 7 Jan 2018 19:43:24 +0000 (20:43 +0100)]
avcodec/diracdec: Fix integer overflow with quant

Fixes: signed integer overflow: 2 + 2147483646 cannot be represented in type 'int'
Fixes: 4792/clusterfuzz-testcase-minimized-6322450775146496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eaa93175895568ef6c2542b13104874907d9c4ef)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/opus_parser: Check payload_len in parse_opus_ts_header()
Michael Niedermayer [Fri, 5 Jan 2018 21:12:07 +0000 (22:12 +0100)]
avcodec/opus_parser: Check payload_len in parse_opus_ts_header()

Fixes: clusterfuzz-testcase-minimized-6134545979277312
Fixes: crbug 797469

Reported-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1bcd7fefcb3c1ec47978fdc64a9e8dfb9512ae62)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/jpeg2000dsp: Fix integer overflows in ict_int()
Michael Niedermayer [Sun, 7 Jan 2018 03:12:57 +0000 (04:12 +0100)]
avcodec/jpeg2000dsp: Fix integer overflows in ict_int()

Fixes: signed integer overflow: 46802 * -71230 cannot be represented in type 'int'
Fixes: 4756/clusterfuzz-testcase-minimized-4812495563784192

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b3192c64b5bdcb0474cda437d2d5f9421d68811e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/h264_slice: Do not attempt to render into frames already output
Michael Niedermayer [Wed, 3 Jan 2018 22:42:01 +0000 (23:42 +0100)]
avcodec/h264_slice: Do not attempt to render into frames already output

Fixes: null pointer dereference
Fixes: 4698/clusterfuzz-testcase-minimized-5096956322906112

This testcase does not reproduce the issue before 03b82b3ab9883cef017e513c7d0b3b986b3b3e7b

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 476665d4de989dba48ec1195215ccc8db54538f4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dnxhddec: Check dc vlc
Michael Niedermayer [Wed, 3 Jan 2018 22:42:00 +0000 (23:42 +0100)]
avcodec/dnxhddec: Check dc vlc

Fixes: signed integer overflow: 1024 + 2147483640 cannot be represented in type 'int'
Fixes: 4671/clusterfuzz-testcase-minimized-6027464343027712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b2be76c0a472b729756ed7a91225c209d0dd1d2e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
18 months agoavformat/hvcc: zero initialize the nal buffers past the last written byte
James Almer [Fri, 23 Feb 2018 03:03:15 +0000 (00:03 -0300)]
avformat/hvcc: zero initialize the nal buffers past the last written byte

Prevents use of uninitialized values.

Fixes ticket #7038.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 9482ec1b203e4cf51d7f60c85d261cc13f9a9d2f)

18 months agoswresample/rematrix: fix update of channel matrix if input or output layout is undefined
Tobias Rapp [Wed, 14 Feb 2018 16:01:08 +0000 (17:01 +0100)]
swresample/rematrix: fix update of channel matrix if input or output layout is undefined

Prefer direct in/out channel count values over channel layout, when
available. Fixes a pan filter bug (ticket #6790).

Signed-off-by: Tobias Rapp <t.rapp@noa-archive.com>
(cherry picked from commit 6325bd3717348615adafb52e4da2fd01a3007d0a)

19 months agoconfigure: add support for libnpp* from cuda sdk 9
Timo Rothenpieler [Tue, 29 Aug 2017 11:30:29 +0000 (13:30 +0200)]
configure: add support for libnpp* from cuda sdk 9

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
19 months agoavcodec/nvenc: also clear data pointer after unregistering a resource
Timo Rothenpieler [Sun, 28 Jan 2018 12:05:09 +0000 (13:05 +0100)]
avcodec/nvenc: also clear data pointer after unregistering a resource

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
19 months agoavcodec/nvenc: add some more error case checks
Timo Rothenpieler [Sun, 28 Jan 2018 11:51:20 +0000 (12:51 +0100)]
avcodec/nvenc: add some more error case checks

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
19 months agoavcodec/nvenc: unregister input resource when unmapping
Timo Rothenpieler [Sun, 28 Jan 2018 11:39:03 +0000 (12:39 +0100)]
avcodec/nvenc: unregister input resource when unmapping

Currently the resource is only ever unregistered when the
registered_frames array is fully in use and an unmapped entry is re-used
and cleaned up.
I'm pretty sure the frame will have been cleaned up before that happens,
so I'm kinda surprised this never blew up.

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
19 months agoavcodec/nvenc: refcount input frame mappings
Timo Rothenpieler [Fri, 26 Jan 2018 19:16:53 +0000 (20:16 +0100)]
avcodec/nvenc: refcount input frame mappings

If some logic like vsync in ffmpeg.c duplicates frames, it might pass
the same frame twice, which will result in a crash due it being
effectively mapped and unmapped twice.

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
20 months agoavformat/libssh: check the user provided a password before trying to use it
James Almer [Sun, 11 Jun 2017 17:17:30 +0000 (14:17 -0300)]
avformat/libssh: check the user provided a password before trying to use it

Fixes ticket #6413

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 8ddb6820bd52df6ed616abc3d8be200b126aa8c1)

20 months agochangelog: update with previous commit n3.3.6
James Almer [Sat, 30 Dec 2017 22:38:23 +0000 (19:38 -0300)]
changelog: update with previous commit

Signed-off-by: James Almer <jamrial@gmail.com>
20 months agox264: Support version 153
Luca Barbato [Tue, 26 Dec 2017 11:32:42 +0000 (12:32 +0100)]
x264: Support version 153

It has native simultaneus 8 and 10 bit support.

(cherry picked from commit c6558e8840fbb2386bf8742e4d68dd6e067d262e)

20 months agoUpdate for 3.3.6
Michael Niedermayer [Sat, 30 Dec 2017 20:13:19 +0000 (21:13 +0100)]
Update for 3.3.6

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
20 months agoavcodec/exr: Check buf_size more completely
Michael Niedermayer [Fri, 29 Dec 2017 02:00:19 +0000 (03:00 +0100)]
avcodec/exr: Check buf_size more completely

Fixes: Out of heap array read
Fixes: 4683/clusterfuzz-testcase-minimized-6152313673613312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 903be5e4f66268273dc6e3c42a7fdeaab32066ef)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>