ffmpeg.git
17 months agoUpdate for FFmpeg 4.2.2 n4.2.2
Michael Niedermayer [Tue, 31 Dec 2019 19:53:54 +0000 (20:53 +0100)]
Update for FFmpeg 4.2.2

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agocbs_mpeg2: Fix parsing the last unit
Andreas Rheinhardt [Mon, 29 Jul 2019 19:56:56 +0000 (21:56 +0200)]
cbs_mpeg2: Fix parsing the last unit

There is one way to find out if avpriv_find_start_code has found a start
code or not: One has to check whether the state variable contains a
start code, i.e. whether the three most significant bytes are 0x00 00 01.
Checking for whether the return value is the end of the designated
buffer is not enough: If the last four bytes constitute a start code,
the return value is also the end of the buffer. This happens with
sequence_end_codes which have been ignored for exactly this reason,
although e.g. all three files used for fate tests of cbs_mpeg2 contain
sequence_end_codes.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit fd93d5efe64206d5f1bce8c702602353444c0c1a)

17 months agocbs_mpeg2: Rearrange start code search
Andreas Rheinhardt [Mon, 29 Jul 2019 19:56:55 +0000 (21:56 +0200)]
cbs_mpeg2: Rearrange start code search

1. Currently, cbs_mpeg2_split_fragment uses essentially three variables
to hold the start code values found by avpriv_find_start_code. By
rearranging the code, one of them can be omitted.
2. The return value of avpriv_find_start_code points to the byte after
the byte containing the start code identifier (or to the byte after the
last byte of the fragment's data if no start code was found), but
cbs_mpeg2_split_fragment needs to work with the pointer to the byte
containing the start code identifier; it already did this, but in a
clumsy way. This has been changed.
3. Also use the correct type for the variable holding the
CodedBitstreamUnitType.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 276b21a586900b4692efbb99e4789e05d927708e)

17 months agocbs_mpeg2: Decompose Sequence End
Andreas Rheinhardt [Mon, 29 Jul 2019 19:56:54 +0000 (21:56 +0200)]
cbs_mpeg2: Decompose Sequence End

Sequence End units (or actually, sequence_end_codes) have up until now
not been decomposed; in fact due to a bug in cbs_mpeg2_split_fragment they
have mostly been treated as part of the preceding unit. So implement
decomposing them as preparation for fixing said bug.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 0e66e1b61ea2fd8fd85ebe3b86ff48dad78233dd)

17 months agocbs_mpeg2: Fix parsing of picture and slice headers
Andreas Rheinhardt [Wed, 19 Jun 2019 23:45:12 +0000 (01:45 +0200)]
cbs_mpeg2: Fix parsing of picture and slice headers

1. The extra information in slice headers was parsed incorrectly:
In the first reading pass to derive the length of the extra information,
one should look at bits n, n + 9, n + 18, ... and check whether they
equal one (further extra information) or zero (end of extra information),
but instead bits n, n + 8, n + 16, ... were inspected. The second pass
of reading (where the length is already known and the bytes between the
length-determining bits are copied into a buffer) did not record what
was in bits n, n + 9, n + 18, ..., presuming they equal one. And during
writing, the bytes in the buffer are interleaved with set bits and
written. This means that if the detected length of the extra information
was greater than the real length, the output was corrupted. Fortunately
no sample is known that made use of this mechanism: The extra information
in slices is still marked as reserved in the specifications. cbs_mpeg2
is now ready in case this changes.

2. Furthermore, the buffer is now padded and slightly different, but
very similar code for reading resp. writing has been replaced by code
used for both. This was made possible by a new macro, the equivalent
to cbs_h2645's fixed().

3. These changes also made it possible to remove the extra_bit_slice
element from the MPEG2RawSliceHeader structure. Said element was always
zero except when the detected length of the extra information was less
than the real length.

4. The extra information in picture headers (which uses essentially the
same syntax as the extra information in slice headers) has simply been
forgotten. This meant that if this extra information was present, it was
discarded during reading; and unfortunately writing created invalid
bitstreams in this case (an extra_bit_picture - the last set bit of the
whole unit - indicated that there would be a further byte of data,
although the output didn't contain said data).

This has been fixed; both types of extra information are now parsed via
the same code and essentially passed through.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d9182f04caa59c3ba2659981183238ada340f814)

17 months agocbs: Remove useless initializations
Andreas Rheinhardt [Wed, 19 Jun 2019 23:45:11 +0000 (01:45 +0200)]
cbs: Remove useless initializations

Up until now, a temporary variable was used and initialized every time a
value was read in CBS; if reading turned out to be successfull, this
value was overwritten (without having ever been looked at) with the
value read if reading was successfull; on failure the variable wasn't
touched either. Therefore these initializations can be and have been
removed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b71a0367a6e763d631b8dcd608f98d42c05fa57c)

17 months agompeg2_metadata, cbs_mpeg2: Fix handling of colour_description
Andreas Rheinhardt [Wed, 19 Jun 2019 23:45:10 +0000 (01:45 +0200)]
mpeg2_metadata, cbs_mpeg2: Fix handling of colour_description

If a sequence display extension is read with colour_description equal to
zero, but a user wants to add one or more of the colour_description
elements, then the colour_description elements the user did not explicitly
request to be set are set to zero and not to the value equal to
unknown/unspecified (namely 2). A value of zero is not only inappropriate,
but explicitly forbidden. This is fixed by inferring the right default
values during the reading process if the elements are absent; moreover,
changing any of the colour_description elements to zero is now no longer
possible.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit c2a91645c5b5cd6ed32089ec79cbb667326a8d8a)

17 months agolavc/cbs_h2645_syntax_template: Fix memleak
Andriy Gelman [Fri, 6 Dec 2019 19:22:14 +0000 (14:22 -0500)]
lavc/cbs_h2645_syntax_template: Fix memleak

payload_count is used to track the number of SEI payloads. It is also
used to free the SEIs in cbs_h264_free_sei()/cbs_h265_free_sei().

Currently, payload_count is set after for loop is completed. Hence if
there is an error and the function exits, the payload remains zero
causing a memleak.

This commit keeps track of payload_count inside the for loop to fix the
issue. Note that that the contents of current are initialized with
av_mallocz() so there is no need to zero initialize payload_count.

Found-by: libFuzzer
Reviewed-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Andriy Gelman <andriy.gelman@gmail.com>
(cherry picked from commit c07a77247363eb666a49536af505e7317225ee81)

17 months agoavcodec/cbs: Fix potential overflow
Andreas Rheinhardt [Sun, 17 Nov 2019 07:34:36 +0000 (08:34 +0100)]
avcodec/cbs: Fix potential overflow

The number of bits in a PutBitContext must fit into an int, yet nothing
guaranteed the size argument cbs_write_unit_data() uses in init_put_bits()
to be in the range 0..INT_MAX / 8. This has been changed.

Furthermore, the check 8 * data_size > data_bit_start that there is
data beyond the initial padding when writing mpeg2 or H.264/5 slices
could also overflow, so divide it by 8 to get an equivalent check
without this problem.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit cda3e8ca04c0e343f5b60fda8fb467936e176f33)

17 months agoavcodec/cbs: Factor out common code for writing units
Andreas Rheinhardt [Sun, 17 Nov 2019 07:34:35 +0000 (08:34 +0100)]
avcodec/cbs: Factor out common code for writing units

All cbs-functions to write units share a common pattern:
1. They check whether they have a write buffer (that is used to store
the unit's data until the needed size becomes known after writing the
unit when a dedicated buffer will be allocated).
2. They use this buffer for a PutBitContext.
3. The (codec-specific) writing takes place through the PutBitContext.
4. The return value is checked. AVERROR(ENOSPC) here always indicates
that the buffer was too small and leads to a reallocation of said
buffer.
5. The final buffer will be allocated and the data copied.

This commit factors this common code out in a single function in cbs.c.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 7c92eaace2b338e0b3acc18e1543b365610578fd)

17 months agoavcodec/ffwavesynth: Fix undefined overflow in wavesynth_synth_sample()
Michael Niedermayer [Mon, 25 Nov 2019 20:50:57 +0000 (21:50 +0100)]
avcodec/ffwavesynth: Fix undefined overflow in wavesynth_synth_sample()

Fixes: signed integer overflow: 2147464192 + 21176 cannot be represented in type 'int'
Fixes: 19042/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5719828090585088

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fa47f6412dbf93b4865adf8c66618906a3274330)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cook: Use 3 stage VLC decoding for channel_coupling
Michael Niedermayer [Mon, 25 Nov 2019 20:39:48 +0000 (21:39 +0100)]
avcodec/cook: Use 3 stage VLC decoding for channel_coupling

Fixes: shift exponent -1 is negative
Fixes: out of array read
Fixes: 19028/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_COOK_fuzzer-5759766471376896
Fixes: 19037/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_COOK_fuzzer-5734106625474560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89fd76db71d9d4f87c51fee2a2edf99662444df7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmalosslessdec: Fixes undefined overflow in dequantization in decode_subframe()
Michael Niedermayer [Fri, 29 Nov 2019 21:45:07 +0000 (22:45 +0100)]
avcodec/wmalosslessdec: Fixes undefined overflow in dequantization in decode_subframe()

Fixes: signed integer overflow: 47875596 * 45 cannot be represented in type 'int'
Fixes: 19082/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5687766512041984

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 53efab44a9d0971c6c12d9b3d1af855ca863c847)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/sonic: Check e in get_symbol()
Michael Niedermayer [Mon, 21 Oct 2019 21:22:05 +0000 (23:22 +0200)]
avcodec/sonic: Check e in get_symbol()

Fixes: signed integer overflow: 1721520852 + 1721520852 cannot be represented in type 'int'
Fixes: 18346/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SONIC_fuzzer-5709623893426176
Fixes: 18753/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SONIC_fuzzer-5663299131932672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aea67556116330d3151e4cd3ef1e266b5d90f388)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/twinvqdec: Correct overflow in block align check
Michael Niedermayer [Tue, 3 Dec 2019 18:48:46 +0000 (19:48 +0100)]
avcodec/twinvqdec: Correct overflow in block align check

Fixes: signed integer overflow: 538976288 * 8 cannot be represented in type 'int'
Fixes: 19126/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TWINVQ_fuzzer-5687464110325760

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4dc93ae3d725e892927f04002021337c2f90252a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vc1dec: Fix "return -1" cases
Michael Niedermayer [Mon, 16 Dec 2019 23:04:23 +0000 (00:04 +0100)]
avcodec/vc1dec: Fix "return -1" cases

Reviewed-by: "mypopy@gmail.com" <mypopy@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26f040bcb4a1db78d1311af2e69de6984ecb43e5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vc1dec: Free sprite_output_frame on error
Michael Niedermayer [Mon, 16 Dec 2019 22:31:22 +0000 (23:31 +0100)]
avcodec/vc1dec: Free sprite_output_frame on error

Fixes: memleaks
Fixes: 19471/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5688035714269184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ee9240be3e4044ae9e60a9a3a68820bf8075299)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/atrac9dec: Clamp band_ext_data to max that can be read if skipped.
Michael Niedermayer [Mon, 16 Dec 2019 23:19:42 +0000 (00:19 +0100)]
avcodec/atrac9dec: Clamp band_ext_data to max that can be read if skipped.

Fixes: out of array read
Fixes: 19327/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5679823087468544

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 18ff210efb8d158f3e8c79508d99a52eaebf9d48)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/agm: Include block size in the MV check for flags == 3
Michael Niedermayer [Mon, 16 Dec 2019 22:09:04 +0000 (23:09 +0100)]
avcodec/agm: Include block size in the MV check for flags == 3

Fixes: out of array read
Fixes: 19331/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5644115983466496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1f2096945709a32315da740691b5716da55893c3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmadec: Keep track of exponent initialization per channel
Michael Niedermayer [Sat, 30 Nov 2019 15:46:46 +0000 (16:46 +0100)]
avcodec/wmadec: Keep track of exponent initialization per channel

Fixes: division by 0
Fixes: 19123/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAV2_fuzzer-5655493121146880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bf5c850b795126d4f60dd9498c06f0492f5726a7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/iff: Check that video_size is large enough for the read parameters
Michael Niedermayer [Sat, 30 Nov 2019 11:22:25 +0000 (12:22 +0100)]
avcodec/iff: Check that video_size is large enough for the read parameters

video is allocated before parameters like bpp are read.

Fixes: out of array access
Fixes: 19084/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5718556033679360
Fixes: 19465/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5759908398235648

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f1b97f62f86d5dca35d01d7a5ebbc5dca2a88ae6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cbs_vp9: Check data_size
Michael Niedermayer [Wed, 25 Dec 2019 23:57:07 +0000 (00:57 +0100)]
avcodec/cbs_vp9: Check data_size

Fixes: out of array access
Fixes: 19542/clusterfuzz-testcase-minimized-ffmpeg_BSF_TRACE_HEADERS_fuzzer-5659498341728256

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4fa2d5a692f40c398a299acf2c6a20f5b98a3708)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cbs_vp9: Check index_size
Michael Niedermayer [Fri, 13 Dec 2019 23:27:09 +0000 (00:27 +0100)]
avcodec/cbs_vp9: Check index_size

Fixes: out of array read
Fixes: 19300/clusterfuzz-testcase-minimized-ffmpeg_BSF_VP9_METADATA_fuzzer-5653911730126848

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d6553e2e60a389296dd2f83a96f944ccfa5877a0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/adpcm: Clip predictor for APC
Michael Niedermayer [Thu, 21 Nov 2019 22:02:56 +0000 (23:02 +0100)]
avcodec/adpcm: Clip predictor for APC

Fixes: signed integer overflow: -2147483648 - 13 cannot be represented in type 'int'
Fixes: 18893/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ADPCM_IMA_APC_fuzzer-5630760442920960

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9fe07908c3f67d59cf4db5668d61b34506189590)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/targa: Check colors vs. available space
Michael Niedermayer [Thu, 21 Nov 2019 21:43:01 +0000 (22:43 +0100)]
avcodec/targa: Check colors vs. available space

Fixes: Timeout (37sec -> 52ms)
Fixes: 18892/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TARGA_fuzzer-5739537854889984

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 01593278cef06dbb4491d50d03b72198d2848adf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dstdec: Use get_ur_golomb_jpegls()
Michael Niedermayer [Mon, 30 Sep 2019 22:43:03 +0000 (00:43 +0200)]
avcodec/dstdec: Use get_ur_golomb_jpegls()

Fixes: shift exponent -4 is negative
Fixes: 17793/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DST_fuzzer-5766088435957760
Fixes: 18989/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DST_fuzzer-5175008116867072

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a76690c02b4fd12d7fac6f753af8bad72c82d55c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmavoice: Check remaining input in parse_packet_header()
Michael Niedermayer [Sat, 23 Nov 2019 08:18:12 +0000 (09:18 +0100)]
avcodec/wmavoice: Check remaining input in parse_packet_header()

Fixes: Infinite loop
Fixes: 18914/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAVOICE_fuzzer-5731902946541568

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 19c41969b26d07519fff8182a0d3266cdb712078)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmalosslessdec: Fix 2 overflows in mclms
Michael Niedermayer [Wed, 20 Nov 2019 21:05:40 +0000 (22:05 +0100)]
avcodec/wmalosslessdec: Fix 2 overflows in mclms

Fixes: signed integer overflow: 2038337026 + 109343477 cannot be represented in type 'int'
Fixes: 18886/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5673660505653248

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 92455c8c65c403ea696cb8c63d474d386d631bbd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmaprodec: Fixes integer overflow with 32bit samples
Michael Niedermayer [Wed, 20 Nov 2019 19:34:55 +0000 (20:34 +0100)]
avcodec/wmaprodec: Fixes integer overflow with 32bit samples

Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 18860/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAPRO_fuzzer-5755223125786624

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a9cc69c0d59057ea172a107e0308fdf5fd8fc04e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/adpcm: Fix invalid shift in xa_decode()
Michael Niedermayer [Wed, 20 Nov 2019 18:13:09 +0000 (19:13 +0100)]
avcodec/adpcm: Fix invalid shift in xa_decode()

Fixes: left shift of negative value -1
Fixes: 18859/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ADPCM_XA_fuzzer-5748474213040128

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 50db30b47d016fc4e7b47067545b15d22d4faddf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmalosslessdec: Fix several integer issues
Michael Niedermayer [Mon, 18 Nov 2019 13:22:57 +0000 (14:22 +0100)]
avcodec/wmalosslessdec: Fix several integer issues

Fixes: shift exponent -1 is negative (and others)
Fixes: 18852/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5660855295541248

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ec3fe67074ad0a6a3a817f6f42175ea63a98092b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmalosslessdec: Check that padding bits is not more than sample bits
Michael Niedermayer [Mon, 18 Nov 2019 11:49:25 +0000 (12:49 +0100)]
avcodec/wmalosslessdec: Check that padding bits is not more than sample bits

Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 18817/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMALOSSLESS_fuzzer-5713317180211200

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9d428265808255ad2fc60355fe641aaa4fd3dae4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/iff: Skip overflowing runs in decode_delta_d()
Michael Niedermayer [Mon, 18 Nov 2019 08:45:29 +0000 (09:45 +0100)]
avcodec/iff: Skip overflowing runs in decode_delta_d()

Fixes: Timeout (107sec - 75ms>
Fixes: 18812/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-6295585225441280

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 185f441ba26a2112725db1e8f218e54ac8068bbb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/pnm: Check that the header is not truncated
Michael Niedermayer [Sat, 14 Dec 2019 18:19:57 +0000 (19:19 +0100)]
avcodec/pnm: Check that the header is not truncated

Fixes: Ticket8430

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c94cb8d9b21baeeecef962c72965dbedc4e0b0e1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mp3_header_decompress_bsf: Check sample_rate_index
Michael Niedermayer [Fri, 13 Dec 2019 23:37:27 +0000 (00:37 +0100)]
avcodec/mp3_header_decompress_bsf: Check sample_rate_index

Fixes: out of array read
Fixes: 19309/clusterfuzz-testcase-minimized-ffmpeg_BSF_MP3_HEADER_DECOMPRESS_fuzzer-5651002950942720

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f064c7c449f162a9011ad890f26ceeca26934d22)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cbs_av1_syntax_template: Check num_y_points
Michael Niedermayer [Wed, 11 Dec 2019 21:03:50 +0000 (22:03 +0100)]
avcodec/cbs_av1_syntax_template: Check num_y_points

"It is a requirement of bitstream conformance that num_y_points is less than or equal to 14."

Fixes: index 24 out of bounds for type 'uint8_t [24]'
Fixes: 19282/clusterfuzz-testcase-minimized-ffmpeg_BSF_AV1_FRAME_MERGE_fuzzer-5747424845103104

Note, also needs a23dd33606d5

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: jamrial
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bbe27890ff7e31e74d024a17123cb073720f2486)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/rmdec: Initialize and sanity check offset in ivr_read_header()
Michael Niedermayer [Fri, 15 Nov 2019 22:00:51 +0000 (23:00 +0100)]
avformat/rmdec: Initialize and sanity check offset in ivr_read_header()

Fixes: signed integer overflow: -9223372036854775808 - 17 cannot be represented in type 'long'
Fixes: 18768/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5674385247830016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7e665e4a81e2e96eb45138a1dfa38617de2631a4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/agm: Do not allow MVs out of the picture area as no edge is allocated
Michael Niedermayer [Sat, 2 Nov 2019 07:35:39 +0000 (08:35 +0100)]
avcodec/agm: Do not allow MVs out of the picture area as no edge is allocated

Fixes: out of array access
Fixes: 18499/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5749038406434816

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7a1b30c871c873e97c93af75f925c854de7b75f2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/apedec: Fix 2 integer overflows
Michael Niedermayer [Thu, 14 Nov 2019 15:38:36 +0000 (16:38 +0100)]
avcodec/apedec: Fix 2 integer overflows

Fixes: signed integer overflow: 2119056926 - -134217728 cannot be represented in type 'int'
Fixes: 18728/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5747539563511808

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6e15ba2d1f688c61759001839811b11903de9ce0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/id3v2: Fix double-free on error
Andreas Rheinhardt [Sun, 10 Nov 2019 04:07:28 +0000 (05:07 +0100)]
avformat/id3v2: Fix double-free on error

ff_id3v2_parse_priv_dict() uses av_dict_set() with the flags
AV_DICT_DONT_STRDUP_KEY and AV_DICT_DONT_STRDUP_VAL. In this case both
key and value are freed on error (and owned by the destination
dictionary on success), so that freeing them again on error is a
double-free and therefore forbidden. But it nevertheless happened.

Fixes CID 1452489 and 1452421.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 67d4940a7795aa3afc8d1e624de33b030e0be51e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmaprodec: Set packet_loss when we error out on a sanity check
Michael Niedermayer [Tue, 12 Nov 2019 19:25:00 +0000 (20:25 +0100)]
avcodec/wmaprodec: Set packet_loss when we error out on a sanity check

Fixes: left shift of negative value -34
Fixes: 18719/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAPRO_fuzzer-5642658173419520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a9cbd25d89dbdf72f7b616fdf672d7da36143cfe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmaprodec: Check offset
Michael Niedermayer [Tue, 12 Nov 2019 19:06:35 +0000 (20:06 +0100)]
avcodec/wmaprodec: Check offset

Fixes: index 33280 out of bounds for type 'float [32768]'
Fixes: 18718/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XMA2_fuzzer-5635373899710464

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5473c7825ea627a115155313a56a907d67a0d0c1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/truemotion2: Fix 2 integer overflows in tm2_low_res_block()
Michael Niedermayer [Tue, 12 Nov 2019 17:47:52 +0000 (18:47 +0100)]
avcodec/truemotion2: Fix 2 integer overflows in tm2_low_res_block()

Fixes: signed integer overflow: 1778647621 + 574372924 cannot be represented in type 'int'
Fixes: 18692/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-6248679635943424

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 93d52a181ec050d3a4fb68f526604d39cd006be5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmaprodec: Check if the channel sum of all internal contexts match the external
Michael Niedermayer [Tue, 12 Nov 2019 17:39:08 +0000 (18:39 +0100)]
avcodec/wmaprodec: Check if the channel sum of all internal contexts match the external

Fixes: NULL pointer dereference
Fixes: 18689/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XMA1_fuzzer-5715114640015360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 090ac5799751c6f52358da4e5201a3845760db93)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/atrac9dec: Check q_unit_cnt more completely before using it to access at9_tab...
Michael Niedermayer [Tue, 3 Dec 2019 20:33:18 +0000 (21:33 +0100)]
avcodec/atrac9dec: Check q_unit_cnt more completely before using it to access at9_tab_band_ext_group

Fixes: index 8 out of bounds for type 'const uint8_t [8][3]'
Fixes: 19127/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5709394985091072

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e1d836d2375c93cbc44a2b0d34e404682c1e8436)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/fitsdec: Use lrint()
Michael Niedermayer [Mon, 30 Sep 2019 16:30:26 +0000 (18:30 +0200)]
avcodec/fitsdec: Use lrint()

Fixes: fate-fitsdec-bitpix-64

Possibly Fixes: -nan is outside the range of representable values of type 'unsigned short'
Possibly Fixes: 17769/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5678314672357376

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 37f31f4e509fe4ccc56a64edaa6fa3d95ee20466)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/g729dec: Avoid using buf_size
Michael Niedermayer [Sat, 9 Nov 2019 20:21:51 +0000 (21:21 +0100)]
avcodec/g729dec: Avoid using buf_size

buf_size is not updated as buf is advanced so it is wrong after the first
iteration

Fixes: Timeout (160sec -> 27sec)
Fixes: 18658/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_G729_fuzzer-5729784269373440

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 336f9461df7d2005db9d1af4f5f81fd033025ce2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/g729dec: Factor block_size out
Michael Niedermayer [Sat, 9 Nov 2019 20:11:02 +0000 (21:11 +0100)]
avcodec/g729dec: Factor block_size out

This will be used in the next commit

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 576746b4e30069a922564e0019ef0758811e771d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/g729dec: require buf_size to be non 0
Michael Niedermayer [Sat, 9 Nov 2019 20:19:24 +0000 (21:19 +0100)]
avcodec/g729dec: require buf_size to be non 0

The 0 case was added with the support for multiple packets. It
appears unintended and causes extra complexity and out of array
accesses (though within padding)

No testcase

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f64be9da4c8b16071ec84056a61d1fc0d5d6728c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/alac: Fix integer overflow in lpc_prediction() with sign
Michael Niedermayer [Fri, 8 Nov 2019 19:40:46 +0000 (20:40 +0100)]
avcodec/alac: Fix integer overflow in lpc_prediction() with sign

Fixes: signed integer overflow: -2147483648 * -1 cannot be represented in type 'int'
Fixes: 18643/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALAC_fuzzer-5672182449700864

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7686ba1f149a94c3bac235589de8aa8db92be4e5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmaprodec: Fix buflen computation in save_bits()
Michael Niedermayer [Fri, 8 Nov 2019 18:20:31 +0000 (19:20 +0100)]
avcodec/wmaprodec: Fix buflen computation in save_bits()

Fixes: Assertion failure
Fixes: 18630/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAPRO_fuzzer-5201588654440448

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 589cb44498b5e9683c95746255a2abd6d1e74f94)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vc1_block: Fix integer overflow in AC rescaling in vc1_decode_i_block_adv()
Michael Niedermayer [Fri, 8 Nov 2019 17:31:02 +0000 (18:31 +0100)]
avcodec/vc1_block: Fix integer overflow in AC rescaling in vc1_decode_i_block_adv()

Fixes: signed integer overflow: 50176 * 262144 cannot be represented in type 'int'
Fixes: 18629/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5182370286403584

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0e010e489b70c044a67c47083cf8eb03209ee89f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vmdaudio: Check chunk counts to avoid integer overflow
Michael Niedermayer [Fri, 8 Nov 2019 16:28:27 +0000 (17:28 +0100)]
avcodec/vmdaudio: Check chunk counts to avoid integer overflow

Fixes: signed integer overflow: 4 * 538976288 cannot be represented in type 'int'
Fixes: 18622/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VMDAUDIO_fuzzer-5092166174507008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47d963335eb2c36c0e6615d7971c762458e813dd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/mxfdec: Clear metadata_sets_count in mxf_read_close()
Michael Niedermayer [Thu, 31 Oct 2019 12:32:55 +0000 (13:32 +0100)]
avformat/mxfdec: Clear metadata_sets_count in mxf_read_close()

This avoids problems if the function is called twice

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13816a1d085fdb6598ea6dc92ed3a1e6aff0cc1f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/nuv: Use ff_set_dimensions()
Michael Niedermayer [Sat, 23 Nov 2019 08:29:58 +0000 (09:29 +0100)]
avcodec/nuv: Use ff_set_dimensions()

Fixes: OOM
Fixes: 18956/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NUV_fuzzer-5766505644163072

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1ca978d6366f3c7d7df6b3d50566e892f8da605a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/vividas: Error out on audio packets in the absence of audio streams
Michael Niedermayer [Tue, 5 Nov 2019 21:03:19 +0000 (22:03 +0100)]
avformat/vividas: Error out on audio packets in the absence of audio streams

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d83002179fb377f1f201b43c9a55cc237695a1fe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/vividas: Check and require 1 video stream
Michael Niedermayer [Tue, 5 Nov 2019 20:52:41 +0000 (21:52 +0100)]
avformat/vividas: Check and require 1 video stream

The decoder hardcodes that audio is stream_id = 1 so it does not
currently work with more or less than 1 video stream at st=0

Fixes: assertion failure
Fixes: 18602/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-6259277199310848

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3e5a528bbe85a3a00640bc2739c11ee07eb05485)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/ffwavesynth: Fix integer overflow with pink_ts_cur/next
Michael Niedermayer [Tue, 5 Nov 2019 21:11:52 +0000 (22:11 +0100)]
avcodec/ffwavesynth: Fix integer overflow with pink_ts_cur/next

Fixes: signed integer overflow: 6175076100092079360 - -5034989061050195840 cannot be represented in type 'long'
Fixes: 18614/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-5704508847423488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d82ab96e76bfec6568d059df7c8591dda4317c62)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/ralf: Fix integer overflows with the filter coefficient in decode_channel()
Michael Niedermayer [Tue, 5 Nov 2019 21:27:04 +0000 (22:27 +0100)]
avcodec/ralf: Fix integer overflows with the filter coefficient in decode_channel()

Fixes: signed integer overflow: 1145975808 - -1146173210 cannot be represented in type 'int'
Fixes: 18616/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5121296757424128

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 721624c2f67545989626ba4413f7b8dbd7dff678)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/g729dec: Use 64bit and clip in scalar product
Michael Niedermayer [Tue, 5 Nov 2019 22:28:35 +0000 (23:28 +0100)]
avcodec/g729dec: Use 64bit and clip in scalar product

The G729 reference decoder clips after each individual operation and keeps track if overflow
occurred (in the fixed point implementation), this here is
simpler and faster but not 1:1 the same what the reference does.

Non fuzzed samples which trigger any such overflow are welcome, so
the need and impact of different clipping solutions can be evaluated.

Fixes: signed integer overflow: 1271483721 + 1073676289 cannot be represented in type 'int'
Fixes: 18617/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ACELP_KELVIN_fuzzer-5137705679978496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bf9c4a12750e593d753011166b066efce208d9e0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/mxpegdec: Check for multiple SOF
Michael Niedermayer [Sun, 3 Nov 2019 11:20:14 +0000 (12:20 +0100)]
avcodec/mxpegdec: Check for multiple SOF

Fixes: Timeout (14sec -> 9ms)
Fixes: 18598/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MXPEG_fuzzer-5726095261564928

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 75b64e5aa36e7796a0460415a1f3fd7372029525)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/nuv: Move comptype check up
Michael Niedermayer [Sat, 2 Nov 2019 13:14:44 +0000 (14:14 +0100)]
avcodec/nuv: Move comptype check up

Fixes: Timeout (23sec -> 5ms)
Fixes: 18517/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NUV_fuzzer-5753135536013312

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1138cdecbe0164ab1f07768418e794fddfdc636d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmavoice: Fix integer overflow in synth_frame()
Michael Niedermayer [Sat, 2 Nov 2019 14:15:46 +0000 (15:15 +0100)]
avcodec/wmavoice: Fix integer overflow in synth_frame()

Fixes: left shift of negative value -3
Fixes: 18518/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAVOICE_fuzzer-6560514359951360

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cf323f4d38f5756ecdb8fb4f72c80a8069da832e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/rawdec: Check bits_per_coded_sample more pedantically for 16bit cases
Michael Niedermayer [Sun, 20 Oct 2019 21:02:27 +0000 (23:02 +0200)]
avcodec/rawdec: Check bits_per_coded_sample more pedantically for 16bit cases

Fixes: shift exponent -14 is negative
Fixes: 18335/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RAWVIDEO_fuzzer-5723267192586240

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5634e2052533fcce46f20c2720b0c8d5f55143ce)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavutil/lfg: Correct index increment type to avoid undefined behavior
Michael Niedermayer [Sat, 19 Oct 2019 19:27:41 +0000 (21:27 +0200)]
avutil/lfg: Correct index increment type to avoid undefined behavior

Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 18333/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_COMFORTNOISE_fuzzer-5668481831272448

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6014bcf1b74e903f535461ade4aa5fb44dbf5d8b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cngdec: Remove AV_CODEC_CAP_DELAY
Michael Niedermayer [Sat, 19 Oct 2019 19:58:26 +0000 (21:58 +0200)]
avcodec/cngdec: Remove AV_CODEC_CAP_DELAY

As is the decoder will never stop, it will cause an infinite loop. The RFC seems only
to speak of non empty packets so endlessly generating noise from the last empty flush
packets seems wrong.

Fixes: infinite loop
Fixes: 18333/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_COMFORTNOISE_fuzzer-5668481831272448

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 327a968817a366c24d1513526258a3dbbcf888a7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/iff: Move index use after check in decodeplane8()
Michael Niedermayer [Tue, 29 Oct 2019 18:12:23 +0000 (19:12 +0100)]
avcodec/iff: Move index use after check in decodeplane8()

Fixes: index 9 out of bounds for type 'const uint64_t [8][256]'
Fixes: 18409/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5767030560522240
Fixes: 18720/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5651995784642560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f8b36cc45406f66aac635a4db32d2a5cc29f43)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/atrac3: Check for huge block aligns
Michael Niedermayer [Sun, 20 Oct 2019 21:51:58 +0000 (23:51 +0200)]
avcodec/atrac3: Check for huge block aligns

The largest documented frame size = block align is 1024 bytes
(https://wiki.multimedia.cx/index.php/ATRAC3)

Without a limit this can allocate arbitrary memory and trigger OOM
Fixes: OOM
Fixes: 18337/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC3_fuzzer-5763861478637568
Fixes: 18556/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC3AL_fuzzer-5646183334936576

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f09151fff9c754fbc1d2560adf18b14957f8b181)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/ralf: use multiply instead of shift to avoid undefined behavior in decode_block()
Michael Niedermayer [Sat, 2 Nov 2019 14:52:52 +0000 (15:52 +0100)]
avcodec/ralf: use multiply instead of shift to avoid undefined behavior in decode_block()

Fixes: left shift of negative value -249
Fixes: 18566/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5649394561187840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1b7d02642b2096622cee6165fea1301bb9ad54ff)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmadec: Require previous exponents for reuse
Michael Niedermayer [Thu, 31 Oct 2019 13:38:16 +0000 (14:38 +0100)]
avcodec/wmadec: Require previous exponents for reuse

Fixes: division by zero
Fixes: 18474/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAV2_fuzzer-5764986962182144

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c54b9fc42fee613e2c4c0dae2052ff94cd15e254)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/vc1_block: Fix undefined behavior in ac prediction rescaling
Michael Niedermayer [Thu, 31 Oct 2019 14:00:32 +0000 (15:00 +0100)]
avcodec/vc1_block: Fix undefined behavior in ac prediction rescaling

The intermediates are required to fit in 12bit (8.1.3.9 Coefficient Scaling)
See SMPTE 421M-2006 and Amendment 1-2007

Fixes: signed integer overflow: -20691 * 262144 cannot be represented in type 'int'
Fixes: 18479/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1_fuzzer-5128912371187712

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7fc1baf0ca83ef06014878290339a59735603959)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/qdm2: The smallest header seems to have 2 bytes so treat 1 as invalid
Michael Niedermayer [Thu, 31 Oct 2019 14:22:53 +0000 (15:22 +0100)]
avcodec/qdm2: The smallest header seems to have 2 bytes so treat 1 as invalid

Fixes: Timeout (217sec -> 2ms)
Fixes: 18488/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QDM2_fuzzer-5708293662310400

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e36ccb5048f052b8b2ef08281cb607fa53a7b7e4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/apedec: Fixes integer overflow of res+*data in do_apply_filter()
Michael Niedermayer [Tue, 29 Oct 2019 17:30:07 +0000 (18:30 +0100)]
avcodec/apedec: Fixes integer overflow of res+*data in do_apply_filter()

Fixes: signed integer overflow: 7400 + 2147482786 cannot be represented in type 'int'
Fixes: 18405/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5708834760294400

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dc3f327e7403a34c88a900f0b8de55b4afd7cf6c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/sonic: Fix integer overflow in predictor_calc_error()
Michael Niedermayer [Mon, 21 Oct 2019 21:41:49 +0000 (23:41 +0200)]
avcodec/sonic: Fix integer overflow in predictor_calc_error()

Fixes: signed integer overflow: 5 * -1094995529 cannot be represented in type 'int'
Fixes: 18346/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SONIC_fuzzer-5709623893426176

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8c17b8cef77dc052e8845e5fd86daf2983fd7dd)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/vividas: Add EOF check in val_1 loop in track_header()
Michael Niedermayer [Tue, 29 Oct 2019 21:17:45 +0000 (22:17 +0100)]
avformat/vividas: Add EOF check in val_1 loop in track_header()

Fixes: Timeout (148sec -> 0.1sec)
Fixes: 18427/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5682124627116032

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit faea5b4462c4325b4ec7c150c3c31929429773cc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/atrac9dec: Check precision_fine/coarse
Michael Niedermayer [Sat, 19 Oct 2019 16:48:03 +0000 (18:48 +0200)]
avcodec/atrac9dec: Check precision_fine/coarse

Clipping is done as it was preferred in review
See: [FFmpeg-devel] [PATCH 1/5] avcodec/atrac9dec: Check precision_fine/coarse

Fixes: out of array access
Fixes: 18330/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5641113058148352

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 19b8db2908bf0fd248da1b2126e2592ade66c40c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/mp3dec: Check that the frame fits within the probe buffer
Michael Niedermayer [Thu, 7 Nov 2019 20:16:32 +0000 (21:16 +0100)]
avformat/mp3dec: Check that the frame fits within the probe buffer

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e9a335150a62bb377a26ce096187b4476145d02b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agovcodec/agm: Alloc based on coded dimensions
Michael Niedermayer [Thu, 14 Nov 2019 14:10:28 +0000 (15:10 +0100)]
vcodec/agm: Alloc based on coded dimensions

Fixes: out of array read
Fixes: 18715/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5659333417500672

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bfa8272f405314582e8f099ec1a9249232553c9c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmaprodec: get frame during frame decode
Michael Niedermayer [Wed, 25 Sep 2019 13:54:45 +0000 (15:54 +0200)]
avcodec/wmaprodec: get frame during frame decode

Fixes: memleak
Fixes: 17615/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XMA2_fuzzer-5681306024804352

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f89a2293ea5f642a67700225d76948ed154418e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/interplayacm: Fix overflow of last unused value
Michael Niedermayer [Fri, 25 Oct 2019 10:44:45 +0000 (12:44 +0200)]
avcodec/interplayacm: Fix overflow of last unused value

Fixes: signed integer overflow: -2147450880 - 65535 cannot be represented in type 'int'
Fixes: 18393/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INTERPLAY_ACM_fuzzer-5667520110919680

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 10eabb8e40df0ad84470d750f903917f4a05cb1f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/adpcm: Fix undefined behavior with negative predictions in IMA OKI
Michael Niedermayer [Fri, 25 Oct 2019 09:12:02 +0000 (11:12 +0200)]
avcodec/adpcm: Fix undefined behavior with negative predictions in IMA OKI

Fixes: left shift of negative value -30
Fixes: 18392/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ADPCM_IMA_OKI_fuzzer-5631771831435264

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7786f6c30e77a393b72ded01baa4250738925509)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cook: Move up and extend block_align check
Michael Niedermayer [Thu, 24 Oct 2019 23:12:15 +0000 (01:12 +0200)]
avcodec/cook: Move up and extend block_align check

Fixes: signed integer overflow: 2046820356 * 8 cannot be represented in type 'int'
Fixes: 18391/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_COOK_fuzzer-5631674666188800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1c63edcdd208bf18a3be66e94deb6ac115f6364e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/sbcdec: Fix integer overflows in sbc_synthesize_four()
Michael Niedermayer [Mon, 21 Oct 2019 22:09:11 +0000 (00:09 +0200)]
avcodec/sbcdec: Fix integer overflows in sbc_synthesize_four()

Fixes: signed integer overflow: 1494495519 + 1494495519 cannot be represented in type 'int'
Fixes: 18347/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SBC_fuzzer-5711714661695488

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 00e469fb6123df92ec3c54ab3b37f77e21d297be)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/twinvq: Check block_align
Michael Niedermayer [Tue, 22 Oct 2019 13:41:51 +0000 (15:41 +0200)]
avcodec/twinvq: Check block_align

Fixes: signed integer overflow: 538976288 * 8 cannot be represented in type 'int'
Fixes: 18348/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_METASOUND_fuzzer-6681325716635648

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 97f778e9c55328e8b48f4b8b4171245e5f2232f6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cook: Enlarge gain table
Michael Niedermayer [Wed, 23 Oct 2019 17:59:57 +0000 (19:59 +0200)]
avcodec/cook: Enlarge gain table

Fixes: index 25 out of bounds for type 'float [23]'
Fixes: 18355/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_COOK_fuzzer-5641398941908992

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 50001cd440ac89ed125f0154dedbcfa2718d2d68)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/cook: Check samples_per_channel earlier
Michael Niedermayer [Wed, 23 Oct 2019 17:41:27 +0000 (19:41 +0200)]
avcodec/cook: Check samples_per_channel earlier

Fixes: division by zero
Fixes: 18362/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_COOK_fuzzer-5653727679086592

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 57750bb629a145326e20b8760f21f1041464a937)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/atrac3plus: Check split point in fill mode 3
Michael Niedermayer [Tue, 22 Oct 2019 20:02:32 +0000 (22:02 +0200)]
avcodec/atrac3plus: Check split point in fill mode 3

Fixes: index 32 out of bounds for type 'int [32]'
Fixes: 18350/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC3P_fuzzer-5643794862571520

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit de5102fd92de8d353fdf060375ed3ce859c83977)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/wmavoice: Check sample_rate
Michael Niedermayer [Wed, 23 Oct 2019 20:32:47 +0000 (22:32 +0200)]
avcodec/wmavoice: Check sample_rate

Fixes: left shift of 538976288 by 8 places cannot be represented in type 'int'
Fixes: 18376/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAVOICE_fuzzer-5741645391200256

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 55c97a763783540ee48a326a3e82fbdea42f8280)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/xsubdec: fix overflow in alpha handling
Michael Niedermayer [Wed, 23 Oct 2019 20:08:37 +0000 (22:08 +0200)]
avcodec/xsubdec: fix overflow in alpha handling

Fixes: left shift of 255 by 24 places cannot be represented in type 'int'
Fixes: 18368/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XSUB_fuzzer-5702665442426880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9ea997395909907f569787d4ba5b96352ad31a80)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/iff: Check available space before entering loop in decode_long_vertical_delta...
Michael Niedermayer [Wed, 23 Oct 2019 21:31:03 +0000 (23:31 +0200)]
avcodec/iff: Check available space before entering loop in decode_long_vertical_delta2() / decode_long_vertical_delta()

Fixes: Timeout (31sec -> 41ms)
Fixes: 18380/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5645210121404416

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 32b3c8ce7d050210d210511cdb8c6644664a70ab)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/apedec: Fix integer overflow in filter_3800()
Michael Niedermayer [Sun, 20 Oct 2019 22:26:25 +0000 (00:26 +0200)]
avcodec/apedec: Fix integer overflow in filter_3800()

Fixes: signed integer overflow: 2117181180 + 60483298 cannot be represented in type 'int'
Fixes: 18344/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5685327791915008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1c038c5c63375883a8a94332cffd701c4cb1301a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavutil/lfg: Document the AVLFG struct
Michael Niedermayer [Sun, 20 Oct 2019 10:12:12 +0000 (12:12 +0200)]
avutil/lfg: Document the AVLFG struct

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d6fea2ef221a2f438cc55e82c61d0375750edf94)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/ffv1dec: Use a different error message for the slice level CRC
Michael Niedermayer [Thu, 17 Oct 2019 21:22:22 +0000 (23:22 +0200)]
avcodec/ffv1dec: Use a different error message for the slice level CRC

This way they can be told apart easily

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit df498cf544fd4690e5a246925e4de1125b57795b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/apedec: Fix undefined integer overflow in long_filter_ehigh_3830()
Michael Niedermayer [Thu, 17 Oct 2019 18:56:23 +0000 (20:56 +0200)]
avcodec/apedec: Fix undefined integer overflow in long_filter_ehigh_3830()

Fixes: signed integer overflow: -1094995529 * 2 cannot be represented in type 'int'
Fixes: 18281/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5692589180715008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1d1719a44dd43b2d9d8ccd26e3b2854e675a7bd7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dstdec: Check that AC probabilities are within range
Michael Niedermayer [Tue, 15 Oct 2019 21:42:50 +0000 (23:42 +0200)]
avcodec/dstdec: Check that AC probabilities are within range

ISO/IEC 14496-3:2005(E): "Each entry of P_one[ ][ ] is in the range of 1 to
128, corresponding to a probability of 1/256 to 128/256 of the next error bit (bit E, See Figure 10.5)..."

Fixes: Timeout (42sec ->1sec)
Fixes: 18181/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DST_fuzzer-5736646250594304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c3e1b395b47fac44397604b2a3343c4bd92561c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/dstdec: Check read_table() for failure
Michael Niedermayer [Tue, 15 Oct 2019 21:40:21 +0000 (23:40 +0200)]
avcodec/dstdec: Check read_table() for failure

Fixes: Timeout (too long -> 42sec)
Fixes: 18181/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DST_fuzzer-5736646250594304

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03ea8d8cd45e55eeb9675c38184dc2149710a557)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/vividas: Fix n_sb_blocks Check
Michael Niedermayer [Mon, 14 Oct 2019 21:17:51 +0000 (23:17 +0200)]
avformat/vividas: Fix n_sb_blocks Check

Fixes: signed integer overflow: 1540265776 * 2 cannot be represented in type 'int'
Fixes: 18160/clusterfuzz-testcase-minimized-ffmpeg_DEMUXER_fuzzer-5758808818712576

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 114ddf64300fa78663ef35decbee89b5492abb1d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/snowenc: Set mb_num to avoid ratecontrol floating point divisions by 0.0
Michael Niedermayer [Mon, 14 Oct 2019 21:03:50 +0000 (23:03 +0200)]
avcodec/snowenc: Set mb_num to avoid ratecontrol floating point divisions by 0.0

Fixes: Ticket7990

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 55279d699fa64d8eb1185d8db04ab4ed92e8dea2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavcodec/snowenc: Fix 2 undefined shifts
Michael Niedermayer [Mon, 14 Oct 2019 20:51:57 +0000 (22:51 +0200)]
avcodec/snowenc: Fix 2 undefined shifts

Fixes: Ticket7990

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8802e329c8317ca5ceb929df48a23eb0f9e852b2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
17 months agoavformat/nutenc: Do not pass NULL to memcmp() in get_needed_flags()
Michael Niedermayer [Fri, 1 Nov 2019 09:02:29 +0000 (10:02 +0100)]
avformat/nutenc: Do not pass NULL to memcmp() in get_needed_flags()

This compared to the other suggestions is cleaner and easier to understand
keeping the condition in the if() simple.

This affects alot of fate tests.

See: [FFmpeg-devel] [PATCH 05/11] avformat/nutenc: Don't pass NULL to memcmp
See: [FFmpeg-devel] [PATCH]lavf/nutenc: Do not call memcmp() with NULL argument

Fixes: Ticket 7980

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e4fdeb3fcefeb98f2225f7ccded156fb175959c5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>