avcodec/apedec: Fix multiple integer overflows and undefined behaviorin filter_3800()
authorMichael Niedermayer <michael@niedermayer.cc>
Sun, 16 Jun 2019 09:39:15 +0000 (11:39 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Sun, 4 Aug 2019 19:16:28 +0000 (21:16 +0200)
commita33fd082661faa0a6eda7a4ede722fdb865ddc8b
tree95a71e470963ba9433609ac9167b4eca1dc96267
parent5fa0b18c95a0324a933101a727106e7cbf6597b6
avcodec/apedec: Fix multiple integer overflows and undefined behaviorin filter_3800()

Fixes: left shift of negative value -4
Fixes: signed integer overflow: -15091694 * 167 cannot be represented in type 'int'
Fixes: signed integer overflow: 1898547155 + 453967445 cannot be represented in type 'int'
Fixes: 15258/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5759095564402688
Fixes: signed integer overflow: 962196438 * 31 cannot be represented in type 'int'
Fixes: 15364/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5718799845687296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 267eb2ab7f87696e1a156ca9a5ff1b1628d170c1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/apedec.c