avcodec/4xm: Fix signed integer overflows in idct()
authorMichael Niedermayer <michael@niedermayer.cc>
Wed, 26 Jun 2019 22:15:03 +0000 (00:15 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Mon, 8 Jul 2019 09:49:51 +0000 (11:49 +0200)
Fixes: signed integer overflow: 20242 * 121095 cannot be represented in type 'int'
Fixes: 15310/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5737051745419264

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2bbea155bf7c6ce6d5ae53cc41e44798cad2f39c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/4xm.c

index 8e05a4c..7e6a15e 100644 (file)
@@ -158,7 +158,7 @@ typedef struct FourXContext {
 #define FIX_1_847759065 121095
 #define FIX_2_613125930 171254
 
-#define MULTIPLY(var, const) (((var) * (const)) >> 16)
+#define MULTIPLY(var, const) ((int)((var) * (unsigned)(const)) >> 16)
 
 static void idct(int16_t block[64])
 {