avcodec/aacdec_fixed: Fix integer overflow in predict()
authorMichael Niedermayer <michael@niedermayer.cc>
Fri, 27 Oct 2017 00:23:20 +0000 (02:23 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Fri, 1 Dec 2017 23:28:59 +0000 (00:28 +0100)
Fixes: runtime error: signed integer overflow: -2110708110 + -82837504 cannot be represented in type 'int'
Fixes: 3547/clusterfuzz-testcase-minimized-6009386439802880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0976752420706c0a8b3cb8fd61497a47c7d7270f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/aacdec_fixed.c

index 8c8eeaf..21d81e0 100644 (file)
@@ -307,9 +307,9 @@ static av_always_inline void predict(PredictorState *ps, int *coef,
 
         if (shift < 31) {
             if (shift > 0) {
-                *coef += (pv.mant + (1 << (shift - 1))) >> shift;
+                *coef += (unsigned)((pv.mant + (1 << (shift - 1))) >> shift);
             } else
-                *coef += pv.mant << -shift;
+                *coef += (unsigned)(pv.mant << -shift);
         }
     }