avcodec/snowdec: Fix integer overflow in decode_subband_slice_buffered()
authorMichael Niedermayer <michael@niedermayer.cc>
Sun, 27 Aug 2017 22:30:33 +0000 (00:30 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Tue, 29 Aug 2017 19:21:05 +0000 (21:21 +0200)
Fixes: runtime error: signed integer overflow: 267 * 8388608 cannot be represented in type 'int'
Fixes: 2743/clusterfuzz-testcase-minimized-5820652076400640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 732f9764561558a388c05483ed6a722a5c67b05c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/snowdec.c

index 4ebfa07..0ac0b55 100644 (file)
@@ -140,7 +140,7 @@ static inline void decode_subband_slice_buffered(SnowContext *s, SubBand *b, sli
         v = b->x_coeff[new_index].coeff;
         x = b->x_coeff[new_index++].x;
         while(x < w){
-            register int t= ( (v>>1)*qmul + qadd)>>QEXPSHIFT;
+            register int t= (int)( (v>>1)*(unsigned)qmul + qadd)>>QEXPSHIFT;
             register int u= -(v&1);
             line[x] = (t^u) - u;