avcodec/shorten: Fix signed 32bit overflow in shift in shorten_decode_frame()
authorMichael Niedermayer <michael@niedermayer.cc>
Sun, 12 Aug 2018 21:06:55 +0000 (23:06 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Sun, 28 Oct 2018 00:22:42 +0000 (02:22 +0200)
Fixes: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 9480/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-6647324284551168 -rss_limit_mb=2000

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9b604e96a51a1fca92bbabfe4f7ac53f0470ee41)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/shorten.c

index 054494f..1ffb7d8 100644 (file)
@@ -715,7 +715,7 @@ static int shorten_decode_frame(AVCodecContext *avctx, void *data,
                 if (s->version < 2)
                     s->offset[channel][s->nmean - 1] = sum / s->blocksize;
                 else
-                    s->offset[channel][s->nmean - 1] = s->bitshift == 32 ? 0 : (sum / s->blocksize) * (1 << s->bitshift);
+                    s->offset[channel][s->nmean - 1] = s->bitshift == 32 ? 0 : (sum / s->blocksize) * (1LL << s->bitshift);
             }
 
             /* copy wrap samples for use with next block */