avcodec/mjpegdec: Fix integer overflow in ljpeg_decode_rgb_scan()
authorMichael Niedermayer <michael@niedermayer.cc>
Tue, 5 Jun 2018 00:17:24 +0000 (02:17 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Mon, 9 Jul 2018 23:52:56 +0000 (01:52 +0200)
Fixes: signed integer overflow: 32768 + 2147450880 cannot be represented in type 'int'
Fixes: 7885/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_THP_fuzzer-5298834394578944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 936f4a2c2e14ec753e8835f2e820b4cd9aec9a56)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/mjpegdec.c

index 2e10122..e061742 100644 (file)
@@ -1002,7 +1002,7 @@ static int ljpeg_decode_rgb_scan(MJpegDecodeContext *s, int nb_components, int p
                     return -1;
 
                 left[i] = buffer[mb_x][i] =
-                    mask & (pred + (dc * (1 << point_transform)));
+                    mask & (pred + (unsigned)(dc * (1 << point_transform)));
             }
 
             if (s->restart_interval && !--s->restart_count) {