avcodec/exr: fix undefined shift in pxr24_uncompress()
authorMichael Niedermayer <michael@niedermayer.cc>
Sat, 4 Nov 2017 00:19:19 +0000 (01:19 +0100)
committerMichael Niedermayer <michael@niedermayer.cc>
Sat, 30 Dec 2017 20:11:31 +0000 (21:11 +0100)
Fixes: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'
Fixes: 3787/clusterfuzz-testcase-minimized-5728764920070144

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 66f0c958bfd5475658b432d1af4d2e174b2dfcda)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/exr.c

index ec94022..b4063f8 100644 (file)
@@ -866,7 +866,7 @@ static int pxr24_uncompress(EXRContext *s, const uint8_t *src,
                 in     = ptr[2] + td->xsize;
 
                 for (j = 0; j < td->xsize; ++j) {
-                    uint32_t diff = (*(ptr[0]++) << 24) |
+                    uint32_t diff = ((unsigned)*(ptr[0]++) << 24) |
                                     (*(ptr[1]++) << 16) |
                                     (*(ptr[2]++) << 8);
                     pixel += diff;