avcodec/diracdec: Fix integer overflow in divide3()
authorMichael Niedermayer <michael@niedermayer.cc>
Thu, 27 Jul 2017 21:49:27 +0000 (23:49 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Sat, 29 Jul 2017 12:23:25 +0000 (14:23 +0200)
Fixes: runtime error: signed integer overflow: -1073746548 * 21845 cannot be represented in type 'int'
Fixes: 2729/clusterfuzz-testcase-minimized-5902915464069120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c0220c768c7fc933a76c863ebbb0abdf68a88533)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/diracdec.c

index 3965861..450a5bb 100644 (file)
@@ -231,7 +231,7 @@ enum dirac_subband {
 /* magic number division by 3 from schroedinger */
 static inline int divide3(int x)
 {
-    return ((x+1)*21845 + 10922) >> 16;
+    return (int)((x+1U)*21845 + 10922) >> 16;
 }
 
 static DiracFrame *remove_frame(DiracFrame *framelist[], int picnum)