avutil/integer: Fix integer overflow in av_mul_i()
authorMichael Niedermayer <michael@niedermayer.cc>
Tue, 23 Oct 2018 23:44:12 +0000 (01:44 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Wed, 24 Oct 2018 22:24:54 +0000 (00:24 +0200)
Found-by: fate
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3cc3cb663bf3061e40356392d2f7638de6a479fe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavutil/integer.c

index 6d6855f..ba4aa77 100644 (file)
@@ -74,7 +74,7 @@ AVInteger av_mul_i(AVInteger a, AVInteger b){
 
         if(a.v[i])
             for(j=i; j<AV_INTEGER_SIZE && j-i<=nb; j++){
-                carry= (carry>>16) + out.v[j] + a.v[i]*b.v[j-i];
+                carry= (carry>>16) + out.v[j] + a.v[i]*(unsigned)b.v[j-i];
                 out.v[j]= carry;
             }
     }