avcodec/takdec: Fix integer overflow
authorMichael Niedermayer <michael@niedermayer.cc>
Thu, 22 Jun 2017 19:21:56 +0000 (21:21 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Fri, 23 Jun 2017 01:05:44 +0000 (03:05 +0200)
Fixes: runtime error: signed integer overflow: 512 + 2147483146 cannot be represented in type 'int'
Fixes: 2314/clusterfuzz-testcase-minimized-4519333877252096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c2ef4f6b4d52a7b7184c747ffea3576926ea1b1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/takdec.c

index 5797af9..4154f0a 100644 (file)
@@ -472,7 +472,7 @@ static int decode_subframe(TAKDecContext *s, int32_t *decoded,
             int v = 1 << (filter_quant - 1);
 
             if (filter_order & -16)
-                v += s->adsp.scalarproduct_int16(&s->residues[i], s->filter,
+                v += (unsigned)s->adsp.scalarproduct_int16(&s->residues[i], s->filter,
                                                  filter_order & -16);
             for (j = filter_order & -16; j < filter_order; j += 4) {
                 v += s->residues[i + j + 3] * s->filter[j + 3] +