avcodec/wavpack: Fix integer overflow in FFABS
authorMichael Niedermayer <michael@niedermayer.cc>
Wed, 31 Jan 2018 01:50:18 +0000 (02:50 +0100)
committerMichael Niedermayer <michael@niedermayer.cc>
Mon, 19 Feb 2018 01:40:54 +0000 (02:40 +0100)
Fixes: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 5396/clusterfuzz-testcase-minimized-6558555529281536

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e50bd61e4ff97bd7fc6cbd7ec4ca514e17a70c4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/wavpack.c

index 0429adc..34f73fe 100644 (file)
@@ -474,7 +474,7 @@ static inline int wv_unpack_stereo(WavpackFrameContext *s, GetBitContext *gb,
         }
 
         if (type == AV_SAMPLE_FMT_S16P) {
-            if (FFABS(L) + (unsigned)FFABS(R) > (1<<19)) {
+            if (FFABS((int64_t)L) + FFABS((int64_t)R) > (1<<19)) {
                 av_log(s->avctx, AV_LOG_ERROR, "sample %d %d too large\n", L, R);
                 return AVERROR_INVALIDDATA;
             }