avutil/common: Fix integer overflow in av_clip_uint8_c() and av_clip_uint16_c()
authorMichael Niedermayer <michael@niedermayer.cc>
Wed, 14 Feb 2018 02:54:13 +0000 (03:54 +0100)
committerMichael Niedermayer <michael@niedermayer.cc>
Sat, 17 Feb 2018 15:11:19 +0000 (16:11 +0100)
Fixes: 5567/clusterfuzz-testcase-minimized-5769966247739392
Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ab6f571ef71967da7c7c1cfba483d3597c7357d5)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavutil/common.h

index 7fe3ccc..81f5fed 100644 (file)
@@ -162,7 +162,7 @@ static av_always_inline av_const int64_t av_clip64_c(int64_t a, int64_t amin, in
  */
 static av_always_inline av_const uint8_t av_clip_uint8_c(int a)
 {
-    if (a&(~0xFF)) return (-a)>>31;
+    if (a&(~0xFF)) return (~a)>>31;
     else           return a;
 }
 
@@ -184,7 +184,7 @@ static av_always_inline av_const int8_t av_clip_int8_c(int a)
  */
 static av_always_inline av_const uint16_t av_clip_uint16_c(int a)
 {
-    if (a&(~0xFFFF)) return (-a)>>31;
+    if (a&(~0xFFFF)) return (~a)>>31;
     else             return a;
 }