avcodec/h264_cabac: Tighten allowed coeff_abs range
authorMichael Niedermayer <michael@niedermayer.cc>
Tue, 13 Feb 2018 23:32:30 +0000 (00:32 +0100)
committerMichael Niedermayer <michael@niedermayer.cc>
Mon, 19 Feb 2018 01:52:16 +0000 (02:52 +0100)
Fixes: integer overflows
Reported-by: "Xiaohan Wang (王消寒)" <xhwang@chromium.org>
Based on limits in "8.5 Transform coefficient decoding process and picture
construction process prior to deblocking  filter process"

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f26a63c4ee1bdbe21d7ab462cd66f8ba20b14244)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/h264_cabac.c

index 649fa82..a450b4e 100644 (file)
@@ -1736,7 +1736,7 @@ decode_cabac_residual_internal(const H264Context *h, H264SliceContext *sl,
 \
             if( coeff_abs >= 15 ) { \
                 int j = 0; \
-                while (get_cabac_bypass(CC) && j < 30) { \
+                while (get_cabac_bypass(CC) && j < 16+7) { \
                     j++; \
                 } \
 \