avcodec/mdct_*: Fix integer overflow in addition in RESCALE()
authorMichael Niedermayer <michael@niedermayer.cc>
Sun, 5 Nov 2017 20:20:06 +0000 (21:20 +0100)
committerMichael Niedermayer <michael@niedermayer.cc>
Mon, 13 Nov 2017 19:05:25 +0000 (20:05 +0100)
commit8b46a951f9f7c1b5cefdf7620cc4d60b0a236a41
tree4b2d073c2a020d856999e1d7aa0e20a5cc62db3c
parentccc81f846e324c45e55da1ad03e36b5962f4d6cc
avcodec/mdct_*: Fix integer overflow in addition in RESCALE()

Fixes: runtime error: signed integer overflow: 1219998458 - -1469874012 cannot be represented in type 'int'
Fixes: 3443/clusterfuzz-testcase-minimized-5369987105554432

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 770c934fa1635f4fadf5db4fc5cc5ad15d82455a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/mdct_fixed.c
libavcodec/mdct_template.c