avcodec/mlpdsp: Fix signed integer overflow, 2nd try
authorMichael Niedermayer <michael@niedermayer.cc>
Mon, 20 Nov 2017 17:45:45 +0000 (18:45 +0100)
committerMichael Niedermayer <michael@niedermayer.cc>
Mon, 27 Nov 2017 02:31:54 +0000 (03:31 +0100)
commit97c00edaa043043c29d985653e7e1687b56dfa23
tree706fe5c0444c3ad692b690867e0072a3df1769a1
parent883de7e8b4fee252464b15e0351c6e5733e36e54
avcodec/mlpdsp: Fix signed integer overflow, 2nd try

The outputted bits should match what is used in the lossless check

Fixes: runtime error: signed integer overflow: -538697856 * 256 cannot be represented in type 'int'
Fixes: 4326/clusterfuzz-testcase-minimized-5689449645080576

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/mlpdsp.c