avcodec/aacenc: Tighter input checks
authorMichael Niedermayer <michael@niedermayer.cc>
Tue, 23 Aug 2016 09:00:29 +0000 (11:00 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Thu, 25 Aug 2016 01:29:36 +0000 (03:29 +0200)
Fixes occurance of NaN/Inf leading to assertion failures and out of array access
Fixes: d1c38a09acc34845c6be3a127a5aacaf/signal_sigsegv_3982225_6121_d18bd5451d4245ee09408f04badd1b83.wmv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 77bf96b04710b98a52aaddb93bfd32da0d506191)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/aacenc.c

index 2653cef..4b80d38 100644 (file)
@@ -622,8 +622,8 @@ static int aac_encode_frame(AVCodecContext *avctx, AVPacket *avpkt,
             }
 
             for (k = 0; k < 1024; k++) {
-                if (!isfinite(cpe->ch[ch].coeffs[k])) {
-                    av_log(avctx, AV_LOG_ERROR, "Input contains NaN/+-Inf\n");
+                if (!(fabs(cpe->ch[ch].coeffs[k]) < 1E16)) { // Ensure headroom for energy calculation
+                    av_log(avctx, AV_LOG_ERROR, "Input contains (near) NaN/+-Inf\n");
                     return AVERROR(EINVAL);
                 }
             }