avformat/nutdec: check for allocation failures