avcodec/libtheoraenc: Check for av_malloc failure