avcodec/nvenc: add some more error case checks