avcodec/h264_refs: Do not set reference to things which dont exist