avcodec/loco: Limit lossy parameter so it is sane and does not overflow