avcodec/utvideodec: Check subsample factors