avformat/rawvideodec: Rework packet size calculation