avformat/utils: Fix integer overflow in discontinuity check