avformat/aviobuf: Delay buffer downsizing until asserts are met