ff_convert_matrix: fix integer overflow