avfilter/formats: Avoid using non public AV_SAMPLE_FMT_NB