avcodec/prores_ks: Fix luma quantization if q >= MAX_STORED_Q
authorAlex Mogurenko <alex@mogurenko.com>
Fri, 28 Dec 2018 20:30:08 +0000 (22:30 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Wed, 13 Mar 2019 23:24:44 +0000 (00:24 +0100)
commit7e204f72607c95a7d7f791e085ba48882e778a9b
tree26b840f6af0d36d6f108337f94be17342bc53335
parent53f3f5233f38aef0b544ebfbb2c450134aae0639
avcodec/prores_ks: Fix luma quantization if q >= MAX_STORED_Q

The problem occurs in slice quant estimation and slice encoding:

If the slice quant is larger than  MAX_STORED_Q we don't use pre-calculated
quant matrices, but generate a new one, but both qmat and qmat_chroma both
point to the same table, so the luma table ends up having chroma table
values.

Add custom_chroma_q the same way as custom_q.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
(cherry picked from commit e4788ae31b2e9af45d11f4bf4498c075dcc25a6c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/proresenc_kostya.c