libopenjpegenc: fix out-of-bounds reads when filling the edges
authorAndreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Thu, 13 Oct 2016 20:14:46 +0000 (22:14 +0200)
committerAndreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fri, 14 Oct 2016 14:56:14 +0000 (16:56 +0200)
commit56706ac0d5723cb549fec2602e798ab1bf6004cd
tree3da6bcbaaaf751afb5b66d27481f1ed77f49ef24
parent69c8505f3bf54f316e9dc8bec1c71dfa1febec63
libopenjpegenc: fix out-of-bounds reads when filling the edges

The calculation of width/height should round up, not round down to
prevent setting width or height to 0.

Also image->comps[compno].w is unsigned (at least in openjpeg2), so the
calculation could silently wrap around without the explicit cast to int.

Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
libavcodec/libopenjpegenc.c