msrledec: use signed pixel_ptr in msrle_decode_pal4
authorAndreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Thu, 16 Apr 2015 12:49:08 +0000 (14:49 +0200)
committerMichael Niedermayer <michaelni@gmx.at>
Fri, 17 Apr 2015 13:20:30 +0000 (15:20 +0200)
This fixes segmentation faults, when pic->linesize[0] is negative.
In that case 'line * pic->linesize[0] + pixel_ptr' is treated as
unsigned and wraps around.

This reverts commit 7d78a964.
The problem was introduced in commit f7e1367f, which should obsolete
that commit.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ae6fd7300b4e9f81d3b5ba201096ffe7cccf26fb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
libavcodec/msrledec.c

index deb6f86..200221a 100644 (file)
@@ -36,7 +36,7 @@ static int msrle_decode_pal4(AVCodecContext *avctx, AVPicture *pic,
     unsigned char rle_code;
     unsigned char extra_byte, odd_pixel;
     unsigned char stream_byte;
-    unsigned int pixel_ptr = 0;
+    int pixel_ptr = 0;
     int line = avctx->height - 1;
     int i;