ffmpeg.git
3 years agobrstm: make sure an ADPC chunk was read for adpcm_thp
Andreas Cadhalpun [Mon, 4 Jan 2016 11:57:38 +0000 (12:57 +0100)]
brstm: make sure an ADPC chunk was read for adpcm_thp

This fixes NULL pointer dereferencing.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d7d37c479fa71639650751648275615e979beb33)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agovorbisdec: reject rangebits 0 with non-0 partitions
Andreas Cadhalpun [Sun, 3 Jan 2016 18:11:24 +0000 (19:11 +0100)]
vorbisdec: reject rangebits 0 with non-0 partitions

This causes non-unique elements in floor_setup->data.t1.list, which
makes the stream undecodable according to the specification.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit e7a7b3135a4e5ba4bd2e144444d95a7563f53e9b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agovorbisdec: reject channel mapping with less than two channels
Andreas Cadhalpun [Sun, 3 Jan 2016 18:20:54 +0000 (19:20 +0100)]
vorbisdec: reject channel mapping with less than two channels

It causes the angle channel number to equal the magnitude channel
number, which makes the stream undecodable according to the
specification.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b4b13848dec5420fa5dd9e1a7d4dfae5de1932d5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoffmdec: reset packet_end in case of failure
Andreas Cadhalpun [Sat, 2 Jan 2016 15:27:02 +0000 (16:27 +0100)]
ffmdec: reset packet_end in case of failure

This fixes segmentation faults caused by passing a packet_ptr of NULL to
memcpy.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 40eb2531b279abe008012c5c2c292552d3e62449)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoavformat/ipmovie: put video decoding_map_size into packet and use it in decoder
Paul B Mahol [Sun, 1 Nov 2015 16:02:26 +0000 (17:02 +0100)]
avformat/ipmovie: put video decoding_map_size into packet and use it in decoder

The size of decoding map can differ from one calculated
internally, producing artifacts while decoding video.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit c293ef258cbb2c058e23651a26edf46e3bc05050)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoavcodec/wavpackenc: print channel count in av_log call n2.7.5
James Almer [Wed, 13 Jan 2016 22:26:40 +0000 (19:26 -0300)]
avcodec/wavpackenc: print channel count in av_log call

Fixes a warning with -Wformat-extra-args
(cherry picked from commit 17e7fdf61a04f52c499e2d06eab2cf2d22343aa9)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoUpdate for 2.7.5
Michael Niedermayer [Fri, 15 Jan 2016 15:29:16 +0000 (16:29 +0100)]
Update for 2.7.5

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoconfigure: bump copyright year to 2016
James Almer [Sat, 2 Jan 2016 19:28:31 +0000 (16:28 -0300)]
configure: bump copyright year to 2016

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 78129978f02f27d76ecaf2cd1a7bf7a47253fdab)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/hls: Even stricter URL checks
Michael Niedermayer [Fri, 15 Jan 2016 14:29:22 +0000 (15:29 +0100)]
avformat/hls: Even stricter URL checks

This fixes a null pointer dereference at least

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cfda1bea4c18ec1edbc11ecc465f788b02851488)

Conflicts:

libavformat/hls.c

3 years agoavformat/hls: More strict url checks
Michael Niedermayer [Fri, 15 Jan 2016 12:29:38 +0000 (13:29 +0100)]
avformat/hls: More strict url checks

No case is known where these are needed

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6ba42b6482c725a59eb468391544dc0c75b8c6f0)

Conflicts:

libavformat/hls.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/utils: Detect and skip unneeded sws_setColorspaceDetails() calls
Michael Niedermayer [Thu, 14 Jan 2016 14:11:48 +0000 (15:11 +0100)]
swscale/utils: Detect and skip unneeded sws_setColorspaceDetails() calls

This avoids running various table inits unnecessarily

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc538e9dbd14b61d1ac8c9fa687d83289673fe90)

Conflicts:

libswscale/utils.c

3 years agoswscale/yuv2rgb: Increase YUV2RGB table headroom
Michael Niedermayer [Thu, 14 Jan 2016 02:05:11 +0000 (03:05 +0100)]
swscale/yuv2rgb: Increase YUV2RGB table headroom

This makes SWS more robust
Fixes: 07650a772d98aa63b0fed6370dc89037/asan_heap-oob_27ddeaf_2657_2c81ff264dee5d9712cb3251fb9c3bbb.264
Fixes: out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8f3a9a8c278acf886f70a1d743bc07b6f9c7b51a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/yuv2rgb: Factor YUVRGB_TABLE_LUMA_HEADROOM out
Michael Niedermayer [Thu, 14 Jan 2016 11:36:41 +0000 (12:36 +0100)]
swscale/yuv2rgb: Factor YUVRGB_TABLE_LUMA_HEADROOM out

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e5f82a28737fba4402259617500911cc37e3674)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/hls: forbid all protocols except http(s) & file
Maxim Andreev [Wed, 13 Jan 2016 08:51:12 +0000 (11:51 +0300)]
avformat/hls: forbid all protocols except http(s) & file

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7145e80b4f78cff5ed5fee04d4c4d53daaa0e077)

Conflicts:

libavformat/hls.c

3 years agoavformat/aviobuf: Fix end check in put_str16()
Michael Niedermayer [Wed, 13 Jan 2016 01:31:59 +0000 (02:31 +0100)]
avformat/aviobuf: Fix end check in put_str16()

Fixes out of array read
Fixes: 03c406ec9530e594a074ce2979f8a1f0/asan_heap-oob_7dec26_4664_37c52495b2870a2eaac65f53958e76c1.flac

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 115fb6d03ef6310732b42258d8c3cd1839cfb74b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/asfenc: Check pts
Michael Niedermayer [Tue, 12 Jan 2016 17:49:20 +0000 (18:49 +0100)]
avformat/asfenc: Check pts

Fixes integer overflow
Fixes: 0063df8be3aaa30dd6d76f59c8f818c8/signal_sigsegv_7b7b59_3634_bf418b6822bbfa68734411d96b667be3.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c0b84d89911b2035161f5ef51aafbfcc84aa9e2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mpeg4video: Check time_incr
Michael Niedermayer [Tue, 12 Jan 2016 02:03:01 +0000 (03:03 +0100)]
avcodec/mpeg4video: Check time_incr

Fixes assertion failure
Fixes out of memory access

Fixes: test_casex.ivf

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c97946d6131b31340954a3f603b6bf92590a9a5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/wavpackenc: Check the number of channels
Michael Niedermayer [Mon, 11 Jan 2016 17:58:08 +0000 (18:58 +0100)]
avcodec/wavpackenc: Check the number of channels

They are stored in a byte, thus more than 255 is not possible

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59c915a403af32c4ff5126625b0cc7e38f4beff9)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/wavpackenc: Headers are per channel
Michael Niedermayer [Mon, 11 Jan 2016 17:32:32 +0000 (18:32 +0100)]
avcodec/wavpackenc: Headers are per channel

Fixes: 1b8b83a53bfa751f01b1daa65a4758db/signal_sigabrt_7ffff6ae7cb7_7488_403f71d1a2565b598d01b6cb110fac8f.aiff
Fixes: assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26757b0279b4b93c6066c2151d4d3dbd2ec266bf)

Conflicts:

libavcodec/wavpackenc.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/aacdec_template: Check id_map
Michael Niedermayer [Sun, 10 Jan 2016 18:29:39 +0000 (19:29 +0100)]
avcodec/aacdec_template: Check id_map

Fixes index out of bounds error
Fixes: aac_index_out_of_bounds.wmv

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 590863876d1478547640304a31c15809c3618090)

Conflicts:

libavcodec/aacdec_template.c

3 years agoavcodec/dvdec: Fix "left shift of negative value -254"
Michael Niedermayer [Sun, 10 Jan 2016 16:43:56 +0000 (17:43 +0100)]
avcodec/dvdec: Fix "left shift of negative value -254"

Fixes: dvdec_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 93ac72a98dff592ffc174cfb36a8975dfbf145ae)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mjpegdec: Fix negative shift
Michael Niedermayer [Sun, 10 Jan 2016 14:52:09 +0000 (15:52 +0100)]
avcodec/mjpegdec: Fix negative shift

Fixes: mjpeg_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d86d7b2486cd5c31db8e820d8a89554abf19567e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mss2: Check for repeat overflow
Michael Niedermayer [Sun, 10 Jan 2016 11:19:48 +0000 (12:19 +0100)]
avcodec/mss2: Check for repeat overflow

Fixes: mss2_left_shift.wmv

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e273dade78943e22b71d0ddb67cd0d737fc26edf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat: Add integer fps from 31 to 60 to get_std_framerate()
Michael Niedermayer [Sat, 9 Jan 2016 09:49:23 +0000 (10:49 +0100)]
avformat: Add integer fps from 31 to 60 to get_std_framerate()

Fixes Ticket 5106

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2039b3e7511ef183dae206575114e15b6d99c134)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mpegvideo_enc: Clip bits_per_raw_sample within valid range
Michael Niedermayer [Wed, 6 Jan 2016 23:22:56 +0000 (00:22 +0100)]
avcodec/mpegvideo_enc: Clip bits_per_raw_sample within valid range

Fixes out of array read
Fixes: test_case-mdc.264 (b47be15a120979f5a1a945c938cbef33)

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13f266b50cc7554028d22480b7e4383968e64a63)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavfilter/vf_scale: set proper out frame color range
Thomas Mundt [Wed, 30 Dec 2015 23:01:21 +0000 (00:01 +0100)]
avfilter/vf_scale: set proper out frame color range

Prevents that following scalers in the filter chain will do unintentional color range conversions.
Fixes Ticket #5096

Signed-off-by: Thomas Mundt <loudmax@yahoo.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 73ce8162f3499cf0e86d1d80dea53324bd62bcb3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/motion_est: Fix mv_penalty table size
Michael Niedermayer [Tue, 5 Jan 2016 13:41:04 +0000 (14:41 +0100)]
avcodec/motion_est: Fix mv_penalty table size

Fixes out of array read

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5b4da8a38a5ed211df9504c85ce401c30af86b97)

Conflicts:

libavcodec/motion_est.h

3 years agoavcodec/h264_slice: Fix integer overflow in implicit weight computation
Michael Niedermayer [Tue, 5 Jan 2016 00:06:18 +0000 (01:06 +0100)]
avcodec/h264_slice: Fix integer overflow in implicit weight computation

Fixes mozilla bug 1230423

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7cc01c25727a96eaaa0c177234b626e47c8ea491)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/utils: Use normal bilinear scaler if fast cannot be used due to tiny dimensions
Michael Niedermayer [Mon, 4 Jan 2016 22:22:25 +0000 (23:22 +0100)]
swscale/utils: Use normal bilinear scaler if fast cannot be used due to tiny dimensions

Fixes Ticket4960

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edf129cbc897447a289ca8b045853df5df1bab3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/put_bits: Always check buffer end before writing
Michael Niedermayer [Fri, 1 Jan 2016 01:41:06 +0000 (02:41 +0100)]
avcodec/put_bits: Always check buffer end before writing

This causes a overall slowdown of 0.1 % (tested with mpeg4 single thread encoding of matrixbench at QP=3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cccb0ffccc3723acc7aab3a859b24743596dd9c0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agomjpegdec: extend check for incompatible values of s->rgb and s->ls
Andreas Cadhalpun [Thu, 31 Dec 2015 15:55:43 +0000 (16:55 +0100)]
mjpegdec: extend check for incompatible values of s->rgb and s->ls

This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.

In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.

This fixes an out-of-bounds read in ls_decode_line.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7ea2db6eafa0a8a9497aab20be2cfc8742a59072)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/utils: Fix intermediate format for cascaded alpha downscaling
Michael Niedermayer [Thu, 24 Dec 2015 20:46:15 +0000 (21:46 +0100)]
swscale/utils: Fix intermediate format for cascaded alpha downscaling

Fixes Ticket4926

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b83d8be6bff7d645469a623aee0b380541da15cf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agox86/float_dsp: zero extend offset from ff_scalarproduct_float_sse
James Almer [Fri, 8 Jan 2016 15:08:56 +0000 (12:08 -0300)]
x86/float_dsp: zero extend offset from ff_scalarproduct_float_sse

Reviewed-by: Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit dc79824deb6ac0ce236589c618744b33629201cd)

3 years agoavfilter/vf_zoompan: do not free frame we pushed to lavfi
Paul B Mahol [Sat, 2 Jan 2016 17:51:11 +0000 (18:51 +0100)]
avfilter/vf_zoompan: do not free frame we pushed to lavfi

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 8bcd1997eadb0d79a049227a1d1afe6111397baa)

Fixes ticket #5113.

3 years agoUpdate for 2.7.4 n2.7.4
Michael Niedermayer [Mon, 21 Dec 2015 13:52:20 +0000 (14:52 +0100)]
Update for 2.7.4

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agonuv: sanitize negative fps rate
Andreas Cadhalpun [Wed, 16 Dec 2015 19:52:39 +0000 (20:52 +0100)]
nuv: sanitize negative fps rate

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f6830cf5ba03fdcfcd81a0358eb32d4081a2fcce)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agorawdec: only exempt BIT0 with need_copy from buffer sanity check
Andreas Cadhalpun [Sat, 19 Dec 2015 22:45:06 +0000 (23:45 +0100)]
rawdec: only exempt BIT0 with need_copy from buffer sanity check

Otherwise the too small buffer is directly used in the frame, causing
segmentation faults, when trying to use the frame.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 699e68371ec7e381e5cc48e3d96e29c669261af7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agomlvdec: check that index_entries exist
Andreas Cadhalpun [Sat, 19 Dec 2015 22:44:53 +0000 (23:44 +0100)]
mlvdec: check that index_entries exist

This fixes NULL pointer dereferencing.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9fcfe4a3cdf9a5af0c37758b178965b7b99582d4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agonutdec: reject negative value_len in read_sm_data
Andreas Cadhalpun [Sat, 19 Dec 2015 11:02:56 +0000 (12:02 +0100)]
nutdec: reject negative value_len in read_sm_data

If it is negative, it can cause the byte position to move backwards in
avio_skip, which in turn makes sm_size negative and thus size larger
than the size of the packet buffer, causing invalid writes in avio_read.

Also fix potential overflow of avio_tell(bc) + value_len.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ce10f572c12b0d172c72d31d8c979afce602bf0c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoxwddec: prevent overflow of lsize * avctx->height
Andreas Cadhalpun [Fri, 18 Dec 2015 18:28:51 +0000 (19:28 +0100)]
xwddec: prevent overflow of lsize * avctx->height

This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9d38f06d05efbb9d6196c27668eb943e934943ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agonutdec: only copy the header if it exists
Andreas Cadhalpun [Fri, 18 Dec 2015 14:18:47 +0000 (15:18 +0100)]
nutdec: only copy the header if it exists

Fixes ubsan runtime error: null pointer passed as argument 2, which is
declared to never be null

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9f82506c79874edd7b09707ab63d9e72078de8f9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoexr: fix out of bounds read in get_code
Andreas Cadhalpun [Sun, 13 Dec 2015 22:17:09 +0000 (23:17 +0100)]
exr: fix out of bounds read in get_code

This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 90b99a81071d10e6b5efe86a4602d54d4f45bbcb)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoon2avc: limit number of bits to 30 in get_egolomb
Andreas Cadhalpun [Wed, 16 Dec 2015 15:48:19 +0000 (16:48 +0100)]
on2avc: limit number of bits to 30 in get_egolomb

More don't fit into the integer output.

Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 4d5c3b02e9d2c9a630ca433fabca43285879e0b8)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoavcodec/mpeg4videodec: also for empty partitioned slices
Michael Niedermayer [Sat, 19 Dec 2015 22:21:33 +0000 (23:21 +0100)]
avcodec/mpeg4videodec: also for empty partitioned slices

Fixes assertion failure
Fixes: id_acf3e47f864e1ee4c7b86c0653e0ff31e5bde56e.m4v

Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 70f13abb4f9a376ddc0d2c566739bc3c6a0c47e7)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/h264_refs: Fix long_idx check
Michael Niedermayer [Sat, 19 Dec 2015 20:59:42 +0000 (21:59 +0100)]
avcodec/h264_refs: Fix long_idx check

Fixes out of array read
Fixes mozilla bug 1233606

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b92b4775a0d07cacfdd2b4be6511f3cb362c977b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/h264_mc_template: prefetch list1 only if it is used in the MB
Michael Niedermayer [Thu, 17 Dec 2015 23:20:51 +0000 (00:20 +0100)]
avcodec/h264_mc_template: prefetch list1 only if it is used in the MB

Fixes ubsan warning
Fixes Mozilla bug 1230276

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8ea57664fe3ad611c9ecd234670544ddff7ca55)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/h264_slice: Simplify ref2frm indexing
Michael Niedermayer [Thu, 17 Dec 2015 21:51:00 +0000 (22:51 +0100)]
avcodec/h264_slice: Simplify ref2frm indexing

This also suppresses a ubsan warning
Fixes Mozilla bug 1230247

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef8f6464a55db730cab8c48a1a51fa4e6ca12107)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoRevert "avcodec/aarch64/neon.S: Update neon.s for transpose_4x4H"
Michael Niedermayer [Thu, 17 Dec 2015 20:14:45 +0000 (21:14 +0100)]
Revert "avcodec/aarch64/neon.S: Update neon.s for transpose_4x4H"

The change was not correct and broke H264

This reverts commit cd83f899c94f691b045697d12efa21f83eb2329f.
(cherry picked from commit 95b59bfb9d9e47de8438183a035e02667946f27c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavfilter/vf_mpdecimate: Add missing emms_c()
Michael Niedermayer [Mon, 14 Dec 2015 17:56:13 +0000 (18:56 +0100)]
avfilter/vf_mpdecimate: Add missing emms_c()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 997de2e8107cc4256e50611463d609b18fe9619f)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agosonic: make sure num_taps * channels is not larger than frame_size
Andreas Cadhalpun [Tue, 15 Dec 2015 22:43:03 +0000 (23:43 +0100)]
sonic: make sure num_taps * channels is not larger than frame_size

If that is the case, the loop setting predictor_state in
sonic_decode_frame causes out of bounds reads of int_samples, which has
only frame_size number of elements.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9637c2531f7eb040ad1c3cb46cb40a63dfc77b80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoopus_silk: fix typo causing overflow in silk_stabilize_lsf
Andreas Cadhalpun [Tue, 15 Dec 2015 21:00:31 +0000 (22:00 +0100)]
opus_silk: fix typo causing overflow in silk_stabilize_lsf

Due to this typo max_center can be too large, causing nlsf to be set to
too large values, which in turn can cause nlsf[i - 1] + min_delta[i] to
overflow to a negative value, which is not allowed for nlsf and can
cause an out of bounds read in silk_lsf2lpc.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f61d44b74aaae1d306d8a0d38b7b3d4292c89ced)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoffm: reject invalid codec_id and codec_type
Andreas Cadhalpun [Mon, 14 Dec 2015 21:11:55 +0000 (22:11 +0100)]
ffm: reject invalid codec_id and codec_type

A negative codec_id cannot be handled by the found_decoder API of
AVStream->info: if the codec_id is not recognized, found_decoder is set
to -codec_id, which has to be '<0' according to the API documentation.

This can cause NULL pointer dereferencing in try_decode_frame.

Also make sure the codec_type matches the expected one for codec_id.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ecf63b7cc24b9fd3e6d604313325dd1ada4db662)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agogolomb: always check for invalid UE golomb codes in get_ue_golomb
Andreas Cadhalpun [Sun, 13 Dec 2015 20:02:16 +0000 (21:02 +0100)]
golomb: always check for invalid UE golomb codes in get_ue_golomb

Also correct the check to reject log < 7, because UPDATE_CACHE only
guarantees 25 meaningful bits.

This fixes undefined behavior:
runtime error: shift exponent is negative

Testing with START/STOP timers in get_ue_golomb, one for the first
branch (A) and one for the second (B), shows that there is practically no
slowdown, e.g. for the cavs decoder:

With the check in the B branch:
    629 decicycles in get_ue_golomb B, 4194260 runs,     44 skips
    433 decicycles in get_ue_golomb A,268434102 runs,   1354 skips

Without the check:
    624 decicycles in get_ue_golomb B, 4194273 runs,     31 skips
    433 decicycles in get_ue_golomb A,268434203 runs,   1253 skips

Since the B branch is executed far less often than the A branch, this
change is negligible, even more so for the h264 decoder, where the ratio
B/A is a lot smaller.

Fixes: mozilla bug 1230239
Fixes: fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit

Found-by: Tyson Smith
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 22e960ad478e568f4094971a58c6ad8f549c0180)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoaaccoder: prevent crash of anmr coder
Andreas Cadhalpun [Fri, 4 Dec 2015 17:13:07 +0000 (18:13 +0100)]
aaccoder: prevent crash of anmr coder

If minq is negative, the range of sf_idx can be larger than
SCALE_MAX_DIFF allows, causing assertion failures later in
encode_scale_factors.

Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a4652dd5da0502ff21c183b5ca7d76b1cfd6c51)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoffmdec: reject zero-sized chunks
Andreas Cadhalpun [Wed, 2 Dec 2015 21:47:12 +0000 (22:47 +0100)]
ffmdec: reject zero-sized chunks

If size is zero, avio_get_str fails, leaving the buffer uninitialized.
This causes invalid reads in av_set_options_string.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a611375db532c3d5363d97b10fadd0211811a4fd)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoswscale/x86/rgb2rgb_template: Fallback to mmx in interleaveBytes() if the alignment...
Michael Niedermayer [Tue, 15 Dec 2015 01:50:20 +0000 (02:50 +0100)]
swscale/x86/rgb2rgb_template: Fallback to mmx in interleaveBytes() if the alignment is insufficient for SSE*

This also as a sideeffect fixes the non aligned case

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a066ff89bcbae6033c2ffda9271cad84f6c1b807)

3 years agoswscale/x86/rgb2rgb_template: Do not crash on misaligend stride
Michael Niedermayer [Tue, 15 Dec 2015 01:06:04 +0000 (02:06 +0100)]
swscale/x86/rgb2rgb_template: Do not crash on misaligend stride

Fixes Ticket5013

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 80bfce35ccd11458e97f68f417fc094c5347070c)

3 years agoavformat/mxfenc: Do not crash if there is no packet in the first stream
Michael Niedermayer [Sun, 13 Dec 2015 15:13:22 +0000 (16:13 +0100)]
avformat/mxfenc: Do not crash if there is no packet in the first stream

Fixes: Ticket4914

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b51e7554e74cbf007a1cab83c7bed3ad9fa2793a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/aarch64/neon.S: Update neon.s for transpose_4x4H
zjh8890 [Sat, 21 Nov 2015 16:07:35 +0000 (00:07 +0800)]
avcodec/aarch64/neon.S: Update neon.s for transpose_4x4H

The transpose_4x4H is wrong which cost me much time to find this bug. The orders of r2 and r3 are wrong,
this bug waste me much time while I make aarch64 arm instruction which used the function.
(cherry picked from commit c18176bd551b4616757080376707637e30547fd0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/utils: estimate_timings_from_pts - increase retry counter, fixes invalid...
Rainer Hochecker [Sun, 15 Nov 2015 12:58:50 +0000 (13:58 +0100)]
avformat/utils: estimate_timings_from_pts - increase retry counter, fixes invalid duration for ts files with hevc codec

Fixes a mpegts file with hevc that fails estimating duration. Increasing number of
retries fixes the issue.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d8c2f1a28073d451c7db31291c333cb15ca3d0b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/matroskaenc: Check codecdelay before use
Michael Niedermayer [Wed, 9 Dec 2015 15:16:46 +0000 (16:16 +0100)]
avformat/matroskaenc: Check codecdelay before use

Fixes CID1238790

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e6971db12b8ae49712b77378fa8141de4904082b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavutil/mathematics: Fix division by 0
Michael Niedermayer [Wed, 9 Dec 2015 16:39:38 +0000 (17:39 +0100)]
avutil/mathematics: Fix division by 0

Fixes: CID1341571

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc8b1e694cc395fdf5e2917377ef11263c937d85)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agomjpegdec: consider chroma subsampling in size check
Andreas Cadhalpun [Wed, 2 Dec 2015 20:52:23 +0000 (21:52 +0100)]
mjpegdec: consider chroma subsampling in size check

If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.

This fixes out of bounds writes causing segmentation faults or memory
corruption.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/hevc: Check max ctb addresses for WPP
Michael Niedermayer [Sat, 28 Nov 2015 12:42:05 +0000 (13:42 +0100)]
avcodec/hevc: Check max ctb addresses for WPP

Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dad354f38ddc9bfc834bc21358a1d0ad41532ca0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/vp3: ensure header is parsed successfully before tables
Michael Niedermayer [Wed, 2 Dec 2015 21:59:56 +0000 (22:59 +0100)]
avcodec/vp3: ensure header is parsed successfully before tables

Fixes assertion failure
Fixes: 266ee543812e934f7b4a72923a2701d4/signal_sigabrt_7ffff6ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26379d4fddc17cac853ef297ff327b58c44edbad)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/jpeg2000dec: Check bpno in decode_cblk()
Michael Niedermayer [Fri, 4 Dec 2015 15:23:24 +0000 (16:23 +0100)]
avcodec/jpeg2000dec: Check bpno in decode_cblk()

Fixes: undefined shift
Fixes: c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a85b02dcf70f62a6a433a607143f1f78fa5648bb)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int
Michael Niedermayer [Fri, 4 Dec 2015 20:38:12 +0000 (21:38 +0100)]
avcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int

Fixes: b293a6479bb4b5286cff24d356bfd955/asan_generic_225c3c9_7819_cc526b657450c6cdef1371b526499626.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f2419888ba49245761f4ab343679c38e7880cfe)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/utils: Fix for runtime error: left shift of negative value -1
Michael Niedermayer [Fri, 4 Dec 2015 20:44:05 +0000 (21:44 +0100)]
swscale/utils: Fix for runtime error: left shift of negative value -1

Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 325b59368dae3c3f2f5cc39873002b4cf133ccbc)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/hevc: Fix integer overflow of entry_point_offset
Michael Niedermayer [Sat, 5 Dec 2015 21:08:59 +0000 (22:08 +0100)]
avcodec/hevc: Fix integer overflow of entry_point_offset

Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 214085852491448631dcecb008b5d172c11b8892)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/dirac_parser: Check that there is a previous PU before accessing it
Michael Niedermayer [Sat, 5 Dec 2015 16:15:38 +0000 (17:15 +0100)]
avcodec/dirac_parser: Check that there is a previous PU before accessing it

Fixes out of array read
Fixes: 99d142c47e6ba3510a74b872a1a2ae72/asan_heap-oob_11b36f4_3811_0f5c69e7609a88a580135678de1df844.dxa

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a08681f1e614152184615e2bcd71c3d63835f810)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset
Michael Niedermayer [Sat, 5 Dec 2015 16:14:36 +0000 (17:14 +0100)]
avcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c7d6ec947c053699950af90f695413a5640b3872)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/dirac_parser: Fix potential overflows in pointer checks
Michael Niedermayer [Sat, 5 Dec 2015 16:11:54 +0000 (17:11 +0100)]
avcodec/dirac_parser: Fix potential overflows in pointer checks

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 79798f7c57b098c78e0bbc6becd64b9888b013d1)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/wmaprodec: Check bits per sample to be within the range not causing integer...
Michael Niedermayer [Sat, 5 Dec 2015 12:48:06 +0000 (13:48 +0100)]
avcodec/wmaprodec: Check bits per sample to be within the range not causing integer overflows

Fixes: 549d5aab1480d10f2a775ed90b0342f1/signal_sigabrt_7ffff6ae7cc9_5643_96bbb0cfe3e28be1dadfce1075016345.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 66e05f6ff5e5c105bdd7bf3a49234ddac1b592c5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/wmaprodec: Fix overflow of cutoff
Michael Niedermayer [Sat, 5 Dec 2015 12:11:23 +0000 (13:11 +0100)]
avcodec/wmaprodec: Fix overflow of cutoff

Fixes: 129ca3e28d73af7b1e24a9d4118e7a2d/signal_sigabrt_7ffff6ae7cc9_836_762b310fc3ef6087bd7771e5d8e90b9b.asf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c56f8303e676556ea09bfac73d881c6c9057259)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/smacker: fix integer overflow with pts_inc
Michael Niedermayer [Sat, 5 Dec 2015 12:06:16 +0000 (13:06 +0100)]
avformat/smacker: fix integer overflow with pts_inc

Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ed47e97297fd5ef473d0cc93f0455adbadaac83)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/vp3: Fix "runtime error: left shift of negative value"
Michael Niedermayer [Fri, 4 Dec 2015 11:47:20 +0000 (12:47 +0100)]
avcodec/vp3: Fix "runtime error: left shift of negative value"

Fixes: 5c6129154b356b80bcab86f9e3ee5d29/signal_sigabrt_7ffff6ae7cc9_7322_d26ac6d7cb6567db1b8be0159b387d0b.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 18268f761bffb37552f59f87542fef3d5c80618c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agompegencts: Fix overflow in cbr mode period calculations
Timo Teräs [Sat, 28 Nov 2015 06:27:39 +0000 (08:27 +0200)]
mpegencts: Fix overflow in cbr mode period calculations

ts->mux_rate is int (signed 32-bit) type. The period calculations
will start to overflow when mux_rate > 5mbps. This fixes overflows
by converting first to 64-bit type.

Fixes #5044.

Signed-off-by: Timo Teräs <timo.teras@iki.fi>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 64f7db554ee83846f207e82a08946a6a5a6acfe2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavutil/timecode: Fix fps check
Michael Niedermayer [Thu, 3 Dec 2015 02:14:11 +0000 (03:14 +0100)]
avutil/timecode: Fix fps check

The fps variable is explicitly set to -1 in case of some errors, the check must
thus be signed or the code setting it needs to use 0 as error code
the type of the field could be changed as well but its in an installed header

Fixes: integer overflow
Fixes: 9982cc157b1ea90429435640a989122f/asan_generic_3ad004a_3799_22cf198d9cd09928e2d9ad250474fa58.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b46dcd5209a77254345ae098b83a872634c5591b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavutil/mathematics: return INT64_MIN (=AV_NOPTS_VALUE) from av_rescale_rnd() for...
Michael Niedermayer [Tue, 1 Dec 2015 12:32:31 +0000 (13:32 +0100)]
avutil/mathematics: return INT64_MIN (=AV_NOPTS_VALUE) from av_rescale_rnd() for overflows

Fixes integer overflow
Fixes: mozilla bug 1229167

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f03c2ceec174877e03bb302f5971fbe9ffbe4856)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/apedec: Check length in long_filter_high_3800()
Michael Niedermayer [Wed, 2 Dec 2015 20:16:27 +0000 (21:16 +0100)]
avcodec/apedec: Check length in long_filter_high_3800()

Fixes out of array read
Fixes: 0a7ff0c1d93da9cef28a315ec91b692a/asan_heap-oob_4a52e5_3604_9c56dbb20e308f4faeef7b35f688521a.ape

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cd7524fdd13dc8d0cf22e2cfd8300a245542b13a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/vp3: always set pix_fmt in theora_decode_header()
Michael Niedermayer [Mon, 30 Nov 2015 02:32:36 +0000 (03:32 +0100)]
avcodec/vp3: always set pix_fmt in theora_decode_header()

Fixes assertion failure
Fixes: d0bb0662da342ec65f8f2a081222e6b9/signal_sigabrt_7ffff6ae7cc9_5471_82964f0a9ac2f4d3d59390c15473f6f7.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a814f1d364ba912adf61adef158168c5f7604e93)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mpeg4videodec: Check available data before reading custom matrix
Michael Niedermayer [Sun, 29 Nov 2015 22:44:40 +0000 (23:44 +0100)]
avcodec/mpeg4videodec: Check available data before reading custom matrix

Fixes: out of array read
Fixes: 76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 891dc8f87536ac2ec695c70d081345224524ad99)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavutil/mathematics: Do not treat INT64_MIN as positive in av_rescale_rnd
Michael Niedermayer [Tue, 1 Dec 2015 11:44:23 +0000 (12:44 +0100)]
avutil/mathematics: Do not treat INT64_MIN as positive in av_rescale_rnd

The code expects actual positive numbers and gives completely wrong
results if INT64_MIN is treated as positive
Instead clip it into the valid range that is add 1 and treat it as
negative

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 25e37f5ea92d4201976a59ae306ce848d257a7e6)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavutil/integer: Fix av_mod_i() with negative dividend
Michael Niedermayer [Tue, 1 Dec 2015 11:41:43 +0000 (12:41 +0100)]
avutil/integer: Fix av_mod_i() with negative dividend

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3a9cb18855d29c96a5d9d2f5ad30448cae3a2ddf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/dump: Fix integer overflow in av_dump_format()
Michael Niedermayer [Tue, 1 Dec 2015 11:40:32 +0000 (12:40 +0100)]
avformat/dump: Fix integer overflow in av_dump_format()

Fixes part of mozilla bug 1229167

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e7f4520226d2d9ad6a58ad6c32d1455a8b244b2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/h264_refs: Check that long references match before use
Michael Niedermayer [Sun, 29 Nov 2015 02:25:41 +0000 (03:25 +0100)]
avcodec/h264_refs: Check that long references match before use

Fixes out of array read
Fixes: 59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aa427537b529cd584cd73222980286d36a00fe28)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/utils: Clear dimensions in ff_get_buffer() on failure
Michael Niedermayer [Sat, 28 Nov 2015 19:08:46 +0000 (20:08 +0100)]
avcodec/utils: Clear dimensions in ff_get_buffer() on failure

Fixes out of array access
Fixes: 482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abee0a1c60612e8638640a8a3738fffb65e16dbf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()
Michael Niedermayer [Sat, 28 Nov 2015 16:26:05 +0000 (17:26 +0100)]
avcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()

Fixes integer overflow
Fixes: 3a45b2ae02f2cf12b7bd99543cdcdae5/asan_heap-oob_1dff502_8022_899f75e1e81046ebd7b6c2394a1419f4.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f03bebc79f76df3a3e5bb9e1bc32baabfb7797c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/vp3: Clear context on reinitialization failure
Michael Niedermayer [Fri, 27 Nov 2015 23:23:54 +0000 (00:23 +0100)]
avcodec/vp3: Clear context on reinitialization failure

Fixes null pointer dereference
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_2039_8d1797aeb823ea43858d0fa45c9eb899.ogv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6105b7219a90438deae71b0dc5a034c71ee30fc0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/hevc: allocate entries unconditionally
Michael Niedermayer [Fri, 27 Nov 2015 22:33:03 +0000 (23:33 +0100)]
avcodec/hevc: allocate entries unconditionally

Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d85aa76115214183e7e3b7d65e950da61474959a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/hevc_cabac: Fix multiple integer overflows
Michael Niedermayer [Fri, 27 Nov 2015 21:45:46 +0000 (22:45 +0100)]
avcodec/hevc_cabac: Fix multiple integer overflows

Fixes: 04ec80eefa77aecd7a49a442cc02baea/asan_heap-oob_19544fa_3303_1905796cd9d8e15f86d664332caabc00.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d5028f61e44b7607b6a547f218f7d85217490a5b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()
Michael Niedermayer [Fri, 27 Nov 2015 20:02:13 +0000 (21:02 +0100)]
avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit feb3f39614b88c113211a98dda1bc2fe5c3c6957)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()
Michael Niedermayer [Fri, 27 Nov 2015 19:52:39 +0000 (20:52 +0100)]
avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()

Fixes out of array access
Fixes: 01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 75422280fbcdfbe9dc56bde5525b4d8b280f1bc5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/hevc: Check entry_point_offsets
Michael Niedermayer [Fri, 27 Nov 2015 17:30:05 +0000 (18:30 +0100)]
avcodec/hevc: Check entry_point_offsets

Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef9f7bbfa47317f9d46bf46982a394d2be78503c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/cabac: Check initial cabac decoder state
Michael Niedermayer [Fri, 27 Nov 2015 12:37:50 +0000 (13:37 +0100)]
avcodec/cabac: Check initial cabac decoder state

Fixes integer overflows
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8000d484b83aafa752d84fbdbfb352ffe0dc64f8)

Conflicts:

libavcodec/cabac.h

3 years agoavcodec/cabac_functions: Fix "left shift of negative value -31767"
Michael Niedermayer [Fri, 27 Nov 2015 11:11:29 +0000 (12:11 +0100)]
avcodec/cabac_functions: Fix "left shift of negative value -31767"

Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f6b05f5228979dab0e149deca7a30d22e98af5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/h264_slice: Limit max_contexts when slice_context_count is initialized
Michael Niedermayer [Tue, 24 Nov 2015 21:12:37 +0000 (22:12 +0100)]
avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized

Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4ea4d2f438c9a7eba37980c9a87be4b34943e4d5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/vp8: Do not use num_coeff_partitions in thread/buffer setup
Michael Niedermayer [Wed, 30 Sep 2015 11:10:48 +0000 (13:10 +0200)]
avcodec/vp8: Do not use num_coeff_partitions in thread/buffer setup

The variable is not a constant and can lead to race conditions

Fixes: repro.webm (not reproducable with FFmpeg alone)

Found-by: Dale Curtis <dalecurtis@google.com>
Tested-by: Dale Curtis <dalecurtis@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dabea74d0e82ea80cd344f630497cafcb3ef872c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/ffv1dec: Clear quant_table_count if its invalid
Michael Niedermayer [Sat, 14 Nov 2015 12:21:58 +0000 (13:21 +0100)]
avcodec/ffv1dec: Clear quant_table_count if its invalid

Fixes deallocation of corrupted pointer
Fixes: 343dfbe142a38b521ed069dc4ea7c03b/signal_sigsegv_421427_4074_ffb11959610278cd40dbc153464aa254.avi
No releases affected

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e04126072e984f8db5db9da9303c89ae01f7d6bb)

Fixes ticket #5052.

3 years agoavcodec/ffv1dec: Print an error if the quant table count is invalid
Michael Niedermayer [Thu, 5 Nov 2015 00:25:50 +0000 (01:25 +0100)]
avcodec/ffv1dec: Print an error if the quant table count is invalid

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a8b254e436dce2f5c8c6459108dab4b02cc6b79b)