ffmpeg.git
3 years agoavcodec/jpeg2000dec: More completely check cdef
Michael Niedermayer [Wed, 27 Jan 2016 16:13:10 +0000 (17:13 +0100)]
avcodec/jpeg2000dec: More completely check cdef

Fixes out of array access
Fixes: j2k-poc.bin

Found-by: Lucas Leong <wmliang.tw@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0aada30510d809bccfd539a90ea37b61188f2cb4)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavutil/opt: check for and handle errors in av_opt_set_dict2()
Michael Niedermayer [Sun, 24 Jan 2016 02:42:46 +0000 (03:42 +0100)]
avutil/opt: check for and handle errors in av_opt_set_dict2()

Previously errors could result in random entries to be lost.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f3ace85d8869c3dddd2d28d064002d0d912e3624)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/flacenc: fix calculation of bits required in case of custom sample rate
Paul B Mahol [Sun, 24 Jan 2016 19:47:49 +0000 (20:47 +0100)]
avcodec/flacenc: fix calculation of bits required in case of custom sample rate

Sample rate of 11025 takes 16 bits but previous code would pick only 8.
Fixes assertion failure.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 3e7d6849120d61bb354376d52786c26f20e20835)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat: Document urls a bit
Michael Niedermayer [Fri, 22 Jan 2016 23:35:46 +0000 (00:35 +0100)]
avformat: Document urls a bit

Spell-checked-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3130556c0eb09f3da3c9de6473a97937a4648d62)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/libquvi: Set default demuxer and protocol limitations
Michael Niedermayer [Wed, 20 Jan 2016 14:25:32 +0000 (15:25 +0100)]
avformat/libquvi: Set default demuxer and protocol limitations

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 15cc98a0f38ac45444d177186cfbf28e14bd5f1f)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/concat: Check protocol prefix
Michael Niedermayer [Wed, 20 Jan 2016 10:10:27 +0000 (11:10 +0100)]
avformat/concat: Check protocol prefix

Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e32d014322eada1812af268d7ea9d53169d279c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agodoc/demuxers: Document enable_drefs and use_absolute_path
Michael Niedermayer [Wed, 20 Jan 2016 15:49:43 +0000 (16:49 +0100)]
doc/demuxers: Document enable_drefs and use_absolute_path

Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9a8034b8bc1d1cd7a8889dc385d41744be47b159)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mjpegdec: Check for end for both bytes in unescaping
Michael Niedermayer [Thu, 21 Jan 2016 20:01:47 +0000 (21:01 +0100)]
avcodec/mjpegdec: Check for end for both bytes in unescaping

Fixes assertion failure
Fixes: c40c779601b77dc6e19aaea0b04b9751/signal_sigabrt_7ffff6ae7cb7_5769_b94f6ec70caecb2d3d76b4771b109ac1.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 509c9e74e548139285f30ed8dcc9baf1d64359fa)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mpegvideo_enc: Check for integer overflow in ff_mpv_reallocate_putbitbuffer()
Michael Niedermayer [Thu, 21 Jan 2016 14:39:43 +0000 (15:39 +0100)]
avcodec/mpegvideo_enc: Check for integer overflow in ff_mpv_reallocate_putbitbuffer()

Fixes assertion failure
Fixes: 6568d187979ce17878b6fe5fbbb89142/signal_sigabrt_7ffff6ae7cb7_7176_564bbc6741bdcf907f5c4e685c9a77a2.mpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b65efbc0f4195421c15d2a6c228d331eec5b31c3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/avformat: Replace some references to filenames by urls
Michael Niedermayer [Wed, 20 Jan 2016 20:01:08 +0000 (21:01 +0100)]
avformat/avformat: Replace some references to filenames by urls

Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 41e07390e04cf369d84f0cc7ff5858c273290770)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/wmaenc: Check ff_wma_init() for failure
Michael Niedermayer [Thu, 21 Jan 2016 01:38:05 +0000 (02:38 +0100)]
avcodec/wmaenc: Check ff_wma_init() for failure

Fixes null pointer dereference
Fixes: c4faf8280ba366bf00a79d425f2910a8/signal_sigsegv_1f96477_5177_1448ba7e4125faceb966f44ceb69abfa.qcp
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 19e456d48c90a1e3ceeb9e6241383384cc73dfdf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mpeg12enc: Move high resolution thread check to before initializing threads
Michael Niedermayer [Wed, 20 Jan 2016 23:36:51 +0000 (00:36 +0100)]
avcodec/mpeg12enc: Move high resolution thread check to before initializing threads

Cleaner solution is welcome!

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a53fbda9dc92273054a103db7539d2bb6e9632b2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/img2dec: Use AVOpenCallback
Michael Niedermayer [Wed, 20 Jan 2016 01:35:56 +0000 (02:35 +0100)]
avformat/img2dec: Use AVOpenCallback

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b750b67d13696fdbcd62ce7238eb2826f2be4686)

Conflicts:

libavformat/img2dec.c

3 years agoavformat/avio: Limit url option parsing to the documented cases
Michael Niedermayer [Wed, 20 Jan 2016 08:43:54 +0000 (09:43 +0100)]
avformat/avio: Limit url option parsing to the documented cases

This feature is not know much or used much AFAIK, and it might be helpfull in
exploits.
No specific case is known where it can be used in an exploit though
subsequent commits depend on this commit though

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 984d58a3440d513f66344b5332f6b589c0a6bbc6)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/img2dec: do not interpret the filename by default if a IO context has been...
Michael Niedermayer [Wed, 20 Jan 2016 10:21:44 +0000 (11:21 +0100)]
avformat/img2dec: do not interpret the filename by default if a IO context has been opened

With this, user applications which use custom IO and have set a IO context will not have
their already opened IO context ignored and glob/seq being interpreted

Comments and tests from maintainers of user apps are welcome!

Liked-by: wm4 <nfxjfg@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7ccedc1c78c9a5140758f515d46ce23de6e6a7d2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/ass_split: Fix null pointer dereference in ff_ass_style_get()
Michael Niedermayer [Sun, 17 Jan 2016 14:39:11 +0000 (15:39 +0100)]
avcodec/ass_split: Fix null pointer dereference in ff_ass_style_get()

Fixes: 55d71971da50365d542ed14b65565fe1/signal_sigsegv_4765a4_8499_f146af090a94f591d6254515c7700ef5.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 158f0545d81b2aca1c936490f80d13988616910e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agomov: Add an option to toggle dref opening
Derek Buitenhuis [Fri, 15 Jan 2016 17:03:49 +0000 (17:03 +0000)]
mov: Add an option to toggle dref opening

This feature is mostly only used by NLE software, and is
both of dubious value being enabled by default, and a
possible security risk.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 712d962a6a29b1099cd872cfb07867175a93ac4c)

Conflicts:

libavformat/isom.h
libavformat/mov.c
libavformat/version.h

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/gif: Fix lzw buffer size
Michael Niedermayer [Mon, 18 Jan 2016 18:20:03 +0000 (19:20 +0100)]
avcodec/gif: Fix lzw buffer size

Fixes out of array access
Fixes: aaa479088e6fb40b04837b3119f47b04/asan_heap-oob_e38c68_8576_9d653078b2470700e2834636f12ff557.tga

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03d83ba34b2070878909eae18dfac0f519503777)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/put_bits: Assert buf_ptr in flush_put_bits()
Michael Niedermayer [Mon, 18 Jan 2016 16:13:55 +0000 (17:13 +0100)]
avcodec/put_bits: Assert buf_ptr in flush_put_bits()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ef5de0f19774e2c3dd9b08ba2e8ab7241a4862a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/tiff: Check subsample & rps values more completely
Michael Niedermayer [Mon, 18 Jan 2016 02:31:25 +0000 (03:31 +0100)]
avcodec/tiff: Check subsample & rps values more completely

Fixes out of array access
Fixes: 83aedfb29af669c4d6e10f1bfad974d2/asan_heap-oob_1ab42fe_4984_9f6ec14462f8d8a00ea24b320572a963.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89f464e9c229006e16f6bb5403c5529fdd0a9edd)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/swscale: Add some sanity checks for srcSlice* parameters
Michael Niedermayer [Sun, 17 Jan 2016 17:57:01 +0000 (18:57 +0100)]
swscale/swscale: Add some sanity checks for srcSlice* parameters

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 321e85e1769ca1fc1567025ae264760790ee7fc9)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/x86/rgb2rgb_template: Fix planar2x() for short width
Michael Niedermayer [Sun, 17 Jan 2016 11:33:50 +0000 (12:33 +0100)]
swscale/x86/rgb2rgb_template: Fix planar2x() for short width

Fixes: 451b3e0cf956c0bd2f27ed753ac24050/asan_heap-oob_2873c01_3231_7ed10a9464d15f0d57277f5917c566a8.AVI

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8a9aaab2695e0f9921db946a3b9f14bea880167)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/swscale_unscaled: Fix odd height inputs for bayer_to_yv12_wrapper()
Michael Niedermayer [Sat, 16 Jan 2016 23:55:44 +0000 (00:55 +0100)]
swscale/swscale_unscaled: Fix odd height inputs for bayer_to_yv12_wrapper()

Fixes: 372d2df1f04b49e25f109f07f90b1505/asan_heap-oob_2835d2e_8501_99e0114d7ba3a6db885d0b4684d200c1.cine
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 757248ea3cd917a7755cb15f817a9b1f15578718)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/swscale_unscaled: Fix odd height inputs for bayer_to_rgb24_wrapper()
Michael Niedermayer [Sat, 16 Jan 2016 23:55:44 +0000 (00:55 +0100)]
swscale/swscale_unscaled: Fix odd height inputs for bayer_to_rgb24_wrapper()

Fixes: 372d2df1f04b49e25f109f07f90b1505/asan_heap-oob_2835d2e_8501_99e0114d7ba3a6db885d0b4684d200c1.cine
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ad3b6fa7d83db7de951ed891649af93a47e74be5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/aacenc: Check both channels for finiteness
Michael Niedermayer [Sat, 16 Jan 2016 17:32:07 +0000 (18:32 +0100)]
avcodec/aacenc: Check both channels for finiteness

Fixes null pointer dereference
Fixes: 10412fc52ecc6eab40ed67f82ca7b372/signal_sigsegv_2618c99_2129_f808373959e46afb165593332799ffbc.aif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 057549a9ccc9fd32df71678e6abe69e10668186a)

Conflicts:

libavcodec/aacenc.c

3 years agoswscale/swscale-test: Fix slice height in random reference data creation.
Michael Niedermayer [Mon, 17 Aug 2015 01:08:10 +0000 (03:08 +0200)]
swscale/swscale-test: Fix slice height in random reference data creation.

Found-by: Pedro Arthur <bygrandao@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agodca: fix misaligned access in avpriv_dca_convert_bitstream
Andreas Cadhalpun [Tue, 12 Jan 2016 23:52:58 +0000 (00:52 +0100)]
dca: fix misaligned access in avpriv_dca_convert_bitstream

src and dst are only 8-bit-aligned, so accessing them as uint16_t causes
SIGBUS crashes on architectures like sparc.

This fixes ubsan runtime error: load of misaligned address for type
'const uint16_t', which requires 2 byte alignment

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 44ac13eed49593f4f8efdb72ab0d5b48e05aa305)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agobrstm: fix missing closing brace
Andreas Cadhalpun [Mon, 4 Jan 2016 12:44:16 +0000 (13:44 +0100)]
brstm: fix missing closing brace

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1cb2331eca0dbde1bc63bc715a0e98771dda8b80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agobrstm: also allocate b->table in read_packet
Andreas Cadhalpun [Mon, 4 Jan 2016 11:53:20 +0000 (12:53 +0100)]
brstm: also allocate b->table in read_packet

This fixes NULL pointer dereferencing if the codec is forced to
adpcm_thp even though a different one was detected.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit bcf4ee26a0a1ed349ec7489925540401002b87cc)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agobrstm: make sure an ADPC chunk was read for adpcm_thp
Andreas Cadhalpun [Mon, 4 Jan 2016 11:57:38 +0000 (12:57 +0100)]
brstm: make sure an ADPC chunk was read for adpcm_thp

This fixes NULL pointer dereferencing.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d7d37c479fa71639650751648275615e979beb33)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agovorbisdec: reject rangebits 0 with non-0 partitions
Andreas Cadhalpun [Sun, 3 Jan 2016 18:11:24 +0000 (19:11 +0100)]
vorbisdec: reject rangebits 0 with non-0 partitions

This causes non-unique elements in floor_setup->data.t1.list, which
makes the stream undecodable according to the specification.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit e7a7b3135a4e5ba4bd2e144444d95a7563f53e9b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agovorbisdec: reject channel mapping with less than two channels
Andreas Cadhalpun [Sun, 3 Jan 2016 18:20:54 +0000 (19:20 +0100)]
vorbisdec: reject channel mapping with less than two channels

It causes the angle channel number to equal the magnitude channel
number, which makes the stream undecodable according to the
specification.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b4b13848dec5420fa5dd9e1a7d4dfae5de1932d5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoffmdec: reset packet_end in case of failure
Andreas Cadhalpun [Sat, 2 Jan 2016 15:27:02 +0000 (16:27 +0100)]
ffmdec: reset packet_end in case of failure

This fixes segmentation faults caused by passing a packet_ptr of NULL to
memcpy.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 40eb2531b279abe008012c5c2c292552d3e62449)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoavformat/ipmovie: put video decoding_map_size into packet and use it in decoder
Paul B Mahol [Sun, 1 Nov 2015 16:02:26 +0000 (17:02 +0100)]
avformat/ipmovie: put video decoding_map_size into packet and use it in decoder

The size of decoding map can differ from one calculated
internally, producing artifacts while decoding video.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit c293ef258cbb2c058e23651a26edf46e3bc05050)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoavcodec/wavpackenc: print channel count in av_log call n2.7.5
James Almer [Wed, 13 Jan 2016 22:26:40 +0000 (19:26 -0300)]
avcodec/wavpackenc: print channel count in av_log call

Fixes a warning with -Wformat-extra-args
(cherry picked from commit 17e7fdf61a04f52c499e2d06eab2cf2d22343aa9)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoUpdate for 2.7.5
Michael Niedermayer [Fri, 15 Jan 2016 15:29:16 +0000 (16:29 +0100)]
Update for 2.7.5

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoconfigure: bump copyright year to 2016
James Almer [Sat, 2 Jan 2016 19:28:31 +0000 (16:28 -0300)]
configure: bump copyright year to 2016

Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 78129978f02f27d76ecaf2cd1a7bf7a47253fdab)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/hls: Even stricter URL checks
Michael Niedermayer [Fri, 15 Jan 2016 14:29:22 +0000 (15:29 +0100)]
avformat/hls: Even stricter URL checks

This fixes a null pointer dereference at least

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cfda1bea4c18ec1edbc11ecc465f788b02851488)

Conflicts:

libavformat/hls.c

3 years agoavformat/hls: More strict url checks
Michael Niedermayer [Fri, 15 Jan 2016 12:29:38 +0000 (13:29 +0100)]
avformat/hls: More strict url checks

No case is known where these are needed

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6ba42b6482c725a59eb468391544dc0c75b8c6f0)

Conflicts:

libavformat/hls.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/utils: Detect and skip unneeded sws_setColorspaceDetails() calls
Michael Niedermayer [Thu, 14 Jan 2016 14:11:48 +0000 (15:11 +0100)]
swscale/utils: Detect and skip unneeded sws_setColorspaceDetails() calls

This avoids running various table inits unnecessarily

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc538e9dbd14b61d1ac8c9fa687d83289673fe90)

Conflicts:

libswscale/utils.c

3 years agoswscale/yuv2rgb: Increase YUV2RGB table headroom
Michael Niedermayer [Thu, 14 Jan 2016 02:05:11 +0000 (03:05 +0100)]
swscale/yuv2rgb: Increase YUV2RGB table headroom

This makes SWS more robust
Fixes: 07650a772d98aa63b0fed6370dc89037/asan_heap-oob_27ddeaf_2657_2c81ff264dee5d9712cb3251fb9c3bbb.264
Fixes: out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8f3a9a8c278acf886f70a1d743bc07b6f9c7b51a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/yuv2rgb: Factor YUVRGB_TABLE_LUMA_HEADROOM out
Michael Niedermayer [Thu, 14 Jan 2016 11:36:41 +0000 (12:36 +0100)]
swscale/yuv2rgb: Factor YUVRGB_TABLE_LUMA_HEADROOM out

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e5f82a28737fba4402259617500911cc37e3674)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/hls: forbid all protocols except http(s) & file
Maxim Andreev [Wed, 13 Jan 2016 08:51:12 +0000 (11:51 +0300)]
avformat/hls: forbid all protocols except http(s) & file

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7145e80b4f78cff5ed5fee04d4c4d53daaa0e077)

Conflicts:

libavformat/hls.c

3 years agoavformat/aviobuf: Fix end check in put_str16()
Michael Niedermayer [Wed, 13 Jan 2016 01:31:59 +0000 (02:31 +0100)]
avformat/aviobuf: Fix end check in put_str16()

Fixes out of array read
Fixes: 03c406ec9530e594a074ce2979f8a1f0/asan_heap-oob_7dec26_4664_37c52495b2870a2eaac65f53958e76c1.flac

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 115fb6d03ef6310732b42258d8c3cd1839cfb74b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/asfenc: Check pts
Michael Niedermayer [Tue, 12 Jan 2016 17:49:20 +0000 (18:49 +0100)]
avformat/asfenc: Check pts

Fixes integer overflow
Fixes: 0063df8be3aaa30dd6d76f59c8f818c8/signal_sigsegv_7b7b59_3634_bf418b6822bbfa68734411d96b667be3.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c0b84d89911b2035161f5ef51aafbfcc84aa9e2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mpeg4video: Check time_incr
Michael Niedermayer [Tue, 12 Jan 2016 02:03:01 +0000 (03:03 +0100)]
avcodec/mpeg4video: Check time_incr

Fixes assertion failure
Fixes out of memory access

Fixes: test_casex.ivf

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c97946d6131b31340954a3f603b6bf92590a9a5)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/wavpackenc: Check the number of channels
Michael Niedermayer [Mon, 11 Jan 2016 17:58:08 +0000 (18:58 +0100)]
avcodec/wavpackenc: Check the number of channels

They are stored in a byte, thus more than 255 is not possible

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 59c915a403af32c4ff5126625b0cc7e38f4beff9)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/wavpackenc: Headers are per channel
Michael Niedermayer [Mon, 11 Jan 2016 17:32:32 +0000 (18:32 +0100)]
avcodec/wavpackenc: Headers are per channel

Fixes: 1b8b83a53bfa751f01b1daa65a4758db/signal_sigabrt_7ffff6ae7cb7_7488_403f71d1a2565b598d01b6cb110fac8f.aiff
Fixes: assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26757b0279b4b93c6066c2151d4d3dbd2ec266bf)

Conflicts:

libavcodec/wavpackenc.c

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/aacdec_template: Check id_map
Michael Niedermayer [Sun, 10 Jan 2016 18:29:39 +0000 (19:29 +0100)]
avcodec/aacdec_template: Check id_map

Fixes index out of bounds error
Fixes: aac_index_out_of_bounds.wmv

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 590863876d1478547640304a31c15809c3618090)

Conflicts:

libavcodec/aacdec_template.c

3 years agoavcodec/dvdec: Fix "left shift of negative value -254"
Michael Niedermayer [Sun, 10 Jan 2016 16:43:56 +0000 (17:43 +0100)]
avcodec/dvdec: Fix "left shift of negative value -254"

Fixes: dvdec_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 93ac72a98dff592ffc174cfb36a8975dfbf145ae)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mjpegdec: Fix negative shift
Michael Niedermayer [Sun, 10 Jan 2016 14:52:09 +0000 (15:52 +0100)]
avcodec/mjpegdec: Fix negative shift

Fixes: mjpeg_left_shift.avi

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d86d7b2486cd5c31db8e820d8a89554abf19567e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mss2: Check for repeat overflow
Michael Niedermayer [Sun, 10 Jan 2016 11:19:48 +0000 (12:19 +0100)]
avcodec/mss2: Check for repeat overflow

Fixes: mss2_left_shift.wmv

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e273dade78943e22b71d0ddb67cd0d737fc26edf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat: Add integer fps from 31 to 60 to get_std_framerate()
Michael Niedermayer [Sat, 9 Jan 2016 09:49:23 +0000 (10:49 +0100)]
avformat: Add integer fps from 31 to 60 to get_std_framerate()

Fixes Ticket 5106

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2039b3e7511ef183dae206575114e15b6d99c134)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/mpegvideo_enc: Clip bits_per_raw_sample within valid range
Michael Niedermayer [Wed, 6 Jan 2016 23:22:56 +0000 (00:22 +0100)]
avcodec/mpegvideo_enc: Clip bits_per_raw_sample within valid range

Fixes out of array read
Fixes: test_case-mdc.264 (b47be15a120979f5a1a945c938cbef33)

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13f266b50cc7554028d22480b7e4383968e64a63)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavfilter/vf_scale: set proper out frame color range
Thomas Mundt [Wed, 30 Dec 2015 23:01:21 +0000 (00:01 +0100)]
avfilter/vf_scale: set proper out frame color range

Prevents that following scalers in the filter chain will do unintentional color range conversions.
Fixes Ticket #5096

Signed-off-by: Thomas Mundt <loudmax@yahoo.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 73ce8162f3499cf0e86d1d80dea53324bd62bcb3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/motion_est: Fix mv_penalty table size
Michael Niedermayer [Tue, 5 Jan 2016 13:41:04 +0000 (14:41 +0100)]
avcodec/motion_est: Fix mv_penalty table size

Fixes out of array read

Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5b4da8a38a5ed211df9504c85ce401c30af86b97)

Conflicts:

libavcodec/motion_est.h

3 years agoavcodec/h264_slice: Fix integer overflow in implicit weight computation
Michael Niedermayer [Tue, 5 Jan 2016 00:06:18 +0000 (01:06 +0100)]
avcodec/h264_slice: Fix integer overflow in implicit weight computation

Fixes mozilla bug 1230423

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7cc01c25727a96eaaa0c177234b626e47c8ea491)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/utils: Use normal bilinear scaler if fast cannot be used due to tiny dimensions
Michael Niedermayer [Mon, 4 Jan 2016 22:22:25 +0000 (23:22 +0100)]
swscale/utils: Use normal bilinear scaler if fast cannot be used due to tiny dimensions

Fixes Ticket4960

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edf129cbc897447a289ca8b045853df5df1bab3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/put_bits: Always check buffer end before writing
Michael Niedermayer [Fri, 1 Jan 2016 01:41:06 +0000 (02:41 +0100)]
avcodec/put_bits: Always check buffer end before writing

This causes a overall slowdown of 0.1 % (tested with mpeg4 single thread encoding of matrixbench at QP=3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cccb0ffccc3723acc7aab3a859b24743596dd9c0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agomjpegdec: extend check for incompatible values of s->rgb and s->ls
Andreas Cadhalpun [Thu, 31 Dec 2015 15:55:43 +0000 (16:55 +0100)]
mjpegdec: extend check for incompatible values of s->rgb and s->ls

This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.

In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.

This fixes an out-of-bounds read in ls_decode_line.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7ea2db6eafa0a8a9497aab20be2cfc8742a59072)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/utils: Fix intermediate format for cascaded alpha downscaling
Michael Niedermayer [Thu, 24 Dec 2015 20:46:15 +0000 (21:46 +0100)]
swscale/utils: Fix intermediate format for cascaded alpha downscaling

Fixes Ticket4926

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b83d8be6bff7d645469a623aee0b380541da15cf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agox86/float_dsp: zero extend offset from ff_scalarproduct_float_sse
James Almer [Fri, 8 Jan 2016 15:08:56 +0000 (12:08 -0300)]
x86/float_dsp: zero extend offset from ff_scalarproduct_float_sse

Reviewed-by: Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit dc79824deb6ac0ce236589c618744b33629201cd)

3 years agoavfilter/vf_zoompan: do not free frame we pushed to lavfi
Paul B Mahol [Sat, 2 Jan 2016 17:51:11 +0000 (18:51 +0100)]
avfilter/vf_zoompan: do not free frame we pushed to lavfi

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit 8bcd1997eadb0d79a049227a1d1afe6111397baa)

Fixes ticket #5113.

3 years agoUpdate for 2.7.4 n2.7.4
Michael Niedermayer [Mon, 21 Dec 2015 13:52:20 +0000 (14:52 +0100)]
Update for 2.7.4

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agonuv: sanitize negative fps rate
Andreas Cadhalpun [Wed, 16 Dec 2015 19:52:39 +0000 (20:52 +0100)]
nuv: sanitize negative fps rate

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f6830cf5ba03fdcfcd81a0358eb32d4081a2fcce)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agorawdec: only exempt BIT0 with need_copy from buffer sanity check
Andreas Cadhalpun [Sat, 19 Dec 2015 22:45:06 +0000 (23:45 +0100)]
rawdec: only exempt BIT0 with need_copy from buffer sanity check

Otherwise the too small buffer is directly used in the frame, causing
segmentation faults, when trying to use the frame.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 699e68371ec7e381e5cc48e3d96e29c669261af7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agomlvdec: check that index_entries exist
Andreas Cadhalpun [Sat, 19 Dec 2015 22:44:53 +0000 (23:44 +0100)]
mlvdec: check that index_entries exist

This fixes NULL pointer dereferencing.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9fcfe4a3cdf9a5af0c37758b178965b7b99582d4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agonutdec: reject negative value_len in read_sm_data
Andreas Cadhalpun [Sat, 19 Dec 2015 11:02:56 +0000 (12:02 +0100)]
nutdec: reject negative value_len in read_sm_data

If it is negative, it can cause the byte position to move backwards in
avio_skip, which in turn makes sm_size negative and thus size larger
than the size of the packet buffer, causing invalid writes in avio_read.

Also fix potential overflow of avio_tell(bc) + value_len.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ce10f572c12b0d172c72d31d8c979afce602bf0c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoxwddec: prevent overflow of lsize * avctx->height
Andreas Cadhalpun [Fri, 18 Dec 2015 18:28:51 +0000 (19:28 +0100)]
xwddec: prevent overflow of lsize * avctx->height

This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9d38f06d05efbb9d6196c27668eb943e934943ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agonutdec: only copy the header if it exists
Andreas Cadhalpun [Fri, 18 Dec 2015 14:18:47 +0000 (15:18 +0100)]
nutdec: only copy the header if it exists

Fixes ubsan runtime error: null pointer passed as argument 2, which is
declared to never be null

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9f82506c79874edd7b09707ab63d9e72078de8f9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoexr: fix out of bounds read in get_code
Andreas Cadhalpun [Sun, 13 Dec 2015 22:17:09 +0000 (23:17 +0100)]
exr: fix out of bounds read in get_code

This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 90b99a81071d10e6b5efe86a4602d54d4f45bbcb)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoon2avc: limit number of bits to 30 in get_egolomb
Andreas Cadhalpun [Wed, 16 Dec 2015 15:48:19 +0000 (16:48 +0100)]
on2avc: limit number of bits to 30 in get_egolomb

More don't fit into the integer output.

Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 4d5c3b02e9d2c9a630ca433fabca43285879e0b8)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoavcodec/mpeg4videodec: also for empty partitioned slices
Michael Niedermayer [Sat, 19 Dec 2015 22:21:33 +0000 (23:21 +0100)]
avcodec/mpeg4videodec: also for empty partitioned slices

Fixes assertion failure
Fixes: id_acf3e47f864e1ee4c7b86c0653e0ff31e5bde56e.m4v

Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 70f13abb4f9a376ddc0d2c566739bc3c6a0c47e7)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/h264_refs: Fix long_idx check
Michael Niedermayer [Sat, 19 Dec 2015 20:59:42 +0000 (21:59 +0100)]
avcodec/h264_refs: Fix long_idx check

Fixes out of array read
Fixes mozilla bug 1233606

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b92b4775a0d07cacfdd2b4be6511f3cb362c977b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/h264_mc_template: prefetch list1 only if it is used in the MB
Michael Niedermayer [Thu, 17 Dec 2015 23:20:51 +0000 (00:20 +0100)]
avcodec/h264_mc_template: prefetch list1 only if it is used in the MB

Fixes ubsan warning
Fixes Mozilla bug 1230276

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8ea57664fe3ad611c9ecd234670544ddff7ca55)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/h264_slice: Simplify ref2frm indexing
Michael Niedermayer [Thu, 17 Dec 2015 21:51:00 +0000 (22:51 +0100)]
avcodec/h264_slice: Simplify ref2frm indexing

This also suppresses a ubsan warning
Fixes Mozilla bug 1230247

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef8f6464a55db730cab8c48a1a51fa4e6ca12107)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoRevert "avcodec/aarch64/neon.S: Update neon.s for transpose_4x4H"
Michael Niedermayer [Thu, 17 Dec 2015 20:14:45 +0000 (21:14 +0100)]
Revert "avcodec/aarch64/neon.S: Update neon.s for transpose_4x4H"

The change was not correct and broke H264

This reverts commit cd83f899c94f691b045697d12efa21f83eb2329f.
(cherry picked from commit 95b59bfb9d9e47de8438183a035e02667946f27c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavfilter/vf_mpdecimate: Add missing emms_c()
Michael Niedermayer [Mon, 14 Dec 2015 17:56:13 +0000 (18:56 +0100)]
avfilter/vf_mpdecimate: Add missing emms_c()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 997de2e8107cc4256e50611463d609b18fe9619f)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agosonic: make sure num_taps * channels is not larger than frame_size
Andreas Cadhalpun [Tue, 15 Dec 2015 22:43:03 +0000 (23:43 +0100)]
sonic: make sure num_taps * channels is not larger than frame_size

If that is the case, the loop setting predictor_state in
sonic_decode_frame causes out of bounds reads of int_samples, which has
only frame_size number of elements.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9637c2531f7eb040ad1c3cb46cb40a63dfc77b80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoopus_silk: fix typo causing overflow in silk_stabilize_lsf
Andreas Cadhalpun [Tue, 15 Dec 2015 21:00:31 +0000 (22:00 +0100)]
opus_silk: fix typo causing overflow in silk_stabilize_lsf

Due to this typo max_center can be too large, causing nlsf to be set to
too large values, which in turn can cause nlsf[i - 1] + min_delta[i] to
overflow to a negative value, which is not allowed for nlsf and can
cause an out of bounds read in silk_lsf2lpc.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f61d44b74aaae1d306d8a0d38b7b3d4292c89ced)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoffm: reject invalid codec_id and codec_type
Andreas Cadhalpun [Mon, 14 Dec 2015 21:11:55 +0000 (22:11 +0100)]
ffm: reject invalid codec_id and codec_type

A negative codec_id cannot be handled by the found_decoder API of
AVStream->info: if the codec_id is not recognized, found_decoder is set
to -codec_id, which has to be '<0' according to the API documentation.

This can cause NULL pointer dereferencing in try_decode_frame.

Also make sure the codec_type matches the expected one for codec_id.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ecf63b7cc24b9fd3e6d604313325dd1ada4db662)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agogolomb: always check for invalid UE golomb codes in get_ue_golomb
Andreas Cadhalpun [Sun, 13 Dec 2015 20:02:16 +0000 (21:02 +0100)]
golomb: always check for invalid UE golomb codes in get_ue_golomb

Also correct the check to reject log < 7, because UPDATE_CACHE only
guarantees 25 meaningful bits.

This fixes undefined behavior:
runtime error: shift exponent is negative

Testing with START/STOP timers in get_ue_golomb, one for the first
branch (A) and one for the second (B), shows that there is practically no
slowdown, e.g. for the cavs decoder:

With the check in the B branch:
    629 decicycles in get_ue_golomb B, 4194260 runs,     44 skips
    433 decicycles in get_ue_golomb A,268434102 runs,   1354 skips

Without the check:
    624 decicycles in get_ue_golomb B, 4194273 runs,     31 skips
    433 decicycles in get_ue_golomb A,268434203 runs,   1253 skips

Since the B branch is executed far less often than the A branch, this
change is negligible, even more so for the h264 decoder, where the ratio
B/A is a lot smaller.

Fixes: mozilla bug 1230239
Fixes: fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit

Found-by: Tyson Smith
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 22e960ad478e568f4094971a58c6ad8f549c0180)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoaaccoder: prevent crash of anmr coder
Andreas Cadhalpun [Fri, 4 Dec 2015 17:13:07 +0000 (18:13 +0100)]
aaccoder: prevent crash of anmr coder

If minq is negative, the range of sf_idx can be larger than
SCALE_MAX_DIFF allows, causing assertion failures later in
encode_scale_factors.

Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a4652dd5da0502ff21c183b5ca7d76b1cfd6c51)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoffmdec: reject zero-sized chunks
Andreas Cadhalpun [Wed, 2 Dec 2015 21:47:12 +0000 (22:47 +0100)]
ffmdec: reject zero-sized chunks

If size is zero, avio_get_str fails, leaving the buffer uninitialized.
This causes invalid reads in av_set_options_string.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a611375db532c3d5363d97b10fadd0211811a4fd)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoswscale/x86/rgb2rgb_template: Fallback to mmx in interleaveBytes() if the alignment...
Michael Niedermayer [Tue, 15 Dec 2015 01:50:20 +0000 (02:50 +0100)]
swscale/x86/rgb2rgb_template: Fallback to mmx in interleaveBytes() if the alignment is insufficient for SSE*

This also as a sideeffect fixes the non aligned case

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a066ff89bcbae6033c2ffda9271cad84f6c1b807)

3 years agoswscale/x86/rgb2rgb_template: Do not crash on misaligend stride
Michael Niedermayer [Tue, 15 Dec 2015 01:06:04 +0000 (02:06 +0100)]
swscale/x86/rgb2rgb_template: Do not crash on misaligend stride

Fixes Ticket5013

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 80bfce35ccd11458e97f68f417fc094c5347070c)

3 years agoavformat/mxfenc: Do not crash if there is no packet in the first stream
Michael Niedermayer [Sun, 13 Dec 2015 15:13:22 +0000 (16:13 +0100)]
avformat/mxfenc: Do not crash if there is no packet in the first stream

Fixes: Ticket4914

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b51e7554e74cbf007a1cab83c7bed3ad9fa2793a)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/aarch64/neon.S: Update neon.s for transpose_4x4H
zjh8890 [Sat, 21 Nov 2015 16:07:35 +0000 (00:07 +0800)]
avcodec/aarch64/neon.S: Update neon.s for transpose_4x4H

The transpose_4x4H is wrong which cost me much time to find this bug. The orders of r2 and r3 are wrong,
this bug waste me much time while I make aarch64 arm instruction which used the function.
(cherry picked from commit c18176bd551b4616757080376707637e30547fd0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/utils: estimate_timings_from_pts - increase retry counter, fixes invalid...
Rainer Hochecker [Sun, 15 Nov 2015 12:58:50 +0000 (13:58 +0100)]
avformat/utils: estimate_timings_from_pts - increase retry counter, fixes invalid duration for ts files with hevc codec

Fixes a mpegts file with hevc that fails estimating duration. Increasing number of
retries fixes the issue.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d8c2f1a28073d451c7db31291c333cb15ca3d0b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/matroskaenc: Check codecdelay before use
Michael Niedermayer [Wed, 9 Dec 2015 15:16:46 +0000 (16:16 +0100)]
avformat/matroskaenc: Check codecdelay before use

Fixes CID1238790

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e6971db12b8ae49712b77378fa8141de4904082b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavutil/mathematics: Fix division by 0
Michael Niedermayer [Wed, 9 Dec 2015 16:39:38 +0000 (17:39 +0100)]
avutil/mathematics: Fix division by 0

Fixes: CID1341571

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc8b1e694cc395fdf5e2917377ef11263c937d85)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agomjpegdec: consider chroma subsampling in size check
Andreas Cadhalpun [Wed, 2 Dec 2015 20:52:23 +0000 (21:52 +0100)]
mjpegdec: consider chroma subsampling in size check

If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.

This fixes out of bounds writes causing segmentation faults or memory
corruption.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/hevc: Check max ctb addresses for WPP
Michael Niedermayer [Sat, 28 Nov 2015 12:42:05 +0000 (13:42 +0100)]
avcodec/hevc: Check max ctb addresses for WPP

Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dad354f38ddc9bfc834bc21358a1d0ad41532ca0)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/vp3: ensure header is parsed successfully before tables
Michael Niedermayer [Wed, 2 Dec 2015 21:59:56 +0000 (22:59 +0100)]
avcodec/vp3: ensure header is parsed successfully before tables

Fixes assertion failure
Fixes: 266ee543812e934f7b4a72923a2701d4/signal_sigabrt_7ffff6ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 26379d4fddc17cac853ef297ff327b58c44edbad)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/jpeg2000dec: Check bpno in decode_cblk()
Michael Niedermayer [Fri, 4 Dec 2015 15:23:24 +0000 (16:23 +0100)]
avcodec/jpeg2000dec: Check bpno in decode_cblk()

Fixes: undefined shift
Fixes: c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a85b02dcf70f62a6a433a607143f1f78fa5648bb)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int
Michael Niedermayer [Fri, 4 Dec 2015 20:38:12 +0000 (21:38 +0100)]
avcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int

Fixes: b293a6479bb4b5286cff24d356bfd955/asan_generic_225c3c9_7819_cc526b657450c6cdef1371b526499626.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f2419888ba49245761f4ab343679c38e7880cfe)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/utils: Fix for runtime error: left shift of negative value -1
Michael Niedermayer [Fri, 4 Dec 2015 20:44:05 +0000 (21:44 +0100)]
swscale/utils: Fix for runtime error: left shift of negative value -1

Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 325b59368dae3c3f2f5cc39873002b4cf133ccbc)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/hevc: Fix integer overflow of entry_point_offset
Michael Niedermayer [Sat, 5 Dec 2015 21:08:59 +0000 (22:08 +0100)]
avcodec/hevc: Fix integer overflow of entry_point_offset

Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 214085852491448631dcecb008b5d172c11b8892)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/dirac_parser: Check that there is a previous PU before accessing it
Michael Niedermayer [Sat, 5 Dec 2015 16:15:38 +0000 (17:15 +0100)]
avcodec/dirac_parser: Check that there is a previous PU before accessing it

Fixes out of array read
Fixes: 99d142c47e6ba3510a74b872a1a2ae72/asan_heap-oob_11b36f4_3811_0f5c69e7609a88a580135678de1df844.dxa

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a08681f1e614152184615e2bcd71c3d63835f810)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset
Michael Niedermayer [Sat, 5 Dec 2015 16:14:36 +0000 (17:14 +0100)]
avcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c7d6ec947c053699950af90f695413a5640b3872)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>