ffmpeg.git
2 years agoavcodec/kmvc: Check side data size before use
Michael Niedermayer [Sun, 30 Oct 2016 14:12:12 +0000 (15:12 +0100)]
avcodec/kmvc:  Check side data size before use

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d99101d0964f754822fb4af121c4abc69047dba)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/idcinvideo: Check side data size before use
Michael Niedermayer [Sun, 30 Oct 2016 14:12:12 +0000 (15:12 +0100)]
avcodec/idcinvideo: Check side data size before use

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a2b8dde65947bfabf42269e124ef83ecf9c5974a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/cinepak: Check side data size before use
Michael Niedermayer [Sun, 30 Oct 2016 12:47:38 +0000 (13:47 +0100)]
avcodec/cinepak: Check side data size before use

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 121be310607879841d19a34d9f16d4fe9ba7f18c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/8bps: Check side data size before use
Michael Niedermayer [Sun, 30 Oct 2016 12:44:52 +0000 (13:44 +0100)]
avcodec/8bps: Check side data size before use

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 042faa847feea820451c474af0034fd3de9cff82)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dvdsubdec: Fix off by 1 error
Michael Niedermayer [Tue, 25 Oct 2016 22:11:52 +0000 (00:11 +0200)]
avcodec/dvdsubdec: Fix off by 1 error

Fixes out of array read

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c92f55847a3d9cd12db60bfcd0831ff7f089c37c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/dvdsubdec: Fix buf_size check
Michael Niedermayer [Wed, 26 Oct 2016 14:29:57 +0000 (16:29 +0200)]
avcodec/dvdsubdec: Fix buf_size check

Fixes out of array access

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 25ab1a65f3acb5ec67b53fb7a2463a7368f1ad16)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agovp9: change order of operations in adapt_prob().
Ronald S. Bultje [Fri, 14 Oct 2016 17:01:27 +0000 (13:01 -0400)]
vp9: change order of operations in adapt_prob().

This is intended to workaround bug "665 Integer Divide Instruction May
Cause Unpredictable Behavior" on some early AMD CPUs, which causes a
div-by-zero in this codepath, such as reported in Mozilla bug #1293996.

Note that this isn't guaranteed to fix the bug, since a compiler is free
to reorder instructions that don't depend on each other. However, it
appears to fix the bug in Firefox, and a similar patch was applied to
libvpx also (see Chrome bug #599899).

(cherry picked from commit be885da3427c5d9a6fa68229d16318afffe67193)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/interplayvideo: Check side data size before use
Michael Niedermayer [Tue, 25 Oct 2016 01:51:17 +0000 (03:51 +0200)]
avcodec/interplayvideo: Check side data size before use

Fixes out of array read

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 85d23e5cbc9ad6835eef870a5b4247de78febe56)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/mxfdec: Check size to avoid integer overflow in mxf_read_utf16_string()
Michael Niedermayer [Fri, 21 Oct 2016 17:45:21 +0000 (19:45 +0200)]
avformat/mxfdec: Check size to avoid integer overflow in mxf_read_utf16_string()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fecb3e82a4ba09dc11a51ad0961ab491881a53a1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/mpegvideo_enc: Clear mmx state in ff_mpv_reallocate_putbitbuffer()
Michael Niedermayer [Fri, 21 Oct 2016 12:05:00 +0000 (14:05 +0200)]
avcodec/mpegvideo_enc: Clear mmx state in ff_mpv_reallocate_putbitbuffer()

This function must be called from the mb or slice encoding loop and MMX state may not
be clean there

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03ec6b780cfae85b8bf0f32b2eda201063ad061b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavcodec/utils: Clear MMX state before returning from avcodec_default_execute*()
Michael Niedermayer [Fri, 21 Oct 2016 11:40:18 +0000 (13:40 +0200)]
avcodec/utils: Clear MMX state before returning from avcodec_default_execute*()

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f96f9d1118e073d346d16be157fa5075434e7f2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2 years agoavformat/icodec: Fix crash probing fuzzed file
Mark Harris [Tue, 16 Feb 2016 07:52:13 +0000 (23:52 -0800)]
avformat/icodec: Fix crash probing fuzzed file

Avoid invalid memory read/crash when frame offset >= 0xfffffff8.
Base64-encoded example: AAABADAwMDAwMAAAMAAwMDAw/P///w==
(The previous commit verifies that p->buf_size >= 22.)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 56e2cd9c042e05255aa28487694c29aaec023263)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agodcstr: fix division by zero
Andreas Cadhalpun [Thu, 20 Oct 2016 18:13:54 +0000 (20:13 +0200)]
dcstr: fix division by zero

Also check for possible overflows.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b0a043f51b8cc3b420dc3ceaa38fe9aa344799aa)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agorsd: limit number of channels
Andreas Cadhalpun [Wed, 19 Oct 2016 21:40:41 +0000 (23:40 +0200)]
rsd: limit number of channels

Negative values don't make sense and too large values can cause
overflows. For AV_CODEC_ID_ADPCM_THP this leads to a too small extradata
buffer being allocated, causing out-of-bounds writes.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ee5f0f1d355fa0fd9194ac97a2c8598c93ed328b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agomss2: only use error correction for matching block counts
Andreas Cadhalpun [Thu, 24 Nov 2016 22:57:46 +0000 (23:57 +0100)]
mss2: only use error correction for matching block counts

This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2
with coded_width/coded_height larger than width/height.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2566ad98b01538ea589e5ee07b69fc566aadc348)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agosoftfloat: decrease MIN_EXP to cover full float range
Andreas Cadhalpun [Thu, 24 Nov 2016 23:26:51 +0000 (00:26 +0100)]
softfloat: decrease MIN_EXP to cover full float range

floats are not necessarily normalized, so a normalized softfloat needs
MIN_EXP lowered by 23 to cover that range.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2d6f46d801bab990b7e742b8a8e5c5b0cb70a80e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agolibopusdec: default to stereo for invalid number of channels
Andreas Cadhalpun [Mon, 14 Nov 2016 20:41:45 +0000 (21:41 +0100)]
libopusdec: default to stereo for invalid number of channels

This fixes an out-of-bounds read if avc->channels is 0.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 8c8f543b81aa2b50bb6a6cfd370a0061281492a3)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agopgssubdec: only set w/h/linesize when allocating data
Andreas Cadhalpun [Wed, 9 Nov 2016 22:23:16 +0000 (23:23 +0100)]
pgssubdec: only set w/h/linesize when allocating data

Rects with positive w/h/linesize but no data are invalid.

Reviewed-by: Petri Hintukainen <phintuka@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 995512328ed84bb737bc364e4ef6fba1994f062a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agosbgdec: prevent NULL pointer access
Andreas Cadhalpun [Thu, 10 Nov 2016 21:21:20 +0000 (22:21 +0100)]
sbgdec: prevent NULL pointer access

Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit dbefbb61b785cd77810c032f5cdb499d2a92df07)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agosmacker: limit recursion depth of smacker_decode_bigtree
Andreas Cadhalpun [Sat, 19 Nov 2016 13:21:11 +0000 (14:21 +0100)]
smacker: limit recursion depth of smacker_decode_bigtree

This fixes segmentation faults due to stack-overflow caused by too deep
recursion.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 946ecd19ea752399bccc751c9339ff74b815587e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agomxfdec: fix NULL pointer dereference in mxf_read_packet_old
Andreas Cadhalpun [Thu, 17 Nov 2016 21:53:51 +0000 (22:53 +0100)]
mxfdec: fix NULL pointer dereference in mxf_read_packet_old

Metadata streams have priv_data set to NULL.

Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdb8c455b637f86e2e85503b7e090fa448164398)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agolibschroedingerdec: fix leaking of framewithpts
Andreas Cadhalpun [Sun, 13 Nov 2016 22:10:06 +0000 (23:10 +0100)]
libschroedingerdec: fix leaking of framewithpts

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3c0328d58d98664b05efdd377d3fe66a569d385e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agolibschroedingerdec: don't produce empty frames
Andreas Cadhalpun [Sun, 13 Nov 2016 21:59:47 +0000 (22:59 +0100)]
libschroedingerdec: don't produce empty frames

They are not valid and can cause problems/crashes for API users.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a86ebbf7f641bc797002ddea7fb517759722cd1b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agosoftfloat: handle -INT_MAX correctly
Andreas Cadhalpun [Sun, 13 Nov 2016 19:52:02 +0000 (20:52 +0100)]
softfloat: handle -INT_MAX correctly

This is similar to commit 9ac61e73d0843ec4b83f4e3d47eded73234e406e.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0edd569466eb45b134690b9f4efbb57eda86f58d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agofilmstripdec: correctly check image dimensions
Andreas Cadhalpun [Sun, 13 Nov 2016 17:22:12 +0000 (18:22 +0100)]
filmstripdec: correctly check image dimensions

This prevents a division by zero in read_packet.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 25012c56448a48487cdc9699465e640871dbcd60)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agopnmdec: make sure v is capped by maxval
Andreas Cadhalpun [Wed, 9 Nov 2016 00:09:35 +0000 (01:09 +0100)]
pnmdec: make sure v is capped by maxval

Otherwise put_bits can be called with a value that doesn't fit in the
sample_len, causing an assertion failure.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit cdb5479c9ddc886f0b8661db585405ebab343e80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agosmvjpegdec: make sure cur_frame is not negative
Andreas Cadhalpun [Thu, 10 Nov 2016 21:09:03 +0000 (22:09 +0100)]
smvjpegdec: make sure cur_frame is not negative

This fixes a heap-buffer-overflow detected by AddressSanitizer.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 360bc0d90aa66cf21e9f488e77d21db18e01ec9c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agoicodec: correctly check avio_read return value
Andreas Cadhalpun [Tue, 8 Nov 2016 22:29:28 +0000 (23:29 +0100)]
icodec: correctly check avio_read return value

It can read less than the requested amount, in which case buf contains
uninitialized data, causing problems like segmentation faults later on.

Also make sure that image->size is positive, so that it can't match a
negative error code.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 89eb398c7fc4cb9a15e55bdf2ab6435b5332e377)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agodvbsubdec: fix division by zero in compute_default_clut
Andreas Cadhalpun [Tue, 8 Nov 2016 21:32:42 +0000 (22:32 +0100)]
dvbsubdec: fix division by zero in compute_default_clut

This problem was introduced in commit
4b90dcb8493552c17a811c8b1e6538dae4061f9d.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit c82b8ef0e4f226423ddd644bfe37e6a15d070924)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agoproresdec_lgpl: explicitly check coff[3] against slice_data_size
Andreas Cadhalpun [Wed, 9 Nov 2016 22:49:46 +0000 (23:49 +0100)]
proresdec_lgpl: explicitly check coff[3] against slice_data_size

The implicit checks via v_data_size and a_data_size don't work in the case
'(hdr_size > 7) && !ctx->alpha_info'.

This fixes segmentation faults due to invalid reads.

This problem was introduced in commit
547c2f002a87f4412a83c23b0d60364be5e7ce58.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1e33035ee7a8d9fb7a4b8b6cc54842e72b36ed70)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agoescape124: reject codebook size 0
Andreas Cadhalpun [Tue, 8 Nov 2016 23:38:50 +0000 (00:38 +0100)]
escape124: reject codebook size 0

It causes a cb_depth of 32, leading to assertion failures in get_bits.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 226d35c84591f1901c2a13819031549909faa1f5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agoicodec: add ico_read_close to fix leaking ico->images
Andreas Cadhalpun [Tue, 8 Nov 2016 22:54:41 +0000 (23:54 +0100)]
icodec: add ico_read_close to fix leaking ico->images

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d54c95a1435a8a3fcd599108ec85b7f56a0fcbf9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agoicodec: fix leaking pkt on error
Andreas Cadhalpun [Tue, 8 Nov 2016 22:53:52 +0000 (23:53 +0100)]
icodec: fix leaking pkt on error

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 467eece1bea5c8325c6974190ba61f1bba88a3f3)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agompegts: prevent division by zero
Andreas Cadhalpun [Mon, 7 Nov 2016 22:37:59 +0000 (23:37 +0100)]
mpegts: prevent division by zero

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1bbb18fe82fc77a10d45fa53bd2738d2c54de6c6)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agomatroskadec: fix NULL pointer dereference in webm_dash_manifest_read_header
Andreas Cadhalpun [Mon, 7 Nov 2016 23:42:23 +0000 (00:42 +0100)]
matroskadec: fix NULL pointer dereference in webm_dash_manifest_read_header

The code assumes that s->streams[0] is valid.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ff100c9dd97d2f1f456ff38b192edf84f9744738)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agompegaudio_parser: don't return AVERROR_PATCHWELCOME
Andreas Cadhalpun [Mon, 7 Nov 2016 00:16:14 +0000 (01:16 +0100)]
mpegaudio_parser: don't return AVERROR_PATCHWELCOME

The API does not allow returning AVERROR codes.

It triggers an assert in av_parser_parse2.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5249706e9d2ec5ed1b07d8ffdbb8fb9104261f6d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agomxfdec: fix NULL pointer dereference
Andreas Cadhalpun [Fri, 4 Nov 2016 23:17:53 +0000 (00:17 +0100)]
mxfdec: fix NULL pointer dereference

Metadata streams have priv_data set to NULL.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0efb6106118c17308b3fdc3190f5e5bf84b01d5c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agolzf: update pointer p after realloc
Andreas Cadhalpun [Fri, 4 Nov 2016 21:58:49 +0000 (22:58 +0100)]
lzf: update pointer p after realloc

This fixes heap-use-after-free detected by AddressSanitizer.

Reviewed-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit bb6a7b6f75ac544c956e3eefee297700ef4d3468)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agodiracdec: check return code of get_buffer_with_edge
Andreas Cadhalpun [Fri, 4 Nov 2016 18:00:17 +0000 (19:00 +0100)]
diracdec: check return code of get_buffer_with_edge

If it fails, buffers aren't allocated, causing NULL pointer dereferencing.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit db79dedb1ae5dd38432eee3f09155e26f3f2d95a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agoppc: pixblockdsp: do unaligned block accesses correctly again
Andreas Cadhalpun [Wed, 2 Nov 2016 20:28:49 +0000 (21:28 +0100)]
ppc: pixblockdsp: do unaligned block accesses correctly again

This was broken by the following Libav commit:
4c387c7 ppc: dsputil: do unaligned block accesses correctly

The following tests fail due to this:
fate-checkasm
fate-vsynth1-dnxhd-2k-hr-hq fate-vsynth1-dnxhd-edge1-hr
fate-vsynth1-dnxhd-edge2-hr fate-vsynth1-dnxhd-edge3-hr
fate-vsynth1-dnxhd-hr-sq-mov fate-vsynth1-dnxhd-hr-hq-mov
fate-vsynth2-dnxhd-2k-hr-hq fate-vsynth2-dnxhd-edge1-hr
fate-vsynth2-dnxhd-edge2-hr fate-vsynth2-dnxhd-edge3-hr
fate-vsynth2-dnxhd-hr-sq-mov fate-vsynth2-dnxhd-hr-hq-mov
fate-vsynth3-dnxhd-2k-hr-hq fate-vsynth3-dnxhd-edge1-hr
fate-vsynth3-dnxhd-edge2-hr fate-vsynth3-dnxhd-edge3-hr
fate-vsynth3-dnxhd-hr-sq-mov fate-vsynth3-dnxhd-hr-hq-mov

Fixes trac ticket #5508.

Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3932ccc472ad4f4d370dcfc1c2f574b0f3acb88c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agointerplayacm: increase bitstream buffer size by AV_INPUT_BUFFER_PADDING_SIZE
Andreas Cadhalpun [Sun, 30 Oct 2016 20:18:20 +0000 (21:18 +0100)]
interplayacm: increase bitstream buffer size by AV_INPUT_BUFFER_PADDING_SIZE

This fixes out-of-bounds reads by the bitstream reader.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 60178e78f2fe9a7bfb9da0abc985835e2ebfd2f1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agointerplayacm: validate number of channels
Andreas Cadhalpun [Sun, 30 Oct 2016 20:41:11 +0000 (21:41 +0100)]
interplayacm: validate number of channels

The number of channels is used as divisor in decode_frame, so it must
not be zero to avoid SIGFPE crashes.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5540d6c1343e6d1e06d6601b7d35884761711e3e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agointerplayacm: check for too large b
Andreas Cadhalpun [Sun, 30 Oct 2016 19:47:22 +0000 (20:47 +0100)]
interplayacm: check for too large b

This fixes out-of-bounds reads.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 14e4e26559697cfdea584767be4e68474a0a9c7f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agompeg12dec: unref discarded picture from extradata
Andreas Cadhalpun [Thu, 20 Oct 2016 20:51:55 +0000 (22:51 +0200)]
mpeg12dec: unref discarded picture from extradata

Otherwise another frame gets referenced into picture, triggering an assert
(from commit 13aae8) in av_frame_ref.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a92f8edf0c51781e152651cce2e753ad6e359eb2)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agocavsdec: unref frame before referencing again
Andreas Cadhalpun [Thu, 20 Oct 2016 20:14:22 +0000 (22:14 +0200)]
cavsdec: unref frame before referencing again

This fixes asserts (from commit 13aae8) in av_frame_ref and
av_frame_move_ref.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1966ea012fd72abc8003e95dc3c8ad9e9f197913)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2 years agoavformat: prevent triggering request_probe assert in ff_read_packet
Andreas Cadhalpun [Wed, 19 Oct 2016 17:23:49 +0000 (19:23 +0200)]
avformat: prevent triggering request_probe assert in ff_read_packet

If probe_codec is called with pkt == NULL, it sets probe_packets to 0
and request_probe to -1.
However, request_probe can change when calling s->iformat->read_packet
and thus a probe_packets value of 0 doesn't guarantee a request_probe
value of -1.
In that case calling probe_codec again is necessary to prevent
triggering the assert.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a5b4476a602f31e451b11ca0c18bc92be130a50e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoavcodec/avpacket: fix leak on realloc in av_packet_add_side_data()
James Almer [Sat, 19 Nov 2016 15:38:44 +0000 (12:38 -0300)]
avcodec/avpacket: fix leak on realloc in av_packet_add_side_data()

If realloc fails, the pointer is overwritten and the previously allocated
buffer is leaked, which goes against the expected behavior of keeping the
packet unchanged in case of error.

Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 574929d8b6de32ae712fcca7ab09f01a3e4616be)

3 years agodoc/examples/demuxing_decoding: Drop AVFrame->pts use n3.0.4
Michael Niedermayer [Tue, 18 Oct 2016 02:23:33 +0000 (04:23 +0200)]
doc/examples/demuxing_decoding: Drop AVFrame->pts use

This code is not correct for git master

Reviewed-by: Stefano Sabatini <stefasab@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2bd99564540a365d5b80d9aad6c19264b15955af)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoChangelog: update for recent commits
Andreas Cadhalpun [Mon, 17 Oct 2016 16:15:04 +0000 (18:15 +0200)]
Changelog: update for recent commits

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agolibopenjpegenc: fix out-of-bounds reads when filling the edges
Andreas Cadhalpun [Thu, 13 Oct 2016 20:14:46 +0000 (22:14 +0200)]
libopenjpegenc: fix out-of-bounds reads when filling the edges

The calculation of width/height should round up, not round down to
prevent setting width or height to 0.

Also image->comps[compno].w is unsigned (at least in openjpeg2), so the
calculation could silently wrap around without the explicit cast to int.

Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 56706ac0d5723cb549fec2602e798ab1bf6004cd)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agolibopenjpegenc: stop reusing image data buffer for openjpeg 2
Andreas Cadhalpun [Thu, 13 Oct 2016 19:16:35 +0000 (21:16 +0200)]
libopenjpegenc: stop reusing image data buffer for openjpeg 2

openjpeg 2 sets the data pointers of the image components to NULL,
causing segfaults if the image is reused.

Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 69c8505f3bf54f316e9dc8bec1c71dfa1febec63)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoconfigure: fix detection of libopenjpeg
Andreas Cadhalpun [Tue, 11 Oct 2016 18:28:35 +0000 (20:28 +0200)]
configure: fix detection of libopenjpeg

Use check_lib2 to test the header together with the function. This is
necessary, because '-DOPJ_STATIC' changes what the included header does.

Also add '-DOPJ_STATIC' to CPPFLAGS, so that it isn't necessary to
hardcode this in libavcodec/libopenjpeg{dec,enc}.c.

Finally, check for non-static openjpeg 2.1, too.

Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a65aef00d113a38e0d1a54df49eead9df6aa15c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
3 years agoChangelog: update for recent commits
Michael Niedermayer [Mon, 17 Oct 2016 02:55:52 +0000 (04:55 +0200)]
Changelog: update for recent commits

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agocmdutils: fix typos
Moritz Barsnick [Sun, 9 Oct 2016 10:57:02 +0000 (12:57 +0200)]
cmdutils: fix typos

Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3e5d27d7a7350e096eac9f8999d02bf48c3b3a69)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agolavfi: fix typos
Moritz Barsnick [Sun, 9 Oct 2016 10:57:00 +0000 (12:57 +0200)]
lavfi: fix typos

Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f4e4bde1f4cff99d4ec59ed361ff9228b2050e6b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agolavc: fix typos
Moritz Barsnick [Sun, 9 Oct 2016 10:56:59 +0000 (12:56 +0200)]
lavc: fix typos

Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3305f71025289970fb34473adce5d9c65d1af016)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agotools: fix grammar error
Moritz Barsnick [Sun, 9 Oct 2016 10:56:58 +0000 (12:56 +0200)]
tools: fix grammar error

Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f71c98ee12f9a9e950b4a8fb6b1548fee91ba1f8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoffmpeg: remove unused and errorneous AVFrame timestamp check
Hendrik Leppkes [Sat, 1 Oct 2016 14:15:45 +0000 (16:15 +0200)]
ffmpeg: remove unused and errorneous AVFrame timestamp check

Decoders have previously not used AVFrame.pts, and with the upcoming
deprecation of pkt_pts (in favor of pts), this would lead to an errorneous
interpration of timestamps.

(cherry picked from commit 04a3577263782cd6d70722d4ae18d75fee03dbc4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoSupport for MIPS cpu P6600
Shivraj Patil [Fri, 5 Aug 2016 08:12:44 +0000 (13:42 +0530)]
Support for MIPS cpu P6600

Signed-off-by: Shivraj Patil <shivraj.patil@imgtec.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6803a298f4338c19c3032d2417c6e857eb6d95be)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavutil/mips/generic_macros_msa: rename macro variable which causes segfault for mips r6
Shivraj Patil [Wed, 5 Oct 2016 12:22:24 +0000 (17:52 +0530)]
avutil/mips/generic_macros_msa: rename macro variable which causes segfault for mips r6

Signed-off-by: Shivraj Patil <shivraj.patil@imgtec.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c1cc13cd2a9b8d6d2810ec42454f328a1a0d5efa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoUpdate for 3.0.4
Michael Niedermayer [Wed, 28 Sep 2016 15:13:09 +0000 (17:13 +0200)]
Update for 3.0.4

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/avidec: Check nb_streams in read_gab2_sub()
Michael Niedermayer [Wed, 28 Sep 2016 14:14:08 +0000 (16:14 +0200)]
avformat/avidec: Check nb_streams in read_gab2_sub()

Fixes null pointer dereference
Fixes: 1/null_point.avi

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2679ad4773aa356e7c3da5c68bc81f02a194617f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/avidec: Remove ancient assert
Michael Niedermayer [Wed, 28 Sep 2016 13:47:12 +0000 (15:47 +0200)]
avformat/avidec: Remove ancient assert

This assert can with crafted files fail, a warning is already printed
for this case.

Fixes assertion failure
Fixes:1/assert.avi

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 14bac7e00d72eac687612d9b125e585011a56d4f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/avidec: Fix memleak with dv in avi
Michael Niedermayer [Sun, 25 Sep 2016 09:56:11 +0000 (11:56 +0200)]
avformat/avidec: Fix memleak with dv in avi

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b98dafe04564d5fe3e5bf5073d871dd93a4a62de)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agolavc/movtextdec.c: Avoid infinite loop on invalid data.
Sasi Inguva [Wed, 28 Sep 2016 02:23:20 +0000 (19:23 -0700)]
lavc/movtextdec.c: Avoid infinite loop on invalid data.

Signed-off-by: Sasi Inguva <isasi@google.com>
(cherry picked from commit 7e9e1b7070242a79fa6e3acd749d7fe76e39ea7b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/ansi: Check dimensions
Michael Niedermayer [Mon, 26 Sep 2016 18:25:59 +0000 (20:25 +0200)]
avcodec/ansi: Check dimensions

Fixes: 1.avi

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 69449da436169e7facaa6d1f3bcbc41cf6ce2754)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/cavsdsp: use av_clip_uint8() for idct
Michael Niedermayer [Mon, 19 Sep 2016 13:25:38 +0000 (15:25 +0200)]
avcodec/cavsdsp: use av_clip_uint8() for idct

Fixes out of array read
Fixes: 1.swf

Found-by: 连一汉 <lianyihan@360.cn>
Tested-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0e318f110bcd6bb8e7de9127f2747272e60f48d7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/movenc: Check packet in mov_write_single_packet() too
Michael Niedermayer [Thu, 15 Sep 2016 21:52:54 +0000 (23:52 +0200)]
avformat/movenc: Check packet in mov_write_single_packet() too

Fixes assertion failure

Found-by: durandal117
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 28343139330f557e00293933a4697c7d0fc19c56)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/movenc: Factor check_pkt() out
Michael Niedermayer [Thu, 15 Sep 2016 21:52:42 +0000 (23:52 +0200)]
avformat/movenc: Factor check_pkt() out

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit deabcd2c05b2b01689d91394bbf3908da17234ed)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/utils: fix timebase error in avformat_seek_file()
Xinzheng Zhang [Wed, 14 Sep 2016 08:13:45 +0000 (16:13 +0800)]
avformat/utils: fix timebase error in avformat_seek_file()

When there is only one stream and stream_index has not specified,
The ts has been transferd by the timebase of stream0 without modifying the stream_index
In this condation it cause seek failure.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ecc04b4f2f29ac676e6c1d1ebf20ec45f5385f1e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/g726: Add missing ADDB output mask
Michael Niedermayer [Wed, 14 Sep 2016 11:06:53 +0000 (13:06 +0200)]
avcodec/g726: Add missing ADDB output mask

Fixes: 1.poc
Fixes out of array read

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a5af1240fce845f645440364c1335e0f8e44ee6c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/avpacket: clear side_data_elems
Michael Niedermayer [Mon, 12 Sep 2016 11:13:42 +0000 (13:13 +0200)]
avcodec/avpacket: clear side_data_elems

Fixes null pointer dereference

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e1bf9d8c0d2cdbbf17b06a5dfdf87a635b3203b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/movenc: Check first DTS similar to dts difference
Michael Niedermayer [Fri, 9 Sep 2016 11:11:43 +0000 (13:11 +0200)]
avformat/movenc: Check first DTS similar to dts difference

Fixes assertion failure
Fixes: b84b53855a0b74560e64c6f45f505a13/signal_sigabrt_7ffff6ae7c37_3837_ef4e243ea5b4fa8d0becf4afe9166604.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 68f4c2163ec6d4534ae1756dbcf259845f2e4d2c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/ccaption_dec: Use simple array instead of AVBuffer
Michael Niedermayer [Fri, 9 Sep 2016 08:26:15 +0000 (10:26 +0200)]
avcodec/ccaption_dec: Use simple array instead of AVBuffer

This is simpler and fixes an out of array read, fixing it with AVBuffers
would be more complex

Fixes: e00d9e6e50e5495cc93fea41147b97bb/asan_heap-oob_12dcdbb_8798_b32a97ea722dd37bb5066812cc674552.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 752e6dfa3ea97e7901870bdd9e5a51f860607240)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/mov: Fix potential integer overflow in mov_read_keys
Sergey Volk [Wed, 7 Sep 2016 21:05:35 +0000 (14:05 -0700)]
avformat/mov: Fix potential integer overflow in mov_read_keys

Actual allocation size is computed as (count + 1)*sizeof(meta_keys), so
we need to check that (count + 1) won't cause overflow.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 347cb14b7cba7560e53f4434b419b9d8800253e7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/swscale_unscaled: Try to fix Rgb16ToPlanarRgb16Wrapper() with slices
Michael Niedermayer [Sat, 3 Sep 2016 10:15:24 +0000 (12:15 +0200)]
swscale/swscale_unscaled: Try to fix Rgb16ToPlanarRgb16Wrapper() with slices

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e57d99dd4e0d8fe2992da0d65b563580e35ce728)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoswscale/swscale_unscaled: Fix packed_16bpc_bswap() with slices
Michael Niedermayer [Fri, 2 Sep 2016 18:25:24 +0000 (20:25 +0200)]
swscale/swscale_unscaled: Fix packed_16bpc_bswap() with slices

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47bc1bdafb0950ccf128eaa491d8fd7cc0978813)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agolavf/utils: Avoid an overflow for huge negative durations.
Carl Eugen Hoyos [Sat, 24 Sep 2016 11:07:39 +0000 (13:07 +0200)]
lavf/utils: Avoid an overflow for huge negative durations.

Fixes ticket #5135.
(cherry picked from commit 267da70ea8c36caaa645a3c4f1c5f0ca8bae156a)

3 years agoChangelog: update n3.0.3
Michael Niedermayer [Sun, 4 Sep 2016 23:29:52 +0000 (01:29 +0200)]
Changelog: update

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/avidec: Fix infinite loop in avi_read_nikon()
Michael Niedermayer [Fri, 2 Sep 2016 10:19:29 +0000 (12:19 +0200)]
avformat/avidec: Fix infinite loop in avi_read_nikon()

Fixes: 360/test.poc

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e4e4a9cad7f21593d4bcb1f2404ea0d373c36c43)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/utils: End probing if the expected codec surpasses AVPROBE_SCORE_STREAM_RETRY
Michael Niedermayer [Fri, 26 Aug 2016 23:12:49 +0000 (01:12 +0200)]
avformat/utils: End probing if the expected codec surpasses AVPROBE_SCORE_STREAM_RETRY

Fixes Ticket5800

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c75273310cf1becffee79bab0e2bba0b1606afb7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/aacenc: Tighter input checks
Michael Niedermayer [Tue, 23 Aug 2016 09:00:29 +0000 (11:00 +0200)]
avcodec/aacenc: Tighter input checks

Fixes occurance of NaN/Inf leading to assertion failures and out of array access
Fixes: d1c38a09acc34845c6be3a127a5aacaf/signal_sigsegv_3982225_6121_d18bd5451d4245ee09408f04badd1b83.wmv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 77bf96b04710b98a52aaddb93bfd32da0d506191)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/wtvdec: Check pointer before use
Michael Niedermayer [Sun, 21 Aug 2016 19:30:36 +0000 (21:30 +0200)]
avformat/wtvdec: Check pointer before use

Fixes out of array read
Fixes: 049fdf78565f1ce5665df236d90f8657/asan_heap-oob_10a5a97_1026_42f9d4855547329560f385768de2f3fb.wtv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc5e5548df4af48674c7aef518e831b19e99f9fc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agolibavcodec/wmalosslessdec: Check the remaining bits
Michael Niedermayer [Sun, 21 Aug 2016 18:30:34 +0000 (20:30 +0200)]
libavcodec/wmalosslessdec: Check the remaining bits

Fixes assertion failure
Fixes: 24ebfda03228b5cc1ef792608cfba458/signal_sigabrt_7ffff6ae7c37_6473_3fa8a111dbc752b1a7c411c5ab79aaa4.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 67318187fbba382d887f9581dde48a50842f1bea)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/diracdec: Check numx/y
Michael Niedermayer [Sat, 20 Aug 2016 17:21:07 +0000 (19:21 +0200)]
avcodec/diracdec: Check numx/y

Fixes division by 0
Fixes: 60261c4469ba3e11059890fb2832a515/asan_generic_135e694_2790_beb94eaa0aeb7d11c0437375a8964a99.drc

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a31e08fa1aa5c5f0518b8af850f28eb945268e66)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/cfhd: Increase minimum band dimension to 3
Michael Niedermayer [Fri, 19 Aug 2016 19:34:38 +0000 (21:34 +0200)]
avcodec/cfhd: Increase minimum band dimension to 3

The implementation does not currently support len=2

Fixes out of array accesses
Fixes: 29d1b3db5ba2205e82b0b3a533e057a3/asan_heap-oob_12b650c_9254_3b8c4e4d931eb2c32841c18ebb297f1d.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b8b36717217c6f45db71c77ad4e7c65521e7d9ff)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/indeo2: check ctab
Michael Niedermayer [Fri, 19 Aug 2016 11:07:14 +0000 (13:07 +0200)]
avcodec/indeo2: check ctab

Fixes out of array access
Fixes: 6b73fa392ac808f02e95a4e0a5770026/asan_static-oob_1b15f9a_1969_e7778535e5f27225fe0d6ded14721430.AVI

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9ffe44c5c75c485b4cbb12751e228f18da219df3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/swfdec: Fix inflate() error code check
Michael Niedermayer [Fri, 19 Aug 2016 08:28:22 +0000 (10:28 +0200)]
avformat/swfdec: Fix inflate() error code check

Fixes infinite loop
Fixes endless.poc

Found-by: 连一汉 <lianyihan@360.cn>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a453bbb68f3eec202673728988bba3bc76071761)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/rawdec: Fix bits_per_coded_sample checks
Michael Niedermayer [Fri, 19 Aug 2016 00:07:22 +0000 (02:07 +0200)]
avcodec/rawdec: Fix bits_per_coded_sample checks

Fixes assertion failure
Fixes: 9eb9cf5b8c26dd0fa7107ed0348dcc1f/signal_sigabrt_7ffff6ae7c37_8926_4609a5c3f071d555d2d557625f9687b1.swf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 237207645b36fb79759d313c0399ee93ba467b9d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agolavc/mjpegdec: Do not skip reading quantization tables.
Carl Eugen Hoyos [Fri, 2 Sep 2016 15:10:57 +0000 (17:10 +0200)]
lavc/mjpegdec: Do not skip reading quantization tables.

They may contain 0xFFs, confusing the start code finding algorithm.

Fixes ticket #5819.
(cherry picked from commit cef5bc0e6e2320d3903cf063d59cef83e91dbc3c)

Conflicts:
libavcodec/mjpegdec.c

3 years agocmdutils: fix implicit declaration of SetDllDirectory function
Tobias Rapp [Mon, 29 Aug 2016 13:25:58 +0000 (15:25 +0200)]
cmdutils: fix implicit declaration of SetDllDirectory function

Pre-processor check changed by commiter.

Signed-off-by: James Almer <jamrial@gmail.com>
3 years agocmdutils: check for SetDllDirectory() availability
James Almer [Mon, 22 Aug 2016 22:24:31 +0000 (19:24 -0300)]
cmdutils: check for SetDllDirectory() availability

It's only available on Windows XP or newer.

Should fix compilation with mingw32 using the default OS target.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
3 years agoavcodec/h264: Put context_count check back
Michael Niedermayer [Wed, 8 Jun 2016 10:32:57 +0000 (12:32 +0200)]
avcodec/h264: Put context_count check back

Fixes assertion failure

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f883f0b0bd0dac76b58e49f5c75cf9b497eecaa0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoUpdate for 3.0.3
Michael Niedermayer [Sat, 13 Aug 2016 12:12:19 +0000 (14:12 +0200)]
Update for 3.0.3

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agocmdutils: remove the current working directory from the DLL search path on win32
Hendrik Leppkes [Mon, 8 Aug 2016 13:27:41 +0000 (15:27 +0200)]
cmdutils: remove the current working directory from the DLL search path on win32

Reviewed-by: Matt Oliver <protogonoi@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3bf142c77337814458ed8e036796934032d9837f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/raw: Fix decoding of ilacetest.mov
Michael Niedermayer [Sun, 7 Aug 2016 14:27:31 +0000 (16:27 +0200)]
avcodec/raw: Fix decoding of ilacetest.mov

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bbec14de3126dbc4e1ec2b32ed714dab173386aa)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/ffv1enc: Fix assertion failure with non zero bits per sample
Michael Niedermayer [Fri, 5 Aug 2016 23:53:30 +0000 (01:53 +0200)]
avcodec/ffv1enc: Fix assertion failure with non zero bits per sample

Fixes Ticket5736
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c1bfeda5a34631787e07702f7a3569a41751caeb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavformat/oggdec: Fix integer overflow with invalid pts
Michael Niedermayer [Wed, 3 Aug 2016 11:34:40 +0000 (13:34 +0200)]
avformat/oggdec: Fix integer overflow with invalid pts

If negative pts are possible for some codecs in ogg then the code needs to be
changed to use signed values.

Found-by: Thomas Guilbert <tguilbert@google.com>
Fixes: clusterfuzz_usan-2016-08-02
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c5cc3b08e56fc95665977544486bd9f06e4b7a72)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoffplay: Fix invalid array index
Michael Niedermayer [Wed, 3 Aug 2016 11:15:14 +0000 (13:15 +0200)]
ffplay: Fix invalid array index

Found-by: Thomas Guilbert <tguilbert@google.com>
Fixes: clusterfuzz_usan-2016-08-02
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6cd9a8b67a95a136ea15bfe3c3bab6cf5e6d1cc9)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
3 years agoavcodec/vp9_parser: Check the input frame sizes for being consistent
Michael Niedermayer [Mon, 1 Aug 2016 11:50:21 +0000 (13:50 +0200)]
avcodec/vp9_parser: Check the input frame sizes for being consistent

Suggested-by: BBB
Fixed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 77b0f3f26d33d4f46f274896e0583ad1f5936b7c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>