ffmpeg.git
5 years agoupdate for 0.7.16 n0.7.16
Michael Niedermayer [Sun, 6 Oct 2013 17:07:56 +0000 (19:07 +0200)]
update for 0.7.16

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge tag 'n0.8.15' into release/0.7
Michael Niedermayer [Sun, 6 Oct 2013 16:49:35 +0000 (18:49 +0200)]
Merge tag 'n0.8.15' into release/0.7

FFmpeg 0.8.15 release

* tag 'n0.8.15': (49 commits)
  update for 0.8.15
  avcodec/ffv1enc: update buffer check for 16bps
  avcodec/dsputil: fix signedness in sizeof() comparissions
  avcodec/pngdsp: fix (un)signed type in end comparission
  matroska_read_seek: Fix used streams for subtitle index compensation
  jpeg2000: check log2_cblk dimensions
  avcodec/rpza: Perform pointer advance and checks before using the pointers
  update all trac links to use the trac subdomain
  doc/APIchanges: List merge commit hashes and version numbers
  apichanges: fix 2 wrong hashes
  avcodec/parser: reset indexes on realloc failure
  mpeg12dec: avoid reinitialization on PS changes when possible.
  mpegts: only reopen pmt_cb filter if its different from the previous.
  Autodetect idcin only if audio properties allow decoding.
  alacenc: Fix missing sign_extend()
  h264_cavlc: fix reading skip run
  Update changelog for 0.7.8 release
  aac: check the maximum number of channels
  oggdec: fix faulty cleanup prototype
  qdm2: check that the FFT size is a power of 2
  ...

Conflicts:
Doxyfile
RELEASE
VERSION
libavformat/matroskadec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoupdate for 0.8.15 n0.8.15
Michael Niedermayer [Sun, 6 Oct 2013 15:48:25 +0000 (17:48 +0200)]
update for 0.8.15

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoavcodec/ffv1enc: update buffer check for 16bps
Michael Niedermayer [Mon, 9 Sep 2013 15:58:18 +0000 (17:58 +0200)]
avcodec/ffv1enc: update buffer check for 16bps

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3728603f1854b5c79d1a64dd3b41b80640ef1e7f)

Conflicts:

libavcodec/ffv1enc.c
(cherry picked from commit c900c6e5c26cd86cf34f9c8d4347cedbd01f3935)

5 years agoavcodec/dsputil: fix signedness in sizeof() comparissions
Michael Niedermayer [Fri, 30 Aug 2013 21:40:47 +0000 (23:40 +0200)]
avcodec/dsputil: fix signedness in sizeof() comparissions

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 454a11a1c9c686c78aa97954306fb63453299760)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoavcodec/pngdsp: fix (un)signed type in end comparission
Michael Niedermayer [Fri, 30 Aug 2013 21:14:32 +0000 (23:14 +0200)]
avcodec/pngdsp: fix (un)signed type in end comparission

Fixes out of array accesses
Fixes Ticket2919

Found_by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86736f59d6a527d8bc807d09b93f971c0fe0bb07)

Conflicts:

libavcodec/pngdsp.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agomatroska_read_seek: Fix used streams for subtitle index compensation
Michael Niedermayer [Mon, 20 May 2013 02:00:30 +0000 (04:00 +0200)]
matroska_read_seek: Fix used streams for subtitle index compensation

Might fix Ticket1907 (I have no testcase so i cant test)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4758e32a6c48044f77102a49110c79b4f338f648)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agojpeg2000: check log2_cblk dimensions
Michael Niedermayer [Sat, 24 Aug 2013 01:19:40 +0000 (03:19 +0200)]
jpeg2000: check log2_cblk dimensions

Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9a271a9368eaabf99e6c2046103acb33957e63b7)

Conflicts:

libavcodec/jpeg2000dec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Conflicts:

libavcodec/j2kdec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoavcodec/rpza: Perform pointer advance and checks before using the pointers
Michael Niedermayer [Wed, 21 Aug 2013 23:07:32 +0000 (01:07 +0200)]
avcodec/rpza: Perform pointer advance and checks before using the pointers

Fixes out of array accesses
Fixes Ticket2850

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3819db745da2ac7fb3faacb116788c32f4753f34)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoupdate all trac links to use the trac subdomain
Michael Niedermayer [Sat, 29 Jun 2013 17:48:27 +0000 (19:48 +0200)]
update all trac links to use the trac subdomain

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agodoc/APIchanges: List merge commit hashes and version numbers
Michael Niedermayer [Wed, 13 Feb 2013 02:32:23 +0000 (03:32 +0100)]
doc/APIchanges: List merge commit hashes and version numbers

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoapichanges: fix 2 wrong hashes
Michael Niedermayer [Thu, 14 Feb 2013 20:13:32 +0000 (21:13 +0100)]
apichanges: fix 2 wrong hashes

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2f3bc5122822687dc388f7352c92cf6db456cf7c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoavcodec/parser: reset indexes on realloc failure
Michael Niedermayer [Thu, 26 Sep 2013 19:03:48 +0000 (21:03 +0200)]
avcodec/parser: reset indexes on realloc failure

Fixes Ticket2982

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f31011e9abfb2ae75bb32bc44e2c34194c8dc40a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge remote-tracking branch 'qatar/release/0.7' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:43:33 +0000 (17:43 +0200)]
Merge remote-tracking branch 'qatar/release/0.7' into release/0.8

* qatar/release/0.7:
  Update changelog for 0.7.8 release
  aac: check the maximum number of channels
  oggdec: fix faulty cleanup prototype
  qdm2: check that the FFT size is a power of 2
  rv10: check that extradata is large enough
  lavf: make sure stream probe data gets freed.
  dfa: check for invalid access in decode_wdlt().
  avfiltergraph: check for sws opts being non-NULL before using them.

Conflicts:
Changelog
libavformat/utils.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit 'f844cb9bced3148fca2db5bbb092929526108005' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:36:39 +0000 (17:36 +0200)]
Merge commit 'f844cb9bced3148fca2db5bbb092929526108005' into release/0.8

* commit 'f844cb9bced3148fca2db5bbb092929526108005':
  iff: validate CMAP palette size
  wmaprodec: require block_align to be set.
  lzo: fix overflow checking in copy_backptr()
  flacdec: simplify bounds checking in flac_probe()
  atrac3: avoid oversized shifting in decode_bytes()
  lavf: fix arithmetic overflows in avformat_seek_file()

Conflicts:
libavformat/iff.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit '9c713f30e4913a28d93eb37ea5db7f62be4c0ef6' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:31:56 +0000 (17:31 +0200)]
Merge commit '9c713f30e4913a28d93eb37ea5db7f62be4c0ef6' into release/0.8

* commit '9c713f30e4913a28d93eb37ea5db7f62be4c0ef6':
  parser: fix large overreads
  dsputil: fix invalid array indexing
  shorten: use the unsigned type where needed

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit '5ebb5a32bdd910a8afb316c51ed0b322f5600ae5' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:25:47 +0000 (17:25 +0200)]
Merge commit '5ebb5a32bdd910a8afb316c51ed0b322f5600ae5' into release/0.8

* commit '5ebb5a32bdd910a8afb316c51ed0b322f5600ae5':
  shorten: report meaningful errors
  shorten: set invalid channels count to 0

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit 'd785f6940144eb6ce4c24309ed034056b81395bc' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:22:35 +0000 (17:22 +0200)]
Merge commit 'd785f6940144eb6ce4c24309ed034056b81395bc' into release/0.8

* commit 'd785f6940144eb6ce4c24309ed034056b81395bc':
  shorten: validate that the channel count in the header is not <= 0
  matroskadec: request a read buffer for the wav header
  h264: check for luma and chroma bit depth being equal
  xxan: fix invalid memory access in xan_decode_frame_type0()
  wmadec: require block_align to be set.

Merged-by: Michael Niedermayer <michaelni@gmx.at>
5 years agoMerge commit '5025dbc577c9a9e0109cb363ac630a9eeda6dc1d' into release/0.8
Michael Niedermayer [Sun, 22 Sep 2013 15:17:10 +0000 (17:17 +0200)]
Merge commit '5025dbc577c9a9e0109cb363ac630a9eeda6dc1d' into release/0.8

* commit '5025dbc577c9a9e0109cb363ac630a9eeda6dc1d':
  wmaprodec: return an error, not 0, when the input is too small.
  vorbisdec: Error on bark_map_size equal to 0.
  Update RELEASE file for 0.7.8
  update year to 2013
  oggdec: make sure the private parse data is cleaned up
  indeo5: update AVCodecContext width/height on size change
  doc: filters: Correct BNF FILTER description

Conflicts:
RELEASE

Merged-by: Michael Niedermayer <michaelni@gmx.at>
6 years agompeg12dec: avoid reinitialization on PS changes when possible.
Michael Niedermayer [Mon, 8 Jul 2013 19:46:20 +0000 (21:46 +0200)]
mpeg12dec: avoid reinitialization on PS changes when possible.

Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df73528659925819dec31c4c8c0887f0321)

Conflicts:
libavcodec/mpeg12.c

6 years agompeg12dec: avoid reinitialization on PS changes when possible.
Michael Niedermayer [Mon, 8 Jul 2013 19:46:20 +0000 (21:46 +0200)]
mpeg12dec: avoid reinitialization on PS changes when possible.

Fixes Ticket2574

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 970c8df73528659925819dec31c4c8c0887f0321)

Conflicts:
libavcodec/mpeg12.c

6 years agompegts: only reopen pmt_cb filter if its different from the previous.
Michael Niedermayer [Fri, 5 Jul 2013 01:27:07 +0000 (03:27 +0200)]
mpegts: only reopen pmt_cb filter if its different from the previous.

Fixes Ticket2632

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b009267910df10c004b5f340a090d45da29089a0)

6 years agompegts: only reopen pmt_cb filter if its different from the previous.
Michael Niedermayer [Fri, 5 Jul 2013 01:27:07 +0000 (03:27 +0200)]
mpegts: only reopen pmt_cb filter if its different from the previous.

Fixes Ticket2632

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b009267910df10c004b5f340a090d45da29089a0)

6 years agoAutodetect idcin only if audio properties allow decoding.
Carl Eugen Hoyos [Wed, 19 Jun 2013 14:31:10 +0000 (16:31 +0200)]
Autodetect idcin only if audio properties allow decoding.

Fixes ticket #2688.
(cherry picked from commit 06bede95fcea47d2e51e8ff248c15311f335b898)

6 years agoAutodetect idcin only if audio properties allow decoding.
Carl Eugen Hoyos [Wed, 19 Jun 2013 14:31:10 +0000 (16:31 +0200)]
Autodetect idcin only if audio properties allow decoding.

Fixes ticket #2688.
(cherry picked from commit 06bede95fcea47d2e51e8ff248c15311f335b898)

6 years agoalacenc: Fix missing sign_extend()
Michael Niedermayer [Wed, 12 Jun 2013 22:01:13 +0000 (00:01 +0200)]
alacenc: Fix missing sign_extend()

Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc56f7e7d60767dd27ba8e846a05e8ae)

Conflicts:
libavcodec/alacenc.c

6 years agoalacenc: Fix missing sign_extend()
Michael Niedermayer [Wed, 12 Jun 2013 22:01:13 +0000 (00:01 +0200)]
alacenc: Fix missing sign_extend()

Fixes ticket #2497

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aea2f05dc56f7e7d60767dd27ba8e846a05e8ae)

Conflicts:
libavcodec/alacenc.c

6 years agoh264_cavlc: fix reading skip run
Michael Niedermayer [Thu, 30 May 2013 16:30:42 +0000 (18:30 +0200)]
h264_cavlc: fix reading skip run

Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd295c03720e00d3de83e1abcbedd4b9)

Conflicts:
libavcodec/h264_cavlc.c

6 years agoh264_cavlc: fix reading skip run
Michael Niedermayer [Thu, 30 May 2013 16:30:42 +0000 (18:30 +0200)]
h264_cavlc: fix reading skip run

Fixes Ticket2606

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 826b3a75cd295c03720e00d3de83e1abcbedd4b9)

Conflicts:
libavcodec/h264_cavlc.c

6 years agoUpdate changelog for 0.7.8 release
Reinhard Tartler [Sat, 11 May 2013 10:08:35 +0000 (12:08 +0200)]
Update changelog for 0.7.8 release

6 years agoaac: check the maximum number of channels
Reinhard Tartler [Tue, 7 May 2013 05:13:50 +0000 (07:13 +0200)]
aac: check the maximum number of channels

Broken bitstreams could report a larger than specified number of
channels and cause outbound writes.

CC:libav-stable@libav.org
(cherry picked from commit a943a132f36f4df8fe2f749744677b71984abce7)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/aacdec.c

6 years agooggdec: fix faulty cleanup prototype
Luca Barbato [Wed, 9 Jan 2013 19:49:34 +0000 (20:49 +0100)]
oggdec: fix faulty cleanup prototype

(cherry picked from commit fba8e5b608577fc660989d0057a55818254a3744)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoqdm2: check that the FFT size is a power of 2
Anton Khirnov [Tue, 9 Apr 2013 13:25:20 +0000 (15:25 +0200)]
qdm2: check that the FFT size is a power of 2

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 34f87a58532ed652a6e0283c1d044ee5df0aef0b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agorv10: check that extradata is large enough
Anton Khirnov [Tue, 9 Apr 2013 18:33:25 +0000 (20:33 +0200)]
rv10: check that extradata is large enough

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

(cherry picked from commit 01d376f598fe95478036f5d1e3e5e14ffe32d4bf)

Conflicts:

libavcodec/rv10.c

6 years agoiff: validate CMAP palette size
Kostya Shishkov [Sun, 17 Mar 2013 19:22:19 +0000 (20:22 +0100)]
iff: validate CMAP palette size

Fixes CVE-2013-2495

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
CC: libav-stable@libav.org
(cherry picked from commit 50c449ac24fbb4c03c15d2e2026cef2204b80385)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 31a77177ff323ef83944c60a8654891213ab6691)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agolavf: make sure stream probe data gets freed.
Anton Khirnov [Wed, 27 Mar 2013 16:56:59 +0000 (17:56 +0100)]
lavf: make sure stream probe data gets freed.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit dbb1425811a672eddf4acf0513237cdf20f83756)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agowmaprodec: require block_align to be set.
Anton Khirnov [Wed, 6 Mar 2013 08:58:00 +0000 (09:58 +0100)]
wmaprodec: require block_align to be set.

Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit cacad1c058f66558ec727faac3b277d2dee264d4)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 20373a66ec68d958c266f643a7d0e5ec254c0fcc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agodfa: check for invalid access in decode_wdlt().
Anton Khirnov [Wed, 27 Mar 2013 17:18:38 +0000 (18:18 +0100)]
dfa: check for invalid access in decode_wdlt().

This can happen when the number of skipped lines is not consistent with
the number of coded lines.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 3623589edc7b1257bb45aa9e52c9631e133f22b6)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoavfiltergraph: check for sws opts being non-NULL before using them.
Anton Khirnov [Sun, 17 Mar 2013 15:14:58 +0000 (16:14 +0100)]
avfiltergraph: check for sws opts being non-NULL before using them.

Avoid snprintfing a NULL pointer.

CC: libav-stable@libav.org
(cherry picked from commit 6e3c13a559e9ff300b5ca60e1d503e594d7f055c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoparser: fix large overreads
Michael Niedermayer [Wed, 3 Oct 2012 14:06:23 +0000 (16:06 +0200)]
parser: fix large overreads

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 096abfa15052977eed93f0b5e01afd2d47c53c1f)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
6 years agolzo: fix overflow checking in copy_backptr()
Xi Wang [Fri, 15 Mar 2013 10:59:22 +0000 (06:59 -0400)]
lzo: fix overflow checking in copy_backptr()

The check `src > dst' in the form `&c->out[-back] > c->out' invokes
pointer overflow, which is undefined behavior in C.

Remove the check.  Also replace `&c->out[-back] < c->out_start' with
a safe form `c->out - c->out_start < back' to avoid overflow.

CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit ca6c3f2c53be70aa3c38e8f1292809db89ea1ba6)

Conflicts:
libavutil/lzo.c

6 years agodsputil: fix invalid array indexing
Mans Rullgard [Thu, 26 Apr 2012 13:00:43 +0000 (14:00 +0100)]
dsputil: fix invalid array indexing

Indexing outside an array is invalid and causes errors with
gcc 4.8.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 0a07f2b346433a9a2677c69c6b29a1a827e39109)

Signed-off-by: Diego Biurrun <diego@biurrun.de>
6 years agoflacdec: simplify bounds checking in flac_probe()
Xi Wang [Fri, 15 Mar 2013 11:11:47 +0000 (07:11 -0400)]
flacdec: simplify bounds checking in flac_probe()

Simplify `p->buf > p->buf + p->buf_size - 4' as `p->buf_size < 4'.
Avoid a possible out-of-bounds pointer, which is undefined behavior
in C.

CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 8425d693eefbedbb41f91735614d41067695aa37)

6 years agoatrac3: avoid oversized shifting in decode_bytes()
Xi Wang [Fri, 15 Mar 2013 10:31:21 +0000 (06:31 -0400)]
atrac3: avoid oversized shifting in decode_bytes()

When `off' is 0, `0x537F6103 << 32' in the following expression invokes
undefined behavior, the result of which is not necessarily 0.

    (0x537F6103 >> (off * 8)) | (0x537F6103 << (32 - (off * 8)))

Avoid oversized shifting.

CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit eba1ff31304e407db3cefd7532108408f364367b)

Conflicts:
libavcodec/atrac3.c

6 years agolavf: fix arithmetic overflows in avformat_seek_file()
Mans Rullgard [Fri, 7 Dec 2012 13:53:56 +0000 (13:53 +0000)]
lavf: fix arithmetic overflows in avformat_seek_file()

The values compared here can be more than INT64_MAX apart.  Since the
difference is always positive, converting to uint64_t before subtracting
gives the correct result without overflows.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 91ac403b1316d59b4f43c4ea0f237e24cec2819a)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
6 years agoshorten: use the unsigned type where needed
Luca Barbato [Tue, 5 Mar 2013 16:12:35 +0000 (17:12 +0100)]
shorten: use the unsigned type where needed

get_uint returns an unsigned value, use an unsigned to store
blocksize to make sure the comparison logic is correct and report
correctly the error for the channel count not supported.

CC: libav-stable@libav.org
(cherry picked from commit 5cf7c72757779a740e897a97710aac044fe5258c)
(cherry picked from commit 88089eecfd7e604d40d078b4f4206c647cb2e2b4)
(cherry picked from commit f42d03746afe491dd02bb6372961e85e78299864)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/shorten.c

6 years agoshorten: report meaningful errors
Reinhard Tartler [Tue, 7 May 2013 05:29:06 +0000 (07:29 +0200)]
shorten: report meaningful errors

(cherry picked from commit 4c364eb2b856fc33cf7b42f7c7b979e69fde5f3a)
(cherry picked from commit 0daf1428e82926dc5a8c72a0ff4c93aaa8a84ed9)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/shorten.c

6 years agoshorten: set invalid channels count to 0
Reinhard Tartler [Tue, 7 May 2013 05:26:19 +0000 (07:26 +0200)]
shorten: set invalid channels count to 0

Prevent the loop shorten_decode_close from writing and freeing out of
the array boundary.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
CC: libav-stable@libav.org
(cherry picked from commit c10da30d8426a1f681d99a780b6e311f7fb4e5c5)
(cherry picked from commit 21d568be179c54a1596d1377b4da7fbe755bfe7f)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Conflicts:
libavcodec/shorten.c

6 years agoshorten: validate that the channel count in the header is not <= 0
Justin Ruggles [Tue, 23 Oct 2012 04:40:51 +0000 (00:40 -0400)]
shorten: validate that the channel count in the header is not <= 0

(cherry picked from commit 4c53f4aed3edfa58360c7a2a468782eae31d3176)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/shorten.c

6 years agowmaprodec: return an error, not 0, when the input is too small.
Anton Khirnov [Wed, 6 Mar 2013 09:02:50 +0000 (10:02 +0100)]
wmaprodec: return an error, not 0, when the input is too small.

Returning 0 may result in an infinite loop in valid calling programs. A
decoder should never return 0 without producing any output.

CC:libav-stable@libav.org
(cherry picked from commit 4c0080b7e7d501e2720d2a61f5186a18377f9d63)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 60dd8b5733f9ec4919fbc732ace1be8184dde880)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agomatroskadec: request a read buffer for the wav header
Luca Barbato [Tue, 12 Mar 2013 17:56:28 +0000 (18:56 +0100)]
matroskadec: request a read buffer for the wav header

Solve an infiniloop.

CC: libav-stable@libav.org
(cherry picked from commit 37cb3b180a1dc3d6f123f68e0806585ebc2578b6)

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
6 years agovorbisdec: Error on bark_map_size equal to 0.
Michael Niedermayer [Thu, 10 Jan 2013 23:54:12 +0000 (00:54 +0100)]
vorbisdec: Error on bark_map_size equal to 0.

The value is used to calculate output LSP curve and a division by zero
and out of array accesses would occur.

CVE-2013-0894

CC: libav-stable@libav.org
Reported-by: Dale Curtis <dalecurtis@chromium.org>
Found-by: inferno@chromium.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 11dcecfcca0eca1a571792c4fa3c21fb2cfddddc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 494ddd377ada76ed555f7a3f49391455daa099c9)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoh264: check for luma and chroma bit depth being equal
Reinhard Tartler [Tue, 7 May 2013 05:25:10 +0000 (07:25 +0200)]
h264: check for luma and chroma bit depth being equal

The decoder assumes a single bit depth for all the planes while
the specification allows different bit depths for luma and chroma.

Avoid the possible problems described in CVE-2013-2277

Conflicts:
libavcodec/h264.c

6 years agoUpdate RELEASE file for 0.7.8
Reinhard Tartler [Sun, 17 Feb 2013 08:10:52 +0000 (09:10 +0100)]
Update RELEASE file for 0.7.8

6 years agoxxan: fix invalid memory access in xan_decode_frame_type0()
Reinhard Tartler [Tue, 7 May 2013 05:24:16 +0000 (07:24 +0200)]
xxan: fix invalid memory access in xan_decode_frame_type0()

The loop a few lines below the xan_unpack() call accesses up to
dec_size * 2 bytes into y_buffer, so dec_size must be limited to
buffer_size / 2.

CC:libav-stable@libav.org
(cherry picked from commit 8a49d2bcbe7573bb4b765728b2578fac0d19763f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 62a657de168cf501acb23d48cc1aa00793dc83f3)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
Conflicts:
libavcodec/xxan.c

6 years agowmadec: require block_align to be set.
Anton Khirnov [Wed, 6 Mar 2013 08:58:00 +0000 (09:58 +0100)]
wmadec: require block_align to be set.

Avoids an infinite loop in the calling programs with decoder not
consuming any input and not returning output.

CC:libav-stable@libav.org
(cherry picked from commit ea1136baafb1fe271cb56c3f4d7bff0267e3c70f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit c1f479e8df24284237c80ad959619fc85e29a26d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoupdate year to 2013
Reinhard Tartler [Sun, 17 Feb 2013 08:10:16 +0000 (09:10 +0100)]
update year to 2013

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agooggdec: make sure the private parse data is cleaned up
Luca Barbato [Fri, 4 Jan 2013 15:05:51 +0000 (16:05 +0100)]
oggdec: make sure the private parse data is cleaned up

Related to CVE-2012-2882

(cherry picked from commit d894f74762bc95310ba23f804b7ba8dffc8f6646)

Conflicts:

libavformat/oggdec.h
libavformat/oggparsevorbis.c
(cherry picked from commit b0240165d93d4a08d15d244953219a4d4e725d3f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
6 years agoindeo5: update AVCodecContext width/height on size change
Michael Niedermayer [Sat, 14 Apr 2012 18:04:05 +0000 (20:04 +0200)]
indeo5: update AVCodecContext width/height on size change

Fixes CVE-2012-2787

Note that in 0.7, there is only indeo 5, no indeo 4 decoder

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit b146d74730ab9ec5abede9066f770ad851e45fbc)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit 2bc1e4fcb96c470e2ccb2a0a78a415d5eab960c8)

Conflicts:

libavcodec/ivi_common.c

6 years agoFix type of shared flac table ff_flac_blocksize_table[].
Carl Eugen Hoyos [Sun, 5 May 2013 17:29:00 +0000 (19:29 +0200)]
Fix type of shared flac table ff_flac_blocksize_table[].

Fixes ticket #2533.
(cherry picked from commit a07ac1f7888fd08e42da2bed0421e74f1cfac177)

6 years agoFix type of shared flac table ff_flac_blocksize_table[].
Carl Eugen Hoyos [Sun, 5 May 2013 17:29:00 +0000 (19:29 +0200)]
Fix type of shared flac table ff_flac_blocksize_table[].

Fixes ticket #2533.
(cherry picked from commit a07ac1f7888fd08e42da2bed0421e74f1cfac177)

6 years agosmacker: fix off by one error
Paul B Mahol [Wed, 3 Apr 2013 12:57:58 +0000 (12:57 +0000)]
smacker: fix off by one error

Regression since a93b572ae4f517ce0c35cf085167c318e9215908.

Fixes #2426.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit e3cc92a623a6ece42816c7a692c8815688a99ab0)

6 years agosmacker: fix off by one error
Paul B Mahol [Wed, 3 Apr 2013 12:57:58 +0000 (12:57 +0000)]
smacker: fix off by one error

Regression since a93b572ae4f517ce0c35cf085167c318e9215908.

Fixes #2426.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
(cherry picked from commit e3cc92a623a6ece42816c7a692c8815688a99ab0)

6 years agodoc: filters: Correct BNF FILTER description
Vicente Jimenez Aguilar [Wed, 20 Feb 2013 01:35:00 +0000 (02:35 +0100)]
doc: filters: Correct BNF FILTER description

Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit b5ad422bf4e671a8b30ce73ad236cd6b49940af9)

6 years agoupdate for 0.7.15 n0.7.15
Michael Niedermayer [Thu, 21 Feb 2013 02:28:32 +0000 (03:28 +0100)]
update for 0.7.15

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoMerge branch 'release/0.8' into release/0.7
Michael Niedermayer [Thu, 21 Feb 2013 02:28:00 +0000 (03:28 +0100)]
Merge branch 'release/0.8' into release/0.7

* release/0.8:
  cook: check js_subband_start for validity
  avcodec_align_dimensions2: Ensure cinepak has large enough buffers.
  Update for 0.8.14
  qdm2: increase noise_table size
  wma: check byte_offset_bits
  tiff: check bppcount
  vqavideo: fix return type

Conflicts:
Doxyfile
RELEASE
VERSION

Merged-by: Michael Niedermayer <michaelni@gmx.at>
6 years agocook: check js_subband_start for validity n0.8.14
Michael Niedermayer [Tue, 19 Feb 2013 23:19:39 +0000 (00:19 +0100)]
cook: check js_subband_start for validity

Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c69315a5deb0f8095e6b4746b69171d6f3059b2f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoavcodec_align_dimensions2: Ensure cinepak has large enough buffers.
Michael Niedermayer [Wed, 20 Feb 2013 01:24:30 +0000 (02:24 +0100)]
avcodec_align_dimensions2: Ensure cinepak has large enough buffers.

This is partly redundant with the following patches, but its safer

Found-by: u-bo1b@0w.se
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f5c00b347dc76285c639d9878a014c40395c5228)

Conflicts:

libavcodec/utils.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoUpdate for 0.8.14
Michael Niedermayer [Wed, 20 Feb 2013 00:26:33 +0000 (01:26 +0100)]
Update for 0.8.14

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoqdm2: increase noise_table size
Michael Niedermayer [Mon, 28 Jan 2013 18:34:55 +0000 (19:34 +0100)]
qdm2: increase noise_table size

This prevents out of array reads. An alternative solution would be
to check the index but this would require several checks in the
inner loops

Yet another alternative would be to change the index reset logic
but this likely would introduce a difference to the binary decoder

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8c4aebb58d00fd613f3f684bf0f869966149ae78)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agowma: check byte_offset_bits
Michael Niedermayer [Wed, 30 Jan 2013 21:56:45 +0000 (22:56 +0100)]
wma: check byte_offset_bits

Fixes assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 984add64a41c3296a8a82051cc90bff2eb449609)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agotiff: check bppcount
Michael Niedermayer [Tue, 19 Feb 2013 16:48:56 +0000 (17:48 +0100)]
tiff: check bppcount

Fixes division by 0

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a34418c28e0accd1468ca15fff4d4f138a609f4e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agovqavideo: fix return type
Michael Niedermayer [Tue, 19 Feb 2013 23:47:13 +0000 (00:47 +0100)]
vqavideo: fix return type

Fixes Ticket2281

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoupdate for 0.7.14 n0.7.14
Michael Niedermayer [Mon, 18 Feb 2013 00:12:02 +0000 (01:12 +0100)]
update for 0.7.14

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoMerge branch 'release/0.8' into release/0.7
Michael Niedermayer [Sun, 17 Feb 2013 23:56:01 +0000 (00:56 +0100)]
Merge branch 'release/0.8' into release/0.7

* release/0.8: (92 commits)
  Update for 0.8.13
  pngdec/filter: dont access out of array elements at the end
  aacdec: check channel count
  vqavideo: check chunk sizes before reading chunks
  eamad: fix out of array accesses
  roqvideodec: check dimensions validity
  qdm2: check array index before use, fix out of array accesses
  alsdec: check block length
  huffyuvdec: Skip len==0 cases
  huffyuvdec: Check init_vlc() return codes.
  Update changelog for 0.7.7 release
  mpeg12: do not decode extradata more than once.
  indeo4/5: check empty tile size in decode_mb_info().
  dfa: improve boundary checks in decode_dds1()
  indeo5dec: Make sure we have had a valid gop header.
  rv34: error out on size changes with frame threading
  rtmp: fix buffer overflows in ff_amf_tag_contents()
  rtmp: fix multiple broken overflow checks
  Revert "h264: allow cropping to AVCodecContext.width/height"
  h264: check ref_count validity for num_ref_idx_active_override_flag
  ...

Conflicts:
Doxyfile
RELEASE
VERSION
libavcodec/rv34.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoUpdate for 0.8.13 n0.8.13
Michael Niedermayer [Sun, 17 Feb 2013 22:41:01 +0000 (23:41 +0100)]
Update for 0.8.13

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agopngdec/filter: dont access out of array elements at the end
Michael Niedermayer [Tue, 12 Feb 2013 18:53:40 +0000 (19:53 +0100)]
pngdec/filter: dont access out of array elements at the end

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1ac0fa50eff30d413206cffa5f47f7fe6d4849b1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoaacdec: check channel count
Michael Niedermayer [Sun, 27 Jan 2013 19:37:27 +0000 (20:37 +0100)]
aacdec: check channel count

Prevent out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 96f452ac647dae33c53c242ef3266b65a9beafb6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agovqavideo: check chunk sizes before reading chunks
Michael Niedermayer [Fri, 25 Jan 2013 05:11:59 +0000 (06:11 +0100)]
vqavideo: check chunk sizes before reading chunks

Fixes out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ab6c9332bfa1e20127a16392a0b85a4aa4840889)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoeamad: fix out of array accesses
Michael Niedermayer [Sat, 17 Nov 2012 15:26:55 +0000 (16:26 +0100)]
eamad: fix out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 63ac64864c6e0e84355aa3caa5b92208997a9a8d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoroqvideodec: check dimensions validity
Michael Niedermayer [Thu, 29 Nov 2012 14:18:17 +0000 (15:18 +0100)]
roqvideodec: check dimensions validity

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ae610451170cd5a28b33950006ff0bd23036845)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoqdm2: check array index before use, fix out of array accesses
Michael Niedermayer [Fri, 30 Nov 2012 22:59:40 +0000 (23:59 +0100)]
qdm2: check array index before use, fix out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a7ee6281f7ef1c29284e3a4cadfe0f227ffde1ed)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoalsdec: check block length
Michael Niedermayer [Wed, 12 Dec 2012 11:28:45 +0000 (12:28 +0100)]
alsdec: check block length

Fix writing over the end

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0ceca269b66ec12a23bf0907bd2c220513cdbf16)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoMerge remote-tracking branch 'qatar/release/0.7' into release/0.8
Michael Niedermayer [Thu, 14 Feb 2013 13:12:14 +0000 (14:12 +0100)]
Merge remote-tracking branch 'qatar/release/0.7' into release/0.8

* qatar/release/0.7:
  Update changelog for 0.7.7 release
  mpeg12: do not decode extradata more than once.
  indeo4/5: check empty tile size in decode_mb_info().
  dfa: improve boundary checks in decode_dds1()
  indeo5dec: Make sure we have had a valid gop header.
  rv34: error out on size changes with frame threading

Conflicts:
Changelog

Merged-by: Michael Niedermayer <michaelni@gmx.at>
6 years agohuffyuvdec: Skip len==0 cases
Michael Niedermayer [Tue, 29 Jan 2013 18:22:33 +0000 (19:22 +0100)]
huffyuvdec: Skip len==0 cases

Fixes vlc decoding for hypothetical files that would contain such cases.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0dfc01c2bbf4b71bb56201bc4a393321e15d1b31)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5ff41ffeb4cb9ea6df49757dc859619dc3d3ab4f)

Conflicts:

libavcodec/huffyuv.c
(cherry picked from commit 9bc70fe1ae50fd2faa0b9429d47cfbda01a92ebc)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agohuffyuvdec: Check init_vlc() return codes.
Michael Niedermayer [Tue, 29 Jan 2013 17:29:41 +0000 (18:29 +0100)]
huffyuvdec: Check init_vlc() return codes.

Prevents out of array writes

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f67a0d115254461649470452058fa3c28c0df294)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 95ab8d33e1a680f30a5a9605175112008ab81afc)

Conflicts:

libavcodec/huffyuv.c
(cherry picked from commit 277def59fce10d91e3113e5c0f63e22bc4abfa88)

Conflicts:

libavcodec/huffyuv.c
(cherry picked from commit adf022f458d75e2c8041262e1906a249366ad518)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoUpdate changelog for 0.7.7 release
Reinhard Tartler [Thu, 24 Jan 2013 13:01:42 +0000 (14:01 +0100)]
Update changelog for 0.7.7 release

6 years agompeg12: do not decode extradata more than once.
Anton Khirnov [Thu, 13 Dec 2012 16:53:31 +0000 (17:53 +0100)]
mpeg12: do not decode extradata more than once.

Fixes CVE-2012-2803.

(cherry picked from commit 582368626188c070d4300913c6da5efa4c24cfb2)

Conflicts:

libavcodec/mpeg12.c
libavcodec/mpeg12.h

6 years agoindeo4/5: check empty tile size in decode_mb_info().
Anton Khirnov [Sat, 29 Sep 2012 09:07:58 +0000 (11:07 +0200)]
indeo4/5: check empty tile size in decode_mb_info().

This prevents writing into a too small array if some parameters changed
without the tile being reallocated.

Based on a patch by Michael Niedermayer <michaelni@gmx.at>

Fixes CVE-2012-2800

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit ae3da0ae5550053583a6f281ea7fd940497ea0d1)

Conflicts:

libavcodec/ivi_common.c

6 years agodfa: improve boundary checks in decode_dds1()
Anton Khirnov [Sat, 29 Sep 2012 11:25:28 +0000 (13:25 +0200)]
dfa: improve boundary checks in decode_dds1()

Fixes CVE-2012-2798

CC:libav-stable@libav.org
(cherry picked from commit d05f72c75445969cd7bdb1d860635c9880c67fb6)

Conflicts:

libavcodec/dfa.c

6 years agoindeo5dec: Make sure we have had a valid gop header.
Michael Niedermayer [Sat, 24 Mar 2012 16:43:55 +0000 (17:43 +0100)]
indeo5dec: Make sure we have had a valid gop header.

This prevents decoding happening on a half initialized context.

Fixes CVE-2012-2779

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 891918431db628db17885ed947ee387b29826a64)

Conflicts:

libavcodec/ivi_common.c
libavcodec/ivi_common.h

6 years agorv34: error out on size changes with frame threading
Janne Grunau [Fri, 23 Mar 2012 21:30:38 +0000 (22:30 +0100)]
rv34: error out on size changes with frame threading

(cherry picked from commit cb7190cd2c691fd93e4d3664f3fce6c19ee001dd)

Fixes: CVE-2012-2772 (according to Ubuntu)

6 years agortmp: fix buffer overflows in ff_amf_tag_contents()
Xi Wang [Wed, 23 Jan 2013 02:40:05 +0000 (21:40 -0500)]
rtmp: fix buffer overflows in ff_amf_tag_contents()

A negative `size' will bypass FFMIN().  In the subsequent memcpy() call,
`size' will be considered as a large positive value, leading to a buffer
overflow.

Change the type of `size' to unsigned int to avoid buffer overflow, and
simplify overflow checks accordingly.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4e692374f7962ea358c329de38c380103f8991b6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agortmp: fix multiple broken overflow checks
Xi Wang [Tue, 22 Jan 2013 22:49:29 +0000 (17:49 -0500)]
rtmp: fix multiple broken overflow checks

Sanity checks like `data + size >= data_end || data + size < data' are
broken, because `data + size < data' assumes pointer overflow, which is
undefined behavior in C.  Many compilers such as gcc/clang optimize such
checks away.

Use `size < 0 || size >= data_end - data' instead.

Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 902cfe2f74d777a7dc20ac68f2393b9f84b790c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoRevert "h264: allow cropping to AVCodecContext.width/height"
Michael Niedermayer [Sat, 19 Jan 2013 12:34:41 +0000 (13:34 +0100)]
Revert "h264: allow cropping to AVCodecContext.width/height"

This reverts commit a2ae183a382f063c5403922b5151d865ce7252a2.

This removes a duplicate hunk

Found-by: Joakim Plate <elupus@ecce.se>
6 years agoMerge remote-tracking branch 'qatar/release/0.7' into release/0.8
Michael Niedermayer [Thu, 17 Jan 2013 02:16:46 +0000 (03:16 +0100)]
Merge remote-tracking branch 'qatar/release/0.7' into release/0.8

* qatar/release/0.7:
  h264: check ref_count validity for num_ref_idx_active_override_flag
  h264: check context state before decoding slice data partitions
  oggdec: free the ogg streams on read_header failure
  oggdec: check memory allocation
  Fix uninitialized reads on malformed ogg files.
  rtsp: Recheck the reordering queue if getting a new packet
  alacdec: do not be too strict about the extradata size
  h264: fix sps parsing for SVC and CAVLC 4:4:4 Intra profiles
  h264: check sps.log2_max_frame_num for validity
  ppc: always use pic for shared libraries
  h264: enable low delay only if no delayed frames were seen
  lavf: avoid integer overflow in ff_compute_frame_duration()

Conflicts:
libavformat/oggdec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoMerge commit 'b143844ea0f6246e0d5a938d743e2e8a98453bec' into release/0.8
Michael Niedermayer [Thu, 17 Jan 2013 02:03:39 +0000 (03:03 +0100)]
Merge commit 'b143844ea0f6246e0d5a938d743e2e8a98453bec' into release/0.8

* commit 'b143844ea0f6246e0d5a938d743e2e8a98453bec': (22 commits)
  aacdec: Fix an off-by-one overwrite when switching to LTP profile from MAIN.
  vp6: properly fail on unsupported feature
  h264: Fix parameters to ff_er_add_slice() call
  flacenc: ensure the order is within the min/max range in LPC order search
  yuv4mpeg: reject unsupported codecs
  vp8: reset loopfilter delta values at keyframes.
  vp56: release frames on error
  vp56: make parse_header return standard error codes
  ivi_common: check that scan pattern is set before using it.
  Update RELEASE file for 0.7.7
  tiffenc: Check av_malloc() results.
  mpegaudiodec: fix short_start calculation
  h264: avoid stuck buffer pointer in decode_nal_units
  yuv4mpeg: return proper error codes.
  smacker audio: sign-extend the initial 16-bit predicted value
  vf_pad: don't give up its own reference to the output buffer.
  avidec: return 0, not packet size from read_packet().
  wmapro: prevent division by zero when sample rate is unspecified
  alsdec: fix number of decoded samples in first sub-block in BGMC mode.
  alsdec: remove dead assignments
  ...

Conflicts:
RELEASE
libavformat/avidec.c
libavformat/yuv4mpeg.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoMerge commit 'aa45b90804ab21175b8c116bd8e5eb4b4e85fbcb' into release/0.8
Michael Niedermayer [Thu, 17 Jan 2013 01:56:12 +0000 (02:56 +0100)]
Merge commit 'aa45b90804ab21175b8c116bd8e5eb4b4e85fbcb' into release/0.8

* commit 'aa45b90804ab21175b8c116bd8e5eb4b4e85fbcb': (22 commits)
  alsdec: Check k used for rice decoder.
  cavsdec: check for changing w/h.
  avidec: use actually read size instead of requested size
  wmaprodec: check num_vec_coeffs for validity
  lagarith: check count before writing zeros.
  indeo5: check tile size in decode_mb_info().
  indeo5: prevent null pointer dereference on broken files
  indeo: check for invalid motion vectors
  indeo: clear allocated band buffers
  indeo: check custom Huffman tables for errors
  dfa: add some checks to ensure that decoder won't write past frame end
  dfa: check that the caller set width/height properly.
  bytestream: add a new set of bytestream functions with overread checking
  avsdec: Set dimensions instead of relying on the demuxer.
  lavfi: avfilter_merge_formats: handle case where inputs are same
  rv34: use AVERROR return values in ff_rv34_decode_frame()
  h263: Add ff_ prefix to nonstatic symbols
  eval: fix swapping of lt() and lte()
  bmpdec: only initialize palette for pal8.
  vc1dec: add flush function for WMV9 and VC-1 decoders
  ...

Conflicts:
libavcodec/avs.c
libavcodec/mpegvideo_enc.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
6 years agoh264: check ref_count validity for num_ref_idx_active_override_flag
Janne Grunau [Sat, 12 Jan 2013 16:22:50 +0000 (17:22 +0100)]
h264: check ref_count validity for num_ref_idx_active_override_flag

Fixes segfault in the fuzzed sample bipbop234.ts_s226407.
CC: libav-stable@libav.org
(cherry-picked from commit 6e5cdf26281945ddea3aaf5eca4d127791f23ca8)
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
6 years agoh264: check context state before decoding slice data partitions
Janne Grunau [Wed, 28 Nov 2012 21:17:14 +0000 (22:17 +0100)]
h264: check context state before decoding slice data partitions

Fixes mov_h264_aac__Demo_FlagOfOurFathers.mov.SIGSEGV.4e9.656.

Found-by: Mateusz "j00ru" Jurczyk
CC: libav-stable@libav.org
(cherry-picked from commit c1fcf563b13051f280db169ba41c6a1b21b25e08)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>