tools/target_dec_fuzzer: Adjust VP7 threshold

Fixes: Timeout (110sec -> 10sec)
Fixes: 17705/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP7_fuzzer-5765834135306240

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/qtrleenc: fix undefined behaviour

Fixes #7991.

doc/examples/muxing: fix underflow in duration of encoded streams

Now they are exactly 10 seconds long.

Fixes #5684.

avcodec/cbs_av1: Make overread check more robust

When performing a comparison of a signed int and an unsigned int, the
signed int is first converted to an unsigned int, so that negative
values are being treated as big, positive values. This can become a
problem in an overread check, namely when an overread already happened.
So change the type of the variable containing the amount of bits that
need to be left to signed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avcodec/cbs_h2645: Fix potential out-of-bounds array access

The maximum allowed index for an array access is FF_ARRAY_ELEMS - 1; yet
the current code allowed FF_ARRAY_ELEMS. This wasn't dangerous in practice,
as parameter sets with invalid ids were already filtered out during
reading.

Found via PVS-Studio (see ticket #8156).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avcodec/fitsdec: fix use of uninitialised values

header.data_max and header.data_min are not necessarely set on all decoding scenarios.

Fixes a Valgrind reported regression since cfa193779103c97bbfc28273a0ab12c114b6786d.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>

avformat/utils: Remove unnecessary initializations

Up until now, read_frame_internal always initialized the packet it
received. But since the recent changes to ff_read_packet, this is no
longer needed: If the parsing queue is initially empty upon entering
read_frame_internal, the packet will now either contain content upon
success or be blank upon failure of ff_read_packet. If the parsing
queue is initially not empty, the packet will be overwritten with the
oldest one from the parsing queue.

Similarly, it is unnecessary to initialize ret in read_frame_internal.

In parse_packet, it is easily possible to only initialize the packet
used as temporary storage for the output if said packet is used at all;
furthermore, this packet doesn't need to be zero-initialized, because
av_init_packet will initialize every field except size and data and
those fields will be set by av_parser_parse2.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avformat/utils: Improve parsing packets

Up until now, parse_packet() used a stack packet in case the stream is
flushed. But using such a packet is unnecessary as there is an AVPacket
readily available, it just needs to be used. Whether flushing is intended
or not will now be signalled by an explicit parameter rather than by
whether the packet parameter is NULL. This removes a few checks in
parse_packet(), gets rid of the initialization of the stack packet and
also reduces usage of sizeof(AVPacket) in libavformat.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avformat/utils: Avoid copying packets unnecessarily

Up until now, read_frame_internal in avformat/utils.c uses a spare
packet on the stack that serves no real purpose: At no point in this
function is there a need for another packet besides the packet destined
for output:
1. If the packet doesn't need a parser, but is output as is, the content
of the spare packet (that at this point contains a freshly read packet)
is simply copied into the output packet (via simple assignment, not
av_packet_move_ref, thereby confusing ownership).
2. If the packet needs parsing, the spare packet will be reset after
parsing and any packets resulting from the packet read will be put into
a packet list; the output packet is not used here at all.
3. If the stream should be discarded, the spare packet will be
unreferenced; the output packet is not used here at all either.

Therefore the spare packet and the copies can be removed in principle.
In practice, one more thing needs to be taken care of: If ff_read_packet
failed, the output packet was not affected, now it is. But given that
ff_read_packet returns a blank (as if reset via av_packet_unref) packet
on failure, there is no problem from this side either.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avformat/utils: Don't create unnecessary references

When AVFMT_FLAG_GENPTS is set, av_read_frame would put a reference to a
packet in the packet list (via av_packet_ref) and then immediately
thereafter unreference the original packet. This has been changed to
move the reference instead.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avformat/utils: Fix memleaks II

Up until now, avformat_find_stream_info had a potential for memleaks:
When everything was fine, it read packets and (depending upon whether
AVFMT_FLAG_NOBUFFER was set) put them in a packet list or unreferenced
them when they were no longer needed. But upon failure, said packets
would leak if they were not already on the packet list. This patch fixes
this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avformat/utils: Don't initialize in loops

Since the recent changes to ff_packet_list_put, the source packet will
be automatically reset when the reference is moved to the packet list,
so that it is unnecessary to reinitialize the packet in the loops in
parse_packet and ff_read_packet; initializing once at the beginning is
enough.

This also fixes a potential, but currently unexisting problem: If the
raw packet buffer was initially not empty and probe_codec() failed,
then the packet returned would not be initialized. But given that
probe_codec() currently can't fail (always returns 0) this was not an
acute danger.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

doc/filters: add more advanced silenceremove example

avfilter/fifo: use the name 's' for the pointer to the private context

This is shorter and consistent across filters.

avfilter/fifo: cosmetics

avfilter/vf_framerate: remove duplicate code with macro-based function

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>

avfilter/vf_framerate: refine the filter_slice code for better readiablity

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>

avfilter/vf_framerate: limit the scene level max range

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>

avcodec/motionpixels: Mark 2 functions as always_inline

Fixes: Timeout (30sec -> 25sec)
Fixes: 17050/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MOTIONPIXELS_fuzzer-5719149803732992

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/ituh263dec: Make the condition for the studio slice start code match between ff_h263_resync() and ff_mpeg4_decode_studio_slice_header()

If they mismatch an infinite loop can occur
Fixes: Timeout (infinite loop)
Fixes: 17043/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5695051748868096

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/ralf: Fix integer overflow in decode_channel()

Fixes: signed integer overflow: -1094995519 * 64 cannot be represented in type 'int'
Fixes: 17030/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_RALF_fuzzer-5640695838146560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

vcodec/vc1: compute rangex/y only for P/B frames

Fixes: left shift of 1073741824 by 1 places cannot be represented in type 'int'
Fixes: 16976/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1_fuzzer-4847262047404032

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/vc1_pred: Fix invalid shifts in scaleforopp()

Fixes: left shift of negative value -2
Fixes: 16964/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC1IMAGE_fuzzer-5757853565976576

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/vc1_block: Fix invalid shift with rangeredfrm

Fixes: left shift of negative value -7
Fixes: 16959/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMV3_fuzzer-5200360825683968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/vc1: Check for excessive resolution

Fixes: overflow in aspect ratio calculation
Fixes: signed integer overflow: 393215 * 14594 cannot be represented in type 'int'
Fixes: 15728/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMV3IMAGE_fuzzer-5661588893204480

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/vc1: check REFDIST

"9.1.1.43 P Reference Distance (REFDIST)"
"The value of REFDIST shall be less than, or equal to, 16."

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/apedec: Allocate decoded_buffer after successful ff_get_buffer()

We need to reset samples in this case to avoid being stuck with incorrect
samples value.

Fixes: OOM
Fixes: Timeout
Fixes: 16627/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5638059583864832
Fixes: 17089/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5672188463546368

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/apedec: Fix several integer overflows in predictor_update_filter() and do_apply_filter()

Fixes: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: signed integer overflow: -14527961 - 2147483425 cannot be represented in type 'int'
Fixes: 16380/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5645957131141120
Fixes: 16968/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5716169901735936
Fixes: 17074/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5198710497083392

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/hevc_cabac: Tighten the limit on k in ff_hevc_cu_qp_delta_abs()

Values larger would fail subsequent tests.

Fixes: signed integer overflow: 5 + 2147483646 cannot be represented in type 'int'
Fixes: 16966/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5695709549953024

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avfilter/vf_xbr: Fix left shift of negative number

Affected every usage of vf_xbr, e.g. the FATE-tests filter-2xbr,
filter-3xbr, filter-4xbr.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avfilter/vf_hqx: Fix undefined left shifts of negative numbers

Affected every usage of this filter; in particular, it affected the
FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/jpeg2000dwt: Fix undefined shifts of negative numbers

Affected the vsynth*-jpeg2000 and the vsynth*-jpeg2000-97 FATE tests
(where * ranges over { 1, 2, 3, _lena }) as well as ticket #7983.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/ituh263dec: Fix undefined left shift of negative number

Fixes ticket #8160.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/dnxhdenc: Fix undefined left shifts of negative numbers

Affected 61 FATE-tests: 60 vsynth tests and lavf-mxf_opatom.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

swscale/utils: Fix invalid left shifts of negative numbers

Affected the FATE-tests vsynth_lena-dv-411, vsynth1-dv-411,
vsynth2-dv-411 and hevc-paramchange-yuv420p.yuv420p10.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/ffv1enc: Fix out-of-bounds-array access

libavcodec/ffv1enc.c accessed an array of uint8_t [32] via array[0][j]
in order to loop over all the uint8_t in this array of arrays. Of course
this implied an out-of-bounds access for array[0] and UBSan complained
about this. So replace this with nested loops; furthermore, factor this
out into a function of its own to easily break out of the nested loops.

This affected the FATE-tests vsynth1-ffv1, vsynth1-ffv1-v3-yuv420p,
vsynth1-ffv1-v3-yuv422p10, vsynth1-ffv1-v3-yuv444p16,
vsynth1-ffv1-v3-bgr0, vsynth1-ffv1-ffv1-v3-rgb48 as well as the
corresponding vsynth2-*, vsynth3-* and the vsynth_lena-* tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

swscale/x86/swscale: Fix undefined left shifts of negative numbers

This affected many FATE-tests: The number of failing tests went down
from 663 to 344. (Both numbers exclude tests that failed because of
unaligned accesses in code that is inside #if HAVE_FAST_UNALIGNED.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

ffmpeg_opt: remove errant space

Signed-off-by: Lou Logan <lou@lrcd.com>

doc/ffmpeg: -timelimit is in user time

Signed-off-by: Lou Logan <lou@lrcd.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/nvenc: make use of new GetLastErrorString function

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>

avcodec/nvenc: pass CUstream to nvenc when available

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>

avcodec/nvenc: add multiple reference frames support

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>

avcodec/nvenc: switch to dedicated dpb_size option

Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>

libavformat/rtsp: return error if rtsp_hd_out is null instead of crash

Signed-off-by: Aman Gupta <aman@tmm1.net>

lavu/qsv: remove redundant version query

Signed-off-by: Zhong Li <zhong.li@intel.com>

lavc/mpeg4audio: add chan_config check to avoid indeterminate channels

add chan_config check to avoid indeterminate channels.

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
Signed-off-by: James Almer <jamrial@gmail.com>

avcodec/mpeg4audio: fix doxy for ff_mpeg4audio_get_config_gb()

Signed-off-by: James Almer <jamrial@gmail.com>

avformat: use avpriv_mpeg4audio_get_config2()

Signed-off-by: James Almer <jamrial@gmail.com>

avcodec: use avpriv_mpeg4audio_get_config2()

Signed-off-by: James Almer <jamrial@gmail.com>

avcodec/mpeg4audio: add avpriv_mpeg4audio_get_config2()

Identical to avpriv_mpeg4audio_get_config() except taking a size argument in
bytes, and featuring a new logging context paremeter.

Schedule avpriv_mpeg4audio_get_config() for removal as soon as major is bumped
as well.

Signed-off-by: James Almer <jamrial@gmail.com>

avformat/dhav: fix demuxer since recent breakage

avformat/g729dec: set packet duration and correctly set timebase info

Fixes #6704.

avformat/nut: add pcm_s64 support

swscale/swscale: cosmetics

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avfilter/vf_scale: cosmetics

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/4xm: Check index in decode_i_block() also in the path where its not used.

Fixes: Infinite loop
Fixes: signed integer overflow: 2147483644 + 16 cannot be represented in type 'int'
Fixes: 16169/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5662570416963584
Fixes: 16782/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5743163859271680
Fixes: 17641/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FOURXM_fuzzer-5711603562971136

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/loco: Check for end of input in the first line

Fixes: Timeout (85sec -> 0.1sec)
Fixes: 17634/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LOCO_fuzzer-5666410809786368

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/atrac3: Check block_align

Fixes: Infinite loop
Fixes: 17620/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC3_fuzzer-5086123012915200

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

tools/target_dec_fuzzer: Print samples decoded like pixels

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

tools/target_dec_fuzzer: Check number of all samples decoded too, like max pixels

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/alsdec: Avoid dereferencing context pointer in inner interleave loop

This makes the decoder faster

Improves/Fixes: Timeout (22sec -> 20sec)
Testcase: 17619/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ALS_fuzzer-5078510820917248

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/hcom: Check that there are dictionary entries

Fixes: out of array read
Fixes: 17617/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HCOM_fuzzer-5674970478280704

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/fitsdec: Prevent division by 0 with huge data_max

Fixes: division by 0
Fixes: 15657/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5738154838982656

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/dstdec: Fix integer overflow in samples_per_frame computation

Fixes: Timeout (? -> 2ms)
Fixes: 17616/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DST_fuzzer-5198057947267072

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/g729_parser: Check block_size

Fixes: Infinite loop
Fixes: 17611/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ACELP_KELVIN_fuzzer-5765134928052224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/sbcdec: Initialize number of channels

Fixes: out of array access
Fixes: 17609/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SBC_fuzzer-5758729319874560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

avcodec/exr: Fix undefined left shifts of negative numbers

Affected the FATE-tests exr-rgb-scanline-pxr24-half-uint32-13x9 and
exr-rgb-scanline-pxr24-uint32.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

aformat/movenc: add missing padding to output track extradata

Fixes ticket #8183.

Tested-by: Thierry Foucu <tfoucu@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

doc/filters: warn about gaps/overlaps in xstack

doc/filters: add 4x4 layout example for xstack filter

Add layouts for existing examples.

Reviewed-by: Gyan <ffmpeg@gyani.pro>
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>

avfilter/vf_eq: fix compilation with x86 asm disabled

Signed-off-by: James Almer <jamrial@gmail.com>

fate: update hashes after ya16 change

swresample/audioconvert: fix invalid left shift for 64bit sample format

Fixes #8002.

avcodec/hevc_ps: Remove dead code in vps_id check

Since reading 4 bits always returns a value in the range [0, 15], the
check for vps_id >= HEVC_MAX_VPS_COUNT, where HEVC_MAX_VPS_COUNT = 16, is redundant.

Signed-off-by: Andriy Gelman <andriy.gelman@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

avcodec/truespeech: fix left shift of negative value

swscale/output: fix signed integer overflow for ya16

Fixes #7666.

avformat/movenc: Fix undefined shift

Fixes the movenc FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avcodec/mpeg12dec: Sanitize start codes earlier

The MPEG-1/2 decoder uses avpriv_find_start_code to search for start
codes and worked with the resulting start code before checking that it
is really a start code of a slice. In particular, if the picture is so
big that a slice_vertical_position_extension is present, it added the
slice_vertical_position_extension as if it had a slice. Then a left
shift is performed, without making sure that the value to be shifted is
nonnegative.
Afterwards the end result is checked, but even if a start code of a
non-slice has been found, it might pass these checks: If
slice_vertical_position_extension is present a start code <
SLICE_MIN_START_CODE can lead to a macroblock-row index that appears
valid. Furthermore, the left shift might make an invalid start code
appear valid by discarding the highest bit.
This has been fixed by checking directly after avpriv_find_start_code
has returned.

Fixes ticket #8162 (which is about the undefined left shifts).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avcodec/pcm: Cosmetics

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avcodec/pcm: Fix undefined shifts

Fixes the acodec-pcm-u16[lb]e FATE-tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

avcodec/wavpackenc: Fix undefined shifts

Fixes ticket #8161 and the acodec-wavpack FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

lavc/qsv: add memory type message

Signed-off-by: Zhong Li <zhong.li@intel.com>

lavc/qsv: Fix MSDK initialization failure in system memory mode

MSDK does not create internal acceleration device on Linux,
So MFXVideoCORE_SetHandle() is necessary.
It has been added for ff_qsv_init_session_device().
But missed for ff_qsv_init_internal_session() due to commit
1f26a23 overwrited commit db89f45

Fix #7030

Signed-off-by: Zhong Li <zhong.li@intel.com>

avformat/utils: ensure that all packets in AVPacketList are reference counted

This is done so that its data is really owned by the packet.
This was already true for the current callers.

Signed-off-by: James Almer <jamrial@gmail.com>

avfilter/x86/vf_eq: add SSE2 version

Signed-off-by: Ting Fu <ting.fu@intel.com>

avfilter/x86/vf_eq: Change inline assembly into nasm code

Signed-off-by: Ting Fu <ting.fu@intel.com>

checkasm/vf_eq: add test for vf_eq

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Ruiling Song <ruiling.song@intel.com>

avformat/utils: Move the reference to the packet list

Up until now, ff_packet_list_put had a flaw: When it moved a packet to
the list (meaning, when it ought to move the reference to the packet
list instead of creating a new one via av_packet_ref), it did not reset
the original packet, confusing the ownership of the data in the packet.
This has been done because some callers of this function were not
compatible with resetting the packet.

This commit changes these callers and fixes this flaw. In order to
indicate that the ownership of the packet has moved to the packet list,
pointers to constant AVPackets are used whenever the target of the
pointer might already be owned by the packet list.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

avformat/utils: unref packet on AVInputFormat.read_packet() failure

Demuxers may have allocated a packet before encountering an error and aborting.

Fixes ticket #8150

Signed-off-by: James Almer <jamrial@gmail.com>

avformat/utils: Fix memleaks

ff_read_packet had potential memleaks:
1. If av_packet_make_refcounted fails, it means that the packet is not
refcounted, but it could nevertheless carry side data and therefore
needs to be unreferenced.
2. If putting a packet on a packet list fails, it wasn't unreferenced.

Furthermore, read_frame_internal leaked a packet's (side) data if a
context update was required and failed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

avformat/utils: Assert that stream_index is valid

There is currently an ordinary check for this (which would lead to a
memleak), but given that no demuxer should ever return a packet with an
invalid stream_index it is more appropriate for this to be an assert.

FATE passes with this change.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

lavf/mov: Fix timestamp rescale on sidx atom

Fix #5090
Fix the timestamp rescale issue, from sidx timebase to
stream's timebase.

Revert "lavc/tiff: correct the default value of YCbCrSubsampling to 2x2"

This reverts commit eb5d0f18ff609ba2280cea4e2c6286d216c8756b.

Together with 89f464e9, it breaks decoding of tiff files like fate-suite/exif/image_small.tiff

avcodec/dnxhdenc: return error if av_malloc failed

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>

avfilter/vf_zoompan: fix leaks

avfilter/vf_zoompan: forward status back from output to input

avfilter/vf_stack: add slice threading

avfilter/vf_stack: simplify main processing path

avformat/sdsdec: fix undefined behaviour

Fixes #8163, #8164, #8165.

avformat/aiffdec: fix signed integer overflow

Fixes #8151