Audio is longer than original audio when using write_video

[MKlmt]

🐛 Describe the bug

Hi, I think I found a bug in torchvision's write_video.
When reading a video (with audio) and writing it again without any modifications, the number of audio frames differs by 1024 frames between the original video(/audio) and the newly saved video(/audio).
Using torchaudio's save and load function works fine.
This is my minimal working example:

from torchvision.io import read_video, write_video
import torchaudio

vid, audio, info = read_video("200-4090.mp4", pts_unit="sec")
print("Original video shape:", vid.shape)
print("Original audio shape:", audio.shape)
write_video(
    "tmp.mp4",
    vid,
    30,
    audio_array=audio,
    audio_fps=48000,
    video_codec="libx264",
    audio_codec="aac",
)

vid, audio_vid, _ = read_video("tmp.mp4", pts_unit="sec")
print("saving with torchvision:", vid.shape)
print("saving with torchvision:", audio_vid.shape)

torchaudio.save("tmp.wav", audio, 48000, channels_first=True)
only_audio, t = torchaudio.load("tmp.wav")
print("saving with torchaudio:", only_audio.shape)

I checked the fps and audio_fps numbers. They are correct for my video.
I tested it for different mp4 files with the same results.

In addition, I found out that the written video is always stereo (2 audio channels) even though my original audio is mono (1 audio channel) in my case.

I hope someone can help me with that, Thank you.

Versions

Collecting environment information...
PyTorch version: 2.2.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: Could not collect
CMake version: version 3.22.4
Libc version: glibc-2.31

Python version: 3.9.18 (main, Sep 11 2023, 13:41:44) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-4.18.0-513.5.1.el8_9.x86_64-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 11.3.109
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA A100-SXM4-80GB
GPU 1: NVIDIA A100-SXM4-80GB
GPU 2: NVIDIA A100-SXM4-80GB
GPU 3: NVIDIA A100-SXM4-80GB

Nvidia driver version: 555.42.06
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 48 bits physical, 48 bits virtual
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 1
Core(s) per socket: 16
Socket(s): 2
NUMA node(s): 8
Vendor ID: AuthenticAMD
CPU family: 25
Model: 1
Model name: AMD EPYC 7313 16-Core Processor
Stepping: 1
CPU MHz: 2994.359
BogoMIPS: 5988.71
Virtualization: AMD-V
L1d cache: 1 MiB
L1i cache: 1 MiB
L2 cache: 16 MiB
L3 cache: 256 MiB
NUMA node0 CPU(s): 0-3
NUMA node1 CPU(s): 4-7
NUMA node2 CPU(s): 8-11
NUMA node3 CPU(s): 12-15
NUMA node4 CPU(s): 16-19
NUMA node5 CPU(s): 20-23
NUMA node6 CPU(s): 24-27
NUMA node7 CPU(s): 28-31
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd amd_ppin brs arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm

Versions of relevant libraries:
[pip3] facenet-pytorch==2.6.0
[pip3] numpy==1.26.4
[pip3] pytorch-lightning==1.9.0
[pip3] pytorchvideo==0.1.5
[pip3] torch==2.2.0
[pip3] torch-tb-profiler==0.4.1
[pip3] torchaudio==2.2.0
[pip3] torchinfo==1.8.0
[pip3] torchmetrics==1.4.0.post0
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.17.2
[pip3] triton==2.2.0
[conda] numpy 1.26.2 pypi_0 pypi
[conda] torch 1.12.0+cu113 pypi_0 pypi
[conda] torch-tb-profiler 0.4.1 pypi_0 pypi
[conda] torchaudio 0.12.0+cu113 pypi_0 pypi
[conda] torchvision 0.13.0+cu113 pypi_0 pypi

[NicolasHug]

Thanks for the report @MKlmt . We'll be consolidating the video decoding/encoding efforts of pytorch within the https://github.com/pytorch/torchcodec/ repo. Torchcodec doesn't support writing videos yet, but that's in scope. That means we won't be updating the video capabilities of torchvision - hopefully you can still use the torchaudio API in the mean time for the correct behavior

[MKlmt]

Alright. Thank you! Looking forward to Torchcodec video writing function :)