Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[youtube] Support --download-sections with formats --live-from-start #6498

Open
wants to merge 70 commits into
base: master
Choose a base branch
from
Open

[youtube] Support --download-sections with formats --live-from-start #6498

wants to merge 70 commits into from
+111 −34

Conversation

elyse0
Copy link
Contributor

@elyse0 elyse0 commented Mar 10, 2023 •
edited
Loading

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

Like #5822, this PR adds support for downloading a specific section of a YT livestream format with --live-from-start, but with a different approach.

The Dash parser now extracts the start and end timestamps of each fragment and the fragment_generator code uses them to filter based on the selected range. A big advantage over #5822 is that these changes could allow support for --download-sections on any Dash livestream, if/when that downloader is implemented.

Section selector

The current approach for specifying a range is using a new # syntax and negative durations, where these durations are relative to "now".

--download-sections "#-2hours - -1hour"  // Start downloading from 2 hours ago and stop until 1 hour ago
--download-sections "#-1hour - 0" // Start downloading from 1 hour ago and stop until now
--download-sections "#-1hour - 1hour" // Start downloading from 1 hour ago and stop until 1 hour from now

Fixes #3451

yt-dlp "https://www.youtube.com/watch?v=w_DfTc7F5oQ" --verbose --live-from-start -f '140+134' --download-sections "#-15minutes - -5minutes" -o venice
[debug] Command-line config: ['https://www.youtube.com/watch?v=w_DfTc7F5oQ', '--verbose', '--live-from-start', '-f', '140+134', '--download-sections', '#-15minutes - -5minutes', '-o', 'venice']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] [392389b7d] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: 4e93198ae
[debug] Python 3.10.7 (CPython x86_64 64bit) - Linux-5.19.0-41-generic-x86_64-with-glibc2.36 (OpenSSL 3.0.5 5 Jul 2022, glibc 2.36)
[debug] exe versions: ffmpeg N-108931-g4dda3b1653-20221104 (setts), ffprobe N-108931-g4dda3b1653-20221104, phantomjs broken, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Extractor Plugins: ElyseNebulaChannelIE, ElyseNebulaIE, ElyseNebulaSubscriptionsIE, OnlineRadioBoxIE
[debug] Plugin directories: ['/home/amish/.yt-dlp/plugins/elyse0/yt_dlp_plugins']
[debug] Loaded 1790 extractors
[youtube] Extracting URL: https://www.youtube.com/watch?v=w_DfTc7F5oQ
[youtube] w_DfTc7F5oQ: Downloading webpage
[youtube] w_DfTc7F5oQ: Downloading android player API JSON
[youtube] w_DfTc7F5oQ: Downloading MPD manifest
[youtube] w_DfTc7F5oQ: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[info] w_DfTc7F5oQ: Downloading 1 format(s): 140+134
[info] w_DfTc7F5oQ: Downloading 1 time ranges: 1683437142.1-1683437742.1
[debug] Invoking dashsegments downloader on "https://manifest.googlevideo.com/api/manifest/dash/expire/1683459643/ei/2zlXZPjBC_q_lu8PuYuUiAk/ip/xxx/id/w_DfTc7F5oQ.1/source/yt_live_broadcast/requiressl/yes/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/qEK7B547rhoxXCE0iglM36M7YD7ipy8/vprv/1/pacing/0/itag_bl/376%2C377%2C384%2C385%2C612%2C613%2C617%2C619%2C623%2C628%2C655%2C656%2C660%2C662%2C666%2C671/keepalive/yes/fexp/24007246/beids/24512781/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AOq0QJ8wRgIhAOCkqQCbdOaAIMN4LCSjBVodFi1xP6kQFxMWOTjxSlreAiEA9xqSRAoPw0nuZ1K0ZpcEsO-7RCt3VrKZ1DLrpXq_F3A%3D", "https://manifest.googlevideo.com/api/manifest/dash/expire/1683459643/ei/2zlXZPjBC_q_lu8PuYuUiAk/ip/xxx/id/w_DfTc7F5oQ.1/source/yt_live_broadcast/requiressl/yes/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/qEK7B547rhoxXCE0iglM36M7YD7ipy8/vprv/1/pacing/0/itag_bl/376%2C377%2C384%2C385%2C612%2C613%2C617%2C619%2C623%2C628%2C655%2C656%2C660%2C662%2C666%2C671/keepalive/yes/fexp/24007246/beids/24512781/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AOq0QJ8wRgIhAOCkqQCbdOaAIMN4LCSjBVodFi1xP6kQFxMWOTjxSlreAiEA9xqSRAoPw0nuZ1K0ZpcEsO-7RCt3VrKZ1DLrpXq_F3A%3D"
[dashsegments] Total fragments: unknown (live)
[download] Destination: venice.f140.mp4
[dashsegments] Total fragments: unknown (live)
[download] Destination: venice.f134.mp4
[debug] [youtube] Selected section: 1683437142.123764 -> 1683437742.123764
WARNING: [youtube] Starting download from the last 120 hours of the live stream since YouTube does not have data before that. If you think this is wrong, please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[debug] [youtube] [w_DfTc7F5oQ] Generating fragments for format 140
[debug] [youtube] Selected section: 1683437142.123764 -> 1683437742.123764
[debug] [youtube] [w_DfTc7F5oQ] Generating fragments for format 134
[download] 100% of   12.17MiB in 00:00:57 at 216.43KiB/s
[download] 100% of   33.73MiB in 00:01:13 at 468.67KiB/s
[Merger] Merging formats into "venice.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i file:venice.f140.mp4 -i file:venice.f134.mp4 -c copy -map 0:a:0 -map 1:v:0 -movflags +faststart file:venice.temp.mp4
Deleting original file venice.f134.mp4 (pass -k to keep)
Deleting original file venice.f140.mp4 (pass -k to keep)
Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

@pukkandan
Copy link
Member

pukkandan commented Mar 10, 2023 •
edited
Loading

The approach looks good. There are a few part that I don't understand, but I'll do a full review when this is undrafted

PS: It would be great if someone can improve --download-sections, because I started using .* just to get it working but it fails in a lot of cases, like *(now-1hour)-(now-45minutes)

We had the plan to support negative values as timestamp from end of stream. So, this would be *-1h--45m. This is trivial to add support for by modifying download_range_func and the validation in __init__.py. If you can implement in this PR, go ahead. Otherwise I'll add it in before merging. I don't particularly like the -- that appears in this syntax, but not much we can do about it. Supporting now-x-now-y would create ambiguity and supporting brackets properly is non-trivial.

coletdjnz
coletdjnz reviewed Mar 11, 2023
View reviewed changes
yt_dlp/__init__.py Outdated Show resolved Hide resolved
D0LLYNH0 and others added 18 commits March 11, 2023 19:51
@D0LLYNH0 @elyse0
[extractor/iq] Set more language codes (yt-dlp#6476)
e3b08ba 
Authored by: D0LLYNH0
@C0D3D3V @elyse0
[extractor/opencast] Add ltitools to _VALID_URL (yt-dlp#6371)
8df4707 
Authored by: C0D3D3V
@pukkandan @elyse0
[downloader/curl] Fix progress reporting
c376b95 
Bug in 8c53322
Closes yt-dlp#6490
@pukkandan @elyse0
[extractor/youtube] Bypass throttling for -f17
2d16554 
and related cleanup

Thanks @AudricV for the finding
@elyse0
[extractor/twitch] Fix is_live (yt-dlp#6500)
c222f6c 
Closes yt-dlp#6494
Authored by: elyse0
@makew0rld @elyse0
[extractor/cbc:gem] Update _VALID_URL (yt-dlp#6499)
96f5d29 
Authored by: makeworld-the-better-one
Closes yt-dlp#6395
@pukkandan @elyse0
Support loading info.json with a list at it's root
34d3df7
@pukkandan @elyse0
[extractor/hidive] Fix login
9f717b6 
Fixes yt-dlp#6493 (comment)
@C0D3D3V @elyse0
[extractor/opencast] Fix format bug (yt-dlp#6512)
e3ffdf7 
Authored by: C0D3D3V
@vampirefrog @elyse0
[extractor/rokfin] Re-construct manifest url (yt-dlp#6507)
f137666 
Authored by: vampirefrog
@Lesmiscore @elyse0
[extractor/youtube] Add client name to format_note when -v (yt-dl…
db62ffd 
…p#6254)

Authored by: Lesmiscore, pukkandan
@pukkandan @elyse0
[extractor/youtube] Add extractor-arg include_duplicate_formats
5ef1a92
@pukkandan @elyse0
[extractor/youtube] Construct fragment list lazily
9fc70f3 
Building fragment list for all formats take significant time for large videos
@elyse0
Support negative durations
e6e2eb0
@elyse0
Revert "[utils] Allow using local timezone for 'now' timestamps"
e40132d 
This reverts commit 1799a6a.
@elyse0
Add fragment count
0ed9a73
@elyse0
Fix unified_timestamp
a43ba2e
@elyse0
Remove tz_aware date code
cdac764
@bashonly
Copy link
Member

@JayXon Try pulling / reinstalling the branch. It's working OK for me after resolving the merge conflict. Double check that you are using the correct syntax; it is different than the regular --download-sections syntax

@JayXon
Copy link
Contributor

JayXon commented Mar 1, 2024

@JayXon Try pulling / reinstalling the branch. It's working OK for me after resolving the merge conflict. Double check that you are using the correct syntax; it is different than the regular --download-sections syntax

same error after pulling, I'm not using --download-sections, because I'm trying to download a clip of a livestream, so I was expecting yt-dlp to get the section from the clip url automatically.
I did see this in the log, but maybe it calculated the section wrong?

[debug] [youtube] Selected section: 523806.302 -> 523821.98500000004

@bashonly
Copy link
Member

bashonly commented Mar 1, 2024

@JayXon I have no idea what's going on if you don't provide the URL and a log

@pukkandan pukkandan mentioned this pull request Mar 26, 2024
Closed
11 tasks
@Kaiman-Doro
Copy link

The implementation does have an issue where sometimes the download will hang on the last fragments and never complete: #8450 (comment)

I keep running into this issue and seem to have found a consistent case where it happens. It seems to happen when the stream has ultra low latency live mode enabled, uses AVC1, and has DVR disabled.

Successful instances (DVR disabled on all):
image
image
image

Failed instances where it froze (DVR disabled on all):
image
image

AVC1 and Ultra Low Latency were used with this stream, but DVR was enabled. It was successful here.
image

Verbose of my executions. The second and last ran are the ones that failed and had AVC1, ULL mode, and DVR disabled.

.\yt-dlp.exe https://www.youtube.com/watch?v=xV97vkCojWI --live-from-start --download-sections '#-10seconds - 0' -o test2.mp4 --verbose
[debug] Command-line config: ['https://www.youtube.com/watch?v=xV97vkCojWI', '--live-from-start', '--download-sections', '#-10seconds - 0', '-o', 'test2.mp4', '--verbose']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ff0779267] (win_exe)
[debug] Python 3.11.9 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 3.0.13 30 Jan 2024)
[debug] exe versions: ffmpeg 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev (setts), ffprobe 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[youtube] Extracting URL: https://www.youtube.com/watch?v=xV97vkCojWI
[youtube] xV97vkCojWI: Downloading webpage
[youtube] xV97vkCojWI: Downloading ios player API JSON
[youtube] xV97vkCojWI: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "xV97vkCojWI")
[youtube] xV97vkCojWI: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] xV97vkCojWI: Downloading 1 format(s): 299+140
[info] xV97vkCojWI: Downloading 1 time ranges: 1714186619.0-1714186629.0
[debug] Invoking dashsegments downloader on "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208229/ei/hWksZqOiL5WAkucPn8uL0Ak/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/xV97vkCojWI.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/tx/51066118/txs/51066116%2C51066117%2C51066118%2C51066119%2C51066120%2C51066121/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9fxTmZ2LGZ4cpiWAeM1T5impU1nOsEePjW9jMhl7hm7Mq_Cgr_kbAinzw/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Ctx%2Ctxs%2Chfr%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRAIgFNnF5UdlVpSd36X65YnE6qU_yl8eWykMUukDioEEVusCIE-Z7uADOHGcrfECCcyBEWg2SA5Pn6WaVM6nJXDXvJ45", "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208229/ei/hWksZqOiL5WAkucPn8uL0Ak/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/xV97vkCojWI.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/tx/51066118/txs/51066116%2C51066117%2C51066118%2C51066119%2C51066120%2C51066121/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9fxTmZ2LGZ4cpiWAeM1T5impU1nOsEePjW9jMhl7hm7Mq_Cgr_kbAinzw/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Ctx%2Ctxs%2Chfr%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRAIgFNnF5UdlVpSd36X65YnE6qU_yl8eWykMUukDioEEVusCIE-Z7uADOHGcrfECCcyBEWg2SA5Pn6WaVM6nJXDXvJ45"
[dashsegments] Total fragments: unknown (live)
[download] Destination: test2.f299.mp4
[debug] File locking is not supported. Proceeding without locking
[dashsegments] Total fragments: unknown (live)
[download] Destination: test2.f140.mp4
[debug] [youtube] Selected section: 1714186619.0477934 -> 1714186629.0477934
[debug] [youtube] [xV97vkCojWI] Generating fragments for format 299
[debug] [youtube] Selected section: 1714186619.0477934 -> 1714186629.0477934
[debug] [youtube] [xV97vkCojWI] Generating fragments for format 140
[download] 100% of    4.42MiB in 00:00:10 at 443.04KiB/s
[download] 100% of  169.36KiB in 00:00:10 at 16.42KiB/s
[Merger] Merging formats into "test2.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i file:test2.f299.mp4 -i file:test2.f140.mp4 -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart file:test2.temp.mp4
Deleting original file test2.f299.mp4 (pass -k to keep)
Deleting original file test2.f140.mp4 (pass -k to keep)


.\yt-dlp.exe https://www.youtube.com/watch?v=q8IOZ2Uk8z4 --live-from-start --download-sections '#-10seconds - 0' -o test.mp4 --verbose
[debug] Command-line config: ['https://www.youtube.com/watch?v=q8IOZ2Uk8z4', '--live-from-start', '--download-sections', '#-10seconds - 0', '-o', 'test.mp4', '--verbose']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ff0779267] (win_exe)
[debug] Python 3.11.9 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 3.0.13 30 Jan 2024)
[debug] exe versions: ffmpeg 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev (setts), ffprobe 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[youtube] Extracting URL: https://www.youtube.com/watch?v=q8IOZ2Uk8z4
[youtube] q8IOZ2Uk8z4: Downloading webpage
[youtube] q8IOZ2Uk8z4: Downloading ios player API JSON
[youtube] q8IOZ2Uk8z4: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "q8IOZ2Uk8z4")
[youtube] q8IOZ2Uk8z4: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] q8IOZ2Uk8z4: Downloading 1 format(s): 299+140
[info] q8IOZ2Uk8z4: Downloading 1 time ranges: 1714186677.4-1714186687.4
[debug] Invoking dashsegments downloader on "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208288/ei/wGksZr6IB6SrkucP0Y-bwAI/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/q8IOZ2Uk8z4.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/tx/51118564/txs/51118561%2C51118562%2C51118563%2C51118564%2C51118565%2C51118566%2C51118567/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9f55WfeRVZYPdxY_U3zgPwdJ_fnJ5ymhi62HrFfb2IwvCFkI36Lyyk1YC/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/LIVE/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Ctx%2Ctxs%2Chfr%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRQIgHU_uvCXqnS07TFIiFpyko0nTGJE7Pt8J4Se1Bko6WmcCIQCYcreiqAuYOrx1cxupkRvVwWT84h5aVoY6lk7KYxPScg%3D%3D", "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208288/ei/wGksZr6IB6SrkucP0Y-bwAI/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/q8IOZ2Uk8z4.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/tx/51118564/txs/51118561%2C51118562%2C51118563%2C51118564%2C51118565%2C51118566%2C51118567/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9f55WfeRVZYPdxY_U3zgPwdJ_fnJ5ymhi62HrFfb2IwvCFkI36Lyyk1YC/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/LIVE/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Ctx%2Ctxs%2Chfr%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRQIgHU_uvCXqnS07TFIiFpyko0nTGJE7Pt8J4Se1Bko6WmcCIQCYcreiqAuYOrx1cxupkRvVwWT84h5aVoY6lk7KYxPScg%3D%3D"
[dashsegments] Total fragments: unknown (live)
[download] Destination: test.f299.mp4
[debug] File locking is not supported. Proceeding without locking
[dashsegments] Total fragments: unknown (live)
[download] Destination: test.f140.mp4
[debug] [youtube] Selected section: 1714186677.4397655 -> 1714186687.4397655
[debug] [youtube] [q8IOZ2Uk8z4] Generating fragments for format 299
[debug] [youtube] Selected section: 1714186677.4397655 -> 1714186687.4397655
[debug] [youtube] [q8IOZ2Uk8z4] Generating fragments for format 140
[download]    6.30MiB at    2.41MiB/s (00:00:05) (frag 9/10)
[download]  190.61KiB at      0.00B/s (00:00:05) (frag 9/10)


.\yt-dlp.exe https://www.youtube.com/watch?v=TWVn10mnXzY --live-from-start --download-sections '#-10seconds - 0' -o test.mp4 --verbose
[debug] Command-line config: ['https://www.youtube.com/watch?v=TWVn10mnXzY', '--live-from-start', '--download-sections', '#-10seconds - 0', '-o', 'test.mp4', '--verbose']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ff0779267] (win_exe)
[debug] Python 3.11.9 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 3.0.13 30 Jan 2024)
[debug] exe versions: ffmpeg 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev (setts), ffprobe 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[youtube] Extracting URL: https://www.youtube.com/watch?v=TWVn10mnXzY
[youtube] TWVn10mnXzY: Downloading webpage
[youtube] TWVn10mnXzY: Downloading ios player API JSON
[youtube] TWVn10mnXzY: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "TWVn10mnXzY")
[youtube] TWVn10mnXzY: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] TWVn10mnXzY: Downloading 1 format(s): 299+140
[info] TWVn10mnXzY: Downloading 1 time ranges: 1714186771.7-1714186781.7
[debug] Invoking dashsegments downloader on "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208382/ei/HmosZtuPFICKkucP55a-sAE/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/TWVn10mnXzY.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9f4PssWGuPgb4O_Fa8lkZ6a9DP4cSPHhstEDlsEd1g_Fyjs-zDNqwECph/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/LIVE/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Chfr%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRgIhAIVpupLzaur-6pJYavF0ha5ImXCBdspEPhkDpzXXM7x3AiEA9nKj77sa9ytPZ7cxE0k7vq8dJjkQOi0DBl9jJVwEPAE%3D", "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208382/ei/HmosZtuPFICKkucP55a-sAE/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/TWVn10mnXzY.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9f4PssWGuPgb4O_Fa8lkZ6a9DP4cSPHhstEDlsEd1g_Fyjs-zDNqwECph/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/LIVE/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Chfr%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRgIhAIVpupLzaur-6pJYavF0ha5ImXCBdspEPhkDpzXXM7x3AiEA9nKj77sa9ytPZ7cxE0k7vq8dJjkQOi0DBl9jJVwEPAE%3D"
[dashsegments] Total fragments: unknown (live)
[download] Destination: test.f299.mp4
[debug] File locking is not supported. Proceeding without locking
[dashsegments] Total fragments: unknown (live)
[download] Destination: test.f140.mp4
[debug] [youtube] Selected section: 1714186771.6965144 -> 1714186781.6965144
[debug] [youtube] [TWVn10mnXzY] Generating fragments for format 299
[debug] [youtube] Selected section: 1714186771.6965144 -> 1714186781.6965144
[debug] [youtube] [TWVn10mnXzY] Generating fragments for format 140
[download] 100% of    6.30MiB in 00:00:05 at 1.24MiB/s
[download] 100% of  190.61KiB in 00:00:05 at 37.57KiB/s
[Merger] Merging formats into "test.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i file:test.f299.mp4 -i file:test.f140.mp4 -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart file:test.temp.mp4
Deleting original file test.f140.mp4 (pass -k to keep)
Deleting original file test.f299.mp4 (pass -k to keep)


.\yt-dlp.exe https://www.youtube.com/watch?v=IklGR6rUqKE --live-from-start --download-sections '#-10seconds - 0' -o test.mp4 --verbose
[debug] Command-line config: ['https://www.youtube.com/watch?v=IklGR6rUqKE', '--live-from-start', '--download-sections', '#-10seconds - 0', '-o', 'test.mp4', '--verbose']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ff0779267] (win_exe)
[debug] Python 3.11.9 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 3.0.13 30 Jan 2024)
[debug] exe versions: ffmpeg 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev (setts), ffprobe 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[youtube] Extracting URL: https://www.youtube.com/watch?v=IklGR6rUqKE
[youtube] IklGR6rUqKE: Downloading webpage
[youtube] IklGR6rUqKE: Downloading ios player API JSON
[youtube] IklGR6rUqKE: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "IklGR6rUqKE")
[youtube] IklGR6rUqKE: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] IklGR6rUqKE: Downloading 1 format(s): 247+140
[info] IklGR6rUqKE: Downloading 1 time ranges: 1714186916.8-1714186926.8
[debug] Invoking dashsegments downloader on "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208527/ei/r2osZt-hG6GBkucPibWV-AE/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/IklGR6rUqKE.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9fx3JDRFPmI3O4uZyFhEz_ER4_Qo3DvvJIQo4gE5Q7cdhyGJG13IQcPEP/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRAIgK138jHQQ5zDUX3jApJQHrXy9K19KaGIPPeLZM49XDroCIBxvwa3yMX6DmF7tPclenRcgjDN6L9lIzu5WVbYV85-j", "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208527/ei/r2osZt-hG6GBkucPibWV-AE/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/IklGR6rUqKE.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9fx3JDRFPmI3O4uZyFhEz_ER4_Qo3DvvJIQo4gE5Q7cdhyGJG13IQcPEP/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRAIgK138jHQQ5zDUX3jApJQHrXy9K19KaGIPPeLZM49XDroCIBxvwa3yMX6DmF7tPclenRcgjDN6L9lIzu5WVbYV85-j"
[dashsegments] Total fragments: unknown (live)
[download] Destination: test.mp4.f247.mkv
[debug] File locking is not supported. Proceeding without locking
[dashsegments] Total fragments: unknown (live)
[download] Destination: test.mp4.f140.mkv
[debug] [youtube] Selected section: 1714186916.8004224 -> 1714186926.8004224
[debug] [youtube] [IklGR6rUqKE] Generating fragments for format 247
[debug] [youtube] Selected section: 1714186916.8004224 -> 1714186926.8004224
[debug] [youtube] [IklGR6rUqKE] Generating fragments for format 140
[download] 100% of    1.80MiB in 00:00:05 at 328.25KiB/s
[download] 100% of  169.54KiB in 00:00:05 at 31.51KiB/s
[Merger] Merging formats into "test.mp4.mkv"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i file:test.mp4.f247.mkv -i file:test.mp4.f140.mkv -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart file:test.mp4.temp.mkv
Deleting original file test.mp4.f247.mkv (pass -k to keep)
Deleting original file test.mp4.f140.mkv (pass -k to keep)


.\yt-dlp.exe https://www.youtube.com/watch?v=8m1KXka32Y4 --live-from-start --download-sections '#-10seconds - 0' -o test2.mp4 --verbose
[debug] Command-line config: ['https://www.youtube.com/watch?v=8m1KXka32Y4', '--live-from-start', '--download-sections', '#-10seconds - 0', '-o', 'test2.mp4', '--verbose']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] from yt-dlp/yt-dlp [ff0779267] (win_exe)
[debug] Python 3.11.9 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 3.0.13 30 Jan 2024)
[debug] exe versions: ffmpeg 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev (setts), ffprobe 2024-04-25-git-cae0f2bc55-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1810 extractors
[youtube] Extracting URL: https://www.youtube.com/watch?v=8m1KXka32Y4
[youtube] 8m1KXka32Y4: Downloading webpage
[youtube] 8m1KXka32Y4: Downloading ios player API JSON
[youtube] 8m1KXka32Y4: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "8m1KXka32Y4")
[youtube] 8m1KXka32Y4: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 8m1KXka32Y4: Downloading 1 format(s): 299+140
[info] 8m1KXka32Y4: Downloading 1 time ranges: 1714186974.0-1714186984.0
[debug] Invoking dashsegments downloader on "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208584/ei/6GosZpXJK_mFkucP_KyMwAk/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/8m1KXka32Y4.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9fwrjy1DWa46OYwZevDqNho-6IvsFeqqlCxKbGcJx8cg9-2G7OyONVeAd/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/LIVE/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Chfr%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRQIhAOFeCeWuUN8-Qs3Yo0hxkgd-CbwrTQ118AwI5ZST0wywAiANCmhY-zwNq10Mrij33hzC89rDUDkg5IWX2vHGmBSpWg%3D%3D", "https://manifest.googlevideo.com/api/manifest/dash/expire/1714208584/ei/6GosZpXJK_mFkucP_KyMwAk/ip/2600%3A1700%3A2ed0%3Af2ef%3A309a%3Aec38%3Af654%3Ad2cb/id/8m1KXka32Y4.1/source/yt_live_broadcast/requiressl/yes/xpc/EgVo2aDSNQ%3D%3D/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/UWF9fwrjy1DWa46OYwZevDqNho-6IvsFeqqlCxKbGcJx8cg9-2G7OyONVeAd/vprv/1/pacing/0/keepalive/yes/itag/0/playlist_type/LIVE/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cxpc%2Chfr%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AJfQdSswRQIhAOFeCeWuUN8-Qs3Yo0hxkgd-CbwrTQ118AwI5ZST0wywAiANCmhY-zwNq10Mrij33hzC89rDUDkg5IWX2vHGmBSpWg%3D%3D"
[dashsegments] Total fragments: unknown (live)
[download] Destination: test2.f299.mp4
[debug] File locking is not supported. Proceeding without locking
[dashsegments] Total fragments: unknown (live)
[download] Destination: test2.f140.mp4
[debug] [youtube] Selected section: 1714186974.0084486 -> 1714186984.0084486
[debug] [youtube] [8m1KXka32Y4] Generating fragments for format 299
[debug] [youtube] Selected section: 1714186974.0084486 -> 1714186984.0084486
[debug] [youtube] [8m1KXka32Y4] Generating fragments for format 140
[download]    5.21MiB at    5.97MiB/s (00:00:00) (frag 9/10)
[download]  190.46KiB at  226.66KiB/s (00:00:00) (frag 9/10)

I reproduced the issue on 4 machines. 3 Windows, one Arch Linux. All of the above were taken on a Windows device. I both tried compiling it myself, which is what these were from and also had the same issues with the bashonly/yt-dlp releases at https://github.com/bashonly/yt-dlp/releases

The part files also appear empty until the terminal window is closed, at which point they will have their proper size. When it is closed, you can rename the part files and manually merge them with ffmpeg for what appears to be the whole specified section. Unlike what the person in the other thread said, it does not appear like the clip ever successfully finishes no matter how much time is given.

@pukkandan pukkandan mentioned this pull request May 3, 2024
Closed
This was referenced Jun 5, 2024
Closed
Open
@azatoth
Copy link

azatoth commented Aug 17, 2024

@bashonly @pukkandan Any progress in getting this PR merged?

@azatoth azatoth mentioned this pull request Sep 4, 2024
Closed
@bashonly bashonly mentioned this pull request Sep 13, 2024
Closed
9 tasks
@coletdjnz coletdjnz mentioned this pull request Nov 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bashonly bashonly bashonly left review comments

@coletdjnz coletdjnz coletdjnz left review comments

Assignees
No one assigned
Labels
site:youtube site-enhancement Feature request for some website
Projects
Major sites
Status: youtube livestream
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Youtube] --live-from-start support for sequence / range download