Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[twitch] Update is_live regex for VODs #6500

Merged
merged 1 commit into from
Mar 10, 2023

Conversation

elyse0
Copy link
Contributor

@elyse0 elyse0 commented Mar 10, 2023

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

Twitch VOD extractor detects if a video is still live by looking for 404_processing_{width}x{height} in the previewThumbnailUrl, but GQL api now returns a slightly diferent url format.

So this PR updates the check to use regex and match both cases

 [
   {
      "data":{
         "user":null,
         "currentUser":null,
         "video":{
            "id":"1760484475",
            "title":"MARIO PARTY CON EL TEAM TRYHARD HOY SE ROMPEN AMISTADES",
            "description":null,
            "previewThumbnailURL":"https://vod-secure.twitch.tv/_404/404_processing_90x60.png",
            "createdAt":"2023-03-09T23:32:00Z",
            "viewCount":1244,
            "publishedAt":"2023-03-09T23:32:00Z",
            "lengthSeconds":19163,
            "broadcastType":"ARCHIVE",
            "owner":{
               "id":"211121982",
               "login":"vickypalami",
               "displayName":"vickypalami",
               "__typename":"User"
            },
            "game":{
               "id":"509658",
               "boxArtURL":"https://static-cdn.jtvnw.net/ttv-boxart/509658-{width}x{height}.jpg",
               ...
[debug] Command-line config: ['https://www.twitch.tv/videos/1760484475', '--verbose', '--skip-download', '--print', 'is_live']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version [email protected] [392389b7d] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: b250878e0
[debug] Python 3.10.7 (CPython x86_64 64bit) - Linux-5.19.0-35-generic-x86_64-with-glibc2.36 (OpenSSL 3.0.5 5 Jul 2022, glibc 2.36)
[debug] exe versions: ffmpeg N-108931-g4dda3b1653-20221104 (setts), ffprobe N-108931-g4dda3b1653-20221104, phantomjs broken, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1788 extractors
[twitch:vod] Extracting URL: https://www.twitch.tv/videos/1760484475
[twitch:vod] 1760484475: Downloading stream metadata GraphQL
[twitch:vod] 1760484475: Downloading video access token GraphQL
[twitch:vod] 1760484475: Downloading m3u8 information
[twitch:vod] 1760484475: Downloading storyboard metadata JSON
WARNING: [twitch:vod] Unable to download JSON metadata: HTTP Error 403: Forbidden
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] v1760484475: Downloading 1 format(s): 936p60
True

Fixes #6494

Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

@pukkandan pukkandan merged commit 0551511 into yt-dlp:master Mar 10, 2023
elyse0 added a commit to elyse0/yt-dlp that referenced this pull request Mar 12, 2023
biolds added a commit to biolds/tubesync that referenced this pull request Mar 28, 2023
This is required to fix Twitch is_live status
(see yt-dlp/yt-dlp#6500)
qa4FKm3mUr added a commit to qa4FKm3mUr/yt-dlp that referenced this pull request Jul 30, 2023
* [utils] Add hackish 'now' support for --download-sections

* [utils] Add microseconds to unified_timestamp

* [common] Extract start and end keys for Dash fragments

* [utils] Allow using local timezone for 'now' timestamps

* Use local timezone for download sections

* Add fixme in modified parse_chapters function

A range like '*(now-1hour)-(now-30minutes)' doesn't work

* [youtube] Support --download-sections for YT Livestream from start

* Create last_segment_url only if necessary

* Improve parse_chapters comments

* Fix linter

* [extractor/iq] Set more language codes (yt-dlp#6476)

Authored by: D0LLYNH0

* [extractor/opencast] Add ltitools to `_VALID_URL` (yt-dlp#6371)

Authored by: C0D3D3V

* [downloader/curl] Fix progress reporting

Bug in 8c53322
Closes yt-dlp#6490

* [extractor/youtube] Bypass throttling for `-f17`

and related cleanup

Thanks @AudricV for the finding

* [extractor/twitch] Fix `is_live` (yt-dlp#6500)

Closes yt-dlp#6494
Authored by: elyse0

* [extractor/cbc:gem] Update `_VALID_URL` (yt-dlp#6499)

Authored by: makeworld-the-better-one
Closes yt-dlp#6395

* Support loading info.json with a list at it's root

* [extractor/hidive] Fix login

Fixes yt-dlp#6493 (comment)

* [extractor/opencast] Fix format bug (yt-dlp#6512)

Authored by: C0D3D3V

* [extractor/rokfin] Re-construct manifest url (yt-dlp#6507)

Authored by: vampirefrog

* [extractor/youtube] Add client name to `format_note` when `-v` (yt-dlp#6254)

Authored by: Lesmiscore, pukkandan

* [extractor/youtube] Add extractor-arg `include_duplicate_formats`

* [extractor/youtube] Construct fragment list lazily

Building fragment list for all formats take significant time for large videos

* Support negative durations

* Revert "[utils] Allow using local timezone for 'now' timestamps"

This reverts commit 1799a6a.

* Add fragment count

* Fix unified_timestamp

* Remove tz_aware date code

* Add debug for selected section

* Add initial documentation

* Fix linter

* Fix linter

* Allow days in parse_duration

* Improve option documentation

* Add some documentation

* Lock less agressively

This gives a speed performance of about 30%

* Fix return values of _extract_sequence_from_mpd

* Always compute last_seq

* Support for epoch timestamps

* Update options docs

* Restore README.md

I think this is auto-generated by some script

* Add warning about --download-sections without --live-from-start

* Fix bug after merge

* Update yt_dlp/options.py

* Cleanup

---------

Co-authored-by: Elyse <[email protected]>
Co-authored-by: Sophire <[email protected]>
Co-authored-by: D0LLYNH0 <[email protected]>
Co-authored-by: Daniel Vogt <[email protected]>
Co-authored-by: pukkandan <[email protected]>
Co-authored-by: makeworld <[email protected]>
Co-authored-by: Daniel Vogt <[email protected]>
Co-authored-by: vampirefrog <[email protected]>
Co-authored-by: Lesmiscore <[email protected]>
Co-authored-by: bashonly <[email protected]>
Co-authored-by: bashonly <[email protected]>
aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this pull request Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Twitch] !is_live filter not excluding vod of current live in latest version
2 participants