Commit Graph

1988 Commits

Author SHA1 Message Date
diogo
0a9e19646a
pass the api/v1/playlists with videos before the offset 2021-10-11 23:43:41 +02:00
diogo
65e45c4079
linting 2021-10-11 23:30:49 +02:00
diogo
440105976f
fix cases when high offset video from playlist has no offset in url 2021-10-11 23:30:49 +02:00
diogo
7eba7fbcc7
add index to playlist item 2021-10-11 23:30:49 +02:00
diogo
d9bfb3d305
playlist starts at the offset 2021-10-11 23:30:49 +02:00
syeopite
b10f37bea9
Use kemal in production mode (#2455) 2021-10-11 14:42:22 +02:00
mastihios
0947c26612
Fix URL-encoding in href strings (#2460)
* hrefs: replace HTML.escape w/ URI.encode_www_form

* Fix search_query_encoded
2021-10-11 05:18:20 -07:00
syeopite
21e29411af
Fix extractor bugs (#2454)
* Add debug/trace logging to extract_items
* Handle invalid timestamps for livestreams extraction
* Make use of author_fallback in playlist extractor
* Don't use extract_text for video length extraction

The extract_text function attempts to extract from both the simpleText and
the runs route. This is typically what we'd want for text extraction as
it could appear in both locations. However, while this still holds true,
the thumbnailOverlayTimeStatusRenderer writes a numerical length (when
present on the video) to the simpleText route and uses runs for a
text overlay like "LIVE" or "PREMIERE".

Therefore, when a video has a text overlay instead of a numerical one,
Invidious still passes it onto decode_length_seconds, which obviously
raises since it cannot be converted into integers.

In the future, if more routes requires one text route over the other, we
should go ahead and add an argument to extract_text itself. Though for
now, this is sufficient.

* Handle unsupported "special" categories
2021-10-07 23:39:21 +02:00
Samantaz Fox
adc12addfa
Add config option to display source code URL in footer (#2450) 2021-10-07 13:53:12 +02:00
Samantaz Fox
d806310665
Revert "Fix typo (#2456)" (#2457)
This reverts commit 4982bff74d.
2021-10-04 19:51:57 +02:00
Jorge Maldonado Ventura
4982bff74d
Fix typo (#2456) 2021-10-04 17:47:57 +00:00
syeopite
81b12b8001
Add support to make invidious API-only via flag (#2363) 2021-10-02 20:04:02 +00:00
syeopite
3e781b7d13
Merge pull request #2443 from iv-org/syeopite-patch-1
Use make_client when querying instance api
2021-10-02 20:00:37 +00:00
syeopite
9c44e41a4c
Merge pull request #2228 from syeopite/extract-items-overhaul
Overhaul extract_item(s) functions and add infrastructure for parsing YouTube categories
2021-10-02 20:00:16 +00:00
syeopite
73eabb6ca2
Actually fix lint 2021-10-02 08:53:58 -07:00
Samantaz Fox
5a52b4fe45
Fix lint 2021-10-02 17:41:25 +02:00
Samantaz Fox
caa08a6379
Disable locales with less than 50% of strings translated 2021-10-02 17:38:06 +02:00
Samantaz Fox
cd02078e26
Add Portuguese (pt) to locales
follow up to #2437
2021-10-02 17:19:28 +02:00
syeopite
77131cff91
Use make_client when querying instance api 2021-10-02 15:16:07 +00:00
mastihios
7b2aa5f98e
add icon-buttons to playlist items (#2442) 2021-10-02 11:59:33 +00:00
syeopite
26b28cea49
Use break instead of short-circuit return 2021-10-01 05:39:23 -07:00
syeopite
23049e026f
Improve readabltiy of SearchChannel auto-gen detect 2021-09-28 08:55:02 -07:00
syeopite
9ab242ca2e
Optimize routing logic of extract_item(s) funcs 2021-09-28 08:50:23 -07:00
syeopite
aa59925374
Rename get_browse_endpoint to get_browse_id 2021-09-28 08:39:00 -07:00
syeopite
43ea8fa706
Convert nil for AuthorFallback to empty strings 2021-09-28 08:25:03 -07:00
syeopite
6df85718e6
Apply suggestions from code review
Co-authored-by: Samantaz Fox <coding@samantaz.fr>
2021-09-28 15:23:36 +00:00
syeopite
b20f72b963
Use default timeout (5 seconds) for YT pool (#2430) 2021-09-26 23:03:45 +02:00
syeopite
092b8a4e52
Add documentation to extractors.cr 2021-09-24 21:07:07 -07:00
syeopite
e5f07dedbf
Typos and tiny styling changes 2021-09-24 21:07:07 -07:00
syeopite
ca9eb0d539
Bountiful extractor changes
- Add extract_text to simplify extraction of InnerTube texts
- Add helper extractor methods to reduce repetition in parsing InnerTube
- Change [] more than 2 blocks long to use #dig or #dig?
- Remove useless ?.try blocks for items that always exists
- Add (some) documentation to VideoRendererParser
2021-09-24 21:07:07 -07:00
syeopite
142317c2be
Overhaul extractors.cr to use modules 2021-09-24 21:07:07 -07:00
syeopite
3dea670091
Switch to structs in extractors.cr for performance 2021-09-24 21:07:07 -07:00
syeopite
8435e79913
Improve documentation for extract_item(s) funcs 2021-09-24 21:07:07 -07:00
syeopite
30e85b40f9
Fix extract_videos 2021-09-24 21:07:07 -07:00
syeopite
be1a43a337
Manually extract category refactor from 1b569bbc99
Also fixes some errors caused by cherry-picking
2021-09-24 21:07:07 -07:00
syeopite
abca8f7a7c
Rename invidiousitems.cr 2021-09-24 21:07:07 -07:00
syeopite
7b60dac526
Add description_html field to Category
(cherry picked from commit aa8f15f795)
2021-09-24 21:07:07 -07:00
syeopite
ea6434662d
Change typing of Category contents to only Array
(cherry picked from commit d3384e17f1)
2021-09-24 21:07:06 -07:00
syeopite
0b7a108a59
Move continuation_token out of Category struct
(cherry picked from commit 0e96eda28f)
2021-09-24 21:07:06 -07:00
syeopite
57c63f3598
Rename "items_without_cate_items" to reflect usage 2021-09-24 21:07:06 -07:00
syeopite
ae30f32c36
Unpack search items that are embedded in categories
This is a squash of a bunch of commits
cherry-picked commits

Fix category parse error on search

(cherry picked from commit cc02fed4e6)

Fix category items not being extracted in search

(cherry picked from commit 2605b9c609)

Make search not include category items for now

(cherry picked from commit ca4afd59f4)

Change behavior of categories in search results

(cherry picked from commit cc10675610)

Fix missing search results in extraction

(cherry picked from commit abda6840d5)

Fix miscount of search results

(cherry picked from commit 491e33450e)
2021-09-24 21:07:06 -07:00
syeopite
a50f64f6e9
Add parser for categories (shelfRenderer)
This commit adds a new parser for YT's shelfRenderers which are
typically used to denote different categories.The code for featured
channels parsing has also been moved to use the new parser but some
additional refactoring are needed there.

The ContinuationExtractor has also been improved and is now capable of
extraction continuation data that is packaged under
"appendContinuationItemsAction"

In additional this commit adds some useful helper functions to extract
the current selected tab the continuation token. This is to mainly
reduce code size and repetition.
--
This cherry-picked commit also removes the code for parsing featured
channels present on the original.

(cherry picked from commit 8000d538db)
2021-09-24 21:07:06 -07:00
syeopite
1323b94b7a
Rewrite extract_item and extract_items functions
This commit completely rewrites the extract_item and extract_items
function. Before this commit these two function were an unreadable
mess. The extract_item function was a lengthy if-elsif chain
while the extract_items function contained an incomprehensible
mess of .try, else and ||.

With this commit both of these functions have been pulled into a
separate file with the internal logic being moved to a few classes.

This significantly reduces the size of these two methods, enhances
readability and makes adding new extraction/parse rules much simpler.

See diff for details.

--
This cherry-picked commit also removes the code for parsing featured
channels present on the original.

(cherry picked from commit a027fbf7af)
2021-09-24 21:06:46 -07:00
syeopite
6d68fbc31d
Fix livestream regex regression caused by #2271
Closes #2352

Special thanks to @WaywardHeart for finding this issue!
2021-09-25 02:47:52 +00:00
syeopite
86ca568d6d
Remove login type button from frontend (#2423) 2021-09-23 08:44:26 +02:00
Walkyst
2b0bb69a4f
Fix mixes route (#2421) 2021-09-20 22:39:32 -07:00
syeopite
87f46a7532
Unregister captcha job (#2390) 2021-09-19 10:30:19 +00:00
Samantaz Fox
e655af251c
Try not to log search queries (#2362) 2021-09-16 09:36:22 +00:00
syeopite
5054510d15
Prevent VR from being initialized in listen mode (#2396) 2021-09-15 01:37:23 +02:00
Samantaz Fox
947fe4fbb3
HTML escape video mimetype
Video mimetype may contain code information between double quotes.
If not properly escaped, it breaks the browser's parser. E.g:
```
type="video/mp4; codecs=" avc1.64001f,="" mp4a.40.2""=""
```

Thank Robin for catching this!
2021-09-13 18:20:11 +02:00