Commit Graph

30 Commits

Author SHA1 Message Date
rook1e 9a826bbe6f feat: support searching well-known urls in subdirectory 2023-04-02 17:44:14 -07:00
Frédéric Guillot aaa1625724 Ignore empty link when discovering feeds 2023-02-26 17:19:26 -08:00
James Loh 2f6895e118 Fix finding JSON feeds with new MIME type
The 1.1 version (https://jsonfeed.org/version/1.1) for JSON feeds defines that feeds should have a MIME type of `application/feed+json` which Miniflux wasn't searching for
2021-08-21 13:01:08 -07:00
Alexandros Kosiaris b8b16c3bdf Add /rss/ in finder's wellKnownUrls
ATCOM netvolution WCM, probably alongside others, a CMS powering several
high profile and high traffic Greek news sites, among other sites,
publishes the RSS feed under /rss/. Add it to the list. It's generic
enough to allow us to assume other software might do it to

On a select set of 627 Greek news media sites (the infamous Petsas list),
adding this rule increased discoverability of RSS feeds by a factor of
2.61% (from 498 to 511).
2021-07-22 19:46:40 -07:00
Darius 9242350f0e
Add per feed cookies option 2021-03-22 20:27:58 -07:00
Frédéric Guillot ec3c604a83 Add option to allow self-signed or invalid certificates 2021-02-21 13:58:52 -08:00
Frédéric Guillot 246a48359c Do not follow redirects when trying known feed URLs
Some websites redirects unknown URLs to the home page.
As result, the list of known URLs is returned to the subscription list.
We don't want the user to choose between invalid feed URLs.
2020-11-06 17:46:54 -08:00
Frédéric Guillot 16b7b3bc3e http client: remove dependency on global config options 2020-09-27 14:37:46 -07:00
Kebin Liu cf7712acea
Add HTTP proxy option for subscriptions 2020-09-09 23:28:54 -07:00
Frédéric Guillot fc75b0cd8e Add workaround to get YouTube feed from video page 2020-08-02 12:24:46 -07:00
Frédéric Guillot 7380c64141 Add workaround to find YouTube channel feeds
YouTube doesn't expose RSS links anymore for new-style URLs.
2020-08-02 11:37:07 -07:00
Gabriel Augendre e44b4b2540 Try known urls if no link alternate
I came across a few blogs that didn't have a link rel alternate
but offered a RSS/Atom feed.
This aims at solving this issue for "well known" feed urls, since
these urls are often the same.
2020-06-21 20:34:59 -07:00
Frédéric Guillot 3debf75eb9 Normalize URL query string before executing HTTP requests
- Make sure query strings parameters are encoded
- As opposed to the standard library, do not append equal sign
for query parameters with empty value
- Strip URL fragments like Web browsers
2019-12-26 15:56:59 -08:00
Frédéric Guillot 778346b0b0 Simplify feed fetcher
- Add browser package to handle HTTP errors
- Reduce code duplication
2018-10-14 21:43:48 -07:00
Frédéric Guillot 5870f04260 Simplify feed parser and format detection
- Avoid doing multiple buffer copies
- Move parser and format detection logic to its own package
2018-10-14 11:46:41 -07:00
Patrick 2538eea177 Add the possibility to override default user agent for each feed 2018-09-19 18:19:24 -07:00
Frédéric Guillot dbcc5d8a97 Use canonical imports 2018-08-24 21:56:39 -07:00
Frédéric Guillot 9c0f882ba0 Add specific 404 and 401 error messages 2018-06-30 12:42:12 -07:00
Frédéric Guillot bddca15b69 Add new fields for feed username/password 2018-06-19 22:58:29 -07:00
Frédéric Guillot 1eba1730d1 Move HTTP client to its own package 2018-04-28 10:51:07 -07:00
Frédéric Guillot dda9114692 Improve error handling for HTTP client 2018-02-08 18:16:54 -08:00
Frédéric Guillot 0fb87eba3f Improve error handling when the response is empty 2018-02-07 18:47:47 -08:00
Frédéric Guillot 713b38e34c Handle more encoding edge cases
- Feeds with charset specified only in Content-Type header and not in XML document
- Feeds with charset specified in both places
- Feeds with charset specified only in XML document and not in HTTP header
2018-01-20 13:25:21 -08:00
Frédéric Guillot c39f2e1a8d Rename helper packages 2018-01-02 19:15:08 -08:00
Frédéric Guillot 1d8193b892 Add logger 2017-12-15 18:55:57 -08:00
Frédéric Guillot 84d912c979 Rewrite imports 2017-12-12 21:48:13 -08:00
Frédéric Guillot 6f5350a497 Move packages http and url 2017-12-02 20:26:21 -08:00
Frédéric Guillot fb2a73c91e Proxify image enclosures 2017-12-01 22:29:18 -08:00
Frédéric Guillot aecda64030 Make sure XML feeds are always encoded in UTF-8 2017-11-20 17:12:37 -08:00
Frédéric Guillot 8ffb773f43 First commit 2017-11-19 22:01:46 -08:00