ff67794b02
Now retry on HTTP 520 (Unknown error)
2019-09-04 17:57:15 +01:00
b6777c0478
Bugfix: No longer deadlock after encountering HTTP 429 while filling up a Linktable.
...
- Renamed some functions
- After initialise parse of the HTML file, files are no longer assigned as LINK_FILE. They are now assigned as LINK_UNINITIALISED_FILE.
- Link_req_file_stat() now crashes if the link type is other than LINK_UNINITIALISED_FILE.
2019-09-04 17:43:18 +01:00
cf49bf86b8
improved LinkTable_fill() status message
2019-09-03 15:12:38 +01:00
c7dfa241d4
Backtrace will now be printed when the program crashes
...
- Note that static functions are not included in the printed backtrace.
2019-09-03 14:53:32 +01:00
9ff099cd3a
added a status indicator when filling up the linktable
2019-09-03 14:02:41 +01:00
ed5457c76f
Bugfix: partially fixed the cache lock
...
- now when the same file is opened twice, the fread() output is consistent.
2019-09-01 11:39:47 +01:00
1a44a4d960
Wrapped mutex locking and unlocking functions in error checking macro
2019-09-01 01:21:40 +01:00
92a9658c66
Cache system bug fix
...
- Now keep track of the number of times a file has been opened. The on-disk
cache file no longer gets opened multiple times, if a file is opened multiple
times.
2019-09-01 00:43:50 +01:00
afb2a8fe6c
Directory listing performance improvement while file transfers are going on
...
- Added a LinkTable generation priority lock
- This allows LinkTable generation to be run exclusively. This
effectively gives LinkTable generation priority over file transfer.
2019-08-31 21:21:28 +01:00
ed37aa5ca0
removed CURL_LOCK_DATA_CONNECT to solve #36 without disabling HTTP2
2019-07-21 08:41:02 +01:00
4491377f64
forces HTTP1.1 to fix issue #36
2019-07-20 19:25:25 +01:00
78d816716c
Fixed issue #34 - file / directory detection
2019-07-20 17:19:40 +01:00
283520136c
Closing issue #33 , now set a default cache directory
2019-04-30 08:06:09 +01:00
2a93e95aea
improved debug message formatting
2019-04-27 03:28:28 +01:00
ba8c7232e2
HTTP 429, LinkTable_print(), variable shadowing
...
changed the way HTTP 429 gets handled while querying for file stats
fixed LinkTable_print(), added variable shawdowing warning in Makefile
2019-04-27 02:42:29 +01:00
2a2ac2dce2
rewrote curl_multi_perform_once()
2019-04-27 01:53:09 +01:00
6cef84874c
fixed LinkTable_print()
2019-04-26 19:49:20 +01:00
17587852ff
fixed LinkTable_gap_fill
2019-04-26 19:47:03 +01:00
1dc54afdeb
Improved LinkTable_fill() and LinkTable_gapfill()
2019-04-26 19:30:39 +01:00
245d4a3ff4
removed the restriction for Link_get_stat()
2019-04-26 18:53:26 +01:00
99761b249a
restored LinkTable_print()
2019-04-26 18:45:38 +01:00
8110438cd1
reduce the amount of debug output
2019-04-26 18:30:35 +01:00
e2c47260a0
now calculate the number of invalid links in the linktable
2019-04-26 18:25:40 +01:00
84f2ebc970
improved error messages
2019-04-26 16:24:55 +01:00
9717d016ed
added LinkTable_gap_fill() to fill in the gaps in the link table when it is loaded from the disk
2019-04-26 15:35:48 +01:00
04c0499fae
added HTTP 429 handling for file stat query
2019-04-26 14:14:42 +01:00
93f9701aa8
minor error message formatting
2019-04-26 13:32:59 +01:00
774f14c8e7
if we get HTTP 429, wait for 5 secs
2019-04-26 13:20:55 +01:00
edc3df733c
minor bug fix
2019-04-26 11:42:11 +01:00
85d66adf6c
added command line switch to configure user agent and the maximum number of network connections allowed
2019-04-26 11:39:03 +01:00
f479482d7b
stop Cache_bgdl() from downloading past the last segment, remove excess debug message
2019-04-26 10:36:25 +01:00
5f6fc3f761
Now caches the directory structures by dumping LinkTables to the disk.
2019-04-26 08:26:47 +01:00
9a5f37d91f
replaced strndupcat with path_append, added LinkTable_disk_*() functions
2019-04-26 07:39:45 +01:00
df025b1caf
improved the correctness of Cache_bgdl()
2019-04-25 08:50:04 +01:00
0892cd0a8a
changed header inclusion order
2019-04-24 04:28:54 +01:00
6536163f98
improve cached dataset creation process, better error handling
2019-04-24 04:28:46 +01:00
ac61ddd1c0
stability improvement, hopefully got rid off SIGFPE
2019-04-24 04:28:46 +01:00
e2d2b0dd28
Cache system has been succesfully implemented
2019-04-24 04:28:27 +01:00
692ad3f00c
Cache dataset creation is working, time write the function to fill in the cache
2019-04-24 04:27:40 +01:00
b6bdf15ad1
removed a bunch of function prototypes
2019-04-24 04:27:38 +01:00
77bb715590
fixed memory leak in link table creation
2019-04-24 04:21:38 +01:00
5af3def3f8
Linked in CacheSystem_init, Cache_open, Cache_close
2019-04-24 04:21:38 +01:00
e9e41165ae
added CacheDir_create() and fs_release()
2019-04-24 04:21:38 +01:00
4b60b4e9be
wrote the cache opening function
2019-04-24 04:21:38 +01:00
21f1cf2a18
changed strlen in link.c to strnlen, and changed LINK_LEN_MAX to P_URL_LEN_MAX
2019-04-24 04:21:37 +01:00
7cafdbf7d4
Completed metadata read/write functions
2019-04-24 04:21:37 +01:00
7b75e1ea41
enable http pipelining, changed max connection limit to 10
2019-04-24 04:21:37 +01:00
f6f50c7cdf
Move source files in src/
2019-04-24 04:21:10 +01:00
Fufu Fang
Jerome Charaoui