Commit Graph

125 Commits

Author SHA1 Message Date
Fufu Fang 84f2ebc970 improved error messages 2019-04-26 16:24:55 +01:00
Fufu Fang 9717d016ed added LinkTable_gap_fill() to fill in the gaps in the link table when it is loaded from the disk 2019-04-26 15:35:48 +01:00
Fufu Fang 04c0499fae added HTTP 429 handling for file stat query 2019-04-26 14:14:42 +01:00
Fufu Fang 93f9701aa8 minor error message formatting 2019-04-26 13:32:59 +01:00
Fufu Fang 774f14c8e7 if we get HTTP 429, wait for 5 secs 2019-04-26 13:20:55 +01:00
Fufu Fang edc3df733c minor bug fix 2019-04-26 11:42:11 +01:00
Fufu Fang 85d66adf6c added command line switch to configure user agent and the maximum number of network connections allowed 2019-04-26 11:39:03 +01:00
Fufu Fang f479482d7b stop Cache_bgdl() from downloading past the last segment, remove excess debug message 2019-04-26 10:36:25 +01:00
Fufu Fang 5f6fc3f761 Now caches the directory structures by dumping LinkTables to the disk. 2019-04-26 08:26:47 +01:00
Fufu Fang 9a5f37d91f replaced strndupcat with path_append, added LinkTable_disk_*() functions 2019-04-26 07:39:45 +01:00
Fufu Fang df025b1caf improved the correctness of Cache_bgdl() 2019-04-25 08:50:04 +01:00
Fufu Fang 0892cd0a8a changed header inclusion order 2019-04-24 04:28:54 +01:00
Fufu Fang 6536163f98 improve cached dataset creation process, better error handling 2019-04-24 04:28:46 +01:00
Fufu Fang ac61ddd1c0 stability improvement, hopefully got rid off SIGFPE 2019-04-24 04:28:46 +01:00
Fufu Fang e2d2b0dd28 Cache system has been succesfully implemented 2019-04-24 04:28:27 +01:00
Fufu Fang 692ad3f00c Cache dataset creation is working, time write the function to fill in the cache 2019-04-24 04:27:40 +01:00
Fufu Fang b6bdf15ad1 removed a bunch of function prototypes 2019-04-24 04:27:38 +01:00
Fufu Fang 77bb715590 fixed memory leak in link table creation 2019-04-24 04:21:38 +01:00
Fufu Fang 5af3def3f8 Linked in CacheSystem_init, Cache_open, Cache_close 2019-04-24 04:21:38 +01:00
Fufu Fang e9e41165ae added CacheDir_create() and fs_release() 2019-04-24 04:21:38 +01:00
Fufu Fang 4b60b4e9be wrote the cache opening function 2019-04-24 04:21:38 +01:00
Fufu Fang 21f1cf2a18 changed strlen in link.c to strnlen, and changed LINK_LEN_MAX to P_URL_LEN_MAX 2019-04-24 04:21:37 +01:00
Fufu Fang 7cafdbf7d4 Completed metadata read/write functions 2019-04-24 04:21:37 +01:00
Fufu Fang 7b75e1ea41 enable http pipelining, changed max connection limit to 10 2019-04-24 04:21:37 +01:00
Jerome Charaoui f6f50c7cdf Move source files in src/ 2019-04-24 04:21:10 +01:00