9e3e4747ae
fixed Cache_bgdl()
...
- Cache_bgdl() used to corrupt the cache file
2019-09-02 09:04:20 +01:00
ed5457c76f
Bugfix: partially fixed the cache lock
...
- now when the same file is opened twice, the fread() output is consistent.
2019-09-01 11:39:47 +01:00
20f30a0e38
Tidied up some of the comments and formatting
2019-09-01 08:52:18 +01:00
1a44a4d960
Wrapped mutex locking and unlocking functions in error checking macro
2019-09-01 01:21:40 +01:00
92a9658c66
Cache system bug fix
...
- Now keep track of the number of times a file has been opened. The on-disk
cache file no longer gets opened multiple times, if a file is opened multiple
times.
2019-09-01 00:43:50 +01:00
afb2a8fe6c
Directory listing performance improvement while file transfers are going on
...
- Added a LinkTable generation priority lock
- This allows LinkTable generation to be run exclusively. This
effectively gives LinkTable generation priority over file transfer.
2019-08-31 21:21:28 +01:00
23611b8b1c
Version 1.1.7
...
- Now mutex associated debugging outputs are not compiled by default.
- Tagging version 1.1.7
2019-08-23 23:15:01 +01:00
45cee81e15
reverting segment size back to 8MB
2019-07-28 09:38:54 +01:00
71da71148f
removed excess fflush
2019-07-28 09:31:49 +01:00
ab15800b3a
improved error message
2019-07-28 08:20:17 +01:00
c72b0d4a7f
added pthread_detach() for cleanup
2019-07-27 02:49:38 +01:00
91af975ff8
changed default segment size, moved the locking sequence of bgt_lock and rw_lock
2019-07-26 00:06:36 +01:00
64cd6ca2c6
added more debug messages
2019-07-24 17:55:32 +01:00
c660159a18
added mutex locking / unlocking messages
2019-07-22 09:25:30 +01:00
ee2d3edcbf
Fixed issue #37
...
If a cache file already exists, the download segment size is set
to the size set by the metadata. The user specified download
segment size is ignored for existing cache files.
2019-07-21 17:53:35 +01:00
283520136c
Closing issue #33 , now set a default cache directory
2019-04-30 08:06:09 +01:00
4acf91a2b8
added --max-seg-count command line option
2019-04-27 04:52:39 +01:00
2cee4b33a7
cosmetic changes
2019-04-26 23:29:16 +01:00
3c0a5fdf65
removed excess #include directive
2019-04-26 23:10:09 +01:00
774f14c8e7
if we get HTTP 429, wait for 5 secs
2019-04-26 13:20:55 +01:00
80a98a79f9
improved thread synchronisation, to stop Cache_read() from downloading the segment that is being downloaded in the background by Cache_bgdl()
2019-04-26 10:57:07 +01:00
f479482d7b
stop Cache_bgdl() from downloading past the last segment, remove excess debug message
2019-04-26 10:36:25 +01:00
9a5f37d91f
replaced strndupcat with path_append, added LinkTable_disk_*() functions
2019-04-26 07:39:45 +01:00
825bd4d3fb
added the command line switch to control the download segment size
2019-04-25 21:34:07 +01:00
b6b101bc63
removed unnecessary goto statements
2019-04-25 13:14:54 +01:00
df025b1caf
improved the correctness of Cache_bgdl()
2019-04-25 08:50:04 +01:00
f5acebafe2
Merge branch 'master' of github.com:fangfufu/httpdirfs
2019-04-25 01:17:18 +01:00
f628939713
Now dynamically allocate the receive buffers on the heap.
2019-04-25 00:58:29 +01:00
86999915e8
Merge pull request #29 from Lewiscowles1986/patch-1
...
Make larger block size
2019-04-24 23:59:02 +01:00
e442871899
added Cache_background_download() to enable the download of the next block
2019-04-24 23:58:26 +01:00
6d02b0554b
Make larger block size
...
The queue is inherently serial, not a queue data structure, and the code has no concurrency / parallelization added, but making the block-size larger should increase supported file-size & use disk cache more efficiently.
2019-04-24 06:22:43 +01:00
12c19e3421
improved Cache_read()
2019-04-24 04:28:54 +01:00
cc1697894b
Resolved race condition - this code is now considered as stable.
...
Reference:
https://wiki.sei.cmu.edu/confluence/display/c/FIO24-C.+Do+not+open+a+file+that+is+already+open
2019-04-24 04:28:54 +01:00
2ebf37fb65
fixed doxygen warnings
2019-04-24 04:28:54 +01:00
797501514d
changed the log output
2019-04-24 04:28:54 +01:00
60b2e6c0c8
minor cosmetic changes
2019-04-24 04:28:54 +01:00
0892cd0a8a
changed header inclusion order
2019-04-24 04:28:54 +01:00
3a0c431bba
Stability improvement
...
Metadata corruption still occasionally happens when it gets read in.
I suspect it is due to some sort of race condition.
2019-04-24 04:28:54 +01:00
cba66c58fd
more stability and error handling improvements
2019-04-24 04:28:54 +01:00
fe079f6266
improved stability / error handling
2019-04-24 04:28:54 +01:00
36ddbe9ec5
fixed bug in fs_open(), now handles cache dataset creation properly
2019-04-24 04:28:46 +01:00
6536163f98
improve cached dataset creation process, better error handling
2019-04-24 04:28:46 +01:00
ac61ddd1c0
stability improvement, hopefully got rid off SIGFPE
2019-04-24 04:28:46 +01:00
f122cf000d
updated BUG section of README.md
2019-04-24 04:28:46 +01:00
e166098162
updated README, also moved some status message around
2019-04-24 04:28:46 +01:00
96a1e4fd55
used a more stupid version of the bitmap, because it works reliably
2019-04-24 04:28:35 +01:00
0d8d5ef329
documented the new feature, and the bug
2019-04-24 04:28:35 +01:00
e2d2b0dd28
Cache system has been succesfully implemented
2019-04-24 04:28:27 +01:00
692ad3f00c
Cache dataset creation is working, time write the function to fill in the cache
2019-04-24 04:27:40 +01:00
b6bdf15ad1
removed a bunch of function prototypes
2019-04-24 04:27:38 +01:00
5af3def3f8
Linked in CacheSystem_init, Cache_open, Cache_close
2019-04-24 04:21:38 +01:00
a02072f548
replaced fseek with fseeko, this should enable large file support
2019-04-24 04:21:38 +01:00
07b8382022
modified cache_create
2019-04-24 04:21:38 +01:00
212e319f20
unified struct member name between link.h and cache.h
2019-04-24 04:21:38 +01:00
c6717e0450
changed a bunch of function to static function
2019-04-24 04:21:38 +01:00
e9e41165ae
added CacheDir_create() and fs_release()
2019-04-24 04:21:38 +01:00
0c55dc5a0c
wrote Seg_exist, Seg_set, refined CacheSystem_init
2019-04-24 04:21:38 +01:00
b1971a7c3f
modified strndupcat, added more comments
2019-04-24 04:21:38 +01:00
e5c77f3680
Changed the way segments work, added more functions
...
Now segments are fixed size block of 128KiB
2019-04-24 04:21:38 +01:00
3d0269e90f
added Cache_create
2019-04-24 04:21:38 +01:00
4b60b4e9be
wrote the cache opening function
2019-04-24 04:21:38 +01:00
794fb9414f
wrote more Cache_open
2019-04-24 04:21:37 +01:00
7ae6d6fe1b
all the functions written so far compile fine, working on Cache_open
2019-04-24 04:21:37 +01:00
7cafdbf7d4
Completed metadata read/write functions
2019-04-24 04:21:37 +01:00
8d10316b95
changed the data type for file size. wrote Data_read()
2019-04-24 04:21:37 +01:00
c710904cfd
wrote Data_create function
2019-04-24 04:21:37 +01:00
90eccb6049
created the skeletal cache.c
2019-04-24 04:21:37 +01:00
a3e2b0bc3b
added some of the prototype for the cache controller
2019-04-24 04:21:37 +01:00
Fufu Fang
Lewis Cowles