I am just having a play around and kind of seeing what my options are. I have a HDD for downloads/torrents etc but Windows/applications all live on a NVMe. Anything download either staying on the HDD for a bit, or gets copied over to my NAS shortly after.
I have ~2GB RAM free and have also tested creating a L2 cache as an image on the NVMe. I had set L1 as 1GB and L2 as 10GB.
Everything appeared alright, but I don't actually know what would be beneficial and wouldn't. Based on what I have said, can anyone suggest anything? I'm thinking that a write cache would definitely help because concurrent downloads tend to put the drive under strain, especially when torrents are involved and they are being seeded. I have Googled a bit and if I read things correctly, L2 for writes could in theory cache a downloaded torrent so that any seeding uses data from there rather than the HDD?
Caching Options for Downloads Drive
Re: Caching Options for Downloads Drive
Also, I don't fully understand whether the L1 (1GB-2GB RAM) ought to be set as read or write? If both, should the space be shared or have a ratio defined from the start? My L2 is a 12G file-disk on a NVNe, and was set up with RAMDisk as a SCSI. It was then formatted as L2STORAGE in PrimoCache. Similarly, should this one be read/write and shared or have the space defined?
There would be more seeding taking place when new torrents are involved, so it would be beneficial if L2 could serve those requests. The HDD obviously struggles with multiple concurrent read/writes, and that becomes really noticable when my scripts copy the downloaded media over to a NAS and there are still multiple download/uploads going on. Could there a brief delayed write set and L! reserved for that?
Not sure if L1 passes through L2 to get to the HDD, especially if L2 has been set as 100% read. Overall, I've not got my ahead around all this yet so any help would be greatly appreciated
There would be more seeding taking place when new torrents are involved, so it would be beneficial if L2 could serve those requests. The HDD obviously struggles with multiple concurrent read/writes, and that becomes really noticable when my scripts copy the downloaded media over to a NAS and there are still multiple download/uploads going on. Could there a brief delayed write set and L! reserved for that?
Not sure if L1 passes through L2 to get to the HDD, especially if L2 has been set as 100% read. Overall, I've not got my ahead around all this yet so any help would be greatly appreciated
Re: Caching Options for Downloads Drive
I think L1 write-only cache (100% write cache space) with defer-write enabled shall help downloading. You may check the cache statistics to see the performance and tune the configuration.
PS. it is not recommended to use a file-disk or virtual disk as level-2 storage. Dead-lock might happen if doing so.
PS. it is not recommended to use a file-disk or virtual disk as level-2 storage. Dead-lock might happen if doing so.
Re: Caching Options for Downloads Drive
I am only able to spare 2GB of RAM at most for L1 so was hoping to back that up with a disk based L2. Was also hoping that placing the image on a NVMe would still benefit the standard HDD.support wrote: ↑Tue Jun 11, 2019 4:29 am I think L1 write-only cache (100% write cache space) with defer-write enabled shall help downloading. You may check the cache statistics to see the performance and tune the configuration.
PS. it is not recommended to use a file-disk or virtual disk as level-2 storage. Dead-lock might happen if doing so.
I could partition the NVMe and allocate L2 to the HDD that way?
I've just noticed that I can't copy some films from the HDD to my NAS... just hangs for ages and then Windows shows some sort of error. I am assuming that is is trying to do something with the cache and then falling over... assuming its down to the file based L2 lol.
Re: Caching Options for Downloads Drive
Just about to try this out now and was wondering what sort of L2 size you would recommend? L1 can only go up to 2GB due to RAM limitations. If it makes a difference, the HDD in question is a 2TB Western Digital Black.
Re: Caching Options for Downloads Drive
With only 2GB for the L1, it's a waste to put that as a read/write L1 on the 2TB drive - the read hitrate on the L1 would be abysmal. Make it a write-only L1 and then as large a read/write L2 as you can use for the rest of that cache task. If you are partitioning off part of your NVMe (boot drive?) for the L2, remember to leave empty space at the end of the NVMe to serve as overprovisioning (typically 5-10% of the total drive space on the SSD/NVMe).
The problem overall is the size of total data residing on the 2TB drive, and the resulting hitrate with small L1/L2 caches. If the 2TB is fairly full, you'll need a good sized L2 to adequately cache frequently used items.
The problem overall is the size of total data residing on the 2TB drive, and the resulting hitrate with small L1/L2 caches. If the 2TB is fairly full, you'll need a good sized L2 to adequately cache frequently used items.
Re: Caching Options for Downloads Drive
Roughly 1.5TB is occupied, but at least 1TB of that is like a garden shed... a dump of old data that may never see the light of day lol. In terms of active use, downloads are written to the drive before being moved elsewhere. Torrents are seeded for a while, so the data is copied elsewhere rather than being moved.
After a few days of testing a 2GB L1 write only cache, the hit rate was around 20%. Previously, when downloading a newly released torrent (~1GB), copying to a NAS took a ridiculously long time, and speeds of any active downloads/uploads significantly dropped during this time. With the L1 in place, the copying had no impact on the downloads/uploads and vice versa.
Funnily enough, I now have more RAM available because I found that storing my Plex/Emby database/cache/metadata on a RamDisk was no faster than the NVMe storage, all whilst adding risk/complications.
The plan:
L1 - 8GB RAM
L2 - 12GB NVMe
After a few days of testing a 2GB L1 write only cache, the hit rate was around 20%. Previously, when downloading a newly released torrent (~1GB), copying to a NAS took a ridiculously long time, and speeds of any active downloads/uploads significantly dropped during this time. With the L1 in place, the copying had no impact on the downloads/uploads and vice versa.
Funnily enough, I now have more RAM available because I found that storing my Plex/Emby database/cache/metadata on a RamDisk was no faster than the NVMe storage, all whilst adding risk/complications.
The plan:
L1 - 8GB RAM
L2 - 12GB NVMe
- Should I now make both read+write and let PrimoCache manage everything?
- I don't understand the empty space part, but have left ~500MB at the end. Before I made changes, C: went all the way to the end
Re: Caching Options for Downloads Drive
Yeah I'd do read/write for both with that much ram, L2 might not even be necessary. Also it's good health for the SSD to leave maybe 10% of the drive unpartitioned you only seem to have 1% unpartitioned.
Re: Caching Options for Downloads Drive
Yep, agree with ferrari on your new changes. Switching from a 2GB L1 to 8GB will dramatically change your hitrate, so give it full read/write on both caches. I'd still do a L2 given the size of your data set (500GB) to keep hitrate high. And it's a NVMe, which means the L2 will give very snappy access times compared to a typical SSD.
The unallocated space is just overprovisioning to help with SSD life. You currently have it set to 1% of the drive space, but it's enough to help even a tiny bit. Typically it would be between 5% and 10% of drive space (5-10 GB).
Give it a test and see how it does, you should be pleased with the results.
The unallocated space is just overprovisioning to help with SSD life. You currently have it set to 1% of the drive space, but it's enough to help even a tiny bit. Typically it would be between 5% and 10% of drive space (5-10 GB).
Give it a test and see how it does, you should be pleased with the results.