More than 2 levels of caching, and RAID

FAQ, getting help, user experience about PrimoCache
Post Reply
Barza
Level 1
Level 1
Posts: 2
Joined: Fri May 17, 2019 5:07 pm

More than 2 levels of caching, and RAID

Post by Barza »

Hi. I have some questions regarding a multiple caching scheme and RAID.

Primocache caches a "slow" drive (eg. SATA) to a "faster" drive (eg. SSD as L2 cache) and then to even faster volatile storage (eg. DDR as L1 cache).

Can you add an additional layer of caching in this process? So for example:
SATA >> SSD >> NVME >> DDR

Let's say you have multiple TBs of data (games for exmaple) stored on a SATA disk, a spare SSD, an M2 NVME drive with a spare partition on your motherboard, and plenty of RAM. Could you set up caching task 1 to cache the SATA to SSD (L2 only, no L1 cache); and then caching task 2 to cache the SATA drive to the NVME as L2 cache and some DDR as L1 cache? If you did so, would you get the multiple levels of caching described above?

A final question. Let's say you have multiple SATA drives, or multiple SSD drives. Either using an intel or other controller, you set these up as a SATA Raid 0 array, and also set the SSD's as a raid 0 array. Each array is faster than if drives are used individually, (although the striping of data across multiple disks increases exposure to hardware failure). If this is all done via hardware controllers, could you then set up Primocache so the sequence was:
SATA RAID 0 >> SSD RAID 0 >> NVME >> DDR

The reason I ask is that data in question is games, of which i have about 5TB and rising. That can all be redownloaded in the event a RAID array went bad. So I have been toying with the idea of the above given that i have the hardware to do it.

All input welcome, and thanks in advance.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: More than 2 levels of caching, and RAID

Post by Jaga »

I haven't heard any talk about implementing an "L3" caching tier, but perhaps Romex has considered it now that we have many different (and faster) layers of hardware to take advantage of. SSD, NVMe, Optane, and so forth could be implemented in this fashion if the software supported it. It might add a bit too much overhead however, and add latency where it isn't desired.
Barza
Level 1
Level 1
Posts: 2
Joined: Fri May 17, 2019 5:07 pm

Re: More than 2 levels of caching, and RAID

Post by Barza »

Hi Jaga

It would be nice to have "L3" explicitly supported. But i was wondering how the software works as already designed if you set up multiple caching tasks.

Cache task 1 caches a SATA disk to ssd as L2. Cache task 2 then caches the same SATA disk to NVME as L2 and DDR as L1.

If you set that it up like that, would task 1 "nest" within the second one, hence when task 2 wanted data from the SATA disk, it would get it from the SSD if cached there be task 1?

Or does Primocache prevent multiple cache taks accessing the same disk and set of files?
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: More than 2 levels of caching, and RAID

Post by Jaga »

I think the SATA disk in question (having been cached in Task 1) would be an invalid target, since it was already spoken for. Primocache has to be in control of all the writes to the target volume, and separation between tasks would throw a monkey wrench in there I'm sure, even if you could set it up.

But a 3-tier caching scheme wouldn't be out of the question - Support would know more if the overhead would be too high, or efficiency too low to consider adding the feature. Some processors actually use 3 tiers of caching, so it's not entirely as inefficient as one might think. RAM -> NVMe/Optane -> SSD -> Disk might be viable. I just don't see how it could currently be done.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: More than 2 levels of caching, and RAID

Post by Support »

Thank you for the suggestion. I'm sorry that so far we don't have the plan to implement a 3-tier caching scheme, considering the overhead and efficiency. PrimoCache supports RAID (either hard RAID or soft RAID) as level-2 cache, so if you have multiple spare SSD/NVMe drives, you may make them a RAID and use it as level-2 cache.
Barza wrote: Sat May 18, 2019 10:23 am does Primocache prevent multiple cache taks accessing the same disk and set of files?
Yes, one volume can only be cached by one cache task.
theolu
Level 1
Level 1
Posts: 1
Joined: Wed Jul 31, 2019 2:06 pm

Re: More than 2 levels of caching, and RAID

Post by theolu »

With regards to this, is it possible to have a NVME drive be regarded as the 'RAM' portion of what would've otherwise been a RAM>SSD>HDD scheme?

This on its own could already, potentially, allow for a significant increase in relative speed whilst not relying on the use of actual system RAM for the fastest tier of storage.

Sorry for the thread necro, but I was considering precisely what Barza was thinking of doing in that I'm about to build a system consisting of a 512GB NVME with a 240GB SSD and 2TB HDD, so a 20GB/240GB/2TB NVME/SSD/HDD scheme seemed like it could work out quite well for my use if it'd be supported.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: More than 2 levels of caching, and RAID

Post by Jaga »

No way that I'm aware of theolu. You could however just dedicate the entire SSD as L2 for the HDD, and use the RAM to L1 cache the NVMe (two separate cache tasks).
libranskeptic612
Level 1
Level 1
Posts: 3
Joined: Tue Jun 02, 2020 9:33 am

Re: More than 2 levels of caching, and RAID

Post by libranskeptic612 »

Just sayin, but when raid1 or raid 10 reads from the least busy mirror copy, its kinda adding a tier of cache?
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: More than 2 levels of caching, and RAID

Post by Support »

libranskeptic612 wrote: Tue Jun 02, 2020 11:42 am Just sayin, but when raid1 or raid 10 reads from the least busy mirror copy, its kinda adding a tier of cache?
Yes, it is. However, this performance boost is limited as you have known that a HDD RAID is still much slower than a fast nvme SSD.
Post Reply