A few quick questions before I install? Topic is solved

[fattipants2016]

I hope someone can answer a few quick questions for me before I install and try PrimoCache.

My current configuration is a single 250GB SSD, and 6x8TB archive disks pooled with DrivePool.

I'm considering migrating my OS to a 1TB HDD and dedicating the 250GB SSD as L2 cache for all other disks in the system.

My questions are 3:

1.) If a file is downloaded to disk A and moved to disk B (where A and B share L2 cache) is the move instant, as if on the same disk?

2.) Will primocache store folder contents, so disks don't have to spin up just to browse folders?

3.) How does the cache behave when disks are deframented? Is every block that's moved during defrag written to cache or is it smarter?

I appreciate any guidance in advance. It's going to require a considerable amount of time to configure, so I'd like to know what to expect.

[Jaga]

Hey fattipants - good to see you here.

1) That's an excellent question, one I had trouble trying to answer without testing. So.. I went and tested. I setup a 465GB L2 against my C: (4KB sectors) and Y: (64KB sectors) drives, and disabled the L1 that was already in the Cache Task to get as clear a result as possible. I then copied a 24GB .mkv file to C: with the L2 active, verifying via Primocache's GUI that the L2 was being filled. Because I have deferred writes on (I use a UPS), I noted the deferred writes statistic climbing while the initial copy to C: was happening.

I then forced a flush of those deferred blocks, and waited until it finished (so the data was actually on C:), and double-checked the free L2 cache space to make sure it still held the file (which it did). At this point, the file is physically on C:, and in the L2 cache as read-cache-contents. The Cache Task was setup as read & write, with 16KB blocks. Moving the file from C: to Y: forced Primocache to duplicate the file inside the L2 - I can only speculate that it had something to do with all the different sector/block sizes. It did not happen instantly in this test case.

So, I went and re-formatted Y: to 4KB sectors, and re-created the Cache Task with a 4KB block size (so all three items were using 4KB unit sizes), and re-tested. Knowing that the large file to move wasn't in the L2, I flushed the cache contents and re-copied it to C: so it populated the L2 cache. I then flushed deferred writes, and proceeded to move the file to Y: again. And again - it did not move the file instantly, though it did max out the L2 SSD's transfer rate.

So it appears blocks are uniquely independent inside the L2 (and L1) caches for each drive, and transfers will not be instant. However had I put a sizable L1 on this Cache Task (that could hold both copies of the file), it would have duplicated the file at the fastest speed RAM could handle.


2) Not by design. That is - Primocache works at the block level, each block holding x number of sectors on the drive. It's theoretically possible that it could hold the entire MFT and deliver that content without spinning the drive up, but I've never browsed a sleeping drive without it waking up first.


3) I actually use a large L1 (RAM) cache (with deferred writes) to help defragment spinner disks more quickly. Every time a block of sectors is read/moved/written to a temp area on the disk, and then re-read/re-moved/re-written, Primocache can defer that set of operations so that there's less disk thrashing and "trims" the repeated writes.

The only downside, is that it won't natively know you're defragmenting, and so it will start filling the cache (if there's space) with every file it accesses. It will of course, still "weigh" the most frequently accessed files so that some of your normally-cached content stays in there. But the overall effect lowers hitrate and dumps some of the cache that would normally be filled with file contents you'd used before.

If you truly want the cache to stay full of the things you use most, pause the Cache Task prior to a defragment, then when done resume the Cache Task. Due to the relocation of certain files, if they were previously in the cache they will have been removed from it since Primocache was paused during the defragment. Those blocks/sectors will have been cleared during the move.


If you find you want any help or tips setting up Primocache, feel free to ask.

[Jaga]

I'm considering migrating my OS to a 1TB HDD and dedicating the 250GB SSD as L2 cache for all other disks in the system.

One thing about this - you may want to consider leaving your OS on the 250GB SSD, and getting a new 1TB SSD for the L2 cache against your 48TB worth of archive disks. The better a % of data coverage you have with the L2 cache, the better your hitrate is going to be (and also the write cache space). a 250GB L2 trying to cache even just 50% used space (24TB) means only 1% of a half-full array would be cached at any one time. On the flipside, a 1TB L2 caching 24TB of data would mean a little more than 4% coverage on the array at half utilization.

If you aren't concerned as much with read caches and are more interested in just the write cache however, a 250GB L2 would be adequate.

[fattipants2016]

Hey fattipants - good to see you here.

Holy smokes!

It's going to be morning before I can digest all of that, but I wanted to pop in and thank you for all the effort!

From what I gleaned from the initial skim, that's about what I expected, since it works at block level.

I thought perhaps it would realize the hashes of blocks being read from disk A were the same as those written to disk B, but didn't expect it to.

I plan to give it more careful consideration tomorrow, but based on what I've read so far I think I'll wait until I build my next box.

I'm using a small form-factor PC as a media server which limits me to 1 x 2.5" SATA and 1 x MSATA, but building a tower is on my short-list.

Thanks again for all the help.

[Jaga]

Most welcome.