L2 Cache Not Used When Needed

FAQ, getting help, user experience about PrimoCache
mell111
Level 5
Level 5
Posts: 48
Joined: Fri Oct 05, 2018 11:16 am

L2 Cache Not Used When Needed

Post by mell111 »

I have one cache task with an L1 cache of 16GB and L2 Cache of 96GB configured for an 8TB drive. Both L1 and L2 are write only (100% write) 256k block size, running Win10 1709 with 32GB RAM.

Most of the time the L1 cache is sufficient to keep up with the disk with Defer Write (Average, 60s), but I recently encountered a situation in which the disk was very slow (due to reading activity) for a little while. I noticed that even though the L1 cache filled up, there was no write activity at all to the L2 cache (0 writes). Instead, the writing app slowed to a crawl. To recover, I had to stop all read and write activity to the affected disk. After I did so, PrimoCache wrote out a few thousand deferred blocks in a few minutes and then the disk recovered. There were no error messages. After I resumed writing, this time without the reading that was slowing down the disk, everything was fine.

I expected that PrimoCache would start using L2 cache when L1 cache filled out and it couldn't write out to disk fast enough. Is that incorrect? Thanks.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: L2 Cache Not Used When Needed

Post by Support »

By current design, when L1 is full, L1 cache will flush certain amount of "dirty data" into disk, at the same time incoming new write-data will go to L2 cache. The problem here is 0 writes in L2 cache. Could you please upload a screenshot of your PrimoCache main dialog which showing the cache configuration and statistics? Thanks.
mell111
Level 5
Level 5
Posts: 48
Joined: Fri Oct 05, 2018 11:16 am

Re: L2 Cache Not Used When Needed

Post by mell111 »

PrimoCache-Stats.PNG
PrimoCache-Stats.PNG (22.85 KiB) Viewed 4687 times

I changed Average to Intelligent and 60s to 90s. It didn't make a difference. Otherwise everything is identical. Thanks.
cichy45
Level 4
Level 4
Posts: 38
Joined: Sun Oct 14, 2018 3:34 pm

Re: L2 Cache Not Used When Needed

Post by cichy45 »

I guess this might be similar issue that I reported here:

viewtopic.php?t=4674

Support said that this might be resolved in future update.
mell111
Level 5
Level 5
Posts: 48
Joined: Fri Oct 05, 2018 11:16 am

Re: L2 Cache Not Used When Needed

Post by mell111 »

I wonder. The following from Support...:

"By current design, when L1 is full, L1 cache will flush certain amount of "dirty data" into disk, at the same time incoming new write-data will go to L2 cache."

... doesn't qualify as applying only with multiple Volumes. Also, in the other thread the potential new mode appears to be writing to L2 data already written to L1, which I agree could be useful, but here it appears PrimoCache is trying to write new data directly to disk at the same time it is trying to flush data from L1 to the underlying disk even though it could easily direct the new data to L2 and avoid thrashing the underlying disk. Hopefully support will clarify.

By the way, I noticed in the above screenshot Total Write (L1/L2) 159.27GB/21.78GB implying writing to L2, but L2Storage Write is 0. In any case, the underlying disk continued to thrash until I stopped all new writing and reading to allow it to recover.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: L2 Cache Not Used When Needed

Post by Support »

Well, "L2Storage Write" and "Total Write (L2)" are indeed confusing.
"L2Storage Write" indicates the amount of data written to L2 Storage from populating read-data. This statistics is related to read caching.
"Total Write (L1/L2)" indicates the amount of data written to L2 Storage from caching write-data. This statistics is related to write caching.
So it is correct that "L2Storage Write" is 0 here because L2 cache is set to write-only. And the defer-write logic in L1 and L2 also works as designed as you can see that "Total Write (L2)" is 21.78GB, indicating handled 21.78GB incoming write-data.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: L2 Cache Not Used When Needed

Post by Support »

PS. In 2.x versions, L2 cache didn't support write caching. There was only the term "L2Storage Write" and no term "Total Write (L2)". So it didn't cause any confusion. Now in 3.x versions, L2 cache supports write caching and the term "Total Write (L2)" is introduced. However, we still keep the term name "L2Storage Write" for back compatibility.
mell111
Level 5
Level 5
Posts: 48
Joined: Fri Oct 05, 2018 11:16 am

Re: L2 Cache Not Used When Needed

Post by mell111 »

First, thanks for clarifying. There's still a problem:

When the disk is thrashing (100% active in windows task manager, *very* slow writing - fluctuating around 1 Mbps) even though there is plenty of both L1 and L2 cache space available (the deferred blocks is at less than 10000 which is about 2.5GB out of 14.75GB L1 and very little L2 written), the writing app is heavily throttled (writing well under 5 Mbps, fluctuating), as if there is no cache. When I pause the writing app (which also does some reading, but only a tiny fraction compared to writing), PrimoCache slowly catches up (several minutes), writing the deferred block and everything gets back to normal. There are no indications of any errors in the disk (checking SMART data) and the disk functions as you'd expect (full speed.)

An additional data point: I changed the mode to native, with 90s interval. While PrimoCache is writing out the deferred blocks, everything is fine - the writing app operates at full speed (30-50 mbps) even though the disk is 100% active writing at around 100 Mbps (which is normal). It's only when the disk appears to be thrashing that the writing app slows to a crawl even though PrimoCache appears to have plenty of cache space left.

I made sure there is no other activity in the underlying disk.

Is this expected (PrimoCache not being able to accept incoming data even with available cache space when it struggles writing to thrashing disk?)
mell111
Level 5
Level 5
Posts: 48
Joined: Fri Oct 05, 2018 11:16 am

Re: L2 Cache Not Used When Needed

Post by mell111 »

Adding: the thrashing I mention is almost certainly a result of the the minimal uncached reading the writing app still needs to do. It's very little, but once a disk starts to thrash, it's so slow that it takes a while to recover. If PrimoCache would temporarily back off writing when it recognizes extremely slow writing speed (when it has cache space available for incoming data), it should recover on its own. I say this because when I manually pause, it recovers.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: L2 Cache Not Used When Needed

Post by Support »

Did you see the "Urgent" writes (increasing) when thrashing?
Post Reply