Journaling / recover write cache data from L2 cache

Suggestions around PrimoCache
Post Reply
Axel Mertes
Level 9
Level 9
Posts: 180
Joined: Thu Feb 03, 2011 3:22 pm

Journaling / recover write cache data from L2 cache

Post by Axel Mertes »

Hi Romex Software team!

We have discussed it a couple of times in the past, but I think its worth to give it special attention:

There is a huge need to consider implementing some kind of journaling L2 write cache in PrimoCache to be able to recover not yet flushed data from L2 SSD write cache to be finally written to target disk.

I know this is way more complex than it may sound in first place. I have put my head around this a couple of times and there are strategies to enable this. I do understand it looks quite complex in first place, but once its done it may end up being a true killer feature.

At best you would employ L1 write caching mainly for reducing L2 writes overhead by combining individual writes into SSD friendly block writes. Thats something you can already do, and which e.g. Condusiv Diskeeper / IntelliWrite is doing as well.

Once write cache data is written to L2 SSD write cache and once you are storing kind of a lookup table of that write cache data, to employ some kind of journaling algorithm to recover not-yet-flushed-write-data from a write cache (after power outage or BSOD), you would solve a big problem.

Most times its said that enabling write cache is risky, because you might end up with never written data in power outage or BSOD scenarios. However, I am very sure the opposite is true, because writes going to SSDs are performed MUCH FASTER than to HDDs. So its way more likely that a lot of data is written to SSD versus HDD, before the crash happens. In best case, all data has been written to SSD, while the HDD might still be in write back process. So SSD write caching might turn from being risky to being a live saver in some scenarios.

We can clearly discuss strategies on how exactly to implement this, but I just want to give it a big thumbs up from my side and hear others comments or concerns about this.

As an example:
Condusive Diskeeper / IntelliWrite is said to be absolutely risk free, but it IS DOING WRITE CACHING at the level of block collection to optimize for SSD writes. As this happens usually faster than writing block by block in the end, they say its not less secure, it is more secure than the "normal" operation approach from the OS in conjunction with writing data from any application. The same is IMHO true for PrimoCache, especially if data is quickly moved from L1 RAM write cache to L2 SSD write cache in conjuction with a journaling system.

What do you think?

Cheers
Axel
Last edited by Axel Mertes on Tue Nov 27, 2018 5:16 pm, edited 1 time in total.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Journaling / recover write cache data from L2 cache

Post by Support »

Hi Axel,
Thank you very much for the suggestion!
Journaling system is one of ways that we're thinking and studying for the L2 recovering. L2 recovering is quite complicated together with current caching design and we have to take more time.
rockingdemon
Level 1
Level 1
Posts: 1
Joined: Tue Jul 16, 2019 2:49 pm

Re: Journaling / recover write cache data from L2 cache

Post by rockingdemon »

Axel Mertes wrote: Tue Nov 20, 2018 7:16 am Most times its said that enabling write cache is risky, because you might end up with never written data in power outage or BSOD scenarios. However, I am very sure the opposite is true, because writes going to SSDs are performed MUCH FASTER than to HDDs. So its way more likely that a lot of data is written to SSD versus HDD, before the crash happens. In best case, all data has been written to SSD, while the HDD might still be in write back process. So SSD write caching might turn from being risky to being a live saver in some scenarios.

Cheers
Axel
Hello, Alex, and Romex Software Team. I'm again on this feature.
Alex has the point.

Writeback should be implemented with the following strategy (or as an option):
1. Upon write request from software we write to SSD and immediately ACK to the system, marking this block as dirty.
2. On gather interval we scavenge all dirty blocks, REORDER them ascending and send to slow device as an almost linear write.
3. When blocks are written back to slow media we can mark them as clean or even discard them (depends on policy).
4. Nothing special should be done on crash - we just read from SSD. So we even don't need to discard L2 cache.
5. Memory (L1) must not be used as a temporary storage to avoid crash data loss. (however, L1 could perfectly be used with writethrough policy. I mean we can save both to SSD and memory on p.1).

This mode is implemented in Linux bcache and works like a charm. The only difference is that bcache use its own fs label, so you just can't write accidentally to the backed file system bypassing cache driver - maybe there should be an option in PrimoCache too?
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Journaling / recover write cache data from L2 cache

Post by Support »

@rockingdemon, thank you for the suggestion. We did study many other implementations including bcache.
Post Reply