Caching a portable drive
Posted: Sun Nov 11, 2018 11:24 am
Hello,
I am relatively new to PrimoCache, so I am seeking your expert opinions on a solution I am trying to implement.
I have a desktop and a laptop, each with internal SSDs. I only use one system at any given time - the laptop is for when I'm working away from my desk.
I like being able to pick up my laptop and have all my data with me. That is, *all* my data - documents, photos, Hyper-V machines, etc.
So, normally, I would:
• Keep my current, frequently-used, or performance-critical files on the internal SSDs. Everything else lives on a portable USB HDD that I switch between systems as needed.
• Sync the internal SSD data between the laptop and desktop somehow (e.g. OneDrive, scripts/utilities, or god forbid, manually). And this is a particular pain with large Hyper-V files which are typically locked by the host.
• Manually demote non-current data to the portable drive to free up internal space.
I want a better solution, and my idea is to store *all* my data on the external drive and use a very large L2 cache on the internal SSDs to bring the performance levels back up.
My goals are:
• Simplicity: Data all in one place, rather than split between internal and external drives. No need to manually demote data out to the external drive to free up space on the internal SSD. And a simpler backup profile as a bonus.
• Convenience: No need to go through the pain and delays of syncing data whenever I want to switch to the laptop or back again. Or having to remember to turn on the laptop in good time to allow background syncing to finish before I leave the house.
• Efficiency: No need for the constant CPU/bandwidth overhead of running a sync client like OneDrive (especially when I only switch systems infrequently - most of the time we are pointlessly re-syncing).
• Concurrency... Is NOT a goal. I have no desire or expectation to be able to access the data on both systems at the same time.
• Performance: I obviously cannot work at portable HDD speeds, hence the need for PrimoCache.
But, I have read all about offline modification. And offline modification is kind of the crux of this idea. It will, of course, occur every time I move the drive between the two machines.
So, on to the practical considerations:
I am interested in the "Volatile" setting applicable to the L2 cache. My understanding is that if the L2 cache is marked "volatile", then it is discarded on reboot. This would seem to allow "offline modification". Great...
...But I reboot far more frequently than I switch between machines, and there is no way that losing the entire L2 cache on reboot is going to be acceptable.
I was optimistic when I saw the "Gather" setting. Having read about it, I figured, great, PrimoCache has a strategy for automatically loading up my L2 cache in the background. That would reduce the pain of having a volatile L2 cache that is wiped on reboot. (I assumed this was the L2 equivalent of the L1 "Prefetch" setting.)
But I have tested, and I'm not sure what "gathering" is supposed to be happening, because my L2 cache remains empty even after a long idle period after boot. And changing the Gather setting has no effect either. Am I missing something here?
So, it would seem I have to set a non-volatile cache, and *explicitly remember* to reset the cache if I move the drive. And if ever I forget, I risk data corruption?! That doesn't sound acceptable either..
I could imagine crafting a little program that would display a prompt upon Windows shutdown to ask if I am going to unplug the portable drive and, if so, command PrimoCache to reset the cache. I haven't yet investigated whether that's feasible. Or, a little sticky tag on my USB cable ("REMEMBER PRIMOCACHE!") Yeah, you can see the "meh" factor.
So, I'm very curious to hear your thoughts, and I'm open to suggestions if I seem to be on the wrong track with this idea!
I am relatively new to PrimoCache, so I am seeking your expert opinions on a solution I am trying to implement.
I have a desktop and a laptop, each with internal SSDs. I only use one system at any given time - the laptop is for when I'm working away from my desk.
I like being able to pick up my laptop and have all my data with me. That is, *all* my data - documents, photos, Hyper-V machines, etc.
So, normally, I would:
• Keep my current, frequently-used, or performance-critical files on the internal SSDs. Everything else lives on a portable USB HDD that I switch between systems as needed.
• Sync the internal SSD data between the laptop and desktop somehow (e.g. OneDrive, scripts/utilities, or god forbid, manually). And this is a particular pain with large Hyper-V files which are typically locked by the host.
• Manually demote non-current data to the portable drive to free up internal space.
I want a better solution, and my idea is to store *all* my data on the external drive and use a very large L2 cache on the internal SSDs to bring the performance levels back up.
My goals are:
• Simplicity: Data all in one place, rather than split between internal and external drives. No need to manually demote data out to the external drive to free up space on the internal SSD. And a simpler backup profile as a bonus.
• Convenience: No need to go through the pain and delays of syncing data whenever I want to switch to the laptop or back again. Or having to remember to turn on the laptop in good time to allow background syncing to finish before I leave the house.
• Efficiency: No need for the constant CPU/bandwidth overhead of running a sync client like OneDrive (especially when I only switch systems infrequently - most of the time we are pointlessly re-syncing).
• Concurrency... Is NOT a goal. I have no desire or expectation to be able to access the data on both systems at the same time.
• Performance: I obviously cannot work at portable HDD speeds, hence the need for PrimoCache.
But, I have read all about offline modification. And offline modification is kind of the crux of this idea. It will, of course, occur every time I move the drive between the two machines.
So, on to the practical considerations:
I am interested in the "Volatile" setting applicable to the L2 cache. My understanding is that if the L2 cache is marked "volatile", then it is discarded on reboot. This would seem to allow "offline modification". Great...
...But I reboot far more frequently than I switch between machines, and there is no way that losing the entire L2 cache on reboot is going to be acceptable.
I was optimistic when I saw the "Gather" setting. Having read about it, I figured, great, PrimoCache has a strategy for automatically loading up my L2 cache in the background. That would reduce the pain of having a volatile L2 cache that is wiped on reboot. (I assumed this was the L2 equivalent of the L1 "Prefetch" setting.)
But I have tested, and I'm not sure what "gathering" is supposed to be happening, because my L2 cache remains empty even after a long idle period after boot. And changing the Gather setting has no effect either. Am I missing something here?
So, it would seem I have to set a non-volatile cache, and *explicitly remember* to reset the cache if I move the drive. And if ever I forget, I risk data corruption?! That doesn't sound acceptable either..
I could imagine crafting a little program that would display a prompt upon Windows shutdown to ask if I am going to unplug the portable drive and, if so, command PrimoCache to reset the cache. I haven't yet investigated whether that's feasible. Or, a little sticky tag on my USB cable ("REMEMBER PRIMOCACHE!") Yeah, you can see the "meh" factor.
So, I'm very curious to hear your thoughts, and I'm open to suggestions if I seem to be on the wrong track with this idea!