Primocache for diskless program (ccboot)

FAQ, getting help, user experience about PrimoCache
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: Primocache for diskless program (ccboot)

Post by InquiringMind »

Jaga wrote:...your budget of 1k is VERY low to accomplish the client-caching strategy. With 30 clients, you'd run most of your budget out just getting 30 copies of PrimoCache, which leaves no room for additional RAM on those clients (which Primo needs), or local SSD drives (to help with local L2 caching to relieve server/network stress)...
Good point - I overlooked the budget limitation. And if this is for business use, then that requires the Server edition of PrimoCache at US$119 per copy - it's very likely that Romex would offer some (and possibly a significant) discount for a 30-license package, but I couldn't see this costing less than $1K.

Nonetheless, it may be worth contacting Romex business sales (email link on the purchase webpage) to see what they can offer as well as experimenting with other options (the only other piece of software I've come across offering similar functionality would be eBoostr, which was last updated in 2010). It may also be worth checking your network bandwidth with a few game-related tests (like timing level loads) to ensure that it is giving maximum bandwidth (enabling jumbo packets may or may not improve matters). If your server has multiple network connections (e.g. dual Gigabit) and your switch/hub supports link aggregation (e.g. 802.1AX) then you can use this to effectively double your server's connection bandwith.

Aside from that, improving the server setup as Jaga has suggested should be a good second choice.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Primocache for diskless program (ccboot)

Post by Support »

Jaga wrote:If you're absolutely stuck with the $1k budget, then I'd recommend a larger RAID 1 SSD drive pair on your server (to give redundancy + speed for reading), along with a copy of PrimoCache and as much RAM as you can afford to drop in the server (speed is negligible for the RAM).
Yes, as Jaga recommended, if you have the budget restraints, a larger SSD cache + PrimoCache shall work for you. Recently we tested the Optane SSD 900P and find it do have the ultra-high read speed with very high endurance. I think it's a good option for this case. See viewtopic.php?f=33&t=4531
cemara
Level 3
Level 3
Posts: 13
Joined: Sun Feb 25, 2018 12:58 am

Re: Primocache for diskless program (ccboot)

Post by cemara »

support wrote:
Jaga wrote:If you're absolutely stuck with the $1k budget, then I'd recommend a larger RAID 1 SSD drive pair on your server (to give redundancy + speed for reading), along with a copy of PrimoCache and as much RAM as you can afford to drop in the server (speed is negligible for the RAM).
Yes, as Jaga recommended, if you have the budget restraints, a larger SSD cache + PrimoCache shall work for you. Recently we tested the Optane SSD 900P and find it do have the ultra-high read speed with very high endurance. I think it's a good option for this case. See viewtopic.php?f=33&t=4531
Can't seem to open the thread.
Sorry for the late reply, i've decided to look into this again after some people mentioning primo compared to intel optane.
I think i have Samsung's 250 gb NVME laying around from my previous project, would that be a good choice?

Also i contacted ccboot technician and he said trhis
random, latency are the relevant benchmark values for gamedisk.
Big game files benefit from sequential, but small files also benefit from 4k

Thinking of using this as setup :
12 TB Gamedisk + 250 gb nvme as cache.
Would this be a good idea?

Also, do i need minimal ram mhz ? cause i will have to plan ahead the build since ram mhz is limited by cpu
oh and i think i'm not gonna need it on client side, clients are already using 2999 Mhz ram now. The bottleneck seems to be on gamedisk
cemara
Level 3
Level 3
Posts: 13
Joined: Sun Feb 25, 2018 12:58 am

Re: Primocache for diskless program (ccboot)

Post by cemara »

From what ive gathered by reading the forums (CMIIW)
1. L2 caching doesnt need ram to be effective
2. L1 caching requires high mhz ram to be effective
3. Effective Caching size depends on size of games frequently accessed > for my case, the biggest one iirc is ~40 gb

Judging from these facts, im gonna try l2 caching on 12 tb + 240gb nvme 960 and see how it goes..
Higher mhz ram requires me to invest more systems, and it'll cost roughly 1500k at least

Just a question on how caching works, assuming im allocating 100 gb for caching and lets say game A is 60 gb and is often accessed, game B is 30 gb game C is 40 gb, so what happens when i have clients on different machines playing all 3 games?
Which one benefits from the cache?
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: Primocache for diskless program (ccboot)

Post by Jaga »

L1 caching benefits very little with faster RAM. Just put in the largest amount of identical ram your board can hold and dedicate a lot of it to the L1 cache. I overclocked my RAM significantly and saw very little (2%) benefit. So slower RAM will work just as well in your case.

A L2 SSD should work well for you, though a large L1 and large L2 would work better. Primocache knows enough to populate the L1 with more frequently accessed blocks, so the most used data that can fit into the L1 would stay there. You could then have the SSD handle all the writes (make the L1 read-only cache) for efficiency. If writes aren't a big deal, make both caches read only.

The cache doesn't work on entire games/folders, it works on the block level. Whatever the client system reads is what Primocache is going to pull into the L1/L2. IF that is the entire game, and the L1/L2 could handle holding all three games, all three would benefit.

More importantly - your most accessed games (their frequently read data anyway) are what end up residing the cache.

If you guesstimate you're going to have ~20 popular games averaging 10 gig each that get loaded daily, you can figure your cache needs to be at least 200 gig. If you use that 240gb NvME you are just -barely- covered. I would still look into adding more similar-speed RAM (don't replace it with fast RAM, that's a waste) and making a large L1 if you can (read-only).
cemara
Level 3
Level 3
Posts: 13
Joined: Sun Feb 25, 2018 12:58 am

Re: Primocache for diskless program (ccboot)

Post by cemara »

Jaga wrote:L1 caching benefits very little with faster RAM. Just put in the largest amount of identical ram your board can hold and dedicate a lot of it to the L1 cache. I overclocked my RAM significantly and saw very little (2%) benefit. So slower RAM will work just as well in your case.

A L2 SSD should work well for you, though a large L1 and large L2 would work better. Primocache knows enough to populate the L1 with more frequently accessed blocks, so the most used data that can fit into the L1 would stay there. You could then have the SSD handle all the writes (make the L1 read-only cache) for efficiency. If writes aren't a big deal, make both caches read only.

The cache doesn't work on entire games/folders, it works on the block level. Whatever the client system reads is what Primocache is going to pull into the L1/L2. IF that is the entire game, and the L1/L2 could handle holding all three games, all three would benefit.

More importantly - your most accessed games (their frequently read data anyway) are what end up residing the cache.

If you guesstimate you're going to have ~20 popular games averaging 10 gig each that get loaded daily, you can figure your cache needs to be at least 200 gig. If you use that 240gb NvME you are just -barely- covered. I would still look into adding more similar-speed RAM (don't replace it with fast RAM, that's a waste) and making a large L1 if you can (read-only).
I forgot to mention gamedisk is read-only,
ccboot uses separate drive for write purpose from clients, it's called writeback drives, and i've assigned an ssd for writeback drive
Hmm guess i'll try it first to see how much is needed.
The current ram is only 32gb ddr3 1333 mhz though, would that be good enough?
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: Primocache for diskless program (ccboot)

Post by Jaga »

Yep, a L1 on 1333mhz RAM is still going to be faster than an off-the-shelf SSD (and might still be faster than the NvME). 64gig would be better, but I know there are cost limits.

You can try the setup with a L1 and L2, and then disable the L1 for a couple days and see how just the L2 does by itself. If you aren't satisfied, turn the L1 back on. :)

Definitely configure both for Read-only, since that's your setup with the game disk.
cemara
Level 3
Level 3
Posts: 13
Joined: Sun Feb 25, 2018 12:58 am

Re: Primocache for diskless program (ccboot)

Post by cemara »

Sorry for late reply, i am now testing the server version to cache the gamedisk,
256 samsung nvme L2 with 10Gb L1 to cache 12tb gamedisk (effectively 10tb)

Mode is set to read ssd only, because gamedisk only needs to be read, not written
WIll monitor it for a few days to see if it's better than ccboot's caching system.
I used ccboot's cache before and seems like the lags still occurs on clients.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: Primocache for diskless program (ccboot)

Post by Jaga »

266GB L1+L2 to cache 10TB means you're effectively covering 2.6% of the disk, but how full is it? Divide 266 by the number of gigs of data you have to find real coverage percentage. If it turns out not to be fast enough most of the time, you might have to double the size of the L2 drive (or just replace it with a 512GB or 1TB SSD which easily saturate most networks anyway). My next server upgrade will have a 1TB L2 SSD covering a 48TB pool, but the actual data size is much smaller so it covers more.

Will be interesting to see your results!
cemara
Level 3
Level 3
Posts: 13
Joined: Sun Feb 25, 2018 12:58 am

Re: Primocache for diskless program (ccboot)

Post by cemara »

Hmm previously ccboot cache only fils like 100gb IIRC,
But i'm adding more AAA games so it might increase even more.
Forgot to mention, i'm using quad-nic which is load balanced, although the switch and the client is only 1Gb of course,
Post Reply