Stripe sizing?

[Panic_MD]

Can anyone here tell me what stripe size i should use for my raid setup, i'm going for a raid 0, i'll be using it mostly for gaming. maybe...in the future for video recording. basicly i'm looking for speed n' proformance. intel has suggested using 16k to 128k for stripe sizing, but i think there more to that, thanks.

 

[DelJo63]

HD performance; lots to be considered.

At the top, you want maximum cluster utilization with minimum last block free space.
In a Raid-0 configuration, read cluster == stripe size.

So, how do you set 'an appropriate stripe size'?

It goes like this. KNOW THY DATA!
you need to know, How Frequently Do I Access Each File? Fq
how big is each file? Fs
here's the hardest, is the file updated or totally replaced? UR
...{a save as without a name collision is a new file, }
...{whereas a name collision is a delete followed by a new file}
...{save is sometimes an update in place}

Performance tuning is always 'an exercise', so be patient with yourself.

To get the Weighted Average data Transfer,
.... Fq X Fs == Effect of Data Transfer
a small file updated daily: 1/day x 1024 x 365 days/year = 372,735/year
a smaller file updated hourly: 24/day x 512 x 365 days/year = 4,485,120/year
a large file updated weekley: 1/wk x 2048 x 52weeks = 106,496/year

add'em and divide by the number of files == WA
WA/365 will be the average I/O traffic per day (in bytes) == IOD

divide IOD by the average number of files / day == avg file size / day AFSD

divide AFSD by {powers of 2} x 1024
eg: 2k, 4k, 8k, 16k, .... 256k
this will be the effective I/O buffer count required to transfer the AFSD workload.
Keep the fractional part as large as possable, eg:
15.9 is a better number that 14.1 (large cluster loss).

Now if this is just a tad too much (Yes I've had to do this for a client puke: ),
take the default. :angel:

 

[nein]

Can anyone here tell me what stripe size i should use for my raid setup, i'm going for a raid 0, i'll be using it mostly for gaming. maybe...in the future for video recording. basicly i'm looking for speed n' proformance. intel has suggested using 16k to 128k for stripe sizing, but i think there more to that, thanks.

Actually it's even simpler than you'd have expected.

As you'll be using Intel hardware, despite the fundamental Distributed Processing nature of RAID hardware, your Intel hardware will need you to set/optimize it for Symmetric Processing - AKA single big humongous chunks.

Which meant you set your stripe size to the largest block size possible without exceeding the maximum bandwidth for the symmetric sequential processing time slice duration. You'll run into data corruption if you did exceed it.

You can ignore the parameters jobeard mentioned above...

One - Most Joe Consumer User will not have the established historical usage data.

Two - They applied fully only to Distributed Processing hardware designs, Distributed Processing hardware required proper bandwidth management for maximum efficiency. Conventional Symmetric Processing hardware had very little use for them, they are not based on bandwidth usage but the maximum data rate in the largest time slice duration (maximum clock rate per time period duration).

If you had AMD64 series hardware instead then based upon the performance profile data below, your best stripe size usage is 64K.

 

[Panic_MD]

Okay..with that said, i'm using a P4 478 socket, i've setup my raid n' it's running as i type, i chosen the 128k stripe size, will that be good for my system? keep mind intel setup won't let me choose no other than the size what they have defaulted.

 

[DelJo63]

give it a buzz and see how it performs. I would opt SMALLISH
to reduce buffer sizes and latency times. My guess would have been toward
smaller sizes, 32-64k, but that's just a guess.

 

[nein]

Okay..with that said, i'm using a P4 478 socket, i've setup my raid n' it's running as i type, i chosen the 128k stripe size, will that be good for my system?

As long as you never encounter data corruption then you are fine, if you do run into them then the drives aren't quite good enough to keep up and you'll need to lower the stripe size down a notch one way or another, unless you can get better drives.

keep mind intel setup won't let me choose no other than the size what they have defaulted.

You can't fault Intel for not knowing what is best for their hardware.

 

[Nodsu]

Uhm.. What has Linpack got to do with disk performance? And even i it had, I don't think Panic_MD will be solving scientific issues with his computer. If you are thinking about swapping the Linpack data then there are things like caches that make your reasoning moot since the graphs will be totally different for every hardware configruation and task. Something like a real life normal use graph would be a lot more useful here.

If you people are going to plot fancy graphs and complex formulas then it would be a good idea to provide some background material to back it up. I am very interested in the subject because of my work and I would apprciate some serious links

 

[nein]

Uhm.. What has Linpack got to do with disk performance? And even i it had, I don't think Panic_MD will be solving scientific issues with his computer. If you are thinking about swapping the Linpack data then there are things like caches that make your reasoning moot since the graphs will be totally different for every hardware configruation and task. Something like a real life normal use graph would be a lot more useful here.

You lack of comprehension here is the exact same lack of comprehension HERE. Linpack_Performance and Disk_Performance, plus Video Streaming_Performance in the old thread all have one thing in common - performance throughput directly related to data chunk_size/block_size/stripe_size.

Intel's time based Symmetric Processing designs are best performing with one single big humongous chunk of data at a time whatever the processing are supposed to be (video, disk or linpak), AMD's bandwidth based Distributed Processing designs are best performing with multiple small data chunks at a time whatever the processing are supposed to be (video, disk or linpak).

Intel's hardware is made for Single-Tasking, AMD's hardware is made for Multi-Tasking.

If you people are going to plot fancy graphs and complex formulas then it would be a good idea to provide some background material to back it up. I am very interested in the subject because of my work and I would apprciate some serious links

The above only require comprehension and intelligence, not backing up.

"I've 20 respected sources, 100 weblinks, and a 3DFx's engineer who said V5's FSAA didn't blur" <--- This isn't backing up, it's only more ignorant dumb****s with important opinions.

"Jittered blending of adjacent pixels caused bluriness in V5's FSAA" <--- This required no backing up, only comprehension and intelligence. If neither available then more time passing by will give better result.

 

[DelJo63]

...your Intel hardware will need you to set/optimize it for Symmetric Processing - AKA single big humongous chunks.

Do you imagine a high population of home users with SMP hardware?
Not very like today, but with dual core chips hitting the market, maybe 5-10 years.

 

[nein]

Do you imagine a high population of home users with SMP hardware?
Not very like today, but with dual core chips hitting the market, maybe 5-10 years.

Not everyone have SMP, only a few relative to the whole. But many have had Distributed Multi-Processing, they simply didn't recognize what they had, he he... they can't even recognize plain old Distributed Processing much less Distributed Multi-Processing.

 

[DelJo63]

I would have posted a Private Message, but Nien has it disabled :blackeye:

..they can't even recognize plain old Distributed Processing much less Distributed Multi-Processing

Commercial sites with a Domain Server, network login, running Lotus Notes or
Exchange Server... yea. But home users with a 3 system lan running File and Print sharing, you got to be kidding.

btw: I did Systems Analysis for Tandem Computers, one of the biggest in
fault tollerant, distributed multi-processing systems so others have some background too.

Please, let's desist from the king of the mountain game and realize that there
is always more than just one answer. you want to flame me, ok, but use the
PM feature and let's go out back and leave the party in peace.

 

[Panic_MD]

Please no fighting. close the thread, we are all friends here.

 

[nein]

I would have posted a Private Message, but Nien has it disabled :blackeye:

Commercial sites with a Domain Server, network login, running Lotus Notes or
Exchange Server... yea. But home users with a 3 system lan running File and Print sharing, you got to be kidding.

btw: I did Systems Analysis for Tandem Computers, one of the biggest in
fault tollerant, distributed multi-processing systems so others have some background too.

Please, let's desist from the king of the mountain game and realize that there
is always more than just one answer. you want to flame me, ok, but use the
PM feature and let's go out back and leave the party in peace.

Flaming you? That's your imagination working overtime.

 

[DelJo63]

Please no fighting. close the thread, we are all friends here.

that's my intent also

 

[Nodsu]

The fact that someone even mentions CPU as some sort of a bottleneck or a factor in disk subsystem performance is ridiculous enough to delete this thread forever as it shines bad light on this community..

Dear nein, even a 500MHz P3 CPU has more bandwidth than any consumer level RAID setups can ever hope to achieve. You might as well say that if i get a faster CPU I will be able to type faster since the computer is able to read more characters from the keyboard buffer.

Thread closed.