Slightly OT: Web server q's - SCSI vs. IDE

[Mark Hughes]

On Tue, Oct 30, 2001 at 01:37:44PM -0000, Mark Hughes wrote:
>> Hi guys. I know this is somewhat off-topic, but I know you're a
>> knowledgable bunch of guys and hope you'll be able to give me some
valuable
>> opinions on this subject.
>
>> I'm still looking into our new server, and was wondering what peoples
>> opinions were on SCSI vs. IDE vs. bucket loads of RAM for a
database-heavy
>> web site running under FreeBSD....

> Bucketloads of RAM :-)

Yeah? cool... that should help then. I think the fun bit is going to be
tuning freebsd, apache, php and mysql to work to their maximum efficiency
on the hardware we end up with, whatever it is. I'm quite worried that
we'll get this hardware and not know properly how to set up FreeBSD to get
the absolute best out of it.

Currently we're doing about 40-55,000 hits per day on a vbulletin php/mysql
forum - and our cobolt raq is very nearly on the verge of not being able to
cope (we had the five minute load average up to 32 on it last night :+) you
could almost hear it scream - I was quite impressed that it didn't crash
though, it normally does when it gets that high).

It's not many hits, but it's killing the old processor/hard drive in the
RaQ. And personally I can't wait to get away from that godawful OS that
cobolt put on them...

>> From what I've heard in the past, the bottleneck with databases (we use
>> MySQL) is almost always the hard drive, so to that end, in terms of
>> performance, how much difference will having SCSI drives make over IDE?

> Depends how frequently you're hitting the disk, what the access patterns
> are like, and how you've layed out the filesystem.  RAID of some sort
> also makes a difference, as does the disks ability to do tagged queuing.

right. What's tagged queueing exactly? googling around a bit, I think I get
it, but is it a software or a hardware thing?

>> Also, would having shedloads of RAM (looking at 1GB initially, because
of
>> the stupid prices at the moment) make any difference to the IDE vs SCSI
>> equation? The entire of our database currently is only 100MB odd, is
there
>> any way to make MySQL cache *all* that data into RAM?

> FreeBSD will cache it on demand.

okay... Is there any way of forcing write-caching through the RAM? if the
system crashes or looses power, a minutes worth of forum posts isn't really
going to matter, so is there any way, at the OS level, to force DB writes
to be cached?

I know softupdates has this effect with metadata - does that affect all
writes or just those creating/deleting/moving files?

>> Likewise, would a vinum software RAID-0 or RAID-1 of two IDE drives make
>> them better than a single SCSI drive? How would two SCSI drives in a
RAID
>> fair?

> It really does depend on your access policy.  You make finger-in-the-air
> estimates (as I'm doing now), but you really need to benchmark to get
> the numbers and work out where the bottlenecks are.  vmstat, systat,
> iostat, and others are all useful in gathering the sort of data you
> need.

unfortunately, being as how we are currently on this evil RaQ server, it
doesn't have these - it's got vmstat, actually. Any tips for logging the
output of vmstat in some meaningful way over time?

I'd expect that fine tuning freebsd for our application we could make it
much more aggressive with it's use of memory, and I think the bottle necks
on our current server are most likely CPU (old K6-450 processor... awful)
and the ATA/33 HDD interface with single hard disc.

> <plug>
> Should you need a consultant to come in and set this sort of thing up
> for you, give me a shout.  My rates are reasonable, and this is
> bread-and-butter stuff for me.
> </plug>

i'd love to, but unfortunately we have very little cash.... (well,
actually, less than even that :+) ).

Thanks for your help.

Mark