l i n u x - u s e r s - g r o u p - o f - d a v i s
Next Meeting:
September 2: Social gathering
Next Installfest:
Latest News:
Aug. 18: Discounts to "Velocity" in NY; come to tonight's "Photography" talk
Page last updated:
2009 Jul 31 14:45

The following is an archive of a post made to our 'vox-tech mailing list' by one of its subscribers.

Report this post as spam:

(Enter your email address)
Re: [vox-tech] Large File Server - File System?
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [vox-tech] Large File Server - File System?

Hash: SHA1

Alex Mandel wrote:
> So I'm putting in a new file server at work.
> RAID 6, 8 TB
> It's purpose is to server large files(1GB+) and large amounts of files
> to local desktops and servers.(About 25 machines and several web services)
> 1. What partition scheme would people recommend?
> Yes I read the whole previous thread on this topic.
> Here's what I'm thinking:
> /   10 GB
> swap - do we even need swap?(we have 12 GB of ram)

Having some is good, if you don't expect to swap having not much seems
reasonable.   I check 10 of our 16 disk servers that do various tasks and the
highest swap used I found was 3GB, even though vmstat was around zero.  If you
think you won't swap much I'd allocate 1 partition, if you think it might be a
performance limiting factor I'd add more.  If you are worried about uptime
keep in mind while linux will automatically use more than one swap partition
that you need to use a raid-1 if you want to survive a swap disk dying.

> /var 10-20 GB
> /tmp	40 GB
> /home Everything else?

If you are going to do /, /var, and /tmp in different partitions, I'd consider
 a LVM.  It might well save you some pain, downtime, or silly games like
making symbolic links from one partition to another.  Some folks enjoy
snapshots as well, pretty handy for trying out a major change and then
deciding to revert.  They can simplify backups as well.  There's advantages
and disadvantages either way.

How about:

Disk    A     B     C     D     E     F   G     H
p1      /     /     swap  /var  /var /tmp /tmp /tmp
p2      everything else

So with LVM:
/ = /dev/sda1 /dev/sdb1 raid-1
swap on /dev/sdc1
them use lvm to manage c1, d1, e1, f1, g1, h1 on whatever raid level you want.

Without lvm:
/ = /dev/sda1 /dev/sdb1  (10-15GB)
swap on /dev/sdc1
/var on d1 e1
/tmp on f1 g1 h1
raid-6 on /dev/sd{abcdefgh]2 (each as big as a disk - 10-15GB)

Pick the raid level for /var and /tmp as you see fit.

> 2. More importantly what file system should we use?
> ext3,ext4,XFS, etc?

For things that aren't actually a performance bottleneck I try to use the most
popular/most conservative decision.  I want as many people as possible to have
a problem before I do, ideally all the problems are fixed before I try it.
For that reason I'd pick whatever is default for your installer.  It's likely
to be heavily tested in a large number of environments.  Doubly so with a LTS
release.   For similar reasons I suggest their default partitioning as well.

Serving large files is rather easy, I'd be surprised if with any file system
you mention would be a bottleneck.  I'd expect it either in the disks (because
of seeks) or the network (which is about the speed of a single disk for
sequential reads).

> Note we'll probably run Ubuntu 8.04 LTS which may effect your answer.

I don't believe ext4 is available on 8.04 LTS.  Keep in mind that while you
can upgrade from ext3 to ext4 you lose some of the benefits of ext4.  As
mentioned I don't think this will be a bottleneck for your usage.  However,
the RAID-6 might well be.  Sure if a single user asks for 1GB and you have
GigE between the user and the file server he will get the better part of

Even if 4 users ask for the same file, the same thing.

Now if 4 users ask for 4 different files there's going to be quite a bit of
seeking, the raid-6 might well become the bottleneck.

To demonstrate, a local fileserver, using a raid-5 across 5 of 16 disks I get
around 280MB/sec for sequential reads.  With 2 running at once I get a total
of 120MB/sec, 4 around 100MB/sec, 6 around 73MB/sec.  I was playing with a
64GB slice at the beginning of each disk, if my 6 threads were spread across
the entire 1T disk I'd expect much lower.

On the flip side if I run 1 per RAID on 3 5 disk raid-5's I get 605 MB/sec.

So depending on what performance you'd like and how many simultaneous users
you have you may want to tweak your large raid-6.

Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

vox-tech mailing list

LUGOD Group on LinkedIn
Sign up for LUGOD event announcements
Your email address:
LUGOD Group on Facebook
'Like' LUGOD on Facebook:

Hosting provided by:
Sunset Systems
Sunset Systems offers preconfigured Linux systems, remote system administration and custom software development.

LUGOD: Linux Users' Group of Davis
PO Box 2082, Davis, CA 95617
Contact Us

LUGOD is a 501(c)7 non-profit organization
based in Davis, California
and serving the Sacramento area.
"Linux" is a trademark of Linus Torvalds.

Sponsored in part by:
Appahost Applications
For a significant contribution towards our projector, and a generous donation to allow us to continue meeting at the Davis Library.