l i n u x - u s e r s - g r o u p - o f - d a v i s
L U G O D
 
Next Meeting:
April 21: Google Glass
Next Installfest:
TBD
Latest News:
Mar. 18: Google Glass at LUGOD's April meeting
Page last updated:
2010 Apr 14 23:56

The following is an archive of a post made to our 'vox-tech mailing list' by one of its subscribers.

Report this post as spam:

(Enter your email address)
Re: [vox-tech] Performance tuning for http file serving
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [vox-tech] Performance tuning for http file serving



On 04/14/2010 11:23 PM, Brian Lavender wrote:
>> What gets ugly is if you have 2 or more clients accessing 2 or more
>> files.  Suddenly it becomes very very important to intelligently handle
>> your I/O.  Say you have 4 clients, reading 4 ISO files, and a relatively
>> stupid/straight forward I/O system.  Say you read 4KB from each file (in
>> user space), then do a send.
>
> That's like operating a disk in PIO mode. I would think something that
> can leverage the DMA controller better would avoid thrashing.

DMA controllers make it easier on the kernel, but it doesn't change the 
nature of random I/O limiting performance.

> Perhaps
> there is a way to tell apache to use memory cache and dump readaheads
> into memory. Linux should be able to handle this fine.

I didn't see anything.

> I was looking to see if there is a config on this, but I can't find one
> at the moment. I was looking at this cache page.
>
> http://httpd.apache.org/docs/2.2/mod/mod_mem_cache.html

Yeah, I looked around, it seemed to be rather focussed on having objects 
smaller than cache.

If you want to quantify the problem I'd just download a 4GB file and 
measure the bandwidth.  Then access enough files so that they sum to 
twice as big as ram.  Say 4 4GB files if you have 8GB ram, or 8 4GB 
files if you have 16.  Launch a client each, then when they get half way 
done fire another download for each file.

If you are saturating the network, or at least within 75% of so it 
sounds like linux is prefetching and/or using a large enough transfer 
size so that random I/O isn't killing performance.

If you are at 5% or less then I'd continue looking for some tuning.

> I imagine someone else has encountered this problem before.

I seem to recall some GigE connected freebsd box that was one of the 
busiest ftp servers in the world working on this kind of thing.  I saw a 
few published articles on the software, hardware, and kernel tuning.  I 
look around for apache tuning for large file transfers and didn't really 
find much.  Maybe it's not a problem.


>
> brian

_______________________________________________
vox-tech mailing list
vox-tech@lists.lugod.org
http://lists.lugod.org/mailman/listinfo/vox-tech



LinkedIn
LUGOD Group on LinkedIn
Sign up for LUGOD event announcements
Your email address:
facebook
LUGOD Group on Facebook
'Like' LUGOD on Facebook:

Hosting provided by:
Sunset Systems
Sunset Systems offers preconfigured Linux systems, remote system administration and custom software development.

LUGOD: Linux Users' Group of Davis
PO Box 2082, Davis, CA 95617
Contact Us

LUGOD is a 501(c)7 non-profit organization
based in Davis, California
and serving the Sacramento area.
"Linux" is a trademark of Linus Torvalds.

Sponsored in part by:
Sunset Systems
Who graciously hosts our website & mailing lists!