l i n u x - u s e r s - g r o u p - o f - d a v i s
L U G O D
 
Next Meeting:
January 6: Social gathering
Next Installfest:
TBD
Latest News:
Nov. 18: Club officer elections
Page last updated:
2001 Dec 30 17:07

The following is an archive of a post made to our 'vox-tech mailing list' by one of its subscribers.

Report this post as spam:

(Enter your email address)
[vox-tech] Redirecting wget output to stdout?
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[vox-tech] Redirecting wget output to stdout?



Is there an undocumented redirect to stdout for wget? Or equally good, a
"bash"ful way to redirect (my bass book is at home)?

The 2 GB file size limit has reared its ugly head once more. The kernel
on the VA Linux 6.2.4 distro supports large files, but not the C
libraries. Thus, typical tools (with the exception of dd) still don't
work with large files. So far, I have been able to use ftp to download
smaller reference database files (all shy of 2 GB when compressed) and
using pipes and redirection to get around the size issue, such as:

	uncompress -c dbEST.FASTA.dailydump.Z | formatdb -i stdin -p F -o T -n
dbEST

Formatdb results in a compressed format ready for homology searching,
but I can also leave it off and create huge files (useful only in
showing the value of stdout/stdin redirects in this system).

Now the reference databases have to be downloaded in ~ 5 GB file chunks
and I am thinking of a pipe which starts with the download itself, such
as:

	wget -q (missing flag) ftp://server/file | gunzip -c stdin |
perlscript.pl

where perlscript.pl will recompress and distribute an equal portion of
the reference db to each node of the cluster and the reformatting can
occur off the master node.

-- 
Eric Engelhard


LinkedIn
LUGOD Group on LinkedIn
Sign up for LUGOD event announcements
Your email address:
facebook
LUGOD Group on Facebook
'Like' LUGOD on Facebook:

Hosting provided by:
Sunset Systems
Sunset Systems offers preconfigured Linux systems, remote system administration and custom software development.

LUGOD: Linux Users' Group of Davis
PO Box 2082, Davis, CA 95617
Contact Us

LUGOD is a 501(c)7 non-profit organization
based in Davis, California
and serving the Sacramento area.
"Linux" is a trademark of Linus Torvalds.

Sponsored in part by:
Sunset Systems
Who graciously hosts our website & mailing lists!