l i n u x - u s e r s - g r o u p - o f - d a v i s
Next Meeting:
July 7: Social gathering
Next Installfest:
Latest News:
Jun. 14: June LUGOD meeting cancelled
Page last updated:
2001 Dec 30 17:07

The following is an archive of a post made to our 'vox-tech mailing list' by one of its subscribers.

Report this post as spam:

(Enter your email address)
[vox-tech] Redirecting wget output to stdout?
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[vox-tech] Redirecting wget output to stdout?

Is there an undocumented redirect to stdout for wget? Or equally good, a
"bash"ful way to redirect (my bass book is at home)?

The 2 GB file size limit has reared its ugly head once more. The kernel
on the VA Linux 6.2.4 distro supports large files, but not the C
libraries. Thus, typical tools (with the exception of dd) still don't
work with large files. So far, I have been able to use ftp to download
smaller reference database files (all shy of 2 GB when compressed) and
using pipes and redirection to get around the size issue, such as:

	uncompress -c dbEST.FASTA.dailydump.Z | formatdb -i stdin -p F -o T -n

Formatdb results in a compressed format ready for homology searching,
but I can also leave it off and create huge files (useful only in
showing the value of stdout/stdin redirects in this system).

Now the reference databases have to be downloaded in ~ 5 GB file chunks
and I am thinking of a pipe which starts with the download itself, such

	wget -q (missing flag) ftp://server/file | gunzip -c stdin |

where perlscript.pl will recompress and distribute an equal portion of
the reference db to each node of the cluster and the reformatting can
occur off the master node.

Eric Engelhard

LUGOD Group on LinkedIn
Sign up for LUGOD event announcements
Your email address:
LUGOD Group on Facebook
'Like' LUGOD on Facebook:

Hosting provided by:
Sunset Systems
Sunset Systems offers preconfigured Linux systems, remote system administration and custom software development.

LUGOD: Linux Users' Group of Davis
PO Box 2082, Davis, CA 95617
Contact Us

LUGOD is a 501(c)7 non-profit organization
based in Davis, California
and serving the Sacramento area.
"Linux" is a trademark of Linus Torvalds.

Sponsored in part by:
Appahost Applications
For a significant contribution towards our projector, and a generous donation to allow us to continue meeting at the Davis Library.