library

technical documents

Mirroring with wget

Posted by: Dan on July 29, 2007 01:04:01 PM +00:00
Making copies was never easier.

Ever needed to pull down a simple website in one fell swoop? Well, there are tons of tools out there for different GUI operating systems like Windoze and MacOS but I prefer the simplicity of wget, especially when I can start it remotely on my *nix server.

The simple command below will archive all the pages of the specified site and pack them neatly into a directory that will work from your browser.

wget -m -k -K -E http://www.domain.tld/ -o wgetlog.txt 

Brief explanation fo the options we're employing:

-m, --mirror shortcut for -N -r -l inf --no-remove-listing.

-k, --convert-links make links in downloaded HTML point to local files.

-K, --backup-converted before converting file X, back up as X.orig.

-E, --html-extension save HTML documents with `.html' extension.

Finally, as I dislike all of the info scrolling down my screen, I instruct wget to write it to a log for later review.

-o, --output-file=FILE log messages to FILE.

Updated: 05 Apr 10 07:33