library
technical documents
All » Software » Linux/Nix (general)
Any of these categories - All of these categories
Mirroring with wget
Ever needed to pull down a simple website in one fell swoop? Well, there are tons of tools out there for different GUI operating systems like Windoze and MacOS but I prefer the simplicity of wget, especially when I can start it remotely on my *nix server.
The simple command below will archive all the pages of the specified site and pack them neatly into a directory that will work from your browser.
wget -m -k -K -E http://www.domain.tld/ -o wgetlog.txt
Brief explanation fo the options we're employing:
-m, --mirror shortcut for -N -r -l inf --no-remove-listing.
-k, --convert-links make links in downloaded HTML point to local files.
-K, --backup-converted before converting file X, back up as X.orig.
-E, --html-extension save HTML documents with `.html' extension.
Finally, as I dislike all of the info scrolling down my screen, I instruct wget to write it to a log for later review.
-o, --output-file=FILE log messages to FILE.