WGET, an indispensable tool for working with the web. Below are a few examples extracted from my CLI cheat-sheet, with explanation on syntax.
WGET & CURL: equivalent examples
wget -O index.html www.exampledomain.com # The -O (upper case) is optional, and if omitted it usually saves as the index.html for the website.
curl -L www.exampledomain.com > index.html # The -L follows all redirects before returning data. # Curl normally spits out to the CLI, but the '>' redirects the named file. Beware '>' overwrites, sometimes useful is '>>' to append.
Sometimes it’s really useful to be able to grab a website in its entirety.
wget -pHk www.exampledomain.com -p fetches all accompanying assets (images, css, js) to view the site -H enables recursive run, to fetch assets from other hosts -k after downloading, this will change all asset links to local/relative
du -sh $pwd
wget -pHk (in an empty directory), use
du -sh $pwd to see the size of your website. I’ve found this to be a good statistic to keep track of for UX/mobile purposes. Though there’s a lot to consider whether it’s CSS, JS, or other and a large website doesn’t necessarily mean it’s slow.