Usages of "wget" command in Linux

Wget, IMO, is the best download manager application I've ever seen for Linux operating system. Apart from the normal usages of wget, there are so many cool usages which can become real handy!

Resume Downloads:

# wget -c download_link

Download in background:

# wget -b download_link

Limit the bandwidth usage:

# wget --limit-rate=10k download_link

This will make wget download the file at a maximum speed of 10Kbps. This can be handy when downloading in background and there are other applications that needs Internet usage.

Download from websites having authentication:

# wget --username=username --password=password download_link

Download from an FTP server that requires authentication:

# wget --ftp-user=username --ftp-password=password download_link

Download a website completely (recursively):

# wget -r website_link

Download a website completely (recursively) up to a certain level:

# wget -r -l 5 website_link

Download up to 5 levels from the website directory. Please make sure you have enough disk space before attempting to download like this.

Download a website and convert the links relative to the local system to make it available for offline use:

# wget --convert-links -r website_link

Torn ON mirroring:

# wget --mirror download_link

Download an HTML page along with its page requisites:

This helps to download a web page (HTML) along with the requirements of that page such as images, sounds, style sheets etc.

# wget --page-requisites download_link

Download Securely from HTTPS sites:

wget --secure-protocol=protocol_name download_link

Valid options for Protocol names are "auto", "SSLv2", "SSLv3", and "TLSv1". The option "auto" can be used if you dont want to specify the protocol to be used.

And.. yes.. that's not all.. there are so many other options for the wget command. Refer its manual pages for more details.. If I find anything more interesting, I will make sure I post it here :-)

Find all files matching a specific pattern and move them to a specified sub-folder

To find all files matching a specific pattern and move them to a specified sub-folder, this command will be handy:

# find . -type f -exec grep -q 'search_string' {} \; -exec mv {} sub_folder_location/ \;

Nvidia driver fails to load after an upgrade from Ubuntu 8.04 (Hardy Heron) to Ubuntu 8.10 (Intrepid Ibex)

Well, I guess it was too early for me for an upgrade but I was waiting restlessly for an upgrade to the latest Ubuntu version, Intrepid Ibex. The upgrade went fine but guess what, my X server failed to load NVIDIA drivers once it booted up. I could get into Safe Graphics mode.. but it was certainly not what I should have.

I had to refer so many blogs and docs to get this working.. I am posting that here so that anyone else having the same issue might find this useful.

• Boot with Xorg server default configuration first.
• Open up Synaptic and remove every installed package having the name "nvidia"
• Reboot again (this is a must)
• Install these Nvidia packages except "nvidia-xconfig" package (which got the bug)

nvidia-glx-177
nvidia-*-modaliases
nvidia-kernel
nvidia-kernel-common
nvidia-177-kernel-source
nvidia-settings

• Reboot again
• Run sudo nvidia-xconfig
• Restart X server.

From many blogs and forums, many have said that this corrected their issue.. but unfortunately, not mine.. I checked the Xorg.log and could see the line that the file "libglx.so" was missing due to some reason. Then I got a suggestion from another blog to link the file libglx.so.169.12 to libglx.so.

# sudo -s
# cd /usr/lib/xorg/modules/extensions
# ln -s libglx.so.169.12 libglx.so
# init 6

That worked! And, I became another happy Intrepid Ibex user :-)

PS: This bug might have fixed with the latest updates so, I also recommend doing a complete update before trying these steps!

A cool usage of "sed" command

Suppose, you have a text file called "input.txt". You want to convert all instances of "thisword" to "anotherword" and produces and output file called "output.txt".

# cat input.txt | sed 's/thisword/anotherword/g' > output.txt

Cool, huh? ;-)

SSH Port forwarding in Linux using IPTables

Port forwarding can be done using IPTables using DNAT. I had a requirement in my enterprise, which is shown in the below chart:

Internet --> Connections made on port 22 on Gateway machine --> re-directed to port 22 of another machine present in the private network

Here the Gateway machine as two NIC cards with one of them having a public IP, say, 202.202.202.202. The machine to which the SSH connections are forwarded are on port 22 of 192.168.1.10 which is on a private network connected via the second NIC card.

So here is how the IPTable rule should be given:

iptables -t nat -A PREROUTING -p tcp -i external_interface -d external_ip --dport 22 -j DNAT --to-destination private_ip

So as per my above example network, here is how the rule should look like:

iptables -t nat -A PREROUTING -p tcp -i eth0 -d 202.202.202.202 --dport 22 -j DNAT --to-destination 192.168.1.10

Note: Please note that the private machine to be connected must be using the same Gateway we are trying to SSH. This should be taken into consideration while port forwarding in a network using more than one Gateway machines.

Reverse SSH Tunneling in Linux

This can be very handy if you want to access a Linux box present inside a NATed Network from a Linux machine outside the network. The hard way is via DNAT using IPTables using Port Forwarding, but this is very, very simple if SSH Tunneling is used.

For example, if you want to access a machine 192.168.1.10 (present under a NAT Network) from another Linux machine having the IP address 202.202.202.202, then here is what you have to do:

1) SSH from the private machine to the public machine using the below command;

ssh -R 18000:localhost:22 remoteuser@202.202.202.202

Here -R specifies that the given port on the remote (server) host is to be forwarded to the given host and port on the local side. 18000 can be any unused port on the machine.

2) Now from the remote machine, SSH into that machine itself to the port we mentioned earlier.

ssh localuser@localhost -p 18000

That should ask for a password and it needs to be the password of the localuser you have specified in the above command.