A Linux Admin's WeBlog!

Oracle Data Pump

2010-12-02T12:26:00.007+05:30

The Data Pump utility was introduced from Oracle 10g R1 onwards. This boasts very high data transfer between databases. The Data Pump provides two utilities as mentioned below:

Data Pump Export : expdp
Data Pump Import : impdp

They might seem similar to the traditional exp and imp commands but they actually aren't! The export files from expdp cant be imported using the traditional import command imp. Also export files from exp cant be imported using impdp. The utilities are server related rather than client related with regard to the traditional ones.

Okay, enough of theories! Let's look at some examples:

Step 1: Log into the database as SYS user:

splus sys as sysdba

Step 2: Creating directory for dumping and setting permissions

create directory DUMPDIR as '/home/oracle/dumpdir';

Grant necessary permissions for the user who needs the export their schema:

grant read,write on directory dumpdir to user1;

Step 3: Exporting the schema

expdp user1/passwd DIRECTORY=DUMPDIR DUMPFILE=user1.expdp.dmp SCHEMAS=user1;

Step 4: Importing the schema

impdp user1/password DIRECTORY=DUMPDIR DUMPFILE=user1.expdp.dmp SCHEMAS=user1;

Oracle wont start: ORA-27300, ORA-27301, ORA-27302 (No space left on device)

2010-12-02T10:11:00.000+05:30

Ever encountered this error while starting up your Oracle database? At first glance, as the error informs with the line familiar to most administrators "No space left on device", its not actually what you think it is!

Even if you have loads of HDD space and free memory left, this error can happen. Its because you don't have enough semaphores left. A semaphore is a value in a designated place in operating system (or kernel) storage that each process can check and then change (ref).

ORA-27154: post/wait create failed
ORA-27300: OS system dependent operation:semget failed with status: 28
ORA-27301: OS failure message: No space left on device
ORA-27302: failure occurred at: sskgpsemsper

To fix this error, edit your /etc/sysctl.conf file and add/modify the below parameter to the values mentioned below:

kernel.sem = 250 32000 100 128

Now to apply these changes, run the below command:

sysctl -p

Once its done, try starting your database. It should start without the previous error.

Oracle Export and Import - The Traditional Way!

2010-12-02T09:23:00.003+05:30

Even though its possible to pass the requires parameters during exp or imp command run time, its convenient to use single-line full commands when its to be used in an automated export/import scripts:

Complete Database Export:

To export the complete Oracle Database and import it to another machine, it requires SYSTEM user privileges. Here is how its done:

exp SYSTEM/password FULL=y FILE=completedb.dmp LOG=dba.log CONSISTENT=y

Here:

Full=y : denotes a complete database export is required.
FILE=filename.dmp : denotes the file name of the export file.
LOG=logname.log : denotes the log file to be written.
CONSISTENT=y : denotes no further changes shall be affected in the export file.

To import the complete database in another machine, here is how to do it:

imp SYSTEM/password FULL=y FIlE=completedb.dmp commit=y

User Schema Export:

An user can export his/her own schema using the below command:

exp username/password FILE=userdb.dmp OWNER=username

To import the schema, the below command shall be used:

imp SYSTEM/password FROMUSER=username1 TOUSER=username2 FILE=userdb.dmp

Here "username1" is the name of the user who exported the schema and "username2" is the name of the user to which the schema needs to be imported.

User Schema Export (Table specific):

To export a schema with only selected tables:

exp username/password FILE=userdb-tables.dmp TABLES=(table1,table2)

To import that export file back to another user, the below command shall be used:

imp SYSTEM/password FROMUSER=username1 TOUSER=username2 FILE=userdb-tables.dmp TABLES=(table1,table2)

Finding schema size in Oracle using SQLPlus CLI

2010-10-07T16:53:00.008+05:30

To find out the schema size for a particular user, login as the user into the schema using SQLPlus CLI:

sqlplus schemauser/password

Once logged in, execute the below query:

SELECT sum(bytes)/1024/1024 FROM user_segments;

As per the calculation above (1024/1024), the output will be in MB.

Sample output shown below:

SUM(BYTES)/1024/1024

-------------------

12.4375

Tomcat: Deploying an application in direct URL with port 80

2010-10-07T15:50:00.009+05:30

Ever had a requirement of deploying a Tomcat Web Application such that you need to get that under http://example.com instead of http://example.com:8080/webapp.

Here is what needs to be done:

1. Run the Tomcat Server in port 80 instead of port 8080:

This can be done under the server.xml file present under "tomcat_home/conf". Open the server.xml file and find the part that says:

Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443"

Change that to the below form:

Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443"

Once done, the application will be accessible under http://example.com/webapp

2. Setting a web app to Document Root

There are two easier methods for doing this:

a) The ROOT folder:

By default, the ROOT folder of the Tomcat Server is showing the default Tomcat page. So backup that folder to another suitable location (apart from the "webapps" folder. Now either rename your application folder to the name ROOT (please note that it should be in capitals) or create a soft link for that folder with the name ROOT:

$ mv tomcatapp ROOT

$ ln -s tomcatapp ROOT

b) Re-direction method:

Edit the file tomcat_home/webapps/ROOT/index.html, delete everything (after taking a backup) and place the below content in it:

Change the http://example.com to your required URL.

Now, your application shall be accessible under http://example.com itself. Easier for the clients to access ;-)

Note: There are other methods also for doing this.

Oracle: Compression, Partitioning & Pruning

2010-09-30T18:25:00.001+05:30

"What I have written below is actually the notes I have taken when I tried to learn Oracle Partitioning, Table/Tablespace Compression and Partition Pruning. I know its not a technically advanced write up, but I guess it will help beginners like me to get started with this heavily vast topic"

Disclaimer:

Most of what written below has been taken from other websites. The examples mentioned in this blog worked out for me but I give no guarantee that it will work on another machine without proper study.

TABLE and TABLESPACE Compression

Table compression was introduced in Oracle 9i as a space saving feature for data warehousing projects. In 11g it is now considered a mainstream feature that is acceptable for OLTP databases. In addition to saving storage space, compression can result in increased I/O performance and reduced memory use in the buffer cache.

TABLESPACE with OLTP Compression

sql> CREATE TABLESPACE TB_COMPRESSED datafile '/home/oracle/app/oracle/oradata/TB_COMPRESSED.dbf' size 10000M DEFAULT COMPRESS FOR ALL OPERATIONS;

User Creation:

1. Since we created the tablespace with compression enabled, we can create an user on that tablespace.

sql> create user ORAUSER identified by perfomix_test default tablespace TB_COMPRESSED;

2. Once the user is created, necessary grants needs to be provided for the user for normal operations.

sql> grant CREATE TABLE, CREATE SESSION, CREATE TRIGGER, CREATE VIEW, CREATE SEQUENCE, CREATE PROCEDURE to ORAUSER;

3. Setting quota for the user.

sql> alter user ORAUSER quota unlimited on TB_COMPRESSED;

Oracle Partitioning

Partitioning allows a table, index, or index-organized table to be subdivided into smaller pieces, where each piece of such a database object is called a partition. Each partition has its own name, and may optionally have its own storage characteristics.

From the perspective of a database administrator, a partitioned object has multiple pieces that can be managed either collectively or individually. This gives the administrator considerable flexibility in managing partitioned objects. However, from the perspective of the application, a partitioned table is identical to a non-partitioned table; no modifications are necessary when accessing a partitioned table using SQL queries and DML statements.

Partitioning can provide tremendous benefit to a wide variety of applications by improving performance, manageability, and availability. It is not unusual for partitioning to improve the performance of certain queries or maintenance operations by an order of magnitude. Moreover, partitioning can greatly simplify common administration tasks.

Oracle Partitioning offers three fundamental data distribution methods as basic partitioning strategies that control how data is placed into individual partitions:

Range Partitioning: This maps data to partitions based on ranges of values of the partitioning key that you establish for each partition. It is the most common type of partitioning and is often used with dates.

Hash Partitioning: This maps data to partitions based on a hashing algorithm that Oracle applies to the partitioning key that you identify. The hashing algorithm evenly distributes rows among partitions, giving partitions approximately the same size.

List Partitioning: This enables you to explicitly control how rows map to partitions by specifying a list of discrete values for the partitioning key in the description for each partition. The advantage of list partitioning is that you can group and organize unordered and unrelated sets of data in a natural way.

Here I am going to discuss more about RANGE partitioning in this blog.

Creating a Partitioned Table (Range Partitioning) with OLTP Compression:

Below shown is an example of Range Partitioning in Oracle. The CREATE query will have the clause "PARTITION BY RANGE(field_name)" and the partitions are defined one-by-one with a distinctive name. A clause "VALUES LESS THAN" is mandatory for defining the partition end-points.

Example:

CREATE TABLE SAMPLE_ORDERS
(ORDER_NUMBER NUMBER,
ORDER_DATE DATE,
CUST_NUM NUMBER,
TOTAL_PRICE NUMBER,
TOTAL_TAX NUMBER,
TOTAL_SHIPPING NUMBER)
PARTITION BY RANGE(ORDER_DATE)
(
PARTITION P1 VALUES LESS THAN(TO_DATE('01-JAN-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS,
PARTITION P2 VALUES LESS THAN(TO_DATE('01-FEB-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS,
PARTITION P3 VALUES LESS THAN(TO_DATE('01-MAR-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS,
PARTITION P4 VALUES LESS THAN(TO_DATE('01-APR-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS,
PARTITION P5 VALUES LESS THAN(TO_DATE('01-MAY-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS,
PARTITION P6 VALUES LESS THAN(TO_DATE('01-JUN-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS,
PARTITION P7 VALUES LESS THAN(TO_DATE('01-JUL-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS,
PARTITION P8 VALUES LESS THAN(TO_DATE('01-AUG-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS,
PARTITION P9 VALUES LESS THAN(TO_DATE('01-SEP-2010', 'DD-MON-YYYY')) COMPRESS FOR ALL OPERATIONS
);

This should create a table with 9 range partitions along with enabling OLTP compression.

Now we can insert some dummy values to the table for testing. As you can see, there is no changes with the INSERT queries even if its done against a partitioned table. Oracle manages the input data and store them into respective partitions.

sql> INSERT INTO SAMPLE_ORDERS values ('200', '05-FEB-2010', '200', '2000', '2222', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('300', '05-MAR-2010', '300', '3000', '3333', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('400', '05-APR-2010', '400', '4000', '4444', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('500', '05-MAY-2010', '500', '5000', '5555', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('600', '05-JUN-2010', '600', '6000', '6666', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('700', '05-JUL-2010', '700', '7000', '7777', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('800', '05-AUG-2010', '800', '8000', '8888', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('101', '02-JAN-2010', '100', '1000', '1111', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('201', '02-FEB-2010', '200', '2000', '2222', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('301', '02-MAR-2010', '300', '3000', '3333', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('401', '02-APR-2010', '400', '4000', '4444', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('501', '02-MAY-2010', '500', '5000', '5555', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('601', '02-JUN-2010', '600', '6000', '6666', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('701', '02-JUL-2010', '700', '7000', '7777', '1111');
sql> INSERT INTO SAMPLE_ORDERS values ('801', '02-AUG-2010', '800', '8000', '8888', '1111');

SELECT Queries:

Just like the INSERT queries, the SELECT queries can also be used without any kind of modifications relation to partitioning.

SQL> select * from SAMPLE_ORDERS

How ever, its also possible to SELECT data from specific partitions by giving the partition name.

SQL> select * from SAMPLE_ORDERS PARTITION(P2);

ORDER_NUMBER ORDER_DAT     CUST_NUM TOTAL_PRICE TOTAL_TAX TOTAL_SHIPPING
------------ --------- ---------- ----------- ---------- --------------
    101                  02-JAN-10          100    1000        1111       1111

SQL> select * from SAMPLE_ORDERS PARTITION(P3);

ORDER_NUMBER ORDER_DAT     CUST_NUM TOTAL_PRICE TOTAL_TAX TOTAL_SHIPPING
------------ --------- ---------- ----------- ---------- --------------
     200 05-FEB-10          200     2000        2222       1111
     201 02-FEB-10          200     2000        2222       1111

Checking on Partition and Compression Details

Since we created the partitions using "COMPRESS FOR ALL OPERATIONS", its ideal for heavy-transaction databases like OLTP. We can find the details of the Partitions and Compression details using the below query:

SQL> SELECT partition_name, compression, compress_for FROM user_tab_partitions;

PARTITION_NAME               COMPRESS COMPRESS_FOR
------------------------------ -------- ------------
P1                   ENABLED    OLTP
P2                   ENABLED    OLTP
P3                   ENABLED    OLTP
P4                   ENABLED    OLTP
P5                   ENABLED    OLTP
P6                   ENABLED    OLTP
P7                   ENABLED    OLTP
P8                   ENABLED    OLTP
P9                   ENABLED    OLTP

Modifying Partitions in a table

Adding a new partition:

sql> alter table tempabc add partition p2 values less than ('C%');

Note:

You cannot move a partition with sub-partitions
Each subpartition must be individually moved.
We can add a partition to the high end of the table (after the last existing partition). If the first element of the partition bound of the high partition is MAXVALUE, you cannot add a partition to the table. You must split the high partition.
You can add a partition to a table even if one or more of the table indexes or index partitions are marked UNUSABLE.
You must use the SPLIT PARTITION clause to add a partition at the beginning or the middle of the table.
ALTER TABLE DROP PARTITION drops a partition and its data. If you want to drop a partition but keep its data in the table, you must merge the partition into one of the adjacent partitions.
If you drop a partition and later insert a row that would have belonged to the dropped partition, the row will be stored in the next higher partition. However, if you drop the highest partition, the insert will fail because the range of values represented by the dropped partition is no longer valid for the table.

Ways to partition an existing table

Method 1:

create a PARTITIONED table
load the NON-PARTITIONED data into the PARTITIONED table
drop the NON-PARTITIONED table
rename PARTITIONED table to whatever is appropriate

Method 2:

export table and data
drop table
create new partitioned table the way you want it
import table data into new partitioned table (this should also create all the necessary indexes and constraints from your original table)

Partition Pruning

Semantics-based partitioning can allow Adaptive Server to eliminate certain partitions when performing a search. Range-based partitions, for example, contain rows whose partitioning keys are discrete value sets. When a query predicate—a where clause—is based on those partitioning keys, Adaptive Server can quickly ascertain whether rows in a particular partition can satisfy the query. This behavior is called partition pruning, or partition elimination, and it can save considerable time and resources during execution.

For range and list partitioning – Adaptive Server can apply partition pruning on equality (=) and range (>, >=, <, and <=) predicates on partition-key columns on a single table.

Sample Queries:

Example 1:

sql> select * from sample_orders WHERE ORDER_DATE BETWEEN TO_DATE ('01-JAN-2010','DD-MON-YYYY') AND TO_DATE('01-MAR-2010','DD-MON-YYYY');

Result:

ORDER_NUMBER ORDER_DAT     CUST_NUM TOTAL_PRICE TOTAL_TAX TOTAL_SHIPPING
------------ --------- ---------- ----------- ---------- --------------
     100 05-JAN-10          100     1000        1111       1111
     101 02-JAN-10          100     1000        1111       1111
     200 05-FEB-10          200     2000        2222       1111
     201 02-FEB-10          200     2000        2222       1111

Example 2:

SQL> select * from sample_orders where ORDER_DATE > ('01-JUN-2010');

Result:

ORDER_NUMBER ORDER_DAT     CUST_NUM TOTAL_PRICE TOTAL_TAX TOTAL_SHIPPING
     ------------ ---------              ---------- ----------- ---------- --------------
     600                   05-JUN-10      600     6000      6666     1111
     601 02-JUN-10      600     6000      6666     1111
     700                   05-JUL-10      700     7000       7777     1111
     701                   02-JUL-10      700     7000       7777     1111
     800                   05-AUG-10    800     8000        8888    1111
     801                   02-AUG-10    800     8000        8888       1111

More Examples:

Partition on a numeric value range:

CREATE TABLE emp (
   empno NUMBER(4),
   ename VARCHAR2(30),
   sal   NUMBER
)
PARTITION BY RANGE(empno) (
    partition e1 values less than (1000)     tablespace ts1,
    partition e2 values less than (2000)     tablespace ts2,
    partition e3 values less than (MAXVALUE) tablespace ts3
);

Partition on a VARCHAR2 string:

CREATE TABLE emp
   ( id        NUMBER(5)    PRIMARY KEY,
     name      VARCHAR2(50) NOT NULL,
     phone     VARCHAR2(15),
     email     VARCHAR2(100)
   )
   PARTITION BY RANGE ( name )
     (
       PARTITION p1 VALUES LESS THAN ('L')      TABLESPACE ts1,
       PARTITION p2 VALUES LESS THAN (MAXVALUE) TABLESPACE ts2
     );

Time based range partitioning:

CREATE TABLE t1 (id NUMBER, c1 DATE)
PARTITION BY RANGE (c1)
    (PARTITION t1p1 VALUES LESS THAN (TO_DATE('2007-11-01', 'YYYY-MM-DD')),
     PARTITION t1p2 VALUES LESS THAN (TO_DATE('2007-12-01', 'YYYY-MM-DD')),
     PARTITION t1p3 VALUES LESS THAN (TO_DATE('2008-01-01', 'YYYY-MM-DD')),
     PARTITION t1p4 VALUES LESS THAN (MAXVALUE)
    );

Oracle Database : Complete Export and Import

2009-08-22T16:56:00.001+05:30

Suppose, we need to export all the Oracle schema in one step, similar
to "--all-databases" option in MySQL. In such cases, here is how its done:

exp USERID=system/password@sid file=complete_bak.dmp log=/tmp/full.log FULL=Y;

This should export all the databases in one step and will store that to a single file.

Now to import that back, here is what needs to be done.

imp system/password@sid file=complete_bak.dmp log=/tmp/full.log full=y ignore=y

Note:

* Will work only if ORACLE_HOME is same on both servers
* Create tablespace on the new server before importing the database. Else this wont work.

OpenOffice Error : The user interface language cannot be determined-Linux

2009-08-22T16:39:00.000+05:30

Stuck at this error after re-installing or upgrading OpenOffice?

OpenOffice Error : The user interface language cannot be determined-Linux

The error occurs because the folder where the necessary files are
not present are not having enough permissions. The solution
is pretty simple. Get into the user's home directory# cd /home/username
Find the hidden folder which stores OpenOffice files. Its
usually ".openofficeversion" or ".oooversion".
# cd ~/.oooversion
Now execute the below command:# sudo chown --reference=. -R .openoffice.org2
Now give necessary permissions to that folder:# sudo chmod -R 755 .openoffice.org2
Try opening up OpenOffice. It should work just fine now.

Bulk downoading files from a webserver folder accessible via FTP

2009-08-22T16:35:00.000+05:30

Suppose, if we need to download all files from a web folder, accessible via FTP, we can use LFTP command to do that. For example, in one website, under the "clamav" folder, I need to download all RPM files from there. So instead of downloading that one by one, I can do so by the command below:

lftp -c 'open -e "mget -d *.el5.rf.i386.rpm" http://packages.sw.be/clamav/'

This needs to be executed from a terminal. Also packages, LFTP and MGET needs to be present.

Burn an ISO image to a CD right from the command line!

2008-12-16T11:35:00.000+05:30

This is commonly used by system administrators who love simple, efficient and verbose method of writing an image into a Compact Disk. If you have an ISO file "image.iso", then you can use the "cdrecord" command:

cdrecord -v speed=8 dev=/dev/cdrom /path/to/image.iso

Attributes:

-v : Verbose mode, shows a lotta information while writing the disk.. geeky :-D
speed = : you can set the burning speed. 8 means 8x speed
-multi : multisession mode (ooooh, cool feature for a command line tool)

Listing files in Linux sorted according to size

2008-12-16T10:59:00.000+05:30

Have you ever felt the need to list the 10 files that takes the largest space in a specific directory? You might know that its pretty easy do that via GUI but believe me its the same thing when it comes to the Console also.

Here is how its done:

du -a (directory) | sort -n -r | head -n 10

This will list all the files in the directory specified, sorts them and lists the 10 files that takes up maximum size.

Easy, right?

Make files in your webserver download instead of being displayed

2008-12-02T20:27:00.000+05:30

Suppose, you have a text file in your web directory. When you access that file, it will be displayed in the web browser itself. If you want to make that file (or any other file type you want) to be asked to download instead of being displayed, here is what you can do:

Create a .htaccess file in that directory with the following content:

AddType unknown/nothing pdf
AddType unknown/nothing txt
AddType unknown/nothing jpg

Now when you access any file with an extension .jpg or .pdf, your browser will ask you to download the file instead of showing up.

Relaying email from Postfix via another SMTP Server

2008-12-02T19:13:00.000+05:30

Have you ever been in a situation where you want to relay your emails from your Postfix server via another SMTP server? This is certainly possible and its way too simple:

First thing to do is configure Postfix in your machine. Once done, edit the configuration file for Postfix (usually under /etc/main.cf) and editing the below value:

#relayhost =

to

relayhost = smtp.emailprovider.com

Once done, all emails sent via your Postfix server will be relayed through the desired Outgoing server.

Other options that may interest you are as follows:

myhostname = hostname.emailprovider.com
mydomain = emailprovider.com
masquerade_domains = emailprovider.com

The last option will correct the envelopes shows as "user@emailprovider.com".

Hope that helps someone :-)

Limiting Denial of Service (DoS) attacks

2008-12-01T21:24:00.000+05:30

Denial of Service (DoS) attack is an attempt by a malicious (or unwitting) user, process, or system to prevent legitimate users from accessing a resource (usually a network service) by exploiting a weakness or design limitation in an information system. Examples of DoS attacks include flooding network connections, filling disk storage, disabling ports, or removing power. This can be limited by setting timeouts.

# echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
# echo 1800 > /proc/sys/net/ipv4/tcp_keepalive_time
# echo 1 > /proc/sys/net/ipv4/tcp_window_scaling
# echo 0 > /proc/sys/net/ipv4/tcp_sack
# echo 1280 > /proc/sys/net/ipv4/tcp_max_syn_backlog

Source: http://sourcelinux.wikidot.com/firewall-using-iptables

Usages of "wget" command in Linux

2008-11-27T09:33:00.000+05:30

Wget, IMO, is the best download manager application I've ever seen for Linux operating system. Apart from the normal usages of wget, there are so many cool usages which can become real handy!

Resume Downloads:

# wget -c download_link

Download in background:

# wget -b download_link

Limit the bandwidth usage:

# wget --limit-rate=10k download_link

This will make wget download the file at a maximum speed of 10Kbps. This can be handy when downloading in background and there are other applications that needs Internet usage.

Download from websites having authentication:

# wget --username=username --password=password download_link

Download from an FTP server that requires authentication:

# wget --ftp-user=username --ftp-password=password download_link

Download a website completely (recursively):

# wget -r website_link

Download a website completely (recursively) up to a certain level:

# wget -r -l 5 website_link

Download up to 5 levels from the website directory. Please make sure you have enough disk space before attempting to download like this.

Download a website and convert the links relative to the local system to make it available for offline use:

# wget --convert-links -r website_link

Torn ON mirroring:

# wget --mirror download_link

Download an HTML page along with its page requisites:

This helps to download a web page (HTML) along with the requirements of that page such as images, sounds, style sheets etc.

# wget --page-requisites download_link

Download Securely from HTTPS sites:

wget --secure-protocol=protocol_name download_link

Valid options for Protocol names are "auto", "SSLv2", "SSLv3", and "TLSv1". The option "auto" can be used if you dont want to specify the protocol to be used.

And.. yes.. that's not all.. there are so many other options for the wget command. Refer its manual pages for more details.. If I find anything more interesting, I will make sure I post it here :-)

Find all files matching a specific pattern and move them to a specified sub-folder

2008-11-27T09:28:00.000+05:30

To find all files matching a specific pattern and move them to a specified sub-folder, this command will be handy:

# find . -type f -exec grep -q 'search_string' {} \; -exec mv {} sub_folder_location/ \;

Nvidia driver fails to load after an upgrade from Ubuntu 8.04 (Hardy Heron) to Ubuntu 8.10 (Intrepid Ibex)

2008-11-18T09:03:00.000+05:30

Well, I guess it was too early for me for an upgrade but I was waiting restlessly for an upgrade to the latest Ubuntu version, Intrepid Ibex. The upgrade went fine but guess what, my X server failed to load NVIDIA drivers once it booted up. I could get into Safe Graphics mode.. but it was certainly not what I should have.

I had to refer so many blogs and docs to get this working.. I am posting that here so that anyone else having the same issue might find this useful.

Boot with Xorg server default configuration first.
Open up Synaptic and remove every installed package having the name "nvidia"
Reboot again (this is a must)
Install these Nvidia packages except "nvidia-xconfig" package (which got the bug)

nvidia-glx-177
nvidia-*-modaliases
nvidia-kernel
nvidia-kernel-common
nvidia-177-kernel-source
nvidia-settings

Reboot again
Run sudo nvidia-xconfig
Restart X server.

From many blogs and forums, many have said that this corrected their issue.. but unfortunately, not mine.. I checked the Xorg.log and could see the line that the file "libglx.so" was missing due to some reason. Then I got a suggestion from another blog to link the file libglx.so.169.12 to libglx.so.

# sudo -s
# cd /usr/lib/xorg/modules/extensions
# ln -s libglx.so.169.12 libglx.so
# init 6

That worked! And, I became another happy Intrepid Ibex user :-)

PS: This bug might have fixed with the latest updates so, I also recommend doing a complete update before trying these steps!

A cool usage of "sed" command

2008-11-12T10:18:00.000+05:30

Suppose, you have a text file called "input.txt". You want to convert all instances of "thisword" to "anotherword" and produces and output file called "output.txt".

# cat input.txt | sed 's/thisword/anotherword/g' > output.txt

Cool, huh? ;-)

SSH Port forwarding in Linux using IPTables

2008-11-12T09:36:00.000+05:30

Port forwarding can be done using IPTables using DNAT. I had a requirement in my enterprise, which is shown in the below chart:

Internet --> Connections made on port 22 on Gateway machine --> re-directed to port 22 of another machine present in the private network

Here the Gateway machine as two NIC cards with one of them having a public IP, say, 202.202.202.202. The machine to which the SSH connections are forwarded are on port 22 of 192.168.1.10 which is on a private network connected via the second NIC card.

So here is how the IPTable rule should be given:

iptables -t nat -A PREROUTING -p tcp -i external_interface -d external_ip --dport 22 -j DNAT --to-destination private_ip

So as per my above example network, here is how the rule should look like:

iptables -t nat -A PREROUTING -p tcp -i eth0 -d 202.202.202.202 --dport 22 -j DNAT --to-destination 192.168.1.10

Note: Please note that the private machine to be connected must be using the same Gateway we are trying to SSH. This should be taken into consideration while port forwarding in a network using more than one Gateway machines.

Reverse SSH Tunneling in Linux

2008-11-12T09:14:00.000+05:30

This can be very handy if you want to access a Linux box present inside a NATed Network from a Linux machine outside the network. The hard way is via DNAT using IPTables using Port Forwarding, but this is very, very simple if SSH Tunneling is used.

For example, if you want to access a machine 192.168.1.10 (present under a NAT Network) from another Linux machine having the IP address 202.202.202.202, then here is what you have to do:

1) SSH from the private machine to the public machine using the below command;

ssh -R 18000:localhost:22 remoteuser@202.202.202.202

Here -R specifies that the given port on the remote (server) host is to be forwarded to the given host and port on the local side. 18000 can be any unused port on the machine.

2) Now from the remote machine, SSH into that machine itself to the port we mentioned earlier.

ssh localuser@localhost -p 18000

That should ask for a password and it needs to be the password of the localuser you have specified in the above command.

Upgrading from PHP 5.1 to 5.2 in Fedora

2008-09-24T14:54:00.000+05:30

Its not that hard to upgrade PHP from v5.1 to v5.2. Here is how I do this (in detail):

1) Know your current PHP version:

# php -v

PHP 5.1.6 (cli) (built: Feb 23 2007 10:11:28)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies

So the current version is 5.1.6 and this needs to be upgraded.

2) Check out the PHP RPMs that you have installed in your machine:

# rpm -qa | grep php

php-pdo-5.1.6-1.4
php-json-1.2.1-2.fc5
php-mysql-5.1.6-1.4
php-gd-5.1.6-1.4
php-xml-5.1.6-1.4
php-5.1.6-1.4
php-pear-1.4.9-1.2

3) Download the RPM packages for the new PHP Version. I use the website RPM.PBONE.NET since it provides a good customized search interface for finding the right RPM packages. Use "Advanced RPM Search" link near the "Search" button and you will see what I mean. Download the ones required from the desired FTP servers and place them in a folder.

(Since I am doing this for a PHP Development machine, I might use a few extra packages that might not be in your menu ;-)

pcre-6.6-1.fc5.remi.i386.rpm
php-devel-5.2.5-1.fc5.remi.i386.rpm
php-pdo-5.2.5-1.fc5.remi.i386.rpm
php-5.2.5-1.fc5.remi.i386.rpm
php-gd-5.2.5-1.fc5.remi.i386.rpm
php-pear-1.7.1-1.fc5.remi.noarch.rpm
php-cli-5.2.5-1.fc5.remi.i386.rpm
sqlite2-2.8.17-1.fc5.i386.rpm
php-common-5.2.5-1.fc5.remi.i386.rpm
php-mysql-5.2.5-1.fc5.remi.i386.rpm
sqlite2-2.8.17-1.fc5.i386.rpm.html

4) Remove the existing PHP RPMs from the machine:

# rpm -e php php-pear php-pdo php-gd php-mysql php-gd

(If this is successful, you wont get any results. If you see a message like "php-gd not found" then just remove that package from the above remove list)

5) Install the new RPMs in the below order to avoid dependency errors:

# rpm -Uvh pcre-6.6-1.fc5.remi.i386.rpm
# rpm -ivh php-common-5.2.5-1.fc5.remi.i386.rpm
# rpm -ivh php-cli-5.2.5-1.fc5.remi.i386.rpm
# rpm -ivh php-5.2.5-1.fc5.remi.i386.rpm
# rpm -ivh sqlite2-2.8.17-1.fc5.i386.rpm
# rpm -ivh php-pdo-5.2.5-1.fc5.remi.i386.rpm
# rpm -ivh php-mysql-5.2.5-1.fc5.remi.i386.rpm
# rpm -ivh php-pear-1.7.1-1.fc5.remi.noarch.rpm
# rpm -ivh php-gd-5.2.5-1.fc5.remi.i386.rpm

6) This should be over pretty quickly and you wont get any errors. Then just restart your Apache Server (HTTPD) for the changes to take effect.

# /etc/init.d/httpd restart

Now check your PHP Version and see if the change got reflected:

# php -v

PHP 5.2.5 (cli) (built: Nov 10 2007 12:15:39)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies

Voila, Done!