Sunday, August 21, 2011

ORACLE LOCKS

Oracle Database always performs necessary locking to ensure data concurrency,
integrity, and statement-level read consistency. You can override these default locking
mechanisms. For example, you might want to override the default locking of Oracle
Database if:

■ You want transaction-level read consistency or "repeatable reads"—where
transactions query a consistent set of data for the duration of the transaction,
knowing that the data has not been changed by any other transactions. This level
of consistency can be achieved by using explicit locking, read-only transactions,
serializable transactions, or overriding default locking for the system.

■ A transaction requires exclusive access to a resource. To proceed with its
statements, the transaction with exclusive access to a resource does not have to
wait for other transactions to complete.

ROW-LEVEL AND TABLE-LEVEL LOCKS:

DML locks are locks which are acquired automatically by Oracle to protect data in tables and indexes Whenever you issue a DML statement to modify data i.e SELECT..FOR UPDATE,INSERT, UPDATE, MERGE, and DELETE statments.DML locks prevent destructive interference of simultaneous conflicting DML or DDL operations.
DML statments automatically acquire lock on two levels :

i) Row-Level Locks (TX)
ii)Table Level Locks(TM)

These DML locks are always and only (exclusive) at row level but it can be shared as well as exclusive at table level.

Here TX specify lock at row level while TM is at table level.

i) Row Level Locks (TX)
Row-level locks are primarily used to prevent two transactions from modifying the same row. When a transaction needs to modify a row, a row lock is acquired.Any INSERT, DELETE, UPDATE, or SELECT FOR UPDATE statements will automatically issue an exclusive lock on the rows affected by the transaction. This exclusive lock at row means that other transactions can’t modify the affected rows until the original transaction commits or rolls back, thereby releasing the exclusive locks.

ii) Table Level Lock (TM)
Whenever you acquire any row level lock there is ultimately a table level is also acquired to prvent others session to alter, drop this table whose rows are being modified.TM Per table locks are acquired during the execution of a transaction when referencing a table with a DML statement so that the object is not dropped or altered during the execution of the transaction.



Choosing a Locking Strategy :

Different Locking modes:

The v$lock view has an ‘LMODE’ column which indicates the Lock mode in which the session holds the lock. The value of LMODE can vary from 0 to 6.The higher the value of LMODE, stronger is the lock. I am describing below each of these locking modes.

• 0 - none
• 1 - null (NULL)
• 2 - row-S (RS)
• 3 - row-X (RX)
• 4 - share (S)
• 5 - S/Row-X (SRX)
• 6 - exclusive (X)

ROW SHARE (RS)
ROW SHARE permits concurrent access to the locked table but prohibits users from locking the entire table for exclusive access.

ROW EXCLUSIVE (RX)
ROW EXCLUSIVE is the same as ROW SHARE, but it also prohibits locking in SHARE mode. ROW EXCLUSIVE locks are automatically obtained when updating, inserting, or deleting.
The following statements lock a table in ROW-EXCLUSIVE mode.
INSERT INTO table ... ;

UPDATE table ... ; (see example 1).

DELETE FROM table ... ;

LOCK TABLE table IN ROW EXCLUSIVE MODE; (For notes on usage of LOCK TABLE see Example 2).

SHARE (S)
SHARE permits concurrent queries but prohibits updates to the locked table.

SHARE ROW EXCLUSIVE (SRX)
SHARE ROW EXCLUSIVE is used to look at a whole table and to allow others to look at rows in the table but to prohibit others from locking the table in SHARE mode or from updating rows.

EXCLUSIVE (E)
EXCLUSIVE permits queries on the locked table but prohibits any other activity on it.

I am giving below the compatibility matrix of the different locking modes:








compatible RS RX S SRXX
RS yesyesyesyesno
RXyes yes no nono
S yes no yes no no
SRX yes no no no no
X no no no no no


Lets try out an example where we will update the salary column of the EMP table for a particular employee and use the V$LOCK view to check the corresponding locks it acquires:

Example 1.1:


SESSION 1:

SQL> update emp set salary=salary+100 where empno=1;

1 row updated.

SQL> select a.sid,a.type,a.lmode,a.request from v$lock a,v$session b where a.sid=b.sid
and b.username='SCOTT';

SID TY LMODE REQUEST
---------- -- ---------- ----------
162 TX 6 0
162 TM 3 0

So It acquires an EXCLUSIVE row-level(TX) lock to the row it updates and a ROW-EXCLUSIVE table-level(TM) lock on the EMP table.

The session has not committed yet.

Session 2:

Now lets say from another session, the user wants to drop the table EMP which would require an EXCLUSIVE LOCK on the EMP table.

SQL> DROP TABLE EMP;
DROP TABLE EMP
*
ERROR at line 1:
ORA-00054: resource busy and acquire with NOWAIT specified.

It throws an error because the an EXCLUSIVE lock is denied on EMP as another session has already acquired a ROW-EXCLUSIVE(RX) lock on EMP and RX is not compatible with EXCLUSIVE(E) mode(see the compatibility matrix).

To release the locks, session 1 simply needs to commit or rollback.


EXAMPLE 1.2:


Now the session 1 has not yet committed/Rolled back and is still holding the locks.

Session 1:

SQL> select a.sid,a.type,a.lmode,a.request from v$lock a,v$session b where a.sid=b.sid
and b.username='SCOTT';

SID TY LMODE REQUEST
---------- -- ---------- ----------
162 TX 6 0
162 TM 3 0

Session 2:

The user wants to update the salary of the particular employee with employee id 2.

SQL> update emp set salary=salary+100 where empno=2;

1 row updated.
The transaction is allowed.
Both the session have not yet committed/Rolled back. Now lets find out the locks they are holding.

SQL> select a.sid,a.type,a.lmode,a.request from v$lock a,v$session b where a.sid=b.sid
2 and b.username='SCOTT';

SID TY LMODE REQUEST
---------- -- ---------- ----------
149 TX 6 0
149 TM 3 0
162 TX 6 0
162 TM 3 0

As we can see,So both the sessions are holding RX locks on the table level
and EXCLUSIVE locks at row level (but at different rows). As RX is compatible
with RX and their EXCLUSIVE locks are at different rows , so both their TX and
TM locks are compatible and hence the transactions are allowed.


Lock table Statement :

A LOCK TABLE statement manually overrides default locking.
When a LOCK TABLE statement is issued on a view, the underlying base tables are
locked. The following statement acquires exclusive table locks for the
EMP table on behalf of the containing transaction:

Example 2.1: 


SQL>LOCK TABLE EMP IN EXCLUSIVE MODE;

You can specify several tables or views to lock in the same mode; however, only a
single lock mode can be specified for each LOCK TABLE statement.

Lets see what the V$LOCK view has to say about the lock held by the session

SQL>select a.sid,a.type,a.lmode,a.request from v$lock a,v$session b where a.sid=b.sid and
b.username='SCOTT'


SID TY LMODE REQUEST
---------- -- ---------- ----------
162 TM 6 0

LMODE 6 is EXCLUSIVE lock.


EXAMPLE 2.2:


Lets give another example where the table is locked is SHARE ROW EXCLUSIVE (SRX) mode.

SQL> LOCK TABLE EMP IN SHARE ROW EXCLUSIVE MODE;

Table(s) Locked.

SQL> select a.sid,a.type,a.lmode,a.request from v$lock a,v$session b where a.sid=b.sid and
2 b.username='SCOTT';

SID TY LMODE REQUEST
---------- -- ---------- ----------
162 TM 5 0

LMODE 5 is SRX lock mode.


NOWAIT CLAUSE:

You can also indicate if you do or do not want to wait to acquire the lock. If you
specify the NOWAIT option, then you only acquire the table lock if it is immediately
available. Otherwise an error is returned to notify that the lock is not available at this
time. In this case, you can attempt to lock the resource at a later time. If NOWAIT is
omitted, then the transaction does not proceed until the requested table lock is
acquired.



Example 3:


Session 1:

SQL> Lock table EMP IN SHARE ROW EXCLUSIVE MODE;

Session 2:

SQL> LOCK TABLE EMP IN SHARE MODE NOWAIT;
LOCK TABLE EMP IN SHARE MODE NOWAIT
*
ERROR at line 1:
ORA-00054: resource busy and acquire with NOWAIT specified

Note: ALTER TABLE and DROP TABLE commands have implicitly NOWAIT keyword specified (see example 1.1).


Sunday, June 26, 2011

PLAYING WITH ORACLE LOBs

LOBs deal with unstructured data, the ones which are the most difficult to store and retrieve in a relational database.

In this article, I am going to discuss extensively and manipulate the ORACLE


  • LOBs that are stored in the database itself like BLOB,CLOB,NCLOB ,and


  • LOBs like BFILE which are Stored outside the database as Operating System files.
First let us create a directory :

create or replace directory my_dir as 'D:\pics';
Directory created.
BFILE:
BFILEs act as a pointer and store the location of the external OS files in database tables.

1. POPULATING BFILE COLUMNS IN DATABASE TABLES:
Now we insert a row in it.

Insert into lob_test values (1,’dwaipayan’,bfilename(‘MY_DIR’,’ pic1.jpg’));

This bfilename function returns a BFILE locator for a physical LOB binary file.
Now, if we wish to display the contents of the table:

select * from lob_test;
SP2-0678: Column or attribute type can not be displayed by SQL*Plus
We get an error as we cannot display the contents of a bfile like this.

Now, let us insert another row in the table:

insert into lob_test values (2,'shiba',bfilename('MY_DIR','9.jpg'));
1 row created.
Using the DBMS_LOB.GETLENGTH procedure we can get the lengths of the OS files as follows:

select dbms_lob.getlength(PIC) from lob_test;
DBMS_LOB.GETLENGTH(PIC)
---------------------------
180496
15032


Note: PIC is a BFILE column

2. Locating the files in OS from a BFILE:

Now, say I have loaded the tables with data, and I come back after 3 months and I don’t remember the locations of the OS files with which I had loaded the data into the tables.
In that case what should I do…
Simply, we take the help of DBMS_LOB.FILEGETNAME procedure.
CREATE or replace FUNCTION get_dir_name (bf BFILE) RETURN VARCHAR2 IS
DIR_ALIAS VARCHAR2(255);
FILE_NAME VARCHAR2(255);
BEGIN
IF bf is NULL
THEN
RETURN NULL;
ELSE
DBMS_LOB.FILEGETNAME (bf, dir_alias, file_name);
RETURN FILE_NAME;
END IF;
END;

select get_dir_name(pic) from lob_test;

GET_DIR_NAME(PIC)
--------------------------------------------------------------------------------
pic1.jpg
9.jpg


3. OPENING A BFILE AND COPYING IT TO ANOTHER OS LOCATION

File type: image (jpg)


CREATE OR REPLACE procedure imagecopy
AS
h1 bfile;
h2 utl_file.file_type;
len integer;
strt integer;
buf raw(32767);
BEGIN
h2 := utl_file.fopen('MY_DIR','copyof9.jpg','Wb');
select pic into h1 from lob_test where empno=1;
dbms_lob.open(h1,dbms_lob.file_readonly);
strt:=1;
select dbms_lob.getlength(h1) into len from dual;
while strt<=len
loop
buf := dbms_lob.substr(h1,10000,strt) ;
utl_file.put_raw(h2,buf,TRUE);
strt := strt+10000;
end loop;
utl_file.fclose(h2);
END;
/

So, the image file that was pointed to by the BFILE in lob_test table with empno=1, is now copied to the new location in the OS under the directory ’MY_DIR’ as 'copyof9.jpg'.
Note: Here we have used utl_file package to open the database file where we are going to write the data. The mode in which the 'copyof9.jpg' file is opened is ‘Wb’ because we are going to write binary data in it.
Utl_file.put_raw is used to write raw data into the file
Besides these, we have used the dbms_lob.substr function.It’s syntax is a bit different from the ordinary substr function. In dbms_lob.substr, the requested substring length comes first, followed by the offset.


BLOB :
We , first create a table test_blob containing a blob column ‘DOC’ :

create table test_blob (empno number,empname varchar2(30),doc blob);

Table created.
1. Populating Blob column in database from OS file:
create or replace procedure blobfromfile
as
blob_buf blob;
h1 bfile;
begin
h1 := bfilename('MY_DIR','9.jpg');
dbms_lob.open(h1,dbms_lob.file_readonly);
blob_buf := dbms_lob.substr(h1,dbms_lob.getlength(h1),1);
insert into test_blob values (1,'dwaipayan',blob_buf);
commit;
end;


select count(*) from test_blob;

COUNT(*)
----------
1



CLOB :
We , first create a table test_clob containing a clob column ‘ABOUT_ME’ :
create table test_clob (empno number,about_me clob);

Table created.

insert into test_clob values (1,empty_clob());

1 row created.
1. Populating Clob column in database from OS file:
CREATE OR REPLACE procedure CLOB_POPULATE
AS
CLOB_BUF blob;
h1 BFILE;
BEGIN
h1 := BFILENAME('MY_DIR','t3.txt');
dbms_lob.open(h1,dbms_lob.file_readonly);
clob_buf := dbms_lob.substr(h1,dbms_lob.getlength(h1),1) ;
update test_clob set about_me=utl_raw.cast_to_varchar2(clob_buf) where empno=1;
commit;
end;

select substr(about_me,1,100) FROM test_clob;

SUBSTR(ABOUT_ME,1,100)
--------------------------------------------------------------------------------
ooo
hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
ooooooooooooooooooooooooooooooooo
hhh
2. CONVERTING CLOB TO BLOB :
insert into test_blob
2 select 2,'sinha',utl_raw.cast_to_raw(about_me) from test_clob where empno=1;

1 row created.

Displaying The raw data:

select dbms_lob.substr(doc,100,1) from test_blob where empno=2;

DBMS_LOB.SUBSTR(DOC,100,1)
--------------------------------------------------------------------------------
0D0A6F6F6F0D0A686868686868686868686868686868686868686868686868686868686868686868
0D0A6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F6F0D0A686868
6868686868686868686868686868686868686868

Monday, June 6, 2011

DBMS_FGA - ORACLE FINE GRAINED AUDITING

Suppose your boss calls you one day and tells you that there has been some unexpected changes in the employee database.Employee's designation, their salary are being manipulated illegally.Such things have been continuing from a past few days and he asks if you could help getting hold of the culprit.
Don't worry Oracle's DBMS_FGA package will save your day and earn you a raise in your job.

The Oracle DBMS_FGA package provides fine grained auditing on objects.


To have an overview of the summary of dbms_fga subprograms visit :

http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14258/d_fga.htm#i1011920

In this article I am going to add a policy on a table FGA_TEST in the SCOTT schema.
The policy will report on any dml actions on this table affecting its 2 columns 'esal' and 'designation'.
Another user HACKER will execute dml queries on this table and we will try and investigate whether the actions of the HACKER are reported.
The corresponding event handler of this policy will be in a 3rd schema FGA_HANDLER .we will also find out if the audit event was handled properly.

Note: Here it is worthwhile mentioning that the DBMS_FGA package can be used not only to audit records in case of data manipulation(DML) but also in cases where data might have been simply viewed depending upon the policy we define. eg:- in case of selecting particular records from a database table.

SQL> show user;
USER is "SCOTT"



First of all let us create a new schema FGA_HANDLER which will contain the event handler .


SQL> create user fga_handler identified by fga_handler;

User created.

SQL> grant resource,connect to fga_handler;

Grant succeeded.



Now, let us create a new table ,FGA_TEST in SCOTT schema, on which we will enforce the audit conditions(policy) with the help of the DBMS_FGA package.


SQL> create table fga_test
2 (empno number,
3 empname varchar2(30),
4 esal number,
5 designation varchar2(20)
6 );

Table created.



Let us insert some dummy rows in it now.


SQL> insert into FGA_TEST
2 SELECT 1,'dwaipayan',20000,'programmer' from dual
3 union all
4 select 2,'dhruva',30000,'analyst' from dual
5 union all
6 select 3,'shiba',40000,'manager' from dual
7 ;

3 rows created.

SQL> select * from fga_test;

EMPNO EMPNAME ESAL DESIGNATION
---------- ------------------------------ ---------- --------------------
1 dwaipayan 20000 programmer
2 dhruva 30000 analyst
3 shiba 40000 manager



Given below are the parameters of the ADD_POLICY Procedure:
Now, let us add a policy on our FGA_TEST table such that whenever any user tries to insert, update or delete the ‘esal’ or the ‘designation’ columns of any row of FGA_TEST table ,the action will be recorded.



BEGIN
DBMS_FGA.ADD_POLICY (
object_schema => 'SCOTT',
object_name => 'FGA_TEST',
policy_name => 'FGA_TEST_POLICY',
audit_condition => NULL,
audit_column => 'ESAL,DESIGNATION',
handler_schema => 'FGA_HANDLER',
handler_module => 'sp_audit',
enable => true,
statement_types => ‘INSERT,UPDATE,DELETE’
);
end;

Note: the default 'statement_types' is 'SELECT'

Now, sp_audit is the audit procedure, which will act as the alerting mechanism for the administrator.

The required interface for such a procedure is as follows:
PROCEDURE ( object_schema VARCHAR2, object_name VARCHAR2, policy_name VARCHAR2 ) AS ....
Now, let us connect to the FGA_HANDLER schema and design the
the sp_audit procedure.

First let us create the table where the sp_audit procedure will dump the data into.


SQL> conn fga_handler/fga_handler;
Connected.

SQL> create table audit_event
2 (audit_event_no number);

Table created.


We create the sp_audit procedure as follows:



SQL> create or replace procedure sp_audit
(object_schema in varchar2,
object_name in varchar2,
policy_name in varchar2
)
as
count number;
begin

select nvl(max(audit_event_no),0) into count from audit_event;

insert into audit_event
values
(count+1);
commit;

end;


The procedure simply adds a record to the audit_event table each time it is executed and the column audit_event_no acts as counter which displays the number of times the proc has been executed.

Now, finally we create another schema ‘HACKER’ which tries to manipulate the values of the ‘esal’ or ‘designation’ columns of the FGA_TEST table.


SQL> create user hacker identified by hacker;
User created.

SQL> grant resource,connect to hacker;
Grant succeeded.

SQL> grant all on fga_test to hacker;
Grant succeeded.
Now we log onto HACKER and try to change the ‘designation’ of ‘dwaipayan’ from ‘Programmer’ to ‘manager’

SQL> conn hacker/hacker;
Connected.

SQL> update scott.fga_test set designation='manager' where empname='dwaipayan';

1 row updated.



Now, we connect with SCOTT to see the dba_fga_audit_trail view to find if the event was recorded.


SQL> select DB_USER,OS_USER,POLICY_NAME,SQL_TEXT,
TIMESTAMP from dba_fga_audit_trail
where POLICY_NAME='FGA_TEST_POLICY';







DB_USER OS_USER POLICY_NAME SQL_TEXT TIMESTAMP
HACKER HOME-6C286D743C\Dwaipayan FGA_TEST_POLICY update scott.fga_test
set
designation
= 'manager'
where
empname='dwaipayan'
5-Jun-11



Now we connect to the FGA_HANDLER schema to see if the event handler(sp_audit) was called:


SQL> conn fga_HANDLER/fga_handler;
Connected.
SQL> select * from audit_event;

AUDIT_EVENT_NO
--------------
1



We execute the following from HACKER schema:

SQL>update SCOTT.FGA_TEST set designation='HR' where name='shiba';
SQL> conn fga_HANDLER/fga_handler;
Connected.
SQL> select * from audit_event;


AUDIT_EVENT_NO
--------------
1
2



Note: I have just created the sp_audit procedure to add rows to the audit_event table in this testing environment but in a ideal production scenario, we are likely to send emails or Page instead.