Skip to content

Posts tagged ‘oracle’

ORA-1652: unable to extend temp segment by 128 in tablespace TEMP01

Problem:

TEMP tablespace was getting filled when running sql statement.

i increased the temp table size from 25g, 32g, 64 gb. in all cases it throws error.

Solution:

I ran sql tuning advisor on sql and it gave recommendation of profile, index and gather stats.

i created profile, index and gather stats. it did magic sql completed under 5 seconds and did not throw any error.

Oracle DBA Interview Question

Question: User called and said he or she dropped all tables in schema and want to restore it. How will you restore all tables?

Please post your answer in comments.

STARTUP Database failed ORA-38760 to turn on Flashback Database

SYMPTOMS

This is not RAC specific and could happen in Single Instances as well
if Flashback logs required are not available, as for example FLB logs was deleted using OS commands.


The following example shows symptoms seen in a RAC environment.

One of the RAC Instances does not come up, its looking for a Flash back log which has been accidently deleted.

SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-38760: This database instance failed to turn on flashback database

——–From alert log of the thread 2 ——————

Thu May 16 05:17:45 2013
Allocated 15937344 bytes in shared pool for flashback generation buffer
Starting background process RVWR
RVWR started with pid=33, OS id=7292
Thu May 16 05:17:48 2013
Errors in file /<path>/udump/<SID>_ora_7244.trc:
ORA-38701: Flashback database log 25184 seq 14006 thread 1: “+<Disk>/<db unique name>/flashback/log_25184.25633.809806337”
ORA-17503: ksfdopn:2 Failed to open file +<Disk>/<db unique name>/flashback/log_25184.25633.809806337
ORA-15012: ASM file ‘+<Disk>/<db unique name>/flashback/log_25184.25633.809806337’ does not exist
Thu May 16 05:17:48 2013
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: ALTER DATABASE   MOUNT
Thu May 16 05:17:48 2013
ALTER DATABASE OPEN
ORA-38760 signalled during: ALTER DATABASE OPEN…                                                                   <<<< Error on opening database

Oracle is trying to turn on flashback database on startup automatically and failing with ORA-38760 as flashback log is missing.

CHANGES

Current flashback log deleted accidently from OS level.

CAUSE

When Current flashback log is deleted then database open will fail with following error:

Starting ORACLE instance (normal)
…….

Thu May 16 05:17:45 2013
Allocated 15937344 bytes in shared pool for flashback generation buffer
Starting background process RVWR
RVWR started with pid=33, OS id=7292
Thu May 16 05:17:48 2013
Errors in file /<path>/udump/<sid>_ora_7244.trc:
ORA-38701: Flashback database log 25184 seq 14006 thread 1: “+<Disk>/<db unique name>/flashback/log_25184.25633.809806337”
ORA-17503: ksfdopn:2 Failed to open file +<Disk>/<db unique name>/flashback/log_25184.25633.809806337
ORA-15012: ASM file ‘+<Disk>/<db unique name>/flashback/log_25184.25633.809806337’ does not exist
Thu May 16 05:17:48 2013
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: ALTER DATABASE   MOUNT
Thu May 16 05:17:48 2013
ALTER DATABASE OPEN
ORA-38760 signalled during: ALTER DATABASE OPEN…

OR Instance may crash with following error:ORA-38701: Flashback database log 101 seq 101 thread 1: “/<path>/flashback/o1_mf_15fb13pw_.flb”
ORA-27037: unable to obtain file status
Linux Error: 2: No such file or directory
Additional information: 3
Sat Apr 9 03:13:52 2005
RVWR: terminating instance due to error 38701
Instance terminated by RVWR, pid = 5725

SOLUTION

There are 2 possible solutions when a flashback log file was deleted manually:

(1) Turn off Flashback:SQL> alter database flashback off;

The Alert log will have the following entries when you turn off flashback:Thu May 16 05:18:01 2013
alter database flashback off
Thu May 16 05:18:01 2013
Flashback Database Disabled
…….
Completed: alter database flashback off                                                                              <<<<  Flashback disabled
…..

After this turn the flashback on again if required.SQL> alter database flashback on;

Now you can open the database.

(2) In cases where “Guaranteed Restore Point” is defined, RVWR will still try to write to flashback log even though you have disable it and it will fail again with same error:Completed: alter database flashback off                                                                              <<<<  Flashback disabled
…..

Thu May 16 05:19:05 2013
Shutting down instance: further logons disabled
Thu May 16 05:19:05 2013
Stopping background process CJQ0
……

Shutting down instance (immediate)
……..
Thu May 16 05:24:49 2013
Starting ORACLE instance (normal)
…….
Thu May 16 05:25:04 2013
Allocated 15937344 bytes in shared pool for flashback generation buffer
Starting background process RVWR
RVWR started with pid=33, OS id=19851
Thu May 16 05:25:05 2013
Errors in file /<path>/udump/<SID>_ora_19796.trc:                                                  
ORA-38701: Flashback database log 25184 seq 14006 thread 1: “+<Disk>/<db unique name>/flashback/log_25184.25633.809806337”
ORA-17503: ksfdopn:2 Failed to open file +<Disk>/<db unique name>/flashback/log_25184.25633.809806337
ORA-15012: ASM file ‘+<Disk>/<db unique name>/flashback/log_25184.25633.809806337’ does not exist                                <<<< Still looking for deleted flashback log
Thu May 16 05:25:05 2013
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Completed: ALTER DATABASE   MOUNT
Thu May 16 05:25:05 2013
ALTER DATABASE OPEN
ORA-38760 signalled during: ALTER DATABASE OPEN…                                                                   <<<< Again failed with ORA-38760

Check if flashback database is disable or not. When checked the flashback status after disabling flashback, it shows ‘RESTORE POINT ONLY’ instead of ‘NO’.

SQL>  select flashback_on from v$database;

FLASHBACK_ON
——————
RESTORE POINT ONLYFLASHBACK_ON – possible values are as follows:

    YES                – Flashback is on
    NO                 – Flashback is off
    RESTORE POINT ONLY – Flashback is on but one can only flashback to guaranteed restore points

 
So the cause of the issue is Guaranteed Restore Point created on database.
The database would still try to write flashback data to the current flashback log because the database still has at least one Guaranteed Restore Point declared in the controlfile.

Find out the name of Guaranteed Restore Point and delete so that database would not try to write to flashback log on startup:

Now we have 3 options to know the restore point name:

2.1) Check the name from v$restore_point view but that would also fail with same error:SQL> select * from v$restore_point;
select * from v$restore_point
*
ERROR at line 1:
ORA-38701: Flashback database log 2 seq 2 thread 1: “+<Disk>/<db unique name>/flashback/log_2.2286.801367563”
ORA-17503: ksfdopn:2 Failed to open file +<Disk>/<db unique name>/flashback/log_2.2286.801367563
ORA-15012: ASM file ‘+<Disk>/<db unique name>/flashback/log_2.2286.801367563’ does not exist

Here we are not able to query v$restore _point to find out the name.

2.2) Search for restore point name in alert log. In this case customer was purging alert log every year starting so could not find name for the Restore Point.

2.3) Dump the controlfile to get the restore point name:SQL> oradebug setmypid
SQL> alter session set events ‘immediate trace name controlf level 9’;
SQL> oradebug tracefile_name


From trace file of controlfile dump, we could see below information:***************************************************************************
RESTORE POINT RECORDS
***************************************************************************
 (size = 212, compat size = 212, section max = 2048, section in-use = 1,
  last-recid= 1, old-recno = 0, last-recno = 0)
 (extent = 1, blkno = 240, numrecs = 2048)
RESTORE POINT #1:
 restore point name: <restore point test> guarantee flag: 1 incarnation: 2next record 0                            <<<< Name of restore point
 restore point scn: 0x0000.fbff3d87 12/07/2012 02:16:32

Now we have name of Guaranteed Restore Point:

SQL> Drop restore point <restore point test>;

SQL> alter database open;

Database AlteredNote:  This error may also occur if the FRA is full and thus a flashback log cannot be created.  Check V$flash_recovery_area_usage and/or V$recovery_file_dest regarding space availability in FRA

Oracle Script to find unindexed Foreign key that cause blocking

This script suggest index on foreign key that cause blocking

run this code and enable dbms output to see result. it will show existing index on the table and recommended index


SET SERVEROUTPUT ON
DECLARE
    PROCEDURE print_all (s VARCHAR2)
    IS
    BEGIN
        NULL;
        DBMS_OUTPUT.put_line (s);
    END;

    PROCEDURE print_ddl (s VARCHAR2)
    IS
    BEGIN
        NULL;
        DBMS_OUTPUT.put_line (s);
    END;
BEGIN
    DBMS_METADATA.set_transform_param (DBMS_METADATA.session_transform,
                                       'SEGMENT_ATTRIBUTES',
                                       FALSE);

    FOR a
        IN (  SELECT COUNT (*)                                       samples,
                     event,
                     p1,
                     p2,
                     o.owner                                         c_owner,
                     o.object_name                                   c_object_name,
                     p.object_owner                                  p_owner,
                     p.object_name                                   p_object_name,
                     id,
                     operation,
                     MIN (p1 - 1414332420 + 4)                       lock_mode,
                     MIN (sample_time)                               min_time,
                     MAX (sample_time)                               max_time,
                     CEIL (10 * COUNT (DISTINCT sample_id) / 60)     minutes
                FROM dba_hist_active_sess_history
                     LEFT OUTER JOIN dba_hist_sql_plan p USING (dbid, sql_id)
                     LEFT OUTER JOIN dba_objects o ON object_id = p2
                     LEFT OUTER JOIN dba_objects po
                         ON po.object_id = current_obj#
               WHERE     event LIKE 'enq: TM%'
                     AND p1 >= 1414332420
                     AND sample_time > SYSDATE - 15
                     AND p.id = 1
                     AND operation IN ('DELETE', 'UPDATE', 'MERGE')
            GROUP BY event,
                     p1,
                     p2,
                     o.owner,
                     o.object_name,
                     p.object_owner,
                     p.object_name,
                     po.owner,
                     po.object_name,
                     id,
                     operation
            ORDER BY COUNT (*) DESC)
    LOOP
        print_ddl (
               '--  '
            || a.operation
            || ' on '
            || a.p_owner
            || '.'
            || a.p_object_name
            || ' has locked '
            || a.c_owner
            || '.'
            || a.c_object_name
            || ' in mode '
            || a.lock_mode
            || ' for '
            || a.minutes
            || ' minutes between '
            || TO_CHAR (a.min_time, 'dd-mon hh24:mi')
            || ' and '
            || TO_CHAR (a.max_time, 'dd-mon hh24:mi'));

        FOR s
            IN (SELECT DISTINCT
                       REGEXP_REPLACE (
                           CAST (SUBSTR (sql_text, 1, 2000) AS VARCHAR2 (60)),
                           '[^a-zA-Z ]',
                           ' ')    sql_text
                  FROM dba_hist_active_sess_history
                       JOIN dba_hist_sqltext t USING (dbid, sql_id)
                 WHERE     event LIKE 'enq: TM%'
                       AND p2 = a.p2
                       AND sample_time > SYSDATE - 15)
        LOOP
            print_all ('--      ' || 'blocked statement: ' || s.sql_text);
        END LOOP;

        FOR c
            IN (WITH
                    c
                    AS
                        (SELECT p.owner          p_owner,
                                p.table_name     p_table_name,
                                c.owner          c_owner,
                                c.table_name     c_table_name,
                                c.delete_rule,
                                c.constraint_name
                           FROM dba_constraints  p
                                JOIN dba_constraints c
                                    ON (    c.r_owner = p.owner
                                        AND c.r_constraint_name =
                                            p.constraint_name)
                          WHERE     p.constraint_type IN ('P', 'U')
                                AND c.constraint_type = 'R')
                    SELECT c_owner                     owner,
                           constraint_name,
                           c_table_name,
                              CONNECT_BY_ROOT (p_owner || '.' || p_table_name)
                           || SYS_CONNECT_BY_PATH (
                                     DECODE (delete_rule,
                                             'CASCADE', '(cascade delete)',
                                             'SET NULL', '(cascade set null)',
                                             ' ')
                                  || ' '
                                  || c_owner
                                  || '"."'
                                  || c_table_name,
                                  ' referenced by')    foreign_keys
                      FROM c
                     WHERE     LEVEL <= 10
                           AND c_owner = a.c_owner
                           AND c_table_name = a.c_object_name
                CONNECT BY NOCYCLE     p_owner = PRIOR c_owner
                                   AND p_table_name = PRIOR c_table_name
                                   AND (   LEVEL = 1
                                        OR PRIOR delete_rule IN
                                               ('CASCADE', 'SET NULL'))
                START WITH     p_owner = a.p_owner
                           AND p_table_name = a.p_object_name)
        LOOP
            print_all (
                   '--      '
                || 'FK chain: '
                || c.foreign_keys
                || ' ('
                || c.owner
                || '.'
                || c.constraint_name
                || ')'
                || ' unindexed');

            FOR l
                IN (SELECT *
                      FROM dba_cons_columns
                     WHERE     owner = c.owner
                           AND constraint_name = c.constraint_name)
            LOOP
                print_all ('--         FK column ' || l.column_name);
            END LOOP;

            print_ddl (
                   '--      Suggested index: '
                || REGEXP_REPLACE (
                       TRANSLATE (
                           DBMS_METADATA.get_ddl ('REF_CONSTRAINT',
                                                  c.constraint_name,
                                                  c.owner),
                           CHR (10) || CHR (13),
                           '  '),
                       'ALTER TABLE ("[^"]+"[.]"[^"]+") ADD CONSTRAINT ("[^"]+") FOREIGN KEY ([(].*[)]).* REFERENCES ".*',
                       'CREATE INDEX ON \1 \3;'));

            FOR x
                IN (SELECT RTRIM (
                               TRANSLATE (
                                   DBMS_METADATA.get_ddl ('INDEX',
                                                          index_name,
                                                          index_owner),
                                   CHR (10) || CHR (13),
                                   '  '))    ddl
                      FROM dba_ind_columns
                     WHERE     (index_owner, index_name) IN
                                   (SELECT owner, index_name
                                      FROM dba_indexes
                                     WHERE     owner = c.owner
                                           AND table_name = c.c_table_name)
                           AND column_name IN
                                   (SELECT column_name
                                      FROM dba_cons_columns
                                     WHERE     owner = c.owner
                                           AND constraint_name =
                                               c.constraint_name))
            LOOP
                print_ddl ('--      Existing candidate indexes ' || x.ddl);
            END LOOP;

            FOR x
                IN (SELECT RTRIM (
                               TRANSLATE (
                                   DBMS_METADATA.get_ddl ('INDEX',
                                                          index_name,
                                                          index_owner),
                                   CHR (10) || CHR (13),
                                   '  '))    ddl
                      FROM dba_ind_columns
                     WHERE (index_owner, index_name) IN
                               (SELECT owner, index_name
                                  FROM dba_indexes
                                 WHERE     owner = c.owner
                                       AND table_name = c.c_table_name))
            LOOP
                print_all ('--       Other existing Indexes: ' || x.ddl);
            END LOOP;
        END LOOP;
    END LOOP;
END;
/


Sample Ouput

— MERGE on hr.addr has locked hr.address in mode 5 for 5 minutes between 09-aug 15:56 and 13-aug 14:50
— blocked statement: MERGE INTO addr C USING SELECT C CNTC ID FROM ac
— FK chain: hr.addr referenced by hr”.”address (hr.add4) unindexed
— FK column CNTC_ID_PAR
— Suggested index: CREATE INDEX ON “hr”.”address” (“CNTC_ID_PAR”);
— Other existing Indexes: CREATE UNIQUE INDEX “hr”.”addr_hist” ON “hr”.”address” (“add1”, “add2”, “add3”)
— Other existing Indexes: CREATE UNIQUE INDEX “hr”.”addr_hist” ON “hr”.”address” (“add1”, “add2”, “add3”)
— Other existing Indexes: CREATE UNIQUE INDEX “hr”.”addr_hist” ON “hr”.”address” (“add1”, “add2”, “add3”)
— Other existing Indexes: CREATE INDEX “hr”.”ITD_address_NDX1″ ON “hr”.”address” (“CNTC_ID”)

%d bloggers like this: