DPA: Oracle client library not initialised (-1) Oracle-Rman Data Collection Error

Article Number: 490245 Article Version: 4 Article Type: Break Fix



Data Protection Advisor 6.3

Monitoring Oracle RMAN as a Control File results in data collection failing with an error of: “Oracle Client Library not initialised (-1)”

When running a request history report against the RMAN object we can see the error:


User-added image

When Checking the Data Protection Advisor (DPA) Agent log you will also find the same error

In one instance where this was observed, This was a newly discovered version 11.2 Oracle RMAN that was being monitored as a Control File.

The oracle client library files were not set to be executable causing the error “Failed to connect to database testdb01 on server testserver1.local.com: Oracle client library not initialised (-1) ”

Following were how the permissions for the Oracle client libraries were set;

[root@server01 lib]# pwd

/usr/lib/oracle/11.2/client64/lib

[root@server01 lib]# ls -l

total 123456789

-rw-r–r–. 1 root root 123456 OCT 5 2010 libclntsh.so.11.1

-rw-r–r–. 1 root root 1234 OCT 5 2010 libnnz11.so

-rw-r–r–. 1 root root 12345 OCT 5 2010 libocci.so.11.1

-rw-r–r–. 1 root root 123456 OCT 5 2010 libociicus.so

-rw-r–r–. 1 root root 123 OCT 5 2010 libocijdbc11.so

-rw-r–r–. 1 root root 1234567 OCT 5 2010 ojdbc5.jar

-rw-r–r–. 1 root root 1234567 OCT 5 2010 ojdbc6.jar

-rw-r–r–. 1 root root 123456 OCT 5 2010 xstreams.jar

Set the Oracle Client Libraries to the following permissions;

-rwxr-xr-x 1 root root123456 OCT 5 2010 libclntsh.so.11.1

-rwxr-xr-x 1 root root 1234 OCT 5 2010 libnnz11.so

-rwxr-xr-x 1 root root 12345 OCT 5 2010 libocci.so.11.1

-rwxr-xr-x 1 root root 123456 OCT 5 2010 libociicus.so

-rwxr-xr-x 1 root root 123 OCT 5 2010 libocijdbc11.so

-rwxr-xr-x 1 root root 1234567 OCT 5 2010 ojdbc5.jar

-rwxr-xr-x 1 root root 1234567 OCT 5 2010 ojdbc6.jar

-rwxr-xr-x 1 root root 123456 OCT 5 2010 xstreams.jar

Related:

Re: DDBoost Oracle Rman Agent Installation with multiple dbs on one server

Hello David,

Here is the link to the Administrator guide for RMAN. https://support.emc.com/docu86478_Oracle_RMAN_Agent_4.5_Administration_Guide.pdf?language=en_US&source=Coveo If you look on page 16 it list the following:

The Oracle RMAN agent enables database servers to communicate with Data Domain systems in an optimized way, without the need to use a backup application.

Note

In earlier versions of the product, the Oracle RMAN agent was known as Data Domain Boost (DD Boost) for Oracle Recovery Manager (RMAN) or the RMAN plug-in. In version 4.0, the product name changed to Oracle RMAN agent.

The use of the Oracle RMAN agent improves performance while reducing the amount of data that is transferred over the LAN. In the context of Oracle RMAN, the software has two components: l An RMAN agent that you install on each database server. This RMAN agent includes the DD Boost libraries for communicating with the DD Boost server that runs on the Data Domain system. l The DD Boost server that runs on the Data Domain systems. RMAN sets policies that control when backups and replications occur. Administrators manage backup, replication, and restore from a single console and can use all the features of DD Boost, including WAN-efficient replicator software. RMAN manages all the files (collections of data) in the catalog, even those created by the Data Domain system. The Data Domain system exposes pre-made disk volumes called storage units to a DD Boost enabled database server. Multiple database servers, each with the Oracle RMAN agent, can use the same storage unit on a Data Domain system as a storage server. Each database server can run a different operating system, provided that it is supported by Data Domain. The following figure shows an example configuration of the Oracle RMAN agent.

Please let us know if you have any other questions.

Related:

RMAN recovery throws error :’NSR_CLIENT’ was not specified

Oracle DB: 12.2.0.1

Networker: v9.2.1.0

RMAN> show channel;

RMAN configuration parameters for database with db_unique_name TEST are:

CONFIGURE CHANNEL DEVICE TYPE ‘SBT_TAPE’ PARMS ‘ENV=(NSR_SERVER=nw-srv.ilearn.com.co,NSR_CLIENT= hallsdb.ilearn.com.co)’;

[oracle@halldbtest ~]$ rman target /

Recovery Manager: Release 12.2.0.1.0 – Production on Tue Oct 16 10:26:23 2018

Copyright (c) 1982, 2017, Oracle and/or its affiliates. All rights reserved.

connected to target database: TEST (DBID=2264480964)

RMAN> recover table scott.salgrade

until time “to_date(’16/10/2018 10:00:00′,’DD/MM/YYYY hh24:mi:ss’)”

auxiliary destination ‘/u01/temp’

remap table scott.salgrade:salgrade_bkp;

Starting recover at 16-OCT-18

using target database control file instead of recovery catalog current log archived

allocated channel: ORA_SBT_TAPE_1

channel ORA_SBT_TAPE_1: SID=1 device type=SBT_TAPE

channel ORA_SBT_TAPE_1: NMDA Oracle v9.2.1.0

allocated channel: ORA_SBT_TAPE_2

channel ORA_SBT_TAPE_2: SID=505 device type=SBT_TAPE

channel ORA_SBT_TAPE_2: NMDA Oracle v9.2.1.0

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=624 device type=DISK

RMAN-05026: warning: presuming following set of tablespaces applies to specified point-in-time

List of tablespaces expected to have UNDO segments

Tablespace SYSTEM

Tablespace UNDOTBS1

Creating automatic instance, with SID=’elCb’

initialization parameters used for automatic instance:

db_name=TEST

db_unique_name=elCb_pitr_TEST

compatible=12.2.0

db_block_size=8192

db_files=200

diagnostic_dest=/u01/app/oracle

_system_trig_enabled=FALSE

sga_target=9600M

processes=200

db_create_file_dest=/u01/temp

log_archive_dest_1=’location=/u01/temp’

#No auxiliary parameter file used

starting up automatic instance TEST

Oracle instance started

Total System Global Area 10066329600 bytes

Fixed Size 12169992 bytes

Variable Size 1644170488 bytes

Database Buffers 8388608000 bytes

Redo Buffers 21381120 bytes

Automatic instance created

contents of Memory Script:

{

# set requested point in time

set until time “to_date(’16/10/2018 10:00:00′,’DD/MM/YYYY hh24:mi:ss’)”;

# restore the controlfile

restore clone controlfile;

# mount the controlfile

sql clone ‘alter database mount clone database’;

# archive current online log

sql ‘alter system archive log current’;

}

executing Memory Script

executing command: SET until clause

Starting restore at 16-OCT-18

allocated channel: ORA_AUX_SBT_TAPE_1

channel ORA_AUX_SBT_TAPE_1: SID=169 device type=SBT_TAPE

channel ORA_AUX_SBT_TAPE_1: NMDA Oracle v9.2.1.0

allocated channel: ORA_AUX_SBT_TAPE_2

channel ORA_AUX_SBT_TAPE_2: SID=211 device type=SBT_TAPE

channel ORA_AUX_SBT_TAPE_2: NMDA Oracle v9.2.1.0

allocated channel: ORA_AUX_DISK_1

channel ORA_AUX_DISK_1: SID=251 device type=DISK

Removing automatic instance

shutting down automatic instance

Oracle instance shut down

Automatic instance removed

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of recover command at 10/16/2018 10:27:07

RMAN-03015: error occurred in stored script Memory Script

ORA-27191: sbtinfo2 returned error

Additional information: 3620

ORA-19511: non RMAN, but media manager or vendor specific failure, error text:

The value of parameter ‘NSR_CLIENT’ was not specified. (9:13:11)

Any idea ? Thanks

Related:

Avamar Client for Windows: Avamar backup fails with “avtar Error : Out of memory for cache file” on Windows clients

Article Number: 524280 Article Version: 3 Article Type: Break Fix



Avamar Plug-in for Oracle,Avamar Client for Windows,Avamar Client for Windows 7.2.101-31



In this scenario we have the same issue presented in the KB 495969 however the solution does not apply due to an environment issue on a Windows client.

  • KB 495969 – Avamar backup fails with “Not Enough Space” and “Out of Memory for cache file”

The issue could affect any plugin like in this case with the error presented in the following manner:

  • For FS backups:
avtar Info <8650>: Opening hash cache file 'C:Program Filesavsvarp_cache.dat'avtar Error <18866>: Out of memory for cache file 'C:Program Filesavsvarp_cache.dat' size 805306912avtar FATAL <5351>: MAIN: Unhandled internal exception Unix exception Not enough space
  • For VSS backups:
avtar Info <8650>: Opening hash cache file 'C:Program Filesavsvarp_cache.dat'avtar Error <18866>: Out of memory for cache file 'C:Program Filesavsvarp_cache.dat' size 1610613280avtar FATAL <5351>: MAIN: Unhandled internal exception Unix exception Not enough space
  • For Oracle backup:
avtar Info <8650>: Opening hash cache file 'C:Program Filesavsvarclientlogsoracle-prefix-1_cache.dat'avtar Error <18866>: Out of memory for cache file 'C:Program Filesavsvarclientlogsoracle-prefix-1_cache.dat' size 100663840avtar FATAL <5351>: MAIN: Unhandled internal exception Unix exception Not enough spaceor this variant:avtar Info <8650>: Opening hash cache file 'C:Program Filesavsvarclientlogsoracle-prefix-1_cache.dat'avtar Error <18864>: Out of restricted memory for cache file 'C:Program Filesavsvarclientlogsoracle-prefix-1_cache.dat' size 100663840avtar FATAL <5351>: MAIN: Unhandled internal exception Unix exception Not enough space avoracle Error <7934>: Snapup of <oracle-db> aborted due to rman terminated abnormally - check the logs
  • With the RMAN log reporting this:
RMAN-00571: ===========================================================RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============RMAN-00571: ===========================================================RMAN-03002: failure of backup plus archivelog command at 06/14/2018 22:17:40RMAN-03009: failure of backup command on c0 channel at 06/14/2018 22:17:15ORA-04030: out of process memory when trying to allocate 1049112 bytes (KSFQ heap,KSFQ Buffers)Recovery Manager complete. 

Initially it was though the cache file could not grow in size due to incorrect “hashcachemax” value.

The client had plenty of free RAM (48GB total RAM) so we increase the flag’s value from -16 (3GB file size max) to -8 (6GB file size max)

But the issue persisted and the disk space was also not an issue, there was plenty of GBs of free space

Further investigations with a test binary from the engineering team lead to the fact that the MS OS was not releasing enough unused and contiguous memory required to allocate/load into the memory the entire hash cache file for the backup operation.

It was tried a test binary that would allocate the memory in smaller pieces to see if we could reach the point where the OS would allow the full file p_cache.dat to be loaded into memory but that also did not help, the Operative system was still not allowing to load the file into memory for some reason.

The root cause is hided somewhere in the OS however in this case we did not engage the MS team for further investigations on their side.

Instead we found a way to work around the issue setting the cache file to be smaller, see details in the resolution section below.

In order to work around this issue we set the hash cache file to be of a smaller size so that the OS would not have issues in allocating it into memory.

In this case it was noticed that the OS was also having problems in allocating smaller sizes like 200+ MB so we decided to re-size the p_cache.dat to be just 100MB with the use of the following flag:

–hashcachemax=100

This way the hash cache file would never grow beyond 100MB and would overwrite the old entries.

After adding that flag it is requited to recycle the cache file by renaming or deleting the p_cache.dat (renaming is the preferred option)

After the first backup which would take longer than usual as expected (to rebuild the cache file) the issue should be resolved.

  • The Demand-paging cache is not recommended in this scenario since the backup are directed to GSAN storage so the Monolithic paging cache was used.
  • Demand-paging was designed to gain benefit for backup being sent to DataDomain storage.

Related:

Avamar Plug-in for Oracle: Oracle Avamar rman CLI archivelog backup does not run concurrently with database backups when a taskfile is in use

Article Number: 500862 Article Version: 3 Article Type: Break Fix



Avamar Plug-in for Oracle

Oracle Avamar rman CLI archivelog backup does not run concurrently with database backups when a taskfile is in use.

rman script that does the database backup uses the same taskfile than archivelog backups

rman script to do database backups:

run {

allocate channel t1 type sbt PARMS ‘SBT_LIBRARY=/usr/local/avamar/lib/libobk_avamar64.so’ SEND ‘”–flagfile=/home/oracle/flags.txt” “–bindir=/usr/local/avamar/bin” “–cacheprefix=t1” “–vardir=/var/avamar” “–logfile=/var/avamar/t1.log” “–taskfile=/home/oracle/taskflag.txt“‘ trace 5;

rman script that does archivelog backups:

run {

allocate channel t1 type sbt PARMS ‘SBT_LIBRARY=/usr/local/avamar/lib/libobk_avamar64.so’ SEND ‘”–flagfile=/home/oracle/flags.txt” “–bindir=/usr/local/avamar/bin” “–cacheprefix=t1arch” “–vardir=/var/avamar” “–logfile=/var/avamar/t1arch.log” “–taskfile=/home/oracle/taskflag.txt“‘ trace

5;

Note: taskfile is a file that contains info about number of channels and if the operation is backup or restore.

Concurrent operations should use different taskfile names.

  • Create a different file name for the taskfile used for archivelog backups.

Example:


rman script to do database backups:

run {

allocate channel t1 type sbt PARMS ‘SBT_LIBRARY=/usr/local/avamar/lib/libobk_avamar64.so’ SEND ‘”–flagfile=/home/oracle/flags.txt” “–bindir=/usr/local/avamar/bin” “–cacheprefix=t1” “–vardir=/var/avamar” “–logfile=/var/avamar/t1.log” “–taskfile=/home/oracle/taskflag.txt“‘ trace 5;


rman script that does archivelog backups:

allocate channel t1 type sbt PARMS ‘SBT_LIBRARY=/usr/local/avamar/lib/libobk_avamar64.so’ SEND ‘”–flagfile=/home/oracle/flags.txt” “–bindir=/usr/local/avamar/bin” “–cacheprefix=t1arch” “–vardir=/var/avamar” “–logfile=/var/avamar/t1arch.log” “–taskfile=/home/oracle/taskarch.txt“‘ trace 5;

Related:

which license do I need for Mtree replication from DD2200 to DD3300 ?

Hi Guys,



I need your lights



Current config

DD2200

DDboost license

Oracle RMAN backup ( with DDboost plugin ) directly to the DD2200



Customer has purchase a DD3300 ( with DDboost Lic ) with a 16 TB license.



Question:

If I decide to set a Mtree replication between the dd2200 and the dd3300 for the RMAN DDboost storage unit:

DO I need a “replication license ?



If no presence of the replication license:

Is the DDboost license present on both DD boxes NOT sufficient for the Mtree replication ?



Thanks for your help.



JJ.

Related:

Re: how to back up oracle db to storage node?

Hello everyone,

Now I am using Networker 8.2.1 to back up oracle db to a local disk created as an AFTD device of a storage node. Does anyone know how to configure a task to back up oracle database to storage node? for now, the RMAN script on Networker is below.

RUN { ALLOCATE CHANNEL CH1 TYPE DISK;

SEND channel CH1 ‘NSR_ENV=(NSR_SERVER=networker.test.com,NSR_CLIENT=oracle);

BACKUP

FULL

FORMAT ‘%d_%U’

DATABASE INCLUDE CURRENT CONTROLFILE

PLUS ARCHIVELOG;

RELEASE CHANNEL CH1;

}

this script only backed up database to a local disk of oracle host.

regards

Related:

how to back up oracle db to storage node?

Hello everyone,

Now I am using Networker 8.2.1 to back up oracle db to a local disk created as an AFTD device of a storage node. Does anyone know how to configure a task to back up oracle database to storage node? for now, the RMAN script on Networker is below.

RUN { ALLOCATE CHANNEL CH1 TYPE DISK;

SEND channel CH1 ‘NSR_ENV=(NSR_SERVER=networker.test.com,NSR_CLIENT=oracle);

BACKUP

FULL

FORMAT ‘%d_%U’

DATABASE INCLUDE CURRENT CONTROLFILE

PLUS ARCHIVELOG;

RELEASE CHANNEL CH1;

}

this script only backed up database to a local disk of oracle host.

regards

Related:

Re: DD Oracle Agent deployment.

We are in process of replacing Oracle RMAN to NFS (HP) share to DD Agent for Oracle.

Having No / limited knowledge in Oracle domain for which I am asked to integrate it with DD.

DD Model : DD6300 (Replicating to identical box on a remote site)

DD OS : 6.0.2.20-587212

Need to write backup copy to Physical Tape as well.

**(DD is already working perfectly for Veeam using DDBoost and also having a remote replicated copy as mentioned.)

===================================

**Oracle RAC Details from its build document.

Oracle Solaris 11.2 for SPARC 64-bit OS. (11.2.9.5.0)

Recovery Manager: Release 11.2.0.4.0 – Production and configured for daily full backup.

Multi Node HA Cluster.

High Availability Infrastructure: Oracle Cluster-ware and ASM software for Oracle Database.

There are Two DB servers and Two Application Servers configured.

===================================

Questions:

  1. What is the recommended DD topology. (Install agents on all nodes.? / Otherwise) – Currently I understand the backups were cascaded; meaning that more than one oracle servers were dumping data internally to one server and that final server was offloading the data to the NSF Share.
  2. What is the recommended / Compatible Agent version.
  3. Any specific steps / Procedure to achieve the integration.?
  4. Replication should be configured separately using MTree.? Otherwise?
  5. How can we write data to tape.? Veeam Can read from a share and write to tape. I am assuming the DD Agent share can also be assigned with NFS / CIFS share protocol and that should allow Veeam to see the content and take it from there.?!!
  6. Any best practices recommendations are most appreciable.

If there are any further information needed to answer above please let me know.

I can be reached at syedmahdiali @ hotmail.com !

Related:

Re: Restore oracle backup on a different server

Hi Masanc2,

In summary if you want to do this using a networker recovery gui, the only option for you is to use duplicate method, which will create for you a duplicate database.

If you want to recover to another machine with the same oracle SID then you must use oracle RMAN. Before performing the recovery process you must create a client resource for the target machine and you must add the user e.g user@sourcemachine in the ‘remote access’ field in the client resource property.

here’s below sample command to give you an overview. Hope this helps.

#login using the oracle user ( su – <oracleusername>)

%rman target /

set DBID <dbid>;

startup nomount;

run {

allocate channel CH1 type ‘SBT_TAPE’;

send ‘NSR_ENV=(NSR_SERVER=<networkermasterserver>,NSR_CLIENT=<sourceclient>,NSR_RECOVER_POOL=<clonepool>)’;

allocate channel CH2 type ‘SBT_TAPE’;

send ‘NSR_ENV=(NSR_SERVER=<networkermasterserver>,NSR_CLIENT=<sourceclient>,NSR_RECOVER_POOL=<clonepool>)’;

restore controlfile from autobackup;

alter database mount;

RESTORE DATABASE;

RECOVER DATABASE;

alter database open resetlogs;

RELEASE CHANNEL CH1;

RELEASE CHANNEL CH2;

}

Related: