Manual Recovery of an Informix Database on a Redundant Picture Perfect Server That Will Not Startup As Backup
Procedure Steps
Note: Prior to executing these procedures:
- Contact local Database Administrator and local IT and confirm this procedure can be executed
- Make a backup of the current database
Description
Primary - “The Picture Perfect Server that is running in Primary mode”
Backup - “The Picture Perfect server that is being recovered and will run as backup”
NOTE: If the system has experienced a failover, and the Backup Server has become the New Primary Server, you must bring up the failed server as the Backup Server. This insures that you do not lose any activity history being generated by the New Primary Server. Once both servers have been brought back online successfully, a controlled failover can be executed to return the servers to their original role (Primary/Backup).
For all commands listed, the prompt at the start of the line will indicate which system user should execute each command:
# – commands to be executed as root
$ – commands to be run as ppadmin
Procedure Steps
On the server running as the Primary Server, login as root and validate that Picture Perfect is running in the correct mode.
Run the profile for root:
# . /cas/sbin/profile
# tpsmode
tpsmode command should return: Primary
This indicates that the server is running Picture Perfect in “Primary” mode.
2. Cleanup the /ppbackup/db/ directory by removing all Logical Logs and zeroing out the ifxDBArch file on the Primary Server
NOTE: Do not remove the ifxDBArch file; just zero it out as indicated below.
On Primary Server, run the following commands as root:
Navigate to the /ppbackup/db/ directory:
# cd /ppbackup/db
Delete all logical logs in the directory:
# rm -f ifxDBLogical*
Zero out the ifxDBArch file:
# >ifxDBArch
List files in the directory:
# ls -l
At this point, look at the file listing and verify the following:
· There should see no Logical Logs (filenames that start with ifxDBLogical_Log).
· The only file present should be ifxDBArch.
· ifxDBArch should have a file size of 0 bytes.
Cleanup the /ppbackup/db/ directory by removing all Logical Logs and zeroing out the ifxDBArch file on the Backup Server
NOTE: Do not remove the ifxDBArch file; just zero it out as indicated below.
On Backup Server, run the following commands as root:
Navigate to the /ppbackup/db/ directory:
# cd /ppbackup/db
Delete all logical logs in the directory:
# rm -f ifxDBLogical*
Zero out the ifxDBArch file:
# >ifxDBArch
List files in the directory:
# ls -l
At this point, look at the file listing and verify the following:
· There should see no Logical Logs (filenames that start with ifxDBLogical_Log).
· The only file present should be ifxDBArch.
· ifxDBArch should have a file size of 0 bytes.
On the Primary Server, create a new Archive of the Database.
Change user from root to ppadmin
# su - informix
Run the profile for ppadmin
$ . /cas/bin/profile
Use the ontape command to create a new backup [0 = Zero]
$ontape -s -L 0
Screen will display:
Please mount tape 1 on /ppbackup/db/ifxDBArch and press Return to continue ...
Hit the Return key.
The following will display as the archive is being created:
10%
20%
40%
80%
100%
Exit from the ppadmin user session to return to root
$exit
Transfer new ifxDBArch file from the Primary Server to Backup Server
On the Primary Server, as the root user, navigate to the /ppbackup/db/ directory:
# cd /ppbackup/db
b.
# scp -p ifxDBArch :/ppbackup/db/
EXAMPLE: In a configuration where the Backup Server is named pphost2
scp -p ifxDBArch pphost2:/ppbackup/db/
The screen will display the progress on the file transfer as a percentage:
ifxDBArch 3,430Mbs 42345 10%
Once the archive file has completely transferred you are done working on the Primary Server.
On the Backup server as the root user
Navigate to the /ppbackup/db/ directory
# cd /ppbackup/db
List the directory contents
# ls -l
Validate that the ifxDBArch file on the Backup Server matches the size and date of the ifxDBArch file on the Primary Server. If these are not the same, return to Step 6 and transfer the file from the Primary to the Backup again.
Validate that the owner/group of the ifxDBArch file is informix:informix and the permissions are 777 (-rwxrwxrwx). If these are not properly set, use the Unix commands chown and chmod to properly set the owners/permissions.
Now you can recover the database Archive.
On the Backup Server, as root, ensure that informix is not running.
Run root profile
# . /cas/sbin/profile
Run cleanmachine
# cleanmachine
Run cleanmachine a 2nd time to ensure there are no active Shared Memory or Semaphores
# cleanmachine
Change user to ppadmin
# su - ppadmin
Run the profile for ppadmin
$ . /cas/bin/profile
Run the ontape -p command to recover the database archive
$ ontape -p
The output will list the databases that will be restored from the archive.
You will be asked three questions:
Continue with restore? Y/N
Answer: YES
Do you want to backup Log files? Y/N
Answer: NO
Do you want to recover level 1 archives? Y/N
Answer: NO
When you get the command prompt back exit to root and run “cleanamachine” again, twice
Exit to root
$ exit
Run root profile
# . /cas/sbin/profile
Run cleanmachine
# cleanmachine
Run cleanmachine a 2nd time to ensure there are no active Shared Memory or Semaphores
# cleanmachine
Change the database mode to secondary
Change to ppadmin user
# su - ppadmin
Run the profile
$ . /cas/bin/profile
Change the mode of the database to secondary
$ dbmode -m secondary -h
EXAMPLE: In a configuration where the Primary Server is named pphost1
dbmode -m secondary -h pphost1
When this completes, verify the database mode
$ dbmode -iv
This may take a minute to display. When the system returns the database information, validate that the Replication Mode is “Secondary”.
EXAMPLE:
Local Hostname .................... :
Online status ..................... : Online
Replication Mode .................. : Secondary
Remote Hostname ................... :
Last Logical Log ID Number ........ : 198
Last Logical Log Page Number ...... : 10741
Checkpoint Logical Log Position ... : 0xa5b56b4
If the database is in Secondary mode, proceed.
Run cleanmachine and start the database on the Backup Server
Exit to root
$ exit
Run the root profile
# . /cas/sbin/profile
Run cleanmachine
# cleanmachine
Run cleanmachine a 2nd time to ensure there are no active Shared Memory or Semaphores
# cleanmachine
Change to ppadmin
# su - ppadmin
Run the ppadmin profile
$ . /cas/bin/profile
Start the database
$ oninit
Check the status of the database and start Picture Perfect on the Backup Server
Run onstat -g dri to validate database status
$ onstat -g dri
SAMPLE OUTPUT:
IBM Informix Dynamic Server Version 11.50.UC8X4GE – Fast Recovery -- Up 0 days 22:17:24 -- 759924 Kbytes
Data Replication at 0x37087018:
Type State Paired server Last DR CKPT (id/pg) Supports Proxy Writes
secondary on pphosta -1 / -1 NA
---
Run this command every 30 seconds until the text that reads Fast Recovery changes to Updateable. Once the database is updateable you can start Picture Perfect on the Backup Server.
Start Picture Perfect redundant on the Backup Server
$ pprscmd start backup
Applies To
Picture Perfect Redundant systems using an Informix database.
Additional Information
None