This post covers, Online Patching (ADOP) issue reported by one of the trainees in Oracle AppsDBA Training with prepare phase.
Note: There are five phases in Online Patching (ADOP) in R12.2 PREPARE -> APPLY -> FINALIZE -> CUTOVER -> CLEANUP .
- To know more about Online Patching (ADOP) click here
Troubleshooting ADOP Prepare Phase Issue:
1. Run Prepare Phase:
Any Patching Cycle starts with Prepare Phase and to start it run below command
Prepare phase is failing with error message
2. Look at ADOP logs:
First thing in troubleshooting is to look at log file (in this case ADOP log file)
A) Error in adop logs:
- /u01/oracle/PROD is Oracle EBS install base directory
- fs_ne Non-Editioned File System
- 6 – adop session id
- prepare_20171109_032332 – <phase>_<date>_<time>
- PRD122_1705ecloud05 – CONTEXT NAME
Oracle Apps R12.2 has Dual File System (fs1 & fs2), which helps organizations to have less downtime during patching. By introducing duplicate file system, Oracle has reduced patching downtime and improved availability of the system.
We can focus on three major parts of the E-Business Suite 12.2 file system:
- File System 1 (fs1): /u01/oracle/PROD/fs1
- File System 2 (fs2): /u01/oracle/PROD/fs2
- Non-Editioned File System (fs_ne): /u01/oracle/PROD/fs_ne
Thu Nov 9 03:34:15 2017
B) Error in fs_clone logs:
Note: Prepare Phase, creates clone of Patch File System from Run File System and as per previous log, error was during this Cloning so we need to look at clone logs.
C) Error in OHS logs:
Note: Clone process clones, Fusion Midddleware (WebLogic & OHS) as per the previous log, the error was during this startup of OHS server so we need to look at OHS logs.
3. Root Cause:
Root cause as identified in previous OHS log was, wrong owner permission for OHS files
Checked httpd.conf file and wherever it is having applmgr11, changed to applmgr user permission
4. Change in Context File
Note: After updating user to applmgr in httpd.conf, OHS start-up was still failing
So checked $CONTEXT_FILE and below two entries having applmgr11 user on both run (fs1) and patch (fs2) file system and causing the issue
<appsuser oa_var=”s_appsuser” osd=”unix”>applmgr11</appsuser>
<sysadminmail oa_var=”s_sysadmin_mail” osd=”unix”>email@example.com</sysadminmail>
Note: $CONTEXT_FILE, captured all configuration parameters. In Oracle R12, we are having two context file:
Application context file called the <CONTEXT_NAME>.xml contains the configuration information for the application tier & is located in $APPl_TOP/admin for each node of application tier.
Database context file called the <CONTEXT_NAME>.xml contains the configuration information for the database tier & is located in $ORACLE_HOME/appsutil/
1. Changed user to applmgr on both run and patch file system under $CONTEXT_FILE and run autoconfig, then restart prepare phase again
<appsuser oa_var=”s_appsuser” osd=”unix”>applmgr</appsuser>
<sysadminmail oa_var=”s_sysadmin_mail” osd=”unix”>firstname.lastname@example.org</sysadminmail>
Run Prepare Phase Again:
adop phase=prepare restart=yes
The Node Manager is already shutdown
NodeManager log is located at /u01/oracle/PRD122/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1
adnodemgrctl.sh: exiting with status 2
adnodemgrctl.sh: check the logfile /u01/oracle/PRD122/fs2/inst/apps/PRD12111_1705ecloud05/logs/appl/admin/log/adnodemgrctl.txt for more information …
The prepare phase completed successfully.
adop exiting with status = 0 (Success)
Checked ADOP Status
Session Id: 6
Node Name Node Type Phase Status Started Finished Elapsed
————— ———- ————— ————— ——————– ——————– ————
1705ecloud05 master PREPARE COMPLETED 2017/11/06 22:41:13 2017/11/09 08:47:02 58:05:49
APPLY NOT STARTED
FINALIZE NOT STARTED
CUTOVER NOT STARTED
CLEANUP NOT STARTED
File System Synchronization Type: Light
adop exiting with status = 0 (Success)
Next Task For You
Begin your journey towards becoming an Apps DBA by joining our FREE Masterclass on How To Become an Oracle Apps DBA (R12) & Earn A Lot More In 2020.