This post covers, Online Patching (ADOP) issue reported by one of the trainees in Oracle AppsDBA Training with the prepare phase.
Note: There are five phases in Online Patching (ADOP) in R12.2 PREPARE -> APPLY -> FINALIZE -> CUTOVER -> CLEANUP .
- To know more about Online Patching (ADOP) click here
Troubleshooting ADOP Prepare Phase Issue:
1. Run Prepare Phase:
Any Patching Cycle starts with Prepare Phase and to start it run below command
adop phase=prepare
Prepare phase is failing with error message
2. Look at ADOP logs:
First thing in troubleshooting is to look at log file (in this case ADOP log file)
A) Error in adop logs:
- /u01/oracle/PROD is Oracle EBS install base directory
- fs_ne Non-Editioned File System
- 6 – adop session id
- prepare_20171109_032332 – <phase>_<date>_<time>
- PRD122_1705ecloud05 – CONTEXT NAME
Oracle Apps R12.2 has Dual File System (fs1 & fs2), which helps organizations to have less downtime during patching. By introducing duplicate file system, Oracle has reduced patching downtime and improved availability of the system.
We can focus on three major parts of the E-Business Suite 12.2 file system:
- File System 1 (fs1): /u01/oracle/PROD/fs1
- File System 2 (fs2): /u01/oracle/PROD/fs2
- Non-Editioned File System (fs_ne): /u01/oracle/PROD/fs_ne
Thu Nov 9 03:34:15 2017
B) Error in fs_clone logs:
Note: Prepare Phase, creates a clone of Patch File System from Run File System, and as per the previous log, an error was during this Cloning so we need to look at clone logs.
$INST_TOP/admin/log/clone/FSCloneApplyAppsTier_<timestamp>.log
C) Error in OHS logs:
Note: Clone process clones, Fusion Midddleware (WebLogic & OHS) as per the previous log, the error was during this startup of OHS server so we need to look at OHS logs.
/u01/oracle/PRD122/fs2/FMW_Home/webtier/instances/EBS_web_PRD12111_OHS1/diagnostics/logs/OHS/EBS_web_PRD12111/console~OHS~1.log
3. Root Cause:
Root cause as identified in previous OHS log was, wrong owner permission for OHS files
Checked httpd.conf file and wherever it is having applmgr11, changed to applmgr user permission
4. Change in Context File
Note: After updating user to applmgr in httpd.conf, OHS start-up was still failing
So checked $CONTEXT_FILE and below two entries having applmgr11 user on both run (fs1) and patch (fs2) file system and causing the issue
<appsuser oa_var=”s_appsuser” osd=”unix”>applmgr11</appsuser>
<sysadminmail oa_var=”s_sysadmin_mail” osd=”unix”>applmgr11@1705ecloud05.k21academy.com</sysadminmail>
/u01/oracle/PRD122/fs1/inst/apps/PRD12111_1705ecloud05/appl/admin/PRD12111_1705ecloud05.xml
Note: $CONTEXT_FILE, captured all configuration parameters. In Oracle R12, we are having two context file:
Application context file called the <CONTEXT_NAME>.xml contains the configuration information for the application tier & is located in $APPl_TOP/admin for each node of application tier.
Database context file called the <CONTEXT_NAME>.xml contains the configuration information for the database tier & is located in $ORACLE_HOME/appsutil/
Fix:
1. Changed user to applmgr on both run and patch file system under $CONTEXT_FILE and run autoconfig, then restart prepare phase again
<appsuser oa_var=”s_appsuser” osd=”unix”>applmgr</appsuser>
<sysadminmail oa_var=”s_sysadmin_mail” osd=”unix”>applmgr@1705ecloud05.k21academy.com</sysadminmail>
/u01/oracle/PRD122/fs1/inst/apps/PRD12111_1705ecloud05/appl/admin/PRD12111_1705ecloud05.xml
Run Prepare Phase Again:
adop phase=prepare restart=yes
The Node Manager is already shutdown
NodeManager log is located at /u01/oracle/PRD122/fs2/FMW_Home/wlserver_10.3/common/nodemanager/nmHome1
adnodemgrctl.sh: exiting with status 2
adnodemgrctl.sh: check the logfile /u01/oracle/PRD122/fs2/inst/apps/PRD12111_1705ecloud05/logs/appl/admin/log/adnodemgrctl.txt for more information …
The prepare phase completed successfully.
adop exiting with status = 0 (Success)
Checked ADOP Status
adop -status
ADOP (C.Delta.8)
Session Id: 6
Command: status
Output: /u01/oracle/PRD122/fs_ne/EBSapps/log/adop/6/20171109_085910/adzdshowstatus.out
===============================================================
Node Name Node Type Phase Status Started Finished Elapsed
————— ———- ————— ————— ——————– ——————– ————
1705ecloud05 master PREPARE COMPLETED 2017/11/06 22:41:13 2017/11/09 08:47:02 58:05:49
APPLY NOT STARTED
FINALIZE NOT STARTED
CUTOVER NOT STARTED
CLEANUP NOT STARTED
File System Synchronization Type: Light
adop exiting with status = 0 (Success)
Related/Further Readings
- Recommended Browsers for Oracle E-Business Suite Release 12 [Doc ID 389422.1]
- Oracle E-Business Suite Release 12.2 Information Center (Doc ID 1581299.1)
- Oracle E-Business Suite Release 12.2.9 Readme (Doc ID 2495027.1)
- Oracle E-Business Suite 12.2.9 Now Available
- [Troubleshooting] Forms Issue: FRM-92050 failed to connect to server:forms/servlet
- Steps to download Oracle E-Business Suite R12 Software
- Oracle AppsDBA (R12.2) Training: Step by Step Activity Guides/Hands-On Lab Exercise
- Oracle AppsDBA (R12.2): Install | Patch | Clone | Maintain
Next Task For You
We cover Oracle E-Business R12.2 Architecture & concepts in our Oracle Apps DBA For Beginners Training along with the Installation, Patching, Cloning, and Troubleshooting and also, Database upgrade to 19c and much more including the hands-on labs you must perform to upgrade your skills and get a good job with a high package.
Begin your journey towards becoming an Apps DBA by joining our FREE Masterclass on How To Learn Oracle Apps DBA (R12) & It’s New Features.
Devendra Singh says
I am applying 5 patches by ADOP like below and Apply phase failed after 3rd patch and says pre-requisite patch (abcdef) is missing. I have to apply all the patches.
adop phase=apply patches=11111,22222,33333,44444,55555
How can I apply the pre-requisite patch (abcdef)? or How should I handle this situation?
Atul Kumar says
Look at what Giriraj has to say or use
adop phase=apply patches=missedpatch, 33333,44444,55555 abandon=yes restart=no
Devendra Singh says
Thank you, Atul for clarifying my doubt…
sridhar says
adop phase=apply patches=abcdef restart=no abandon=yes
After the above,
adop phase=apply patches=11111,22222,33333,44444,55555
(you can remove the patches that were applied else pass all patches and it will skpp that has already been applied)
Giriraj says
In this scenario all you need to do is apply the pre-requisite patch first.
1)
Adop phase=apply patch=abcdef
Then once this is done continue with the patch you were applying earlier.
Adop phase=apply patches=11111,22222,33333,44444,55555
Note: There is another way to do it if there were no patches appled prior to to the above patches in this patching cycle you could also abort the patching cycle and start fresh patching cycle
2)
Adop phase=abort,cleanup
adop phase=prepare
Then continue with above mentioned prerequisite patch ( abcdef) and the respective patch later (11111,22222,33333,44444,55555)
But in this scenario the second mentioned is not required.