Unable to start Enterprise Manager – Fatal error with server.xml
One of SAND box of SAP of Oracle Database on windows platform, where both the ORACLE_HOME and Archive log location are located in same drive(F:), When there is no space left on disk because of because of archival and caused database hung. without interaction of DBA, Unix admin has performed reboot of server to move the disks to other drive. In process of that even ORACLE_HOME moved to new drive. Now everything is setup properly they able to bring database and listener successfully. But when starting EM services it failed and below is troubleshooting to fix this kind of issue.
Status of Dbconsole after starting EM services from “Services.msc”
C:\Windows\system32>emctl status dbconsole Oracle Enterprise Manager 11g Database Control Release 11.2.0.2.0 Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved. https://ora2-1.com:1158/em/console/aboutApplication Oracle Enterprise Manager 11g is not running. ------------------------------------------------------------------ Logs are generated in directory F:\oracle\product\11.2.0.2\db_1/ora2-1.com_ckpt/sysman/log
Reviewed above log file and found below information
F:\oracle\product\11.2.0.2\db_1/ora2-1.com_ckpt/sysman/log 2012-10-03 05:55:59,486 Thread-4552 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://ora2-1.com:1158/em/upload/: retStatus=-1 2012-10-03 05:56:31,480 Thread-3668 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://ora2-1.com:1158/em/upload/: retStatus=-1 2012-10-03 05:56:33,495 Thread-3668 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://ora2-1.com:1158/em/upload/: retStatus=-1 2012-10-03 05:57:05,552 Thread-4396 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://ora2-1.com:1158/em/upload/: retStatus=-1 2012-10-03 05:57:07,583 Thread-4396 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://ora2-1.com:1158/em/upload/: retStatus=-1 2012-10-03 05:57:39,577 Thread-4024 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://ora2-1.com:1158/em/upload/: retStatus=-1 2012-10-03 05:57:41,608 Thread-4024 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https://ora2-1.com:1158/em/upload/: retStatus=-1
Now Once again stopped EM service from “Services.msc” , and started manually from command prompt.
(pid=3160): starting emagent version 10.2.0.4.3
12/10/03 06:42:05 Error initializing server: /F:/oracle/product/11.2.0.2/db_1/oc4j/j2ee/OC4J_DBConsole_ora2-1.com_ckpt/config/server.xml, Fatal error at line 1 offset 1 in file:/F:/oracle/product/11.2.0.2/db_1/oc4j/j2ee/OC4J_DBConsole_ora2-1.com_ckpt/config/server.xml: .<Line 1, Column 1>: XML-20108: (Fatal Error) Start of root element expected.
12/10/03 06:42:05 Fatal error: server exiting
----- Wed Oct 3 06:42:06 2012::Pid 4560 not found. reaped=4560, processStatus=256 -----
----- Wed Oct 3 06:42:09 2012::Checking status of DBConsole : 4560 -----
----- Wed Oct 3 06:42:09 2012::DBConsole exited at Wed Oct 3 06:42:09 2012 with return value 0. -----
----- Wed Oct 3 06:42:09 2012::DBConsole was shutdown normally. -----
----- Wed Oct 3 06:42:09 2012::Exiting watchdog loop
-----
(pid=3160): emagent started successfully
These issues can be occurred in two of the cases either it can be any Recent upgrades (or) disk space issues. Probably when the services are running or by relocating file location the “server.xml” looks to be either corrupted or unable to find repository.
To fix this issue either restore the file from a binary backup or another similar installation of same OS and DB version or from a new similar fresh installation. if this is ran no Success then consider reconfiguring whole Enterprise Manager.
Now i have renamed file “F:\oracle\product\11.2.0.2\db_1\oc4j\j2ee\OC4J_DBConsole_ora2-1.com_ckpt\config/server.xml” file and copied same file from the location of “F:\oracle\product\11.2.0.2\db_1\oc4j\j2ee\OC4J_DBConsole\config” , Now i started DBConsole. Below is the status of EM.
C:\Windows\system32>emctl status dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.2.0
Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
https://ora2-1.com:1158/em/console/aboutApplication
Oracle Enterprise Manager 11g is running.
------------------------------------------------------------------
Logs are generated in directory F:\oracle\product\11.2.0.2\db_1/ora2-1.com_ckpt/sysman/log
Now ORACLE EM is running without any issues, Later uploaded agent if any pending XML files.
C:\Windows\system32>emctl upload agent
Oracle Enterprise Manager 11g Database Control Release 11.2.0.2.0
Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
EMD upload completed successfully
Now finally check the status of Agent also.
C:\Windows\system32>emctl status agent
Oracle Enterprise Manager 11g Database Control Release 11.2.0.2.0
Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Agent Version : 10.2.0.4.3
OMS Version : 10.2.0.4.3
Protocol Version : 10.2.0.4.3
Agent Home : F:\oracle\product\11.2.0.2\db_1\ora2-1.com_ckpt
Agent binaries : F:\oracle\product\11.2.0.2\db_1
Agent Process ID : 4528
Agent URL : https://ora2-1.com:3938/emd/main
Repository URL : https://ora2-1.com:1158/em/upload/
Started at : 2012-10-03 07:03:22
Started by user : SYSTEM
Last Reload : 2012-10-03 07:03:22
Last successful upload : 2012-10-03 07:10:39
Total Megabytes of XML files uploaded so far : 31.07
Number of XML files pending upload : 0
Size of XML files pending upload(MB) : 0.00
Available disk space on upload filesystem : 82.86%
Data channel upload directory : F:/oracle/product/11.2.0.2/db_1/ora2-1.com_ckpt/sysman/
recv
Last successful heartbeat to OMS : 2012-10-03 07:09:59
---------------------------------------------------------------
Agent is Running and Ready
So far DbConsole and agent both are started without any issues, Now check the EM home page from Browser.
Now The EM can able to check the instance status without any errors.
— Happy Reading, Thank you –
