DevOps SRE Blog

Scripts to monitor and managed Oracle Fusion Middleware applications are available at following github location:  https://github.com/RameshPoonati/oracleMiddlewareScripts Weblogic : Disable Diagnostic Modules Error Report Generation from logs Application/Deployment Monitoring http log monitoring JMS Bridge Monitoring JMS Consumer Monitoring Log Archiving JMS Queue Monitoring Server Monitoring Domain Startup/Shutdown Thread Dump Generation Thread Monitoring SOA Suite : Monitor Adapter Status Reference instance response time from audit trail ODI : Duplicate Execution Monitoring Oracle Service Bus (OSB) : OSB Proxy Status Change Shell Scripts : Disk Usage Monitoring

We have 100+ Oracle middleware servers that are running on Azure cloud. Most of these servers were not used during weekend. So, guys in infra team thought of saving cost by shutting down MW servers. They asked us  to provide a single script to shutdown all machines and another script to start them. They would invoke these scripts through Ansible and shutdown VMs once all processes on the VM are stopped. We came up with a wrapper script that will invoke individual component startup scripts. We have two node cluster and wrapper script is placed on node 1. Node 1 will connect to node 2 using ssh and execute individual component scripts on that node. The order is start managed server node manager -> OHS NM ->  Start OHS etc with delay of 30 seconds. But ssh session that starts managed server node manager never terminates. Found this answer on stackoverflow helped.  We added nohup and routed output to /dev/null. Snippet of the code: echo “`ssh $userName@$secondNodeIP /bin/bash ...

Java and python both have time modules. So when we tried to use strftime function in time, wlst picked up time module present in Java and not that one in python. This is due to name collision. To avoid this error, import time module with an alias name as below and prefix strftime call with alias name. import time as pytime

Server startup failed with below error while trying to start with WLST script:   This Exception occurred at Mon Mar 05 16:28:14 GMT 2018. weblogic.nodemanager.NMException: Received error message from Node Manager Server: [Server start command for WebLogic server ‘osb_server2’ failed due to: [Server failed to start up but Node Manager was not aware of the reason]. Please check Node Manager log and/or server ‘osb_server2’ log for detailed information.]. Please check Node Manager log for details. Problem invoking WLST – Traceback (innermost last):   File “/home/aosbpre/scripts/startMS.py”, line 29, in ?   File “<iostream>”, line 188, in nmStart   File “<iostream>”, line 553, in raiseWLSTException WLSTException: Error occurred while performing nmStart : Error Starting server osb_server2 : Received error message from Node Manager Server: [Server start command for WebLogic server ‘osb_server2’ failed due...