DevOps SRE Blog

Oracle weblogic managed servers (SOA/OSB/ESS) were failing to restart after making few changes. The error written into logs was not straight forward and we spent couple of hours debugging the issue. Sharing this so that others can benefit from it. We are using Exadata as dehydration store. Network domain for applications we are managing got changed. As part of this change we changed datasource, ftp adapter config to refer to new domain. We made all required changes and tried to restart servers. We were able to bring up admin server but all managed servers failed to restart. Following error was written logs. Caused by: java.io.IOException: [DeploymentService:290066]Error occurred while downloading files from Administration Server for deployment request “42,336,427,916,066,804”. Underlying error is: “[DeploymentService:290065]Deployment service servlet encountered an Exception while handling the deployment datatransfer message for request id “42,336,427,916,066,804...

Kubernetes port forwarding feature can be used to access a pod port from local machine without accessing pod. This can be achieved by other means like creating service. But port forwarding is somewhat quick. Port forwarding can be enabled by running following command: kubectl port-forward <pod Name> <local port>: <pod port> e.g., kubectl port-forward my-pod 9000:80 I have a test  k8s cluster with single master node and two worker nodes. I have enabled port forward by using above command and invoked curl by opening another ssh session from master node. vagrant@master-1:~$ curl http://localhost:9000 curl: (52) Empty reply from server I was expecting some output but got “Empty reply from server”. Also observed, in ssh session where port forwarding is enabled, below error message. E0811 13:51:54.597132   13384 portforward.go:391] an error occurred forwarding 9000 -> 80: error forwarding port 80 to pod ce0a72c043baa73a773b0803c9b3dc365229e...