DevOps SRE Blog

This blog covers basics of Vim/vi that are useful in day to day activities of a weblogic/fmw admin. Navigation Commands : h – Move cursor left j – Move cursor down k – Move cursor up l – Move cursor right 0 – Move to first character of real line gj/gk – Move down or up by display lines instead of real lines g0 – To first character of display line (when line spreads multiple lines, useful in navigating log files) gg – Go to beginning of the file G – Go to last line of the file 8G/8gg – Go to 8th line w – Move to start of the word e – Move to end of the word b – Move backward to start of the word Movement by finding characte r: f{char} – Move to the next occurrence of {char} F{char} – Move backward to the previous occurrence of {char} t{char} – Move to the character before the next occurrence of {char} T{char} – Move backward to the character after the previous occurrence of {char} *...

This is the final part of the series. Let us see some miscellaneous commands and tips. Hope this series helped you. 1) The utilities zcat, zgrep, zless, and zdiff, among others, serve the same purpose as cat, grep, less, and diff, respectively, but operate on compressed files. For example, if you want to search inside jar/zip file, you can use zgrep. 2)  apropos command can be used to find the appropriate command if we forget command name.  apropos search # gives all related commands with search string. 3) watch command allows to watch the output of a program change over time, which can be especially convenient when looking for changes to file sizes that reflect debugging output, download or file creation state, memory or disk usage, and so on. 4) History: If you want to remove a sensitive command from your history, you can simply edit your $HISTFILE history file and remove it. There is a trick you can use if you want to fly under the radar and never have a command recor...

Let us learn about tools that can help in troubleshooting performance related problems: 1) top : Top provides statistics about cpu, memory utilization, load averages, resident memory, virtual memory etc. The screen refreshes automatically.  i) top u oracle to see all the processes initiated by user oracle. ii) Press ‘1’ to see the individual cores. iii) top -p <pid> : To monitor only processes with a given process id. iv) top -H -p <pid>: Show all threads.We can press H on top output screen as well to get thread info. v) -b Batch mode. Useful for sending output from top to other programs or to a file. vi) To sort memory usage in ‘top’ output, press ‘SHIFT+m’  vii) List of process are sorted and displayed by default based on CPU utilization. We can change it by pressing “<” or “>”, sort column changes accordingly. top command consumes reasonable cpu. So, don’t run too many top sessions. 2) htop : htop is similar to top...

Let us learn about some miscellaneous commands in this part: 1) To find Linux version of OS: lsb_release -a 2) To find kernel version: uname -a 3) To find number cpu cores: lscpu cat /proc/cpuinfo 4)  more : cat command can be used to view the file contents. But if the file is more than few pages, reading through the buffer is difficult. The commands more and  less are more useful in these cases. We can go to next page by pressing space bar.  By pressing b we can go back by one page. We can search for a string by typing /, followed by the string similar to vi. We can find the next occurrence of the string by typing n. 5) less : less command has similar functionality as more but it is faster. 6) which   command can be used to find the location of a program. which perlwhich java 7) To read a file backwards, we can use tac (reverse cat) and less together: tac nohup.out | less 8) To find packages installed using rpm : rpm -qa | egrep "java-1.7.0-openjdk|java-1.8.0-openjdk 9) fuser i...

Let us learn about tar and ps in this part of the Unix series: tar : tar is derived from Tape Archive when magnetic tapes were used to archive files. tar utility is useful in creating single archive file from multiple files maintaining owners, file types, permissions etc. It is mainly used to create backups and copy files from one computer to other. Let us see few examples: 1) To create tar file from a dir: tar -cvf <tar file name> <Source dir name> #Here, c stands for compress, v for verbose, f - file name of tar file. tar -cvf scripts.tar /opt/oracle/product/admin/scripts 2) To extract files from tar file: tar xvf scripts.tar 3) tar doesn’t compress files, it only creates archive file. It can be compressed using gzip utility. gzip scripts.tar # It creates a compressed file scripts.tar.gz gzip -d scripts.tar.gz  # to unzip 4) We can archive and compress with single command using ‘z’ option: tar -zcvf scripts.tar.gz scripts # Compress tar -zxvf scripts...

Let us learn about df, du, sftp and scp commands in this part of the series: du : 1) To know the percentage of disk space used at mount level:  df -kh 2) To sort disk usage in a directory in human readable format: du -sh * | sort -rh 3) If multiple mounts uses common path. du command on common path includes storage used in all mounts. If we want find disk used by only common directory path and not device mounted on common path, need to use –exclude clause of du as shown above. du -sh --exclude=/directory {target-directory/*} 4) To find largest directories:  du -S | sort -n | tail -20 5) To find directories with size greater than 1 GB: du -h --max-depth 50 /orasoa/common/ 2>/dev/null | grep '[0-9]G>' | sort -h 6) du usually don’t include hidden directories/files during disk usgage calculation. To include hidden files/dirs: du -sk * .[a-zA-Z0-9]* 7) -d option prevents du from following file system boundaries. For example, to determine disk usage on the root fil...

This is in continuation to the first  part in the series. Let us discuss grep and lsof in part 2: grep : 1) Case insensitive search: grep -i "string" FILE   2) Checking for full words, not for sub-strings using grep -w grep -w "serachword" file 3) Displaying lines before/after/around the match using grep -A, -B and -C. This is very useful if you want to see lines around search string: i) grep -A <N> "string" FILENAME # display N lines after match: ii) grep -B <N> "string" FILENAME # display N lines before match iii) grep -C 2 "Example" demo_text # display N lines around match 4) Searching in all files recursively using grep -r grep -r "ramesh" * 5) Invert match using grep -v grep -v -e "pattern" -e "pattern" 6) Counting the number of matches using grep -c. There is no need to pipe the output to wc. grep -c "pattern" filename grep -v -c this demo_file (Count of lines not matching patter...

Scripts to monitor and managed Oracle Fusion Middleware applications are available at following github location:  https://github.com/RameshPoonati/oracleMiddlewareScripts Weblogic : Disable Diagnostic Modules Error Report Generation from logs Application/Deployment Monitoring http log monitoring JMS Bridge Monitoring JMS Consumer Monitoring Log Archiving JMS Queue Monitoring Server Monitoring Domain Startup/Shutdown Thread Dump Generation Thread Monitoring SOA Suite : Monitor Adapter Status Reference instance response time from audit trail ODI : Duplicate Execution Monitoring Oracle Service Bus (OSB) : OSB Proxy Status Change Shell Scripts : Disk Usage Monitoring

We have 100+ Oracle middleware servers that are running on Azure cloud. Most of these servers were not used during weekend. So, guys in infra team thought of saving cost by shutting down MW servers. They asked us  to provide a single script to shutdown all machines and another script to start them. They would invoke these scripts through Ansible and shutdown VMs once all processes on the VM are stopped. We came up with a wrapper script that will invoke individual component startup scripts. We have two node cluster and wrapper script is placed on node 1. Node 1 will connect to node 2 using ssh and execute individual component scripts on that node. The order is start managed server node manager -> OHS NM ->  Start OHS etc with delay of 30 seconds. But ssh session that starts managed server node manager never terminates. Found this answer on stackoverflow helped.  We added nohup and routed output to /dev/null. Snippet of the code: echo “`ssh $userName@$secondNodeIP /bin/bash ...

ctrl-a – move the cursor to the beginning of the current line ctrl-e – move the cursor to the end of the current line alt-b – move the cursor backwards one word alt-f – move the cursor forward one word ctrl-k – delete from cursor to the end of the line ctrl-u – delete from cursor to the beginning of the line alt-d – delete the word in front of the cursor ctrl-w – delete the word behind of the cursor ctrl + xx in command line toggles between start and end of the line. ctrl + ] <char>  – moves cursor to 1st occurrence of <char> to right alt + .   – change command keep last argument. (Same can be achieved using !$) !* – change command keep all arguments