count number of files generated per day

 # count number of files generated per day

find . -type f -printf '%TY-%Tm-%Td\n' | sort | uniq -c

# count number of files generated per hour in last 600 minutes

find . -cmin -600  -type f -printf '%TY-%Tm-%Td-%TH\n' | sort | uniq -c

find . -maxdepth 1 -type f -printf '%TY-%Tm-%Td\n' | awk '{array[$0]+=1}END{ for(val in array) print val" "array[val]   }'|sort

List TOP 10 by size directories

# List TOP 10 by size directories
du -m --max-depth 1 | sort -rn | head -11

PERFORMANCE BOTTLENECK CHECK USING OS TOOLS

A.REAL TIME PERFORMANCE TUNING USING OS TOOLS


TOOL1:-TOP UTILITY

check in TOP with column S where it shows ‘R’ means running and ‘S’ means sleeping.


TOOL 2:-SAR UTILITY WITH -U OPTION TO CHECK CPU AND IO BOTTLENECK
sar -u 10 8

TOOL 3:VMSTAT REPORT
vmstat 1 10
r column is runnable processes

TOOL 4:TO IDENTIFY DISK BOTTLENECK
sar -d 5 2

TOOL 5:-SAR -B TO REPORT DISK USAGE

TOOL 6:-SAR -Q TO FIND PROCESSES UNDER QUEUE.WE NEED TO LOOK BLOCKED SECTION

TOOL 7:-TO IDENTIFY MEMORY USAGE USING SAR -R


TOOL 8. REAL TIME PERFORMANCE TUNING USING ORATOP REPORT
./oratop -f -d -i 10 / as sysdba

Extract range of lines from a text file in Unix

Input File    : orig-data-file
OutPut File : new-file

Line range 100 to 110

sed -n '100,110 p' orig-data-file > new-file

awk 'NR>=100&&NR<=110' orig-data-file > new-file

perl -ne 'print if 110..110' orin-data-file > new_file

-- get alert.log file for particular day
awk '/Jun 26/{c=6}c&&c --' alert_demo.log

grep -B1 -A1 'Jun 26' alert_demo.log




Checking Swap Space Size and Usage

step 1
grep SwapTotal /proc/meminfo

step 2
vmstat 3 100
  The fields si and so show the amount of memory paged in from disk and paged out to disk, respectively

step 3
ls -al /var/log/sa | grep "Oct 12"
sar -W -f /var/log/sa/sa12
The fields pswpin and pswpout show the total number of pages brought in and out per second, respectively.

alternative example for yesterday
sar -f /var/log/sa/sa$(date +%d -d yesterday)

solutions
Adding more RAM.
Reducing the size of the SGA.
Increasing the size of the swap space

Useful command to Check performance on linux server

network
   netstat -ptc

IO
  iostat

 ps commands in order to check for performance probelms:

1) Displaying top CPU_consuming processes:

 # ps aux|head -1; ps aux|sort -rn -k2|head -10
2) Displaying top 10 memory-consuming processes:

# ps aux|head -1; ps aux|sort -rn -k3|head
3) Displaying process in order of being penalized:

# ps -eakl|head -1; ps -eakl|sort -rn +5
4) Displaying process in order of priority:

# ps -eakl|sort -n +6|head
5) Displaying process in order of nice value

# ps -eakl|sort -n +7
6) Displaying the process in order of time

# ps vx|head -1;ps vx|grep -v PID|sort -rn +3|head -10
7) Displaying the process in order of real memory use

# ps vx|head -1; ps vx|grep -v PID|sort -rn +6|head -10
8) Displaying the process in order of I/O

# ps vx|head -1; ps vx|grep -v PID|sort -rn +4|head -10
9) Displaying WLM classes

# ps -a -o pid, user, class, pcpu, pmem, args
10) Determining process ID of wait processes:

# ps vg|head -1; ps vg|grep -w wait
11) Wait process bound to CPU

 # ps -mo THREAD -p
12) CPU usage with priority levels

 # topas -P


# for x in `seq 1 1 10`; do ps -eo state,pid,cmd | grep "^D"; echo "----"; sleep 5; done

# ps auxww --sort=lstart | sort -r -k3,4 | head -20

# ps aux --sort -pcpu

# ps -ef --sort=start_time|head -20

# watch -n 1 "(ps aux | awk '\$8 ~ /D/  { print \$0 }')"

-- -----------------------------------------------------------------------

Linux "tr" Command Examples

Translate upper case to lower
tr A-Z a-z < inputfile > outputfile
cat inputfile|tr A-Z a-z

Translate braces into parenthesis
tr '{}' '()' < inputfile > outputfile

Translate white-space to tabs
 echo "This is for testing" | tr [:space:] '\t'

Squeeze repetition of characters using -s
echo "This   is   for testing" | tr [:space:] '\t'

We can use -s option to squeeze the repetition of characters.
echo "This   is   for testing" | tr -s [:space:] '\t'

convert multiple continuous spaces with a single space
echo "This  is  for testing" | tr -s [:space:] ' '

Delete specified characters using -d option
echo "the geek stuff" | tr -d 't'

To remove all the digits from the string, use
echo "my username is 432234" | tr -d [:digit:]

Complement the sets using -c option
echo "my username is 432234" | tr -cd [:digit:]

Remove all non-printable character from a file
tr -cd [:print:] < file.txt

Join all the lines in a file into a single line
 tr -s '\n' ' ' < file.txt

Deleting tons of files in Linux (Argument list too long)

option 1

find /oramisc01/oracle/demo/adump -name "*aud"  -type f -mtime +1 -exec rm {} \;

option 2

find  /oramisc01/oracle/demo/adump  -name "aud*" -type f  -delete

option 3

find  /oramisc01/oracle/demo/adump  -name "aud*" -type f -mtime +1 |xargs rm -rf

option 4

mkdir empty_dir

rsync -a -delete empty_dir/ yourdirectory/

option 5

perl -e 'for(<*aud>){((stat)[9]<(unlink))}'

Substring - Shell Command

$ export DUMP_PATH=/tmp/products/exp_full_DEMO_1_1.dmp

#delete matching pattern from beginning

$ export TMP=${DUMP_PATH#*/exp_}  $ echo $TMP

full_DEMO_1_1.dmp

#delete matching pattern from end

$ export EXPORT_FILE=${TMP%.dmp}  

$ echo $EXPORT_FILE

full_DEMO_1_1

sheel scripting --- tree like directory structure

-

ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'

swap area in database server

The command to see what’s available is:

free –m This will show you the free space in megabytes

Here’s the output:

             total       used       free     shared    buffers     cached
Mem:         48299      26358      21940          0        647      19719
-/+ buffers/cache:       5992      42306
Swap:         4095          0       4095

So, the swap line shows you that you have 4 gig of swap free…excellent.

For the amount of free memory, you want to look at the -/+ buffers/cache line. So, looking at that line, you have 21940 megabytes (21 gig) free.

Keep in mind the cached value and the buffers value will be reclaimed by the system is needed. You actually want a large cached value. I’m not sure what’s stored in it but….


Caution ! Running out  of SWAP area may  cause  the server to hang

Rupam

count # of processes for each database

Count number of client connections to oracle Database on a server

countproc.sh

#/bin/ksh

# count # of processes for each database

for i in ` pgrep -fl smon|awk -F_ '{print $3}'`

do

echo "database name : " $i

ps -ef|grep $i|grep LOCAL=NO |wc -l

done

Rupam

sudo

How to sudo to oracle account in solaris

/usr/local/bin/sudo /bin/su - oracle

Rupam

Disk IO rate on Oracle Database Sever

From Unix Server

iostat -d -m

iostat -d /dev/sdc -m -x 2

   where

         /dev/sdc is the name of the disk  ( refer my earlier post on how to map ASM device to physical disks)

         await is disk response time

         svctm is time to process i/o request by disk

hostname : tiger

From OEM:

Targets / tiger / performance / under the Disk I/O utilization click ‘Total I/Os per second’ / change view data to last 31 days. See statistics on the left side of the page.

In OEM:

Targets / +ASM_tiger / Performance / Bottom of the page – Disk Group I/O Cumulative Statistics

OEM is Targets / Hosts. Click the Total IO/sec header to sort the list to see what other hosts are similar to tiger. Scroll down to find tiger . This is the same variable as noted above but for the last 24 hours (instead of last 31 days).

Hope this help! Rupam

Disk IO rate on Oracle Database Sever

iostat -d -m

iostat -d /dev/sdc -m -x 2

   where

         /dev/sdc is the name of the disk  ( refer my earlier post on how to map ASM device to physical disks)

         await is disk response time

         svctm is time to process i/o request by disk

Hope this help! Rupam

How to find mapping of ASM disks to Physical Devices?

1.Login as oracle and list ASM diskgroups

$ oracleasm listdisks

DATA1

2. Query Disk

$ oracleasm querydisk -d  DATA1

Disk "DATA1" is a valid ASM disk on device [8, 33]

3. oracleasm querydisk  DATA1  you will get in addition the major - minor numbers, that can be used to match the physical device,

# ls -l /dev |grep 8|grep 33

brw-r----- 1 root disk     8,  33 Aug  2 16:10 sdc1

Hope this help! Rupam

How to remove control characters from text file

tr -d '\015\032' < dosformatfile.txt > unixformatfile.txt

The above command deletes all ^M and ^Z

Using Perl:

perl -i -ep 's/\015//g' filename

perl -pi.bak -0777 -e 's#\r##gi' filename

using sed:

sed -e "s/^M/" filename > newfilename

press ctrl them V & M

Using vi:

vi filename

:s/^M//g

press control then V & M

with:

:%s/^M/\r/g

works perfectly !!!

Heres another little script

#!/bin/sh

FILE="$1"

# Use sed with the -i command line for inline interpreting.

sed -i '' "s/\r//g" $FILE

# the end

Hope this helps. Rupam

CPU utilization on linux

To find out processes, which have exceeded CPU utilization threshold on the server.

In the example,

a) CPU utilization of java processes are checked. You may replace it for any other process.

b) Also, input the threshold value

nfs volumes on Linux

-         Set the mount options explictly. Here are the mount options that need to be used for nfs volumes on Linux.

Use : rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,actimeo=0

default rsize and wsize for NFS is 4096 Blocks so if you have rsize=32k and wsize=32k then NFS would be able to read and write large datagram as compared to deafult one TCP option will make sure that your client are getting the data or not Hard & INTR - The program accessing a file on a NFS mounted file system will hang when the server crashes. The process cannot be interrupted or killed unless you also specify intr. When the NFS server is back online the program will continue undisturbed from where it was. actimeo is for access timeout and it should be 0 .

Hope this Helps!. Rupam

Cron format - crontab

# +---------------- minute (0 - 59)

# |  +------------- hour (0 - 23)

# |  |  +---------- day of month (1 - 31)

# |  |  |  +------- month (1 - 12)

# |  |  |  |   +---- day of week (0 - 6) (Sunday=0 or 7)

# |  |  |  |    |

   *  *  *  *  *  command to be executed

 Examples:

00 23 * * 0,2,4,6 /bin/ksh /home/rupam/rman_hot_backup.sh orcl 1> /home/rupam/BACKUP/backup.log 1<&1

20 30 * * * /bin/ksh /home/rupam/collect_partition_stats.sh 1> /dev/null 2>&1