How to copy directory structure in linux

Sometimes we need to copy only directory structure leaving content ( files )

find . -type d >dirs.txt

xargs mkdir -p

Get CPU RAM MEMORY of all the servers on the network

In order to accomplish this you have to have an account with password less login for all the server on the network you want to retrieve data

Here is my tutorial for password less login

Create a file "ips" which should contain all the ip's you want to retrive data for ( 1 ip per line )


for i in `cat ips`;do ssh -t $i 'echo -e -n `uname -n `"   '$i' " ;cat /proc/cpuinfo | grep "cpu cores" | uniq';done > cpucores

( This one will not work in single core cpu's or vm's )

for i in `cat ips`;do ssh -t $i 'echo -e -n `uname -n `"   '$i'" ;grep MemTotal /proc/meminfo   ';done > Mem_details


for i in `cat ips`;do ssh -t $i 'echo -e -n `uname -n `"   '$i'  " ; sudo /sbin/fdisk -l | grep Disk ';done >hdd_details 

( This one will not work in single core cup's or vm's )

After you run all the three commands you will have three files in your directory

1) cpucores - list servername ip-address and number of cores
2) Mem_details - list servername ip-address and memory details in kb
3) hdd_details - list servername ip-address and drive and size.

Single command to get CPU usage % in linux

top -bn1 | grep "Cpu(s)" | sed "s/.*, *\([0-9.]*\)%\id.*/\1/" | awk '{print 100 - $1"%"}'

Remove dot after file permissions on Linux

annoyed seeing . after file permissions and want to remove it

drwxr-xr-x.  3 testuser1 testuser1  4096 2013-01-27 19:26 testfile

find . -print0 |xargs -0 -n 1 sudo setfattr -h -x security.selinux


Reason for that : dot is actually an selinux security context


Type ls -Z command you will see the SELinux applied security context:
ls -Z /var
ls -Z /home
ls --lcontext /etc


For more read below

http://fedoraproject.org/wiki/Security_context

https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Security-Enhanced_Linux/chap-Security-Enhanced_Linux-SELinux_Contexts.html

Bash script to find average file size in a directory

If you want output in Kb


 ls -l | awk '{s+=$5} END {print "Average file size: " s/NR/1024 "k"}'
Average file size: 28683.4k

If you want output in MB

ls -l | awk '{s+=$5} END {print "Average file size: " s/NR/1024/1024 "MB"}'

Average file size: 28.0112MB

If you want output in GB

ls -l | awk '{s+=$5} END {print "Average file size: " s/NR/1024/1024/1024 "GB"}'

Average file size: 0.0273546GB

Step by Step method to upgrade RHEL5 to RHEL6

First on current RHEL5 backup the list of packages installed just in case if you want to cross verify.

rpm -qa --queryformat "%{NAME}\n" | sort > /root/all_old_packages

--------------------------------
Check Selinux is enabled or not if enable disable it and uninstall it .

command used is

sestatus

SELinux status:                 disabled  ( in my case it is disabled )
---------------------------------
 vi /etc/inittab change to runlevel 3

id:3:initdefault:  ( mine is a server which is already in runlevel 3 )

-------------------------------------

remove old docs

rm -rf /usr/share/doc/HTML/*/docs/common ( this is mandatory else upgrade will fail)

-----------------------------------------

Now boot with RHEL6 DVD on boot menu press esc

then enter " linux upgradeany "

Follow instructions and you are done . IF you face any issues you can got to other run levels and check what caused failure. You can troubleshoot that issue and upgrade. During my installation i faced only one issue . That is the reason i have deleted old documents.

Fix Memory leak errors received while starting/running apache service

export JAVA_OPTS="-Xms2048m -Xmx3056m -XX:MaxPermSize=256m"

How to check syntax of dhcpd (dhcpd.conf file )

/usr/sbin/dhcpd -t -cf /etc/dhcpd.conf

 Some other quick tips:




1. Syntax checking for dhcpd files:



/usr/sbin/dhcpd -t -cf /etc/dhcpd.conf



This will do a basic syntax check to find those missing ; or other odd things.



2. host {} are global. You will see this warning if you place the host in a subnet zone. This is because when ISC will parse the file and make hostnames work globally (you can also make it do neat if then things but that makes my head hurt.) For the most part it is useful to put hosts in a group setting. This keeps them out of subnets, and also allows you to put common items with them (say for PXE booting with cobbler).



group {

next-server rhn.example.com;

filename “pxelinux.0″;



host noah {

hardware ethernet 00:11:00:11:00:11;

fixed-address 192.168.1.121;

}

}

How to get list of all cron jobs ( crontab ) in a server

Run below as root :


for user in $(cut -f1 -d: /etc/passwd); do crontab -u $user -l; done


If you want to see who owns that cron job run below :

for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -u $user -l; done


getent passwd cut -d: -f1 perl -e'while(<>){chomp;$l = `crontab -u $_ -l 2>/dev/null`;print "$_\n$l\n" if $l}'

Apache hardening tips - Subhash C

# Server banner


ServerSignature Off // will not show apache version etc
ServerTokens Prod // Will hide OS version etc

# Disable TRACE requests

TraceEnable off

### Security Fixes

RewriteEngine on

RewriteCond %{REQUEST_METHOD} ^(TRACE
TRACK)

RewriteRule .* - [F]

How to get home directory of a prticular user in bash scripting or a single command

The below command gets the home directory of user if at all the user exists
 
getent passwd naresh | cut -f6 -d:

Bash script to add some line to a file at specific line number

I wrote this example to show how to append JAVA_OPTS parameters to startup.sh of tomcat



#!/bin/bash

NLINE=2
s=" export JAVA_OPTS=\"$JAVA_OPTS -Xms1024m -Xmx1024m -XX:MaxPermSize=256m\""
sp=" "
awk -vn="$NLINE" -vs="$s" -vsp="$sp" 'NR==n{$0=$0 sp s}1' startup.sh >temp
mv temp startup.sh



Sp is space given so that if something exists in that line it will give some space and
append our command or string

I just ran 'chmod -x /bin/chmod'. What did I do? How do I recover?

( or )

What to do if you cannot execute CHMOD?

This is a tricky question.. you have just removed execute permission on chmod.
Result of that is you cannot modify file permissions . Not even for chmod itself .
chmod remains usless on your system.

Here is the solution for the problem

sudo /lib/ld-linux.so.2 /bin/chmod 755 /bin/chmod

or

perl -e 'chmod(0755, "/bin/chmod")'

Solution : Backtrack 5 Live blank screen

I have downloaded backtrack 5 DVD and was trying to live boot from my Laptop.

Once i am in text mode and as root user.

I gave command startx

Just to turn command mode to GUI mode .

My system was showing a blank screen .. Searched online but did not find solutions

So I have tried my own troubleshooting methods and found that it was Graphics related issue.

Fix : Hope problem is because you are using a laptop with graphic card and BT is not able to
load specific drivers for your graphic card.
So go to BIOS either disable graphic card or

Change Graphic mode from Switching to discrete

for me it worked !! let me know if you still have problems will come up with other solutions...

Rsync current directory using SSH port to remote host

/usr/bin/rsync -rave "ssh -p 22222 -l root" --delete . 192.168.1.111:/tmp

how to encrypt your scripts using sch

Dont want to give your scripts to any one but want to give someone to execute for specific period of time then this is what you are looking for .
We can also give timeline in which the script can be used.

Also you will not be revealing you passwords in the script.

SCH will encrypt shell scripts using RC4 and make executable binary.

http://www.datsi.fi.upm.es/~frosal/sources/shc-3.8.7.tgz

shc [ -e date ] [ -m addr ] [ -i iopt ] [ -x cmnd ]
[ -l lopt ] [ -ACDhTv ] -f script

shc -e date ( script will not run afterdate specified )
-m "Please contact support " (message to display after we run the expired scirpt)

shc -e 01/06/2010 -m "You are late to run this script." -f script.sh

./script.sh.x



http://www.datsi.fi.upm.es/~frosal/sources/shc.html

I have tried to hack the encrypted binary that came as output, but was not successful
I used bashdb and GDB

Per user mac based bandwidth monitor for routers

Want to calculate how much bandwidth each user (mac address) is using


    ./wrtbwmon setup
   ./wrtbwmon update /tmp/usage.db offpeak
   ./wrtbwmon publish /tmp/usage.db /www/user/usage.htm /jffs/users.txt

Here is open source solution ClickHere

Below is the script  (wrtbwmon )

---------------------------

LAN_IFACE=$(nvram get lan_ifname)

lock()

{

while [ -f /tmp/wrtbwmon.lock ]; do

if [ ! -d /proc/$(cat /tmp/wrtbwmon.lock) ]; then

echo "WARNING : Lockfile detected but process $(cat /tmp/wrtbwmon.lock) does not exist !"

rm -f /tmp/wrtbwmon.lock

fi

sleep 1

done

echo $$ > /tmp/wrtbwmon.lock

}

unlock()

{

rm -f /tmp/wrtbwmon.lock

}

case ${1} in

"setup" )

#Create the RRDIPT CHAIN (it doesn't matter if it already exists).

iptables -N RRDIPT 2> /dev/null

#Add the RRDIPT CHAIN to the FORWARD chain (if non existing).

iptables -L FORWARD --line-numbers -n | grep "RRDIPT" | grep "1" > /dev/null

if [ $? -ne 0 ]; then

iptables -L FORWARD -n | grep "RRDIPT" > /dev/null

if [ $? -eq 0 ]; then

echo "DEBUG : iptables chain misplaced, recreating it..."

iptables -D FORWARD -j RRDIPT

fi

iptables -I FORWARD -j RRDIPT

fi

#For each host in the ARP table

grep ${LAN_IFACE} /proc/net/arp | while read IP TYPE FLAGS MAC MASK IFACE

do

#Add iptable rules (if non existing).

iptables -nL RRDIPT | grep "${IP} " > /dev/null

if [ $? -ne 0 ]; then

iptables -I RRDIPT -d ${IP} -j RETURN

iptables -I RRDIPT -s ${IP} -j RETURN

fi

done

;;

"update" )

[ -z "${2}" ] && echo "ERROR : Missing argument 2" && exit 1

# Uncomment this line if you want to abort if database not found

# [ -f "${2}" ] || exit 1

lock

#Read and reset counters

iptables -L RRDIPT -vnxZ -t filter > /tmp/traffic_$$.tmp

grep -v "0x0" /proc/net/arp  | while read IP TYPE FLAGS MAC MASK IFACE

do

#Add new data to the graph. Count in Kbs to deal with 16 bits signed values (up to 2G only)

#Have to use temporary files because of crappy busybox shell

grep ${IP} /tmp/traffic_$$.tmp | while read PKTS BYTES TARGET PROT OPT IFIN IFOUT SRC DST

do

[ "${DST}" = "${IP}" ] && echo $((${BYTES}/1000)) > /tmp/in_$$.tmp

[ "${SRC}" = "${IP}" ] && echo $((${BYTES}/1000)) > /tmp/out_$$.tmp

done

IN=$(cat /tmp/in_$$.tmp)

OUT=$(cat /tmp/out_$$.tmp)

rm -f /tmp/in_$$.tmp

rm -f /tmp/out_$$.tmp

if [ ${IN} -gt 0 -o ${OUT} -gt 0 ];  then

echo "DEBUG : New traffic for ${MAC} since last update : ${IN}k:${OUT}k"

LINE=$(grep ${MAC} ${2})

if [ -z "${LINE}" ]; then

echo "DEBUG : ${MAC} is a new host !"

PEAKUSAGE_IN=0

PEAKUSAGE_OUT=0

OFFPEAKUSAGE_IN=0

OFFPEAKUSAGE_OUT=0

else

PEAKUSAGE_IN=$(echo ${LINE} | cut -f2 -s -d, )

PEAKUSAGE_OUT=$(echo ${LINE} | cut -f3 -s -d, )

OFFPEAKUSAGE_IN=$(echo ${LINE} | cut -f4 -s -d, )

OFFPEAKUSAGE_OUT=$(echo ${LINE} | cut -f5 -s -d, )

fi

if [ "${3}" = "offpeak" ]; then

OFFPEAKUSAGE_IN=$((${OFFPEAKUSAGE_IN}+${IN}))

OFFPEAKUSAGE_OUT=$((${OFFPEAKUSAGE_OUT}+${OUT}))

else

PEAKUSAGE_IN=$((${PEAKUSAGE_IN}+${IN}))

PEAKUSAGE_OUT=$((${PEAKUSAGE_OUT}+${OUT}))

fi

grep -v "${MAC}" ${2} > /tmp/db_$$.tmp

mv /tmp/db_$$.tmp ${2}

echo ${MAC},${PEAKUSAGE_IN},${PEAKUSAGE_OUT},${OFFPEAKUSAGE_IN},${OFFPEAKUSAGE_OUT},$(date "+%d-%m-%Y %H:%M") >> ${2}

fi

done

#Free some memory

rm -f /tmp/*_$$.tmp

unlock

;;

"publish" )

[ -z "${2}" ] && echo "ERROR : Missing argument 2" && exit 1

[ -z "${3}" ] && echo "ERROR : Missing argument 3" && exit 1

USERSFILE="/etc/dnsmasq.conf"

[ -f "${USERSFILE}" ] || USERSFILE="/tmp/dnsmasq.conf"

[ -z "${4}" ] || USERSFILE=${4}

[ -f "${USERSFILE}" ] || USERSFILE="/dev/null"

# first do some number crunching - rewrite the database so that it is sorted

lock

touch /tmp/sorted_$$.tmp

cat ${2} | while IFS=, read MAC PEAKUSAGE_IN PEAKUSAGE_OUT OFFPEAKUSAGE_IN OFFPEAKUSAGE_OUT LASTSEEN

do

echo ${PEAKUSAGE_IN},${PEAKUSAGE_OUT},${OFFPEAKUSAGE_IN},${OFFPEAKUSAGE_OUT},${MAC},${LASTSEEN} >> /tmp/sorted_$$.tmp

done

unlock

        # create HTML page

        echo "

Total Usage :

" >> ${3}

        echo "" >> ${3}

        echo "

User	Peak download	Peak upload	Offpeak download	Offpeak upload	Last seen

" >> ${3}

        echo "
This page was generated on `date`" 2>&1 >> ${3}

        echo "" >> ${3}

        #Free some memory

        rm -f /tmp/*_$$.tmp

        ;;

*)

echo "Usage : $0 {setup|update|publish} [options...]"

echo "Options : "

echo "   $0 setup"

echo "   $0 update database_file [offpeak]"

echo "   $0 publish database_file path_of_html_report [user_file]"

echo "Examples : "

echo "   $0 setup"

echo "   $0 update /tmp/usage.db offpeak"

echo "   $0 publish /tmp/usage.db /www/user/usage.htm /jffs/users.txt"

echo "Note : [user_file] is an optional file to match users with their MAC address"

echo "       Its format is : 00:MA:CA:DD:RE:SS,username , with one entry per line"

exit

;;

esac

How to extract 7zip files in ubuntu

First install 7zip package

sudo apt-get install p7zip  

( you will need universal repository to run the above command )


after installing

7z x my7Zzipfile

How to Restore Vyatta grub

 grub> linux /boot/vmlinuz root=/dev/sda1

 grub> initrd /boot/initrd

 grub> boot

once we boot OS using this commands, We have to re-install grub to make it perminant

grub-install /dev/sda

will install grub now just reboot and relax !!

Fastest way of transferring files between 2 server's over network

Below are file transfer tips from the geek..

Fastest way of transferring files between 2 server's over network

-- SCP : (20MB/sec)
-- FTP : requires ftp installed id/password  : 40MB/sec
-- NetCAT : might work if netcat is installed.  : 60MB/sec      

   source : tar -cvzf /dir/ |netcat -l 7777
   client :  netcat source_ip:7777 |tar -xvzf path
-- Using Python / WGet : Ninja Master : 80MB/sec

Server:
 # cd $folder_have the file
 # Start python webserver with this command.
 python -m SimpleHTTPServer
Client :
  wget -c http://x.x.x.239:8000/$file

-c will recover from a file break if something broke half way.