Graphite & statsd installation and Configuration

How to install Graphite and statsd
-----------------------------------

yum install   python-devel.noarch
yum install   pycairo.x86_64 Django.noarch django-tagging.noarch  python-twisted.noarch python-zope-interface.x86_64 python-zope-interface httpd memcached  python-memcached
yum install   fontconfig.x86_64 fontconfig-devel.x86_64
yum install   mod_wsgi.x86_64
yum install   python-pip.noarch

pip-python install whisper
pip-python install carbon
pip-python install graphite-web



cp /opt/graphite/conf/carbon.conf.example /opt/graphite/conf/carbon.conf
 cp /opt/graphite/conf/storage-schemas.conf.example /opt/graphite/conf/storage-schemas.conf
 cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
 cp /opt/graphite/examples/example-graphite-vhost.conf /etc/httpd/conf.d/graphite.conf
 cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py




 python /opt/graphite/webapp/graphite/manage.py syncdb
 chown -R apache:apache /opt/graphite/storage/



wget -O /etc/init.d/carbon https://gist.github.com/raw/3652720/6978bda604c794e21165a81c6b879528d19e8bb4/carbon.init.sh
 chmod 0755 /etc/init.d/carbon
chkconfig --add carbon



 /opt/graphite/bin/carbon-cache.py start
 /etc/init.d/httpd start



yum install nodejs npm


cd /opt/graphite
git clone https://github.com/etsy/statsd.git
cd statsd
cp exampleConfig.js local.js



/etc/init.d/carbon start

/etc/init.d/memcached  start
/etc/init.d/statsd start



Configurations
--------------------
Graphite Configuration: /opt/graphite
Statsd configugration: /opt/graphite/statsd




Graphite/Statsd Logs
---------------------------

Graphite: /opt/graphite/storage/log/webapp
Statsd:  /var/log/statsd/
statsderr.log  statsd.log

Bash script to mask credit card number or any data in logs.

create a temp test file for testing your data

cat test.txt

hi this is my credit card number 1234567891234567
i have 22 this 22 is just for testing 1234567890123456 and this one

Here is the format which varies according to your requirement .

In my case there are 16 digits . 

I want to mask first 6 number here is the regular expression or bash command 

 sed -e :a -e "s/[0-9]\{6\}\([0-9]\{10\}\)/\*\*\*\*\*\*\1/;ta" test.txt 

here 6 and 10 are number of masks and number of visible characters respectively.

you can alter them according to your needs. 

And the stars can be replaced by # or the value you want to mask with .

The number of starts should be equal to number of masks in my case it is 6

Bash script get yesterdays or one day ago's date or date-1

date -d '1 day ago' +"%Y-%m-%d"

Linux list all processes based on memory consumption .

list top 5 processes
ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -5

list all processes in assending order .
ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS

easy ntp time sync between servers

Choose one server from which time need to be synced .

then on clinet open  /etc/ntp.conf

#server 0.rhel.pool.ntp.org ( comment these three lines )
#server 1.rhel.pool.ntp.org
#server 2.rhel.pool.ntp.org
server 10.10.10.180  ( this is the server which i choose so all other servers will be in sync with this server )

 /usr/sbin/ntpdate -s -b -p 8 -u 10.10.10.180

/etc/init.d/ntpd restart

Other commands that are used for NTP are 

ntpdate -u 10.10.10.180

output : ntpdate[13786]: adjust time server 10.10.10.180 offset 0.000221 sec

ntpdate -d 10.10.10.180 

output : output will be more in detail

ntpdate[13806]: ntpdate 4.2.2p1@1.1570-o Thu Nov 26 11:34:35 UTC 2009 (1)

Looking for host 10.10.10.180 and service ntp

host found : 10.10.10.180

transmit(10.10.10.180)

receive(10.10.10.180)

transmit(10.10.10.180)

receive(10.10.10.180)

transmit(10.10.10.180)

receive(10.10.10.180)

transmit(10.10.10.180)

receive(10.10.10.180)

transmit(10.10.10.180)

server 10.10.10.180, port 123

stratum 3, precision -20, leap 00, trust 000

refid [10.10.10.180], delay 0.02583, dispersion 0.00000

transmitted 4, in filter 4

reference time:    d5b8497a.219236b6  Fri, Aug 16 2013 12:17:22.131

originate timestamp: d5b849c8.3769d786  Fri, Aug 16 2013 12:18:40.216

transmit timestamp:  d5b849c8.37669489  Fri, Aug 16 2013 12:18:40.216

filter delay:  0.02588  0.02585  0.02585  0.02583

         0.00000  0.00000  0.00000  0.00000

filter offset: -0.00006 -0.00007 -0.00006 -0.00007

         0.000000 0.000000 0.000000 0.000000

delay 0.02583, dispersion 0.00000

offset -0.000070

16 Aug 12:18:40 ntpdate[13806]: adjust time server 10.10.10.180 offset -0.000070 sec

ntpq -p

How to copy directory structure in linux

Sometimes we need to copy only directory structure leaving content ( files )

find . -type d >dirs.txt

xargs mkdir -p

Script to import csv into dhcpd.conf import multiple mac bindings

make sure your csv file is in this order MAC,IP,NAME and file name of csv is ips.csv
Once that is done just run the script and check mac_out.txt which is file to be imported in dhcpd.conf

!/bin/bash

INPUT=ips.csv
OUTPUT=mac_out.txt
OLDIFS=$IFS
IFS=","
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
[ -f $OUTPUT ] && { rm $OUTPUT; }
[ ! -f $OUTPUT ] && { touch $OUTPUT; }
while read mac ip uname
do
echo "host $uname {" >> $OUTPUT
echo -e "\thardware ethernet $mac;" >> $OUTPUT
echo -e "\tfixed-address $ip;" >> $OUTPUT
echo "}" >> $OUTPUT
done < $INPUT

IFS=$OLDIFS

How to drop cache or in-memory cache on linux and free up some memory ?

$ echo 1 | sudo tee /proc/sys/vm/drop_caches  # drop pagecache
$ echo 2 | sudo tee /proc/sys/vm/drop_caches  # drop dentries and inodes
$ echo 3 | sudo tee /proc/sys/vm/drop_caches  # drop pagecache, dentries and inod

Also we can use 

echo 1 > /proc/sys/vm/drop_caches

echo 2 > /proc/sys/vm/drop_caches

echo 3 > /proc/sys/vm/drop_caches

This is also used when we take benchmarks of server.

Command to get linux dhcp client list

cat /var/log/messages | grep DHCPACK

What Server remote management systems called ?

These are different remote server management tools

Intel uses RMM2 (remote management 2)
Dell uses DRAC (Dell Remote Access Control)
Sun or Oracle uses ILOM (Integrated Lights Out Manager)
IBM uses IMM (Integrated Management Module)
HP uses ILO (Integrated Lights-Out).

Simple encryption and decryption of files using GPG

gpg --yes --passphrase="test" -c test1.txt

encrypted file will be test1.txt.gpg and test1.txt will still exist.

gpg --decrypt  --passphrase "test" --output "test1.txt" "test1.txt.gpg"

Sophos command to exclude directory from on-access scans

/opt/sophos-av/bin/savconfig  ExcludeFilePaths /opt/folder-exclude

Linux bash script for checking and sending email alerts on disk space issues

#!/bin/sh
FILE=/tmp/mailcontent.txt

> /tmp/mailcontent.txt

df -H | awk '{ print $5 " " $6 }'|head -6|tail -5|while read output;

do
#  echo $output
  usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1  )
  partition=$(echo $output | awk '{ print $2 }' )
echo $usep
  if [ $usep -ge 90 ]; then
   echo -e "Running out of space \n\nPartition:\"$partition ($usep%)\" \nHost:$(hostname) \nDate:$(date)\n" >> /tmp/mailcontent.txt

 fi

done

if [[ -s $FILE ]] ; then

mail -s "CRITICAL ALERT:Disk space of $hostname more that 90%" disk-alert@yourcompany.com
else
echo "$FILE is empty."
fi

exit

Install and configure OSSEC ( Host-based Intrusion Detection System ) - By Subhash C

OSSEC is  used to meet PCI Compliance central logging and intrusion monitoring requirements with a free and self-managed solution. OSSEC monitors all types of logs such as syslog, apache, maillogs, mysql logs, ftp logs, and more.

1.     Deploying OSSEC core server:

# Wget  http://www.ossec.net/files/ossec-hids-2.7.tar.gz

2.       Extract the compressed OSSEC package and run the “./install.sh” script (It will guide you through the installation).

# tar -xvf ossec-hids-2.7.tar.gz 

# cd ossec-hids-2.7

# ./install.sh

The interactive installation begins and during this process it prompts for installation type either server or agent or local. However, installation path for OSSEC can be defined but by default it installs in (/var/ossec) directory. It is advised to select default settings and follow accordingly till the end of the installation. Finally, it is compiled and installed successfully according to the specification provided.

Select the language (in this example it is EN)

Select the type of installation as server

Accept the default folder, unless you want to change it:

Enable email notification by specifying the email address which will be used to send email alerts FROM:

If setup fails to determine your SMTP gateway automatically, you may specify it manually (change the IP as necessary to reflect your environment):

Enable integrity check daemon:

Enable rootkit detection daemon:

Whether or not you want to have “Active response” enabled is up to you. Although in prod environment, it can be quite risky.

We will not be using Syslog daemon feature in this example, so disable it.

OSSEC core server installation is now completed.

1.     Generating client keys

OSSEC server with the agent names in order to generate unique authentication keys required to bind the agent to the corresponding OSSEC server.

a. Firstly, create agents.TXT file containing the IPs and names separated by comma, of all target Linux Servers:

192.168.9.101,rwca1

...

…

….

Save:wq!

b. Place this file to /var/ossec folder on OSSEC server.

c. Now execute the following command: /var/ossec/bin/manage_agents -f /agents.txt

More info: http://www.ossec.net/doc/programs/manage_agents.html

1.    Start OSSEC Server


# /var/ossec/bin/ossec-control start
# /var/ossec/bin/ossec-control restart
#/var/ossec/bin/ossec-control stop  

2.     Install the OSSEC Agents

Follow above server installation steps, instead of server choose type agent

            # tar -xvf ossec-hids-2.7.tar.gz 

# cd ossec-hids-2.7

# ./install.sh

Select the type of installation as agent

3.     Add Agents to OSSEC Manager

To add an agent to an OSSEC manager with manage_agents you need to follow the steps below.

1. Run manage_agents on the OSSEC server.
2. Add an agent.
3. Extract the key for the agent.
4. Copy that key to the agent.
5. Run manage_agents on the agent.
6. Import the key copied from the manager.
7. Restart the manager’s OSSEC processes.
8. Start and restart the agent.

Follow online Guide: http://www.ossec.net/doc/manual/agent/agent-management.html#running-manage-agents-and-start-screen

4.     Verify successful installation of the agent

Then verify that the client has been installed successfully and communicating with the OSSEC server.

# tail –f /var/ossec/log/ossec.log

5.     Deploy OSSEC Web UI dashboard

Install PHP for OSSEC UI

#  yum install php-mysql php-devel php-gd php-pecl-memcache php-pspell php-snmp php-xmlrpc php-xml

# Start Apache:  /etc/init.d/httpd  start

# Configure Apache autostart: /sbin/chkconfig httpd on

# Restart Apache: /etc/init.d/httpd restart

Download OSSEC webUI module

# cd /var/ww/html/

#  Wget   http://www.ossec.net/files/ossec-wui-0.8-alpha-0.tar.gz

# tar –xvf ossec-wui-0.8-alpha-0.tar.gz

# mv ossec-wui-0.8-alpha ossec-wui

# Switch to the ossec-wui and run

./setup.sh -  specify user and password to protect Web UI.

# Add apache user to ossec group: 

Change permissions for /var/ossec/tmp directory:

# chmod 770 /var/ossec/ tmp/

# chgrp apache  /var/ossec/tmp/

# /etcinit.d/httpd restart

# /var/ossec/bin/ossec-control restart (restart the OSSEC server)

Access the OSSEC web portal: http://your-ip-address/ossec-wui

in the above URL replace your-ip-address with ip address where you installed ossec-wui

More info: http://www.ossec.net/wiki/index.php/OSSECWUI:Install

Step by step tutorial to configure two node redhat clustering By -- Sojan VM

Two node Red Hat and Clustering

  node1

  node2

  management server

Configure the Shared Storage

Install the clustering software on the nodes

High availability application service “High Availability”

package group  and webserver

service ricci start

provide  passwd for ricci

Install the cluster management software on

the management server

yum groupinstall “High Availability Management”

chkconfig luci on

service luci start

https://manage.example.com:8084

Define a cluster

Node1 and Node2

Then create a partition and format

Define kvm Fence device

Cluster.conf--------------------cluster configuration fine

<?xml version="1.0"?>

<cluster config_version="6" name="cluster">

        <clusternodes>

                <clusternode name="cluster_node1" nodeid="1">

                        <fence>

                                <method name="kvm">

                                        <device domain="cluster_node1" name="kvm"/>

                                </method>

                        </fence>

                </clusternode>

                <clusternode name="cluster_node2" nodeid="2">

                        <fence>

                                <method name="kvm">

                                        <device domain="cluster_node2" name="kvm"/>

                                </method>

                        </fence>

                </clusternode>

        </clusternodes>

        <cman expected_votes="1" two_node="1"/>

        <fencedevices>

                <fencedevice agent="fence_xvm" name="kvm"/>

        </fencedevices>

</cluster>

Kvm Host configuration

Let's start Installing the necessary packages on the host:

yum install   fence-virt fence-virtd fence-virtd-libvirt fence-virtd-multicast

Then create the key needed for the host to authenticate all the fencing requests from the guest cluster:

dd if=/dev/urandom of=/etc/cluster/fence_xvm.key bs=4096 count=1

The key should then be copied to /etc/cluster/fence_xvm.key of every guest.

Now run the configuration tool:

fence_virtd –c

service fence_virtd start

check your fence is working or not:

fence_xvm -o reboot –H node2

Define Resources For Clustered Web Service

Shared Storage (if not in fstab)

IP address

Apache Resource

Define Failover Domains

Node2

Define Clustered Web Service

Define service

Add storage resource (if not in fstab)

Add ip address resource

add script resource

Fix : rpmdb: unable to lock mutex: Invalid argument

 If you are getting this message printed out repeatedly on the terminal screen while trying to install some packages using yum

 

# rpmdb: unable to lock mutex: Invalid argument



 

 yum -y update glibc ( This works sometimes )

 

If that doesn not work then execute below command 



 

rpmdb --rebuilddb -vv

Single command to find and delete all empty files in a directory

find /tmp/testing/ -empty -delete

Install configure sophos antivirus on linux servers

Download sav-linux-xx-xxx.tgz 
from
( http://www.sophos.com )

after untar we will get dir "sophos-av"


install.sh
sav.tar
talpa.tar
uncdownload.tar


Run ./install.sh to install

-------------------------------
Configuration
---------------------------------


/opt/sophos-av/bin/savdstatus


/opt/sophos-av/bin/savdctl enable


Start stop restart disable service as normal linux service

/etc/init.d/sav-protect start

(or)

service sav-protect start


Scanning your computer
-----------------------

savscan /  ( we can use sophos enterprise console to scan all computers at once )

Scanning bootsectors
---------------------

savscan -bs=drivename

scanning masterboot record

savscan -mbr


Enable on reboot
---------------------
/opt/sophos-av/bin/savdctl enableOnBoot savd

Enable onaccess scanning
---------------------------
/opt/sophos-av/bin/savconfig query EnableOnStart

to enable onaccess scanning after reboot

/opt/sophos-av/bin/savconfig set EnableOnStart true


chkconfig --list  ( check if is starting on reboot )



Update sophos antivirus
---------------------------
/opt/sophos-av/bin/savupdate