Graphite & statsd installation and Configuration

How to install Graphite and statsd
-----------------------------------

yum install   python-devel.noarch
yum install   pycairo.x86_64 Django.noarch django-tagging.noarch  python-twisted.noarch python-zope-interface.x86_64 python-zope-interface httpd memcached  python-memcached
yum install   fontconfig.x86_64 fontconfig-devel.x86_64
yum install   mod_wsgi.x86_64
yum install   python-pip.noarch

pip-python install whisper
pip-python install carbon
pip-python install graphite-web



cp /opt/graphite/conf/carbon.conf.example /opt/graphite/conf/carbon.conf
 cp /opt/graphite/conf/storage-schemas.conf.example /opt/graphite/conf/storage-schemas.conf
 cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
 cp /opt/graphite/examples/example-graphite-vhost.conf /etc/httpd/conf.d/graphite.conf
 cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py




 python /opt/graphite/webapp/graphite/manage.py syncdb
 chown -R apache:apache /opt/graphite/storage/



wget -O /etc/init.d/carbon https://gist.github.com/raw/3652720/6978bda604c794e21165a81c6b879528d19e8bb4/carbon.init.sh
 chmod 0755 /etc/init.d/carbon
chkconfig --add carbon



 /opt/graphite/bin/carbon-cache.py start
 /etc/init.d/httpd start



yum install nodejs npm


cd /opt/graphite
git clone https://github.com/etsy/statsd.git
cd statsd
cp exampleConfig.js local.js



/etc/init.d/carbon start

/etc/init.d/memcached  start
/etc/init.d/statsd start



Configurations
--------------------
Graphite Configuration: /opt/graphite
Statsd configugration: /opt/graphite/statsd




Graphite/Statsd Logs
---------------------------

Graphite: /opt/graphite/storage/log/webapp
Statsd:  /var/log/statsd/
statsderr.log  statsd.log

Bash script to mask credit card number or any data in logs.

create a temp test file for testing your data

cat test.txt

hi this is my credit card number 1234567891234567
i have 22 this 22 is just for testing 1234567890123456 and this one

Here is the format which varies according to your requirement .

In my case there are 16 digits . 

I want to mask first 6 number here is the regular expression or bash command 

 sed -e :a -e "s/[0-9]\{6\}\([0-9]\{10\}\)/\*\*\*\*\*\*\1/;ta" test.txt 

here 6 and 10 are number of masks and number of visible characters respectively.

you can alter them according to your needs. 

And the stars can be replaced by # or the value you want to mask with .

The number of starts should be equal to number of masks in my case it is 6