Kibana4 apache configuration with authentication

We need to configure httpd to listen on port 80 which does a reverse proxy to localhost on 5601.
Make sure kibana is listening only on internal port before this settings are done .


        DocumentRoot /var/www/auth

            AuthType Basic
            AuthName "Authentication Required"
            AuthUserFile "/var/www/auth/htpasswd"
            Require valid-user

        ProxyPass / http://localhost:5601/
        ProxyPassReverse / http://localhost:5601/

Use below command to create a htpasswd file and enable authentication 
htpasswd -c /var/www/auth/htpasswd devops

Monitoring of kibana and logstash services and start them if process not found

Kibana and logstash might run out of memory and application might get closed or get terminated. In that case we will be running the below script to monitor services and start them if not running .
There are other parameters that need to be verified before going for this option like java heap size etc.



var1=`ps -ef|grep -i $LOGSTASH|sed -e '/grep/d'`

if [ -z "$var1" ]; then

nohup /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf &

echo "Process logstash not found" | mail -s "Process logstash not found creating a new one"


echo "process logstash  found."


var2=`ps -ef|grep -i $KIBANA|sed -e '/grep/d'`

if [ -z "$var2" ]; then

nohup /opt/kibana/bin/kibana status &

echo "Process kibana  not found" | mail -s "Process kibana not found creating a new one"


echo "process kibana found."


Chef provision a esxi virtual machine or a virtual server

We need to install esxi knife plugin on our workstation

gem install knife-esx

Add below params in knife.rb

knife[:esx_host] = "esx"
knife[:esx_username] = "root"
knife[:esx_password] = "password of your esxi server"

Need to create a ubuntu template ( or custom OS template as per requirement )
Below example shows how to provision a ubuntu template .

ssh root@esxi

mkdir -p /vmfs/volumes/datastore1/esx-gem/templates

vmkfstools -i /vmfs/volumes/datastore1/ubuntu-12.10-x64_template/*.vmdk –diskformat thin /vmfs/volumes/datastore1/esx-gem/templates/ubuntu-12.10-x64_template.vmdk

esx template list ( should display the template we installed )

knife esx vm create --vm-name server1 --use-template ubuntu-12.10-x64_template --verbose true --distro ubuntu12.04-19-gems --vm-memory 2048 -x provision -i ~/.ssh/id_rsa

How to use chef encrypted databags

Note: Please comment the below line in knife.rb file else it will automatically decrypt values
and show when we use "knife data bag show" command
I initially thought knife was not encrypting my values and had to debug ..which took lot of time.

[root@ec2-test .chef]# cat knife.rb | grep secret
#knife[:secret_file] ='/root/encrypted_data_bag_secret'

knife data bag create  --secret-file /root/.chef/encrypted_data_bag_secret testbag password

  "id": "password",
  "password": "this is test password key"


[root@ec2-test .chef]# knife data bag show testbag password

id:       password
  cipher:         aes-256-cbc
  encrypted_data: KMHzeFQwfm0wWeHFymxfJsMo425CP+wlwoZ6xN7waVlgUNOmRrr/+jOtDLIN
  iv:             TYPYnSqYTcmU8ZWE2sIt4A==
  version:        1

Once encrypted if you try to edit the encrypted databag it shows like this 

  "name": "data_bag_item_testbag_password",
  "json_class": "Chef::DataBagItem",
  "chef_type": "data_bag_item",
  "data_bag": "testbag",
  "raw_data": {
    "id": "password",
    "password": {
      "encrypted_data": "KMHzeFQwfm0wWeHFymxfJsMo425CP+wlwoZ6xN7waVlgUNOmRrr/+jOtDLIN\ns7Xl\n",
      "iv": "TYPYnSqYTcmU8ZWE2sIt4A==\n",
      "version": 1,
      "cipher": "aes-256-cbc"

To show the decrypted values we can use 

[root@ec2-test .chef]# knife data bag show  testbag password --secret-file /root/.chef/encrypted_data_bag_secret

id:       password
password: this is test password key

Some important commands related to data bags

Generate random secret key

openssl rand -base64 512 >encrypted_data_bag_secret

Generate random password

date | md5 in mac 
date | md5sum in any linux machine 

Setting a Secure Flag to cookie in httpd – Apache

First check if mod_headers is loaded. Else enable that module

LoadModule headers_module modules/

Once the module is loaded .

Header edit Set-Cookie ^(.*)$ $1;Secure

Add above line to your virtual host in httpd.conf/ssl.conf as per your config

What this does is it adds a secure flag to your cookie. This flag is set by application server when sending
a new cookie to client/user when sending http response.This also mean that server will not send cookie over http. It sends only via https .This prevents unauthorized access to cookie data

This can be done via code in java or can be done @ apache/httpd config level.

Here is more info on code level changes click here..!!

rssh chroot jailroot sftp creation - By Vinodh tiruttani

Steps for chroot jail folder creation:
The chroot command changes its current and root directories to the provided directory and then run command, if supplied, or an interactive copy of the user's login shell. Please note that not every application can be chrooted.
Creating User
[root@learnadmin ~]# /usr/sbin/useradd testrssh
[root@learnadmin ~]# su testrssh
[testrssh@learnadmin root]$ cd /home/testrssh/
[testrssh@learnadmin ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/testrssh/.ssh/id_dsa):
Created directory '/home/testrssh/.ssh'.
Enter passphrase (empty for no passphrase): < Type passphrase >
Enter same passphrase again: <  Re-type passphrase >
Your identification has been saved in /home/testrssh/.ssh/id_dsa.
Your public key has been saved in /home/testrssh/.ssh/
The key fingerprint is:
[testrssh@learnadmin ~]$
[testrssh@learnadmin ~]$ cd .ssh/
[testrssh@learnadmin .ssh]$ pwd
[testrssh@learnadmin .ssh]$ cp authorized_keys2
[testrssh@learnadmin .ssh]$ ls -l
total 24
-rw-r--r-- 1 testrssh testrssh 618 Jun 12 18:12 authorized_keys2
-rw------- 1 testrssh testrssh 736 Jun 12 18:11 id_dsa ( 600 permission)
-rw-r--r-- 1 testrssh testrssh 618 Jun 12 18:11
[testrssh@learnadmin .ssh]$
[testrssh@learnadmin .ssh]$ exit
[root@learnadmin ~]# /usr/sbin/usermod -s /usr/bin/rssh testrssh
Download from net rssh-2.3.3.tar.gz from
[root@learnadmin ~]# tar xvf rssh-2.3.3.tar.gz
[root@learnadmin ~]# cd rssh-2.3.3
[root@learnadmin rssh-2.3.3]# ./configure
[root@learnadmin rssh-2.3.3]# make
[root@learnadmin rssh-2.3.3]# make install
[root@learnadmin rssh-2.3.3]# cd ..
 Download from net rssh-2.3.3-1.x86_64.rpm
 [root@learnadmin ~]# rpm -ivh rssh-2.3.3-1.x86_64.rpm
Preparing...                ########################################### [100%]
   1:rssh                   ########################################### [100%]
[root@learnadmin ~]# pwd
·         Converting rssh chroot Jail folder – Execute the below script from the below location
 [root@learnadmin ~]# cp /usr/share/doc/rssh-2.3.3/ .
[root@learnadmin ~]# chmod 775
 [root@learnadmin ~]# ./ /home/testrssh/
NOT changing owner of root jail.
NOT changing perms of root jail.
setting up /home/testrssh//usr/bin
setting up /home/testrssh//usr/libexec/openssh
setting up /home/testrssh//usr/libexec
Copying libraries for /usr/bin/scp.
Copying libraries for /usr/libexec/openssh/sftp-server.
Copying libraries for /usr/bin/rssh.
Copying libraries for /usr/libexec/rssh_chroot_helper.
copying name service resolution libraries...
tar: Removing leading `/' from member names
tar: /lib/libnss1_files*: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors
Setting up /etc in the chroot jail
cp: omitting directory `/etc/'
Chroot jail configuration completed.
NOTE: if you are not using the passwd file for authentication,
you may need to copy some of the /lib/libnss_* files into the jail.

 NOTE: you must MANUALLY edit your syslog rc script to start syslogd
with appropriate options to log to /home/testrssh//dev/log.  In most cases,
you will need to start syslog as:
    /sbin/syslogd -a /home/testrssh//dev/log
 NOTE: we make no guarantee that ANY of this will work for you... if it
doesn't, you're on your own.  Sorry!
 Note: while executing above command some files are not copied, we need to copy manually.
[root@learnadmin ~]#
[root@learnadmin ~]# cd /home/testrssh/
[root@learnadmin testrssh]# cd dev/
[root@learnadmin dev]# mknod -m 666 zero c 1 5
[root@learnadmin dev]# mknod -m 666 null c 1 3
[root@learnadmin dev]# cd ../lib
[root@learnadmin lib]# cp /lib/ .
[root@learnadmin lib]# cp /lib/ .
[root@learnadmin lib]# cd ../lib64/
[root@learnadmin lib64]# cp /lib64/ .
[root@learnadmin lib64]# cp /lib64/libnss_* .
[root@learnadmin lib64]# cd
[root@learnadmin]# vi /etc/rssh.conf ( add the user and uncomment 2 lines)
[root@learnadmin ~]#
Done rssh chroot jail folder is created
Testing rssh login  from remote system
Login to server  from remote server
 Copy id_dsa key from Server ( user: testrssh )
[root@clientmachine test-rssh-keys]# vi id_dsa_testrssh

Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,42ED97DC52451DE9

 [root@ clientmachine test-rssh-keys]# chmod 600 id_dsa_testrssh
 [root@ clientmachine test-rssh-keys]# sftp -o IdentityFile=id_dsa_testrssh testrssh@sftp server
Connecting to 10.x.x.x...
Enter passphrase for key 'id_dsa_testrssh':
sftp> ls
dev    etc    lib    lib64  usr
sftp> cd /root
Couldn't canonicalise: No such file or directory
sftp> cd /tmp
Couldn't canonicalise: No such file or directory
 rssh chroot jail folder is working.