Adding client certificate file using chef data bags

Below is just an example 

logstash_cert_secret = Chef::EncryptedDataBagItem.load_secret("/etc/chef/secrets/logstash_client_key")

logstash_certs = Chef::EncryptedDataBagItem.load("certs", "logstash_client", logstash_cert_secret)

directory "/etc/pki/tls/certs/" do

  action :create

  mode "0755"

  owner "root"

  group "root"

end

file "/etc/pki/tls/certs/logstash_client.crt" do

  action :create

  mode "0644"

  owner "root"

  group "root"

  content logstash_certs["crt"].join("")

  end

Search and replace in all files in a current directory

grep -rl 'pc961' ./|xargs sed -i 's/pc961/pc/g'

Ansible define a basic windows hosts

[windows]

10.10.10.120

[windows:vars]

ansible_connection=winrm

ansible_ssh_user=Administrator

ansible_ssh_pass=AdminPa$$

Three basic Git commands

1) git add -A

git add -A is equivalent to  git add .; git add -u.

git add .  ( This will add all files but does not remove files )
git add -u ( This will add only changes but will not add any new files created )

So we use

git add -A ( this will take care of both the commands )

2) git commit -m “Commit changes text ex: I have added new function xyz to the file”

3) git push origin  


example of branch name : develop


Another 3 frequently used commands include :

git pull
git status 
git merge origin/Branchname

Installing configuring Ansible and deploying haproxy playbook

  Basic installation 

sudo yum install ansible (or) sudo pip install ansible

Configuring your hosts file with inventory 

cat /etc/ansible/hosts 

[haproxy]

10.0.184.12   



[all_servers:vars]

ansible_connection=ssh 

ansible_user=root

ansible_ssh_private_key_file=/Users/home/yoursshkey.pem


I have only one server in my inventory where i will be setting up an haproxy 
For easy management we will be writing common variables under [all_servers:vars]


When we run ansible play books hosts will be picked from the above inventory file 

Example : hosts: haproxy  ( this will run on all servers under haproxy )

Now test servers doing a ping ( ping in ansible will connect to the server using ssh )

 ansible haproxy -m ping


10.0.184.12 | SUCCESS => {


    "changed": false, 


    "ping": "pong"


}


Then Download ready made packages from ansible galaxy 
https://galaxy.ansible.com/devops/haproxy/
Also download dependencies that will configure EPEL repositories
https://galaxy.ansible.com/sfromm/epel/
Then create a common playbook for both the roles main.yml




- hosts: haproxy

  roles:

     - { role: devops.epel }





- hosts: haproxy

  roles:

     - role: devops.haproxy

       haproxy_stats:

         name: 'global_monitor'

         ip: "{{ ansible_default_ipv4.address }}"

         port: '38888'

         stats:

           enabled: True

           hide_version: true

           uri: /slb_stats_url

           realm: Welcome\ to\ slb\ monitor

           auth: admin:admin

           refresh: 2s

       haproxy_frontends:

       - name: 'fe-testsite'

         ip: '{{ ansible_default_ipv4.address }}'

         port: '80'

         maxconn: '1000'

         default_backend: 'be-testsite'

       haproxy_backends:

       - name: 'be-testsite'

         description: 'testsite'

         servers:

           - name: 'be-testsite-01'

             ip: '192.168.1.100'

sh-3.2# ansible-playbook main.yml 

PLAY [haproxy] *****************************************************************

TASK [setup] *******************************************************************
ok: [10.0.184.12]

TASK [devops.epel : Installs python dependencies] ******************************
changed: [10.0.184.12] => (item=[u'libselinux-python'])

TASK [devops.epel : Disable SElinux] *******************************************
changed: [10.0.184.12]

TASK [devops.epel : create EPEL yum repository] ********************************
changed: [10.0.184.12]

TASK [devops.epel : import EPEL GPG key] ***************************************
changed: [10.0.184.12]

PLAY [haproxy] *****************************************************************

TASK [setup] *******************************************************************
ok: [10.0.184.12]

TASK [devops.haproxy : Include OS-specific variables.] *************************
ok: [10.0.184.12]

TASK [devops.haproxy : Installs haproxy as well as socat for socket api.] ******
changed: [10.0.184.12] => (item=[u'haproxy', u'socat'])

TASK [devops.haproxy : Ensure HAProxy is started and enabled on boot.] *********
changed: [10.0.184.12]

TASK [devops.haproxy : Ensure chroot directory exists.] ************************
ok: [10.0.184.12]

TASK [devops.haproxy : Create directory for the frontend] **********************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the frontends] *********************************
[DEPRECATION WARNING]: Using bare variables is deprecated. Update your playbooks so that the environment value uses the full variable syntax ('{{haproxy_frontends}}').
This feature will be
removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
changed: [10.0.184.12] => (item={u'ip': u'10.0.184.12', u'maxconn': u'1000', u'default_backend': u'be-testsite', u'port': u'80', u'name': u'fe-testsite'})

TASK [devops.haproxy : Create directory for the backends] **********************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the backends] **********************************
[DEPRECATION WARNING]: Using bare variables is deprecated. Update your playbooks so that the environment value uses the full variable syntax ('{{haproxy_backends}}').
This feature will be
removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
changed: [10.0.184.12] => (item={u'servers': [{u'ip': u'192.168.1.100', u'name': u'be-testsite-01'}], u'description': u'testsite', u'name': u'be-testsite'})

TASK [devops.haproxy : Create directory for the listen sections] ***************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the listen sections] ***************************
skipping: [10.0.184.12] => (item=haproxy_listen)

TASK [devops.haproxy : Create directory for the userlists] *********************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the userlist sections] *************************
skipping: [10.0.184.12] => (item=haproxy_userlists)

TASK [devops.haproxy : Create  the compiled folder] ****************************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the global config] *****************************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the default config] ****************************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the stats config] ******************************
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the frontends configuration file] **************
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the backends configuration file] ***************
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the listen sections configuration file] ********
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the userlists sections configuration file] *****
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the final configuration file] ******************
changed: [10.0.184.12]

RUNNING HANDLER [devops.haproxy : restart haproxy] *****************************
changed: [10.0.184.12]

PLAY RECAP *********************************************************************
10.0.184.12                : ok=31   changed=27   unreachable=0    failed=0




This is just a test deployment that will forward request to background test server 192.168.1.100

Kibana4 apache configuration with authentication

We need to configure httpd to listen on port 80 which does a reverse proxy to localhost on 5601.

Make sure kibana is listening only on internal port before this settings are done .

        ServerAdmin devops@learnadmin.com

        ServerName kibana.learnadmin.com

        DocumentRoot /var/www/auth

       

            AuthType Basic

            AuthName "Authentication Required"

            AuthUserFile "/var/www/auth/htpasswd"

            Require valid-user

       

        ProxyPass / http://localhost:5601/

        ProxyPassReverse / http://localhost:5601/

Use below command to create a htpasswd file and enable authentication 

htpasswd -c /var/www/auth/htpasswd devops

Monitoring of kibana and logstash services and start them if process not found

Kibana and logstash might run out of memory and application might get closed or get terminated. In that case we will be running the below script to monitor services and start them if not running .

There are other parameters that need to be verified before going for this option like java heap size etc.

#!/bin/bash

LOGSTASH=logstash

KIBANA=kibana

var1=`ps -ef|grep -i $LOGSTASH|sed -e '/grep/d'`

if [ -z "$var1" ]; then

nohup /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf &

echo "Process logstash not found" | mail -s "Process logstash not found creating a new one" devops@learnadmin.com

else

echo "process logstash  found."

fi

var2=`ps -ef|grep -i $KIBANA|sed -e '/grep/d'`

if [ -z "$var2" ]; then

nohup /opt/kibana/bin/kibana status &

echo "Process kibana  not found" | mail -s "Process kibana not found creating a new one" devops@learnadmin.com

else

echo "process kibana found."

fi