What is Software delivery ?

Implementing CI/CD is part of Devops . Here CD is continouse Delivery .

It include steps we need to do to make code available to customer.

Which means running that code on production servers , making that code resilient .

Which means no bugs , no security issues , No outages and no traffic spikes .

Python script to check if file is locked also display last modified time

To lock file for testing use this command 

The below command will lock /home/ubuntu/fileloc

flock -x -w 5 /home/ubuntu/fileloc echo "4" >  /home/ubuntu/fileloc && sleep 5

------------------------

test.py

------------------------

import os, time

import datetime as dt

def is_locked(filepath):

   

    locked = None

    file_object = None

    if os.path.exists(filepath):

        try:

            print "Trying to open %s." % filepath

            buffer_size = 8

            # Opening file in append mode and read the first 8 characters.

            file_object = open(filepath, 'a', buffer_size)

            if file_object:

                print "%s is not locked." % filepath

                locked = False

        except IOError, message:

            print "File is locked (unable to open in append mode). %s." % \

                  message

            locked = True

        finally:

            if file_object:

                file_object.close()

                print "%s closed." % filepath

    else:

        print "%s not found." % filepath

    return locked

def wait_for_files(filepaths):

   

    for filepath in filepaths:

      

       if is_locked(filepath):

            print "%s is currently in use." % \

                  (filepath)

            st = os.stat(filepath)    

            mtime = dt.datetime.fromtimestamp(st.st_mtime)

            print('%s modified %s'%(filepath, mtime))

                

# Test

if __name__ == '__main__':

    files = [r"/home/ubuntu/filelock"]

    print wait_for_files(files)

-------------------------

python test.py

Install and configure postges-xl

Other than official documentation i did not find much help . It took one hour for me to setup . As i thought its straight forward setup . Download source , compile it and install it . But there is more to it 

Below are steps i followed :

Download a stable branch 

git clone --depth=1  -b XL9_5_STABLE git://git.postgresql.org/git/postgres-xl.git

cd postgres-xl/

./configure

make

cd contrib/pgxc_ctl

make

cd postgres-xl/ ; make install

cd contrib/pgxc_ctl ; make install 


Now installation is done 

Add below in /etc/profile

export LD_LIBRARY_PATH=/usr/local/pgsql/lib
export PGDATA=/usr/local/pgsql/data
export PATH=$PATH:/usr/local/pgsql/bin

Then add postgres user 
and set /user/local/pgsql as its home directory

chown postgres:postgres -R /user/local/pgsql 

Now we need to setup postgres-XL . Since it is distributed database we need atleast two nodes . In general every component sits on its own node and also have stand by nodes .

For this tutorial purpose i am using two nodes node1 and node2


Major components of Postgres-XL

Global Transaction Monitor (GTM) : The Global Transaction Monitor ensures cluster-wide transaction consistency. GTM is responsible for issuing transaction ids and snapshots as part of its Multi-version Concurrency Control.

Coordinator : The Coordinator manages the user sessions and interacts with GTM and the data nodes.

Data Node : The Data Node is where the actual data is stored.
I have tried installing everything on  node1 for demo purpose but default port of coordinator and data node are same also the share same /tmp directory structure.

So we might  need to change port number then temp directories etc

      Below is the error for reference purpose

      postgres@Node1:~$ postgres --datanode -D       /usr/local/pgsql/dbdata -h 0.0.0.0 -p 5432 -c gtm_host=node1 -c gtm_port=6666 &
[3] 35190
postgres@Node1:~$ LOG:  could not bind IPv4 socket: Address already in use
HINT:  Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
WARNING:  could not create listen socket for "0.0.0.0"
FATAL:  could not create any TCP/IP sockets
^C
[3]+  Exit 1                  postgres --datanode -D /usr/local/pgsql/dbdata -h 0.0.0.0 -p 5432 -c gtm_host=node1 -c gtm_port=6666
postgres@Node1:~$ postgres --datanode -D /usr/local/pgsql/dbdata -h 0.0.0.0 -p 5433 -c gtm_host=node1 -c gtm_port=6666 &
[3] 35196
postgres@Node1:~$ LOG:  database system was shut down at 2017-08-01 16:29:12 UTC
FATAL:  lock file "/tmp/.s.PGPOOL.6667.lock" already exists
HINT:  Is another postmaster (PID 35113) using socket file "/tmp/.s.PGPOOL.6667"?
LOG:  pool manager process (PID 35199) exited with exit code 1
LOG:  terminating any other active server processes
LOG:  startup process (PID 35198) exited with exit code 2
LOG:  aborting startup due to startup process failure

[3]+  Exit 1                  postgres --datanode -D /usr/local/pgsql/dbdata -h 0.0.0.0 -p 5433 -c gtm_host=node1 -c gtm_port=6666


Now this two node setup 
Node1 will have  Global Transaction monitor + Coordinator
Node2 will be the Data node .
We need to download and install from source on both the servers then 
One Node1 
Init GTM
Create a directory in /usr/local/pgsql called gtm

postgres@Node1:~/data$ initgtm -D  /usr/local/pgsql/gtm -Z gtm
The files belonging to this GTM system will be owned by user "postgres".
This user must also own the server process.


fixing permissions on existing directory /usr/local/pgsql/gtm ... ok
creating configuration files ... ok
creating control file ... ok

Success.
You can now start the GTM server using:

    gtm -D /usr/local/pgsql/gtm
or
    gtm_ctl -Z gtm -D /usr/local/pgsql/gtm -l logfile start
Then start GTM service 
gtm -D /usr/local/pgsql/gtm -h 0.0.0.0 -n node1 -p 6666 &

Now coordinator service 
Create a directory pgdata or coordinator and run below 

postgres@Node1:~$ initdb -D /usr/local/pgsql/pgdata --nodename=node1
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /usr/local/pgsql/pgdata ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /usr/local/pgsql/pgdata/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
creating cluster information ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
You can now start the database server of the Postgres-XL coordinator using:

    postgres --coordinator -D /usr/local/pgsql/pgdata
or
    pg_ctl start -D /usr/local/pgsql/pgdata -Z coordinator -l logfile

 You can now start the database server of the Postgres-XL datanode using:

    postgres --datanode -D /usr/local/pgsql/pgdata
or 
    pg_ctl start -D /usr/local/pgsql/pgdata -Z datanode -l logfile

Now start the service 

 postgres --coordinator -D /usr/local/pgsql/pgdata -h 0.0.0.0 -p 5432 -c gtm_host=node1 -c gtm_port=6666 &
Now on node2  after you compile from source and followed steps till adding content in /etc/profile and adding of postgres user is done
Create a data folder and initialize as above
initdb -D /usr/local/pgsql/data --nodename=node2
Then start service 
postgres --datanode -D /usr/local/pgsql/data -h 0.0.0.0 -p 5432 -c gtm_host=node1 -c gtm_port=6666 &


Now you can run psql and connect

postgres@Node2:~$ psql
psql (PGXL 9.5r1.5, based on PG 9.5.6 (Postgres-XL 9.5r1.5))
Type "help" for help.

postgres=# 

Creating a dummy process in linux with a process name for testing and other purpose

bash -c 'exec -a ProcessName sleep 1000000' &

nagios https proxy monitoring plugin or Bash website monitoring script

#!/bin/bash

if [ $# -eq 0 ]
  then
    echo "No arguments supplied"
fi

if [ $# -eq 1  ]
then

output=`curl  -x 10.11.10.10:3128 -L $1 -m 10 -o /dev/null || echo "Website_down"`

if [ "$output" == "Website_down" ] ; then

echo "Website is down CRITICAL  "
exit 2

else

echo "Website_up $1  OK"
exit 0
fi

fi

Adding client certificate file using chef data bags

Below is just an example 

logstash_cert_secret = Chef::EncryptedDataBagItem.load_secret("/etc/chef/secrets/logstash_client_key")

logstash_certs = Chef::EncryptedDataBagItem.load("certs", "logstash_client", logstash_cert_secret)

directory "/etc/pki/tls/certs/" do

  action :create

  mode "0755"

  owner "root"

  group "root"

end

file "/etc/pki/tls/certs/logstash_client.crt" do

  action :create

  mode "0644"

  owner "root"

  group "root"

  content logstash_certs["crt"].join("")

  end

Search and replace in all files in a current directory

grep -rl 'pc961' ./|xargs sed -i 's/pc961/pc/g'

Ansible define a basic windows hosts

[windows]

10.10.10.120

[windows:vars]

ansible_connection=winrm

ansible_ssh_user=Administrator

ansible_ssh_pass=AdminPa$$

Three basic Git commands

1) git add -A

git add -A is equivalent to  git add .; git add -u.

git add .  ( This will add all files but does not remove files )
git add -u ( This will add only changes but will not add any new files created )

So we use

git add -A ( this will take care of both the commands )

2) git commit -m “Commit changes text ex: I have added new function xyz to the file”

3) git push origin  


example of branch name : develop


Another 3 frequently used commands include :

git pull
git status 
git merge origin/Branchname

Installing configuring Ansible and deploying haproxy playbook

  Basic installation 

sudo yum install ansible (or) sudo pip install ansible

Configuring your hosts file with inventory 

cat /etc/ansible/hosts 

[haproxy]

10.0.184.12   



[all_servers:vars]

ansible_connection=ssh 

ansible_user=root

ansible_ssh_private_key_file=/Users/home/yoursshkey.pem


I have only one server in my inventory where i will be setting up an haproxy 
For easy management we will be writing common variables under [all_servers:vars]


When we run ansible play books hosts will be picked from the above inventory file 

Example : hosts: haproxy  ( this will run on all servers under haproxy )

Now test servers doing a ping ( ping in ansible will connect to the server using ssh )

 ansible haproxy -m ping


10.0.184.12 | SUCCESS => {


    "changed": false, 


    "ping": "pong"


}


Then Download ready made packages from ansible galaxy 
https://galaxy.ansible.com/devops/haproxy/
Also download dependencies that will configure EPEL repositories
https://galaxy.ansible.com/sfromm/epel/
Then create a common playbook for both the roles main.yml




- hosts: haproxy

  roles:

     - { role: devops.epel }





- hosts: haproxy

  roles:

     - role: devops.haproxy

       haproxy_stats:

         name: 'global_monitor'

         ip: "{{ ansible_default_ipv4.address }}"

         port: '38888'

         stats:

           enabled: True

           hide_version: true

           uri: /slb_stats_url

           realm: Welcome\ to\ slb\ monitor

           auth: admin:admin

           refresh: 2s

       haproxy_frontends:

       - name: 'fe-testsite'

         ip: '{{ ansible_default_ipv4.address }}'

         port: '80'

         maxconn: '1000'

         default_backend: 'be-testsite'

       haproxy_backends:

       - name: 'be-testsite'

         description: 'testsite'

         servers:

           - name: 'be-testsite-01'

             ip: '192.168.1.100'

sh-3.2# ansible-playbook main.yml 

PLAY [haproxy] *****************************************************************

TASK [setup] *******************************************************************
ok: [10.0.184.12]

TASK [devops.epel : Installs python dependencies] ******************************
changed: [10.0.184.12] => (item=[u'libselinux-python'])

TASK [devops.epel : Disable SElinux] *******************************************
changed: [10.0.184.12]

TASK [devops.epel : create EPEL yum repository] ********************************
changed: [10.0.184.12]

TASK [devops.epel : import EPEL GPG key] ***************************************
changed: [10.0.184.12]

PLAY [haproxy] *****************************************************************

TASK [setup] *******************************************************************
ok: [10.0.184.12]

TASK [devops.haproxy : Include OS-specific variables.] *************************
ok: [10.0.184.12]

TASK [devops.haproxy : Installs haproxy as well as socat for socket api.] ******
changed: [10.0.184.12] => (item=[u'haproxy', u'socat'])

TASK [devops.haproxy : Ensure HAProxy is started and enabled on boot.] *********
changed: [10.0.184.12]

TASK [devops.haproxy : Ensure chroot directory exists.] ************************
ok: [10.0.184.12]

TASK [devops.haproxy : Create directory for the frontend] **********************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the frontends] *********************************
[DEPRECATION WARNING]: Using bare variables is deprecated. Update your playbooks so that the environment value uses the full variable syntax ('{{haproxy_frontends}}').
This feature will be
removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
changed: [10.0.184.12] => (item={u'ip': u'10.0.184.12', u'maxconn': u'1000', u'default_backend': u'be-testsite', u'port': u'80', u'name': u'fe-testsite'})

TASK [devops.haproxy : Create directory for the backends] **********************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the backends] **********************************
[DEPRECATION WARNING]: Using bare variables is deprecated. Update your playbooks so that the environment value uses the full variable syntax ('{{haproxy_backends}}').
This feature will be
removed in a future release. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
changed: [10.0.184.12] => (item={u'servers': [{u'ip': u'192.168.1.100', u'name': u'be-testsite-01'}], u'description': u'testsite', u'name': u'be-testsite'})

TASK [devops.haproxy : Create directory for the listen sections] ***************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the listen sections] ***************************
skipping: [10.0.184.12] => (item=haproxy_listen)

TASK [devops.haproxy : Create directory for the userlists] *********************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the userlist sections] *************************
skipping: [10.0.184.12] => (item=haproxy_userlists)

TASK [devops.haproxy : Create  the compiled folder] ****************************
changed: [10.0.184.12]

TASK [devops.haproxy : Empty the folder if not already empty] ******************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the global config] *****************************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the default config] ****************************
changed: [10.0.184.12]

TASK [devops.haproxy : Build up the stats config] ******************************
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the frontends configuration file] **************
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the backends configuration file] ***************
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the listen sections configuration file] ********
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the userlists sections configuration file] *****
changed: [10.0.184.12]

TASK [devops.haproxy : Assemble the final configuration file] ******************
changed: [10.0.184.12]

RUNNING HANDLER [devops.haproxy : restart haproxy] *****************************
changed: [10.0.184.12]

PLAY RECAP *********************************************************************
10.0.184.12                : ok=31   changed=27   unreachable=0    failed=0




This is just a test deployment that will forward request to background test server 192.168.1.100

Kibana4 apache configuration with authentication

We need to configure httpd to listen on port 80 which does a reverse proxy to localhost on 5601.

Make sure kibana is listening only on internal port before this settings are done .

        ServerAdmin devops@learnadmin.com

        ServerName kibana.learnadmin.com

        DocumentRoot /var/www/auth

       

            AuthType Basic

            AuthName "Authentication Required"

            AuthUserFile "/var/www/auth/htpasswd"

            Require valid-user

       

        ProxyPass / http://localhost:5601/

        ProxyPassReverse / http://localhost:5601/

Use below command to create a htpasswd file and enable authentication 

htpasswd -c /var/www/auth/htpasswd devops

Monitoring of kibana and logstash services and start them if process not found

Kibana and logstash might run out of memory and application might get closed or get terminated. In that case we will be running the below script to monitor services and start them if not running .

There are other parameters that need to be verified before going for this option like java heap size etc.

#!/bin/bash

LOGSTASH=logstash

KIBANA=kibana

var1=`ps -ef|grep -i $LOGSTASH|sed -e '/grep/d'`

if [ -z "$var1" ]; then

nohup /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf &

echo "Process logstash not found" | mail -s "Process logstash not found creating a new one" devops@learnadmin.com

else

echo "process logstash  found."

fi

var2=`ps -ef|grep -i $KIBANA|sed -e '/grep/d'`

if [ -z "$var2" ]; then

nohup /opt/kibana/bin/kibana status &

echo "Process kibana  not found" | mail -s "Process kibana not found creating a new one" devops@learnadmin.com

else

echo "process kibana found."

fi

Chef provision a esxi virtual machine or a virtual server

We need to install esxi knife plugin on our workstation


gem install knife-esx

Add below params in knife.rb

knife[:esx_host] = "esx"
knife[:esx_username] = "root"
knife[:esx_password] = "password of your esxi server"

Need to create a ubuntu template ( or custom OS template as per requirement )
Below example shows how to provision a ubuntu template .

ssh root@esxi

mkdir -p /vmfs/volumes/datastore1/esx-gem/templates

vmkfstools -i /vmfs/volumes/datastore1/ubuntu-12.10-x64_template/*.vmdk –diskformat thin /vmfs/volumes/datastore1/esx-gem/templates/ubuntu-12.10-x64_template.vmdk

esx template list ( should display the template we installed )


knife esx vm create --vm-name server1 --use-template ubuntu-12.10-x64_template --verbose true --distro ubuntu12.04-19-gems --vm-memory 2048 -x provision -i ~/.ssh/id_rsa

How to use chef encrypted databags

Note: Please comment the below line in knife.rb file else it will automatically decrypt values
and show when we use "knife data bag show" command
I initially thought knife was not encrypting my values and had to debug ..which took lot of time.

[root@ec2-test .chef]# cat knife.rb | grep secret

#knife[:secret_file] ='/root/encrypted_data_bag_secret'

knife data bag create  --secret-file /root/.chef/encrypted_data_bag_secret testbag password

{

  "id": "password",

  "password": "this is test password key"

}

[root@ec2-test .chef]# knife data bag show testbag password

id:       password

password:

  cipher:         aes-256-cbc

  encrypted_data: KMHzeFQwfm0wWeHFymxfJsMo425CP+wlwoZ6xN7waVlgUNOmRrr/+jOtDLIN

  s7Xl

  

  iv:             TYPYnSqYTcmU8ZWE2sIt4A==

  

  version:        1

  

Once encrypted if you try to edit the encrypted databag it shows like this 

{

  "name": "data_bag_item_testbag_password",

  "json_class": "Chef::DataBagItem",

  "chef_type": "data_bag_item",

  "data_bag": "testbag",

  "raw_data": {

    "id": "password",

    "password": {

      "encrypted_data": "KMHzeFQwfm0wWeHFymxfJsMo425CP+wlwoZ6xN7waVlgUNOmRrr/+jOtDLIN\ns7Xl\n",

      "iv": "TYPYnSqYTcmU8ZWE2sIt4A==\n",

      "version": 1,

      "cipher": "aes-256-cbc"

    }

  }

}

To show the decrypted values we can use 

[root@ec2-test .chef]# knife data bag show  testbag password --secret-file /root/.chef/encrypted_data_bag_secret

id:       password

password: this is test password key


---------------------------
Some important commands related to data bags

Generate random secret key

openssl rand -base64 512 >encrypted_data_bag_secret

Generate random password

date | md5 in mac 

date | md5sum in any linux machine 

Setting a Secure Flag to cookie in httpd – Apache

First check if mod_headers is loaded. Else enable that module


LoadModule headers_module modules/mod_headers.so

Once the module is loaded .

Header edit Set-Cookie ^(.*)$ $1;Secure


Add above line to your virtual host in httpd.conf/ssl.conf as per your config

What this does is it adds a secure flag to your cookie. This flag is set by application server when sending
a new cookie to client/user when sending http response.This also mean that server will not send cookie over http. It sends only via https .This prevents unauthorized access to cookie data

This can be done via code in java or can be done @ apache/httpd config level.

Here is more info on code level changes click here..!!

Get SSL certificate expire date / Openssl get certificate expire date

openssl x509 -in /home/certificates/certname.crt -noout -enddate

rssh chroot jailroot sftp creation - By Vinodh tiruttani

Steps for chroot jail folder creation:

The chroot command changes its current and root directories to the provided directory and then run command, if supplied, or an interactive copy of the user's login shell. Please note that not every application can be chrooted.

Creating User
[root@learnadmin ~]# /usr/sbin/useradd testrssh
[root@learnadmin ~]# su testrssh
[testrssh@learnadmin root]$ cd /home/testrssh/
[testrssh@learnadmin ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/testrssh/.ssh/id_dsa):
Created directory '/home/testrssh/.ssh'.
Enter passphrase (empty for no passphrase): < Type passphrase >
Enter same passphrase again: <  Re-type passphrase >
Your identification has been saved in /home/testrssh/.ssh/id_dsa.
Your public key has been saved in /home/testrssh/.ssh/id_dsa.pub.
The key fingerprint is:
92:c9:fe:9f:47:29:ba:15:29:9c:51:84:0f:c6:d8:0b testrssh@learnadmin.com
[testrssh@learnadmin ~]$
[testrssh@learnadmin ~]$ cd .ssh/
[testrssh@learnadmin .ssh]$ pwd
/home/testrssh/.ssh
[testrssh@learnadmin .ssh]$ cp id_dsa.pub authorized_keys2
[testrssh@learnadmin .ssh]$ ls -l
total 24
-rw-r--r-- 1 testrssh testrssh 618 Jun 12 18:12 authorized_keys2
-rw------- 1 testrssh testrssh 736 Jun 12 18:11 id_dsa ( 600 permission)
-rw-r--r-- 1 testrssh testrssh 618 Jun 12 18:11 id_dsa.pub
[testrssh@learnadmin .ssh]$
[testrssh@learnadmin .ssh]$ exit
[root@learnadmin ~]# /usr/sbin/usermod -s /usr/bin/rssh testrssh
Download from net rssh-2.3.3.tar.gz from http://www.pizzashack.org/rssh/downloads.shtml
[root@learnadmin ~]# tar xvf rssh-2.3.3.tar.gz
[root@learnadmin ~]# cd rssh-2.3.3
[root@learnadmin rssh-2.3.3]# ./configure
[root@learnadmin rssh-2.3.3]# make
[root@learnadmin rssh-2.3.3]# make install
[root@learnadmin rssh-2.3.3]# cd ..
 Download from net rssh-2.3.3-1.x86_64.rpm
 [root@learnadmin ~]# rpm -ivh rssh-2.3.3-1.x86_64.rpm
Preparing...                ########################################### [100%]
   1:rssh                   ########################################### [100%]
[root@learnadmin ~]# pwd
/root

·         Converting rssh chroot Jail folder – Execute the below script from the below location

 [root@learnadmin ~]# cp /usr/share/doc/rssh-2.3.3/mkchroot.sh .
[root@learnadmin ~]# chmod 775 mkchroot.sh
 [root@learnadmin ~]# ./mkchroot.sh /home/testrssh/
NOT changing owner of root jail.
NOT changing perms of root jail.
setting up /home/testrssh//usr/bin
setting up /home/testrssh//usr/libexec/openssh
setting up /home/testrssh//usr/libexec
Copying libraries for /usr/bin/scp.
        /lib64/libcrypto.so.6
        /lib64/libutil.so.1
        /usr/lib64/libz.so.1
        /lib64/libnsl.so.1
        /lib64/libcrypt.so.1
        /lib64/libresolv.so.2
        /usr/lib64/libgssapi_krb5.so.2
        /usr/lib64/libkrb5.so.3
        /usr/lib64/libk5crypto.so.3
        /lib64/libcom_err.so.2
        /usr/lib64/libnss3.so
        /lib64/libc.so.6
        /lib64/libdl.so.2
        /usr/lib64/libkrb5support.so.0
        /lib64/libkeyutils.so.1
        /usr/lib64/libnssutil3.so
        /usr/lib64/libplc4.so
        /usr/lib64/libplds4.so
        /usr/lib64/libnspr4.so
        /lib64/libpthread.so.0
        /lib64/libselinux.so.1
        /lib64/libsepol.so.1
Copying libraries for /usr/libexec/openssh/sftp-server.
        /lib64/libcrypto.so.6
        /lib64/libutil.so.1
        /usr/lib64/libz.so.1
        /lib64/libnsl.so.1
        /lib64/libcrypt.so.1
        /lib64/libresolv.so.2
        /usr/lib64/libgssapi_krb5.so.2
        /usr/lib64/libkrb5.so.3
        /usr/lib64/libk5crypto.so.3
        /lib64/libcom_err.so.2
        /usr/lib64/libnss3.so
        /lib64/libc.so.6
        /lib64/libdl.so.2
        /usr/lib64/libkrb5support.so.0
        /lib64/libkeyutils.so.1
        /usr/lib64/libnssutil3.so
        /usr/lib64/libplc4.so
        /usr/lib64/libplds4.so
        /usr/lib64/libnspr4.so
        /lib64/libpthread.so.0
        /lib64/libselinux.so.1
        /lib64/libsepol.so.1
Copying libraries for /usr/bin/rssh.
        /lib64/libc.so.6
Copying libraries for /usr/libexec/rssh_chroot_helper.
        /lib64/libc.so.6
copying name service resolution libraries...
tar: Removing leading `/' from member names
        lib/libnss_files-2.5.so
tar: /lib/libnss1_files*: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors
        lib/libnss_files.so.2
Setting up /etc in the chroot jail
cp: omitting directory `/etc/ld.so.conf.d'
Chroot jail configuration completed.
NOTE: if you are not using the passwd file for authentication,
you may need to copy some of the /lib/libnss_* files into the jail.

 NOTE: you must MANUALLY edit your syslog rc script to start syslogd
with appropriate options to log to /home/testrssh//dev/log.  In most cases,
you will need to start syslog as:
    /sbin/syslogd -a /home/testrssh//dev/log
 NOTE: we make no guarantee that ANY of this will work for you... if it
doesn't, you're on your own.  Sorry!
 Note: while executing above command some files are not copied, we need to copy manually.
[root@learnadmin ~]#
[root@learnadmin ~]# cd /home/testrssh/
[root@learnadmin testrssh]# cd dev/
[root@learnadmin dev]# mknod -m 666 zero c 1 5
[root@learnadmin dev]# mknod -m 666 null c 1 3
[root@learnadmin dev]# cd ../lib
[root@learnadmin lib]# cp /lib/ld-linux.so.2 .
[root@learnadmin lib]# cp /lib/libc.so.6 .
[root@learnadmin lib]# cd ../lib64/
[root@learnadmin lib64]# cp /lib64/ld-linux-x86-64.so.2 .
[root@learnadmin lib64]# cp /lib64/libnss_* .
[root@learnadmin lib64]# cd
[root@learnadmin]# vi /etc/rssh.conf ( add the user and uncomment 2 lines)
allowscp
allowsftp
user=testrssh:011:00011:"/home/testrssh"
:wq!
[root@learnadmin ~]#
Done rssh chroot jail folder is created
-------------------------------------------------
Testing rssh login  from remote system
Login to server  from remote server
 Copy id_dsa key from Server ( user: testrssh )
[root@clientmachine test-rssh-keys]# vi id_dsa_testrssh

-----BEGIN DSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,42ED97DC52451DE9

UAh2bmigMZ5DN0KVadfR9bRrFL7l0mUMF7iCjgbqHA3wgNfPttytDmt3jJR5Mmb+
uxl0s1D1JwDvNstUgTL9fDC+TnrQXj+0YVQ7bvHPLdcyTEUm3MXNAbqv3PUoFAVa
qfLPtybOHCADOA6wH+vcJ4Mt2NM/M+fbMbcmBsrMw3fYzB2na5Zs2OoVxKLbVPEo
mKP3Eu5Eo1nC6wApQkBzO7ZJDq3x7YjxPPQJZ82j/vVVuZsGj0IFBRm1Esj8gABp
MlQfTzK0wnbbCJ2L3CqRPS7gyyKCpdQtuNYliDh8E+uXAsCjIx+rksfRJkTmKTDj
Huob0no/69che/tnBzhzQV3B6C1BrJ4CPElRVOoq2niQIy4ruku5wK9ONm3J50It
wc3uADpOgBUStavVpuUJTU4Oe8rVraUhEyBB0iOpYySGrAAfEtabwCwB0OvK+bht
1mLleTDGadms07+psexV4L66bmDY6ypu7dtcGWuFXQP5UQdOHqS7ehdEtCpuboer
BVkcrdivDHoc88cScuDOytPsFyMiHJ5fkrMLtKWQBTF3I7SiDqVBq/lKFcubZK3Q
K/PpJyasnxlcaLJLFkFibw==
-----END DSA PRIVATE KEY-----
 :wq!
 [root@ clientmachine test-rssh-keys]# chmod 600 id_dsa_testrssh
 [root@ clientmachine test-rssh-keys]# sftp -o IdentityFile=id_dsa_testrssh testrssh@sftp server
Connecting to 10.x.x.x...
Enter passphrase for key 'id_dsa_testrssh':
sftp> ls
dev    etc    lib    lib64  usr
sftp> cd /root
Couldn't canonicalise: No such file or directory
sftp> cd /tmp
Couldn't canonicalise: No such file or directory
sftp>
 rssh chroot jail folder is working.