Change bash colors

Blue(34) as directory color is difficult to read and strain your eyes.
So, i googled and find using environment variable LS_COLORS one can modify it.

Here is one such google search result:

----------

These three lines in my .bashrc file allow me to configure the color system
by extension.

alias ls="ls --color=auto"
LS_COLORS="di=31;1:ln=36;1:ex=31;1:*~=31;1:*.html=31;1:*.shtml=37;1"
export LS_COLORS

Understanding linux password

Ubuntu / linux stores password in /etc/shadow file not in encrypted form but by hashing it.

Passwords on a linux system are not encrypted, they are hashed which is a huge difference.

It is not possible to reverse a hash function by definition.

Run following commands to get familiar with password security in Ubunutu

 cd /etc

sudo cat passwd

this command will print content of passwd file

bhushan:  x  :   1000 : 1000 : bhushan,,,:/home/bhushan:/bin/bash

1                 :2:        3:        4:                  5                      : 6:                        7:

1. Username: It is used when user logs in. It should be between 1 and 32 characters in length.
2. Password: An x character indicates that encrypted password is stored in /etc/shadow file.
3. User ID (UID): Each user must be assigned a user ID (UID). UID 0 (zero) is reserved for root and UIDs 1-99 are reserved for other predefined accounts. Further UID 100-999 are reserved by system for administrative and system accounts/groups.
4. Group ID (GID): The primary group ID (stored in /etc/group file)
5. User ID Info: The comment field. It allow you to add extra information about the users such as user’s full name, phone number etc. This field use by finger command.
6. Home directory: The absolute path to the directory the user will be in when they log in. If this directory does not exists then users directory becomes /
7. Command/shell: The absolute path of a command or shell (/bin/bash). Typically, this is a shell. Please note that it does not have to be a shell.

This is user information

I want to know password of Bhushan

lets try this command

bhushan@bhushan-desktop:/etc$ cat shadow|grep “bhushan”

cat: shadow: Permission denied

what happened….i can’t run this command without root permission.

bhushan@bhushan-desktop:/etc$ sudo cat shadow|grep “bhushan”

bhushan:

$6$M0jBWc8n$iPADKsrAk67ONNeyfRj56GtAtnkwJ2uulTQEXtc8McCD/aZSB1BgJXgrqBBdyjkeUsP7yIQIjCQtDu.4I.kf91:15632:0:99999:7:::

/etc/shadow file fields

1. User name : It is your login name
2. Password: It your encrypted password. The password should be minimum 6-8 characters long including special characters/digits
3. Last password change (lastchanged): Days since Jan 1, 1970 that password was last changed
4. Minimum: The minimum number of days required between password changes i.e. the number of days left before the user is allowed to change his/her password
5. Maximum: The maximum number of days the password is valid (after that user is forced to change his/her password)
6. Warn : The number of days before password is to expire that user is warned that his/her password must be changed
7. Inactive : The number of days after password expires that account is disabled
8. Expire : days since Jan 1, 1970 that account is disabled i.e. an absolute date specifying when the login may no longer be used

The password hash today comes in three parts, separated by dollar signs ($):

$6$M0jBWc8n$iPADKsrAk67ONNeyfRj56GtAtnkwJ2uulTQEXtc8McCD/aZSB1BgJXgrqBBdyjkeUsP7yIQIjCQtDu.4I.kf91

Here:  $ ID $ SALT $ HASH

Passing arguments to Dockerfile (docker run ...)

CMD and ENTRYPOINT better together

In case you don't know ENTRYPOINT makes your dockerized application behave like a binary. You can pass arguments to the ENTRYPOINT during docker run and not worry about it being overwritten ( unlike CMD ). ENTRYPOINT is even better when used with CMD. Let's checkout my Rethinkdb Dockerfile and see how to use this.

# Dockerfile for Rethinkdb 
# http://www.rethinkdb.com/

FROM ubuntu

MAINTAINER Michael Crosby <michael@crosbymichael.com>

RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
RUN apt-get update
RUN apt-get upgrade -y

RUN apt-get install -y python-software-properties
RUN add-apt-repository ppa:rethinkdb/ppa
RUN apt-get update
RUN apt-get install -y rethinkdb

# Rethinkdb process
EXPOSE 28015
# Rethinkdb admin console
EXPOSE 8080

# Create the /rethinkdb_data dir structure
RUN /usr/bin/rethinkdb create

ENTRYPOINT ["/usr/bin/rethinkdb"]

CMD ["--help"]

This is everything that is required to get Rethinkdb dockerized. We have my standard 5 lines at the top to make sure the base image is updated, ports exposed, etc... With the ENTRYPOINT set, we know that whenever this image is run, all arguments passed during docker run will be arguments to the ENTRYPOINT ( /usr/bin/rethinkdb ).

References:

1. http://crosbymichael.com/dockerfile-best-practices.html

aws:cloudformation templates

Templates

1. auto_scaling_with_instance_profile.template.https://s3.amazonaws.com/cloudformation-templates-us-east-1/auto_scaling_with_instance_profile.template
2. https://securosis.com/blog/using-amazon-iam-roles-to-distribute-security-credentials-for-chef

How to install MySQL 5.6 on Ubuntu 12.04 (Precise)

[EN] How to install MySQL 5.6 on Ubuntu 12.04 (Precise)

This how to will cover the upgrade process of MySQL 5.6 on Ubuntu 12.04 precise. This may also work on Ubuntu 12.10 (quantal) but I haven’t tested it at moment.

DISCLAIMER: I am not responsible for any DATA LOSS you may face when using this how to! Use it at your own risk, create a backup and use your brain! With manual installation you will definitely LOOSE the support of your distribution vendor in case of security updates, be aware of this fact! Thank you!

This how to was created on an up to date (at the day of writing) Ubuntu 12.04.2 LTS machine (use lsb_relase -a to check). MySQL was already running on this machine.

Backing up chef-server-v11

1) Backup Data bags, roles, and nodes

Use this Knife Exec script to perform a backup of an existing Chef Server:

cd ~/path/to/chef/repository   

curl -O https://raw.github.com/jtimberman/knife-scripts/master/chef_server_backup.rb

Run the knife exec script

knife exec chef_server_backup.rb

Quick Docker setup on vagrant machine.

Docker

Docker is a Linux container, written in Go and based on lxc (self-described as “chroot on steroids”) and AUFS. Instead of providing a full VM, like you get with Vagrant, Docker provides you lightweight containers, that share the same kernel and allow to safely execute independent processes.
Docker is attractive for many reasons:

• Lightweight; images are much lighter than full VMs, and spinning off a new instance is lightning fast (in the range of seconds instead of minutes).
• Version control of the images, which makes it much more convenient to handle builds.
• Lots of images (again), just have a look at the docker public index of images.

Runit: process monitoring made easy

Install:

 apt-get install -y runit

# Setup the application's runit directory

/etc/sv/application:
file.directory:
- mode: 0755
- user: someuser
- group: someuser
- recurse:
- user
- group
- mode
 

AWS:Cloudformation : Authentication to AWS::CloudFormation::Init(AWS::CloudFormation::Authentication)

AWS::CloudFormation::Authentication

Use the AWS::CloudFormation::Authentication type to specify authentication credentials for files or sources that you specify with the AWS::CloudFormation::Init type.

{
   "Type" : "AWS::CloudFormation::Authentication" {
      "String" : {
         "accessKeyId" : String,
         "buckets" : [ String, ... ],
         "password" : String,
         "secretKey" : String,
         "type" : String,
         "uris" : [ String, ... ],
         "username" : String,
         "roleName" : String
      },
      ...
   }
}     

aws::cloudformation(AWS::IAM::AccessKey):::How to get access keys (access-key, secret-key)

Make use of  "AWS::IAM::AccessKey" resource type to get credentials and use throughout anywhere in the cloudformation script.
Depends on
  AWS::IAM::User   : for its userName value.

Syntax

{
   "Type": "AWS::IAM::AccessKey",
   "Properties": {
      "Serial": Integer,
      "Status": String,
      "UserName": String
   }
}    

AWS:Cloudformation:::Wait till instance is launched

Resources Used:

• AWS::CloudFormation::WaitConditionHandle
•   has no properties.
• When you reference the WaitConditionHandle resource by using the Ref function, AWS CloudFormation returns a presigned URL. You pass this URL to applications or scripts that are running on your Amazon EC2 instances to send signals to that URL. An associated AWS::CloudFormation::WaitCondition resource checks the URL for the required number of success signals or for a failure signal. 

• AWS::CloudFormation::WaitCondition
• AWS::EC2::Instance  OR   AWS::AutoScaling::LaunchConfiguration

Cloudformation

Reference:

1. https://ivan-site.com/2013/02/auto-scaling-on-amazon-ec2-with-opscode-chef/

Trouobleshooting chef-server and chef

Trouobleshooting chef-server and chef

1. had an error: Net::HTTPServerException: 403 "Forbidden"
1. Solutions
1. long running recipes cause Forbidden errors. So, increase timeout value
1. create a file /etc/chef-server/chef-server.rb that contains "erchef['s3_url_ttl'] = 3600"
    chef-server-ctl reconfigure
2. On "Private Chef" the variable to change turns out to be slightly different:
         opscode_erchef['s3_url_ttl'] = 3600
2. http://stackoverflow.com/questions/11870336/chef-client-connection-errors-after-executing-knife-bootstrap

horizontal and vertical scaling

Vertical scaling is adding more power to the machine that runs our website or application. It is often the easiest option as code only written to run on a single machine - which is usually most code - can be moved easily to a bigger machine, especially if you already have automated server setup with Chef.

At enterprise level you should probably have a server on standby for redundancy in case of hardware or software failure anyway so you may as well use it to handle traffic.

Horizontal scaling is adding more servers and sharing the load between them. It can also have the advantage of adding redundancy. If we have two web servers handling traffic and one breaks our websites will remain online. If we can separate various parts of our website or application into different services we can have even more redundancy.

Chef-client: workstation and Node perspective

Chef-Client usage from the perspective of Administrator( chef workstation)

Run lists specify what recipes or roles the

node should run, along with the order they

should be run in

• Run lists are represented by an array

• Recipes are specified by “recipe[name]”

• Roles are specified by “role[name]”

Virtualbox : launching virtual machines with different IP's with network connectivity

Virtualbox : launching virtual machines with different IP's with network connectivity

1. Create instances as many you like but for all do following changes in the network settings
1. Enable Two Adapters : 
1. Set first one to NAT : To allow internet from host (eth0)
2. Second to Host Only:  to allow different ip for every vm(eth1)
2. Test instance
1. fire 'ifconfig' command not down ip corresponding to eth1. All vm's will have different eth1 ip. Take note of them and use them to communicate with each other.
2. Lauch them and verify that you're able to ping these vm's through 'eth1' IP

Understand Chef-Server Components (Erchef, bookshelf, webui, nginx-load balanceer, rabbitMQ, postregSQL)

Chef-server:

1. The server acts as a hub for configuration data
2. The server stores cookbooks, the policies that are applied to nodes, and metadata that describes each registered node that is being managed by the chef-client
3. front-end is written in erlang and hence the name Erchef

Configuring mail on linus server through ssmtp

Setting SMTP on linux server

Follow through the steps to setup simple smtp on linxu server

Pre-requisite:

1. smtp server credentials
2. ssmtp installed on server

Getting started with Chef-Server and Chef-client

Steps Involved:

1. Chef-server installation and key generation
2. Launching chef-server webui.
3. Transferring keys to admin workstation (admin.pem and chef-validator.pem)
4. Adding new node 
5. Deploying cookbook on newly added node (bootstrap)

Authentication using ssh public (pub) and private keys(pem)

To avoid the need of supplying username and password everytime from trusted machine to login into your server we can generate pub/private keys to drop this authentication step from foreground

Steps involved to generate one such key pair are:

1. Generate key pair (.pub and .pem)
2. Pass .pub file to your server to store it in its authorized_keys file
3. Keep .pem(private key) at yourself whenever to be used to login to server.

Zookeeper Setup Guide -Standalore/Quorum

#Zookeeper Quick Setup Guide

##Zookeeper Download
Visit http://www.apache.org/dyn/closer.cgi/zookeeper/ and dowload the stable zookeeper tar file.
I've downloaded this one: http://www.motorlogy.com/apache/zookeeper/zookeeper-3.4.5/zookeeper-3.4.5.tar.gz

Install docker on ubuntu 64 bit

Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.

Installation of Docker on ubuntu 64 bit machine

Installing Apache Mesos

Apache Mesos: Cluster Management tool , universally used to install analytical frameworks like Hadoop, spark or storm on clusters.
For high availability of Mesos(master) it uses zookeeper.
To install more than analytical framework, for eg. to install ruby, or rails, or pythong on servers it can be used. Twitter Aurora is developer to provide this feature over mesos

To make running and monitoring cron jobs easy, one can install install over mesos cluster (require zookeeper) and can submit cron jobs via this chronos gui
One create its own framework over mesos cluster as well.