AWS:Cloudformation : Authentication to AWS::CloudFormation::Init(AWS::CloudFormation::Authentication)

AWS::CloudFormation::Authentication

Use the AWS::CloudFormation::Authentication type to specify authentication credentials for files or sources that you specify with the AWS::CloudFormation::Init type.

{
   "Type" : "AWS::CloudFormation::Authentication" {
      "String" : {
         "accessKeyId" : String,
         "buckets" : [ String, ... ],
         "password" : String,
         "secretKey" : String,
         "type" : String,
         "uris" : [ String, ... ],
         "username" : String,
         "roleName" : String
      },
      ...
   }
}     

aws::cloudformation(AWS::IAM::AccessKey):::How to get access keys (access-key, secret-key)

Make use of  "AWS::IAM::AccessKey" resource type to get credentials and use throughout anywhere in the cloudformation script.
Depends on
  AWS::IAM::User   : for its userName value.

Syntax

{
   "Type": "AWS::IAM::AccessKey",
   "Properties": {
      "Serial": Integer,
      "Status": String,
      "UserName": String
   }
}    

AWS:Cloudformation:::Wait till instance is launched

Resources Used:

• AWS::CloudFormation::WaitConditionHandle
•   has no properties.
• When you reference the WaitConditionHandle resource by using the Ref function, AWS CloudFormation returns a presigned URL. You pass this URL to applications or scripts that are running on your Amazon EC2 instances to send signals to that URL. An associated AWS::CloudFormation::WaitCondition resource checks the URL for the required number of success signals or for a failure signal. 

• AWS::CloudFormation::WaitCondition
• AWS::EC2::Instance  OR   AWS::AutoScaling::LaunchConfiguration

Cloudformation

Reference:

1. https://ivan-site.com/2013/02/auto-scaling-on-amazon-ec2-with-opscode-chef/

Trouobleshooting chef-server and chef

Trouobleshooting chef-server and chef

1. had an error: Net::HTTPServerException: 403 "Forbidden"
1. Solutions
1. long running recipes cause Forbidden errors. So, increase timeout value
1. create a file /etc/chef-server/chef-server.rb that contains "erchef['s3_url_ttl'] = 3600"
    chef-server-ctl reconfigure
2. On "Private Chef" the variable to change turns out to be slightly different:
         opscode_erchef['s3_url_ttl'] = 3600
2. http://stackoverflow.com/questions/11870336/chef-client-connection-errors-after-executing-knife-bootstrap

horizontal and vertical scaling

Vertical scaling is adding more power to the machine that runs our website or application. It is often the easiest option as code only written to run on a single machine - which is usually most code - can be moved easily to a bigger machine, especially if you already have automated server setup with Chef.

At enterprise level you should probably have a server on standby for redundancy in case of hardware or software failure anyway so you may as well use it to handle traffic.

Horizontal scaling is adding more servers and sharing the load between them. It can also have the advantage of adding redundancy. If we have two web servers handling traffic and one breaks our websites will remain online. If we can separate various parts of our website or application into different services we can have even more redundancy.

Chef-client: workstation and Node perspective

Chef-Client usage from the perspective of Administrator( chef workstation)

Run lists specify what recipes or roles the

node should run, along with the order they

should be run in

• Run lists are represented by an array

• Recipes are specified by “recipe[name]”

• Roles are specified by “role[name]”

Virtualbox : launching virtual machines with different IP's with network connectivity

Virtualbox : launching virtual machines with different IP's with network connectivity

1. Create instances as many you like but for all do following changes in the network settings
1. Enable Two Adapters : 
1. Set first one to NAT : To allow internet from host (eth0)
2. Second to Host Only:  to allow different ip for every vm(eth1)
2. Test instance
1. fire 'ifconfig' command not down ip corresponding to eth1. All vm's will have different eth1 ip. Take note of them and use them to communicate with each other.
2. Lauch them and verify that you're able to ping these vm's through 'eth1' IP

Understand Chef-Server Components (Erchef, bookshelf, webui, nginx-load balanceer, rabbitMQ, postregSQL)

Chef-server:

1. The server acts as a hub for configuration data
2. The server stores cookbooks, the policies that are applied to nodes, and metadata that describes each registered node that is being managed by the chef-client
3. front-end is written in erlang and hence the name Erchef

Configuring mail on linus server through ssmtp

Setting SMTP on linux server

Follow through the steps to setup simple smtp on linxu server

Pre-requisite:

1. smtp server credentials
2. ssmtp installed on server

Getting started with Chef-Server and Chef-client

Steps Involved:

1. Chef-server installation and key generation
2. Launching chef-server webui.
3. Transferring keys to admin workstation (admin.pem and chef-validator.pem)
4. Adding new node 
5. Deploying cookbook on newly added node (bootstrap)

Authentication using ssh public (pub) and private keys(pem)

To avoid the need of supplying username and password everytime from trusted machine to login into your server we can generate pub/private keys to drop this authentication step from foreground

Steps involved to generate one such key pair are:

1. Generate key pair (.pub and .pem)
2. Pass .pub file to your server to store it in its authorized_keys file
3. Keep .pem(private key) at yourself whenever to be used to login to server.

Zookeeper Setup Guide -Standalore/Quorum

#Zookeeper Quick Setup Guide

##Zookeeper Download
Visit http://www.apache.org/dyn/closer.cgi/zookeeper/ and dowload the stable zookeeper tar file.
I've downloaded this one: http://www.motorlogy.com/apache/zookeeper/zookeeper-3.4.5/zookeeper-3.4.5.tar.gz

Install docker on ubuntu 64 bit

Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.

Installation of Docker on ubuntu 64 bit machine

Installing Apache Mesos

Apache Mesos: Cluster Management tool , universally used to install analytical frameworks like Hadoop, spark or storm on clusters.
For high availability of Mesos(master) it uses zookeeper.
To install more than analytical framework, for eg. to install ruby, or rails, or pythong on servers it can be used. Twitter Aurora is developer to provide this feature over mesos

To make running and monitoring cron jobs easy, one can install install over mesos cluster (require zookeeper) and can submit cron jobs via this chronos gui
One create its own framework over mesos cluster as well.