Tuesday, November 3, 2015

AWS - Bootstrap Windows EC2 instance with Chef-Solo


If you have an environment like we do you, fairly small and few service layers, it may not make sense to provision a Chef Server.  Luckily, we can still get the benefits of using Chef to configure our servers by using the included Chef-Solo.  Chef-Solo will run entirely locally on the instance, therefore we must include all required dependencies on the server, ie. cookbooks, run-lists, environments, etc.

The following process I will demonstrate how to launch an AWS EC2 instance and have Chef-Solo configure the instance. I will not go into details about how Chef works with recipes and cookbooks.

Here are the steps we will need to follow to start this process.
  1. Create S3 bucket to store Chef files, this includes your cookbooks
  2. Create IAM Role and define access to the S3 bucket
  3. Configure UserData to run PowerShell on initial launch
    • The UserData will do the following
      • Download the Chef-Client from an S3 Bucket
      • Download the Chef Cookbooks, Recipes, etc
      • Install the Chef-Client
      • Run Chef-solo
Let's get started!

1. Create an S3 bucket and upload the Chef MSI, cookbooks, run scripts, etc

Within the AWS console, create a new S3 bucket to store the Chef installer as well as all the required Chef cookbooks, environment files, run scripts, etc.  For this example we will use,  'examplebucket' for the S3 bucket name, You will need to use your own unique bucket name.

2. Create an IAM role with a policy to allow Read only access to the S3 bucket

By creating an IAM role and assigning the role to the instance we can eliminate the need to use an IAM user account with access keys.  IAM roles utilize temporary credentials to grant access to AWS resources.

Within the AWS console create a new IAM role and Select Role Type: AWS Service Roles > Amazon EC2


Follow the prompts clicking through until the Role is finally created. With the role created, we must now create a new Inline policy which will grant access to the S3 bucket.

Select the newly created Role and expand the 'Inline Policies' to create a new policy:


Choose the option to create a Custom Policy:



For the policy, we will grant ListBucket and GetObject restricted to the S3 bucket we created earlier.

Here is the policy, you must modify the bucket name :

3. Launch a new instance and configure UserData

When launching a new EC2 instance assign the previously created role to the instance.  Also, expand the Advanced details to provide 'UserData'.  This UserData allows you to run scripts when the instance is first launched.  Our instance will need to download and install Chef as well as execute Chef-Solo.

The userdata will utilize PowerShell to execute downloading and installing Chef.  Here is the actual userdata to include in the instance launch.  This assumes the instance AMI you are launching with has the AWS CLI available (the AWS provided AMI's for Windows include this already)

I've provided comments to the code for clarifications.



Troubleshooting

The UserData is only run on instance launch and not on restarts.  This is initiated by the Ec2ConfigService which records log information at:
C:\Program Files\Amazon\Ec2ConfigService\Logs\Ec2ConfigLog.txt

Chef-solo will record log information as defined by the -L option, in this case we are creating the logs at:   C:\chef\log.log

I hope this helps you progress to using Chef and moving more towards Infrastructure as Code in your environment!

2 comments:

Unknown said...

Hi Ryan,

I think I found it. Before creating the AM from the initial instance, I need to enable 'User Data' plugin or shutdown with sysprep.

I'll do the experiment and post the update here.

Thanks -Rafiq

Unknown said...

Thanks for sharing this informative information. You may also refer...Amazon Web Services (AWS) BGP
This video demonstrates how to configure the Amazon Web Services BGP to set up a VPN between a Check Point Security Gateway and Amazon VPC
http://www.s4techno.com/blog/2015/12/24/amazon-web-services-aws-bgp/