AWS Concepts Learning-II

Hypervisor:
•The interaction between the host server/machine and the virtual machine is facilitated by software called Hypervisor (VMM).
•Hypervisor distributes the hardware resources such as CPU, memory to the virtual environments.

What is a Public cloud?
Public cloud is a cloud environment owned by a third-party cloud provider that enables availability of resources such as applications, virtual machines, etc., to the general public over the internet.
Few of such providers are Amazon Web Services, Microsoft Azure.

Private Cloud:Private cloud is also recognized as an internal cloud or corporate cloud. Private cloud is most ideal for the organizations which require direct control over the environments to meet security and business compliance requirements.

Amazon EC2 and Rackspace Cloud are examples of IaaS

Storage Classes: 1) Standard 2) Standard-IA 3) One Zone-IA 4) Glacier
Unlike S3 Standard-IA, this One Zone-IA stores data in a single availability zone. •This is 20% cheaper than the S3 Standard-IA.
•Best suited for secondary backup copy storages or for data that is cross region replicated.
•Data stored in this will be lost in the event of availability zone destruction
Glacier: Amazon Glacier is a data archiving service which is highly durable, extremely low cost, and secure, for varying retrieval needs.
•Amazon Glacier provides three options for access to archives, from a few minutes to several hours.
•Data is resilient in the event of one entire Availability Zone destruction.
You can set the Storage class of an object during uploading it to the bucket and as well as later after uploading. To storage classes can be set using the management console,AWS CLI,and SDK's also using the lifecycle policies.


AWS CLI commands for basic operations

Let's learn some basic CLI commands that are used to perform some important operations:
1.To create a new bucket:
aws s3 mb s3://bucket-name

2.To delete a bucket:
aws s3 rb s3://bucket-name
or
aws s3 rb s3://bucket-name --force

3.To copy files from a device:
aws s3 cp filename s3://bucket-name.

4.To copy files recursively:
aws s3 cp . s3://bucket-name --recursive

To enable versioning:
aws s3api put-bucket-versioning --bucket my-bucket --versioning-configuration Status=Enabled

To disable versioning:
aws s3api put-bucket-versioning --bucket my-bucket --versioning-configuration Status=Suspended

Logging:
If you need to monitor the activities for Security and audit in a bucket you need know about of an important feature of S3 known as logging, This logging is of two types:

Server Access Logging:

Server Access Logging provides detailed records for requests that are made to a bucket, Enabling server access logging provides details about each and every detail about the requester, bucket name, request time, request action, response status, and an error code Stored in a target Bucket.

Object level Logging:

This records all API activities at the object level in a bucket, You do have the ability to control what buckets, prefixes, and objects will be audited, and what types of actions to audit to an AWS API auditing service called cloudTrail


Encryption:
There are three ways of server side encryption depending on how you choose to manage the encryption keys:

1.S3-Managed Encryption Keys (SSE-S3): This server side encryption uses strong multi-factor encryption. Amazon S3 encrypts each object with a unique key. As an additional safeguard, it encrypts the key itself with a master key that it rotates regularly. This uses one of the strongest block ciphers 256-bit Advanced Encryption Standard (AES-256), to encrypt your data.


2.AWS KMS Managed Keys(SSE-KMS): AWS Key Management Service (AWS KMS) is a service that provides a secure key management system for the cloud.KMS uses customer master keys (CMKs) to encrypt your Amazon S3 objects. for the first time you add an SSE-KMS–encrypted object to a bucket in a region, a default CMK is created for you automatically. This key is used for SSE-KMS encryption unless you select a CMK that you created separately using AWS Key Management Service.

To know in detail of this service check out KMS Documentation.


3.Encryption with customer provided Encryption keys (SSE-C): In this customer provides the encryption keys as a part of the request. Amazon S3 manages both the encryption as it writes to disks and decryption. When you upload an object, Amazon S3 uses the encryption key you provide to apply AES-256 encryption to your data and removes the encryption key from memory. When you retrieve an object, you must provide the same encryption key as part of your request only then S3 decrypts the object.


Configuring S3-Object lifecycle from AWS Mgmt console:
1) Go to aws mgmt console  > S3 section
2) Create bucket (if not already created)
3) After creating bucket, go to the bucket properties tab, enable versioning. If you donot enable static website hosting , logging,
cross-region replication, transfer acceleration, it's fine.
4) Go to Management tab > Add lifecycle rule button.
Enter rule name and filter (keep filter field blank if you want to apply rule on entire s3 bucket), Click Next.
Configure Transition screen > Select both checkboxes - Current version (incoming new files in S3) and previous versions (existing files in S3)
NOTE: Standard-IA has 30-day min retention period and 128 KB min obj size; so if you try to set no. of days to less than 30 in Std-IA, it wont allow.
Add Transition: "Transition to Std-IA After" --> 30 days (default)
Add Transition: "Transition to Glacier After"  --> 60 days (default) - again less than 60 days not allowed here

Add similar rule for previous versions also.

Configure Expiration screen > Select both checkboxes - Current version and previous versions
"Expire current version of obj" --> After "61" days from obj creation(default should be greater than 60 i.e only after obj is archived)
"Permanently delete previous versions" --> After "61" days from becoming a previous version.
"Clean up incomplete multipart uploads" --> After "7" days from start of upload.
Save.


Create a Life Cycle Policy using AWS CLI


Amazon S3 Lifecycle configuration is an XML file, But when using CLI we use JSON instead of XML.

consider an example policy:
<LifecycleConfiguration>
    <Rule>
        <ID>ExampleRule</ID>
        <Filter>
           <Prefix>documents/</Prefix>
        </Filter>
        <Status>Enabled</Status>
        <Transition>     
           <Days>365</Days>     
           <StorageClass>GLACIER</StorageClass>    must 
        </Transition> 
        <Expiration>
             <Days>3650</Days>
        </Expiration>
    </Rule>
</LifecycleConfiguration>

The equivalent JSON file is:
{
    "Rules": [
        {
            "Filter": {
                "Prefix": "documents/"
            },
            "Status": "Enabled",
            "Transitions": [
                {
                    "Days": 365,
                    "StorageClass": "GLACIER"
                }
            ],
            "Expiration": {
                "Days": 3650
            },
            "ID": "ExampleRule"
        }
    ]
}

AWS CLI Life Cycle Policy Commands

1.To set life Cycle Policy to a Bucket:
$ aws s3api put-bucket-lifecycle-configuration --bucket bucketname --lifecycle-configuration file://lifecycle.json

2.To view the existing policy:
$ aws s3api get-bucket-lifecycle-configuration --bucket bucketname

3.To delete the existing Policy:
aws s3api delete-bucket-lifecycle --bucket bucketname


To grant full control to an AWS user and read permission to everyone :
aws s3api put-object-acl --bucket MyBucket --key file.txt --grant-full-control emailaddress=user1@example.com,emailaddress=user2@example.com --grant-read uri=http://acs.amazonaws.com/groups/global/AllUsers

NOTE: using --grant-read, --grant-write and public-read,public-read-write we can restrict the IAM users and public respectively


How to attach a policy to a bucket ?
1) Go to aws mgmt console
2) Select your bucket, go to bucket permissions
3) You can set permission through ACL or Bucket policy. Select Bucket policy as with bp you can define more detailed level of permissions on bucket.
4) Click Policy generator to open a new tab where you can select policy type, allow/deny,  and actions you to allow/deny like
DeleteBucket, DeleteObject, DeleteObjectVersion, etc. Give ARN of the s3 bucket (arn_url/*) and click Add Statement.
Click Generate policy. It will give u a json. paste it on the page/editor from where you clicked Policy generator link. Save.

This bucket policy saved will supercede or override all configuration set through ACL(Access Control List)

Accessing Bucket policies through CLI


To set up a bucket policy through CLI you need to have a good knowledge on writing JSON policies,First you need to write the required policy in JSON format first then execute following commands
1.To Put a Bucket Policy:
aws s3api put-bucket-policy --bucket MyBucket --policy file://policy.json

2.To delete existing policy:
aws s3api delete-bucket-policy --bucket my-bucket

3.To get details of existing bucket policy :
aws s3api get-bucket-policy --bucket my-bucket

Cross Origin Resource Sharing (CORS)
Cross-origin resource sharing (CORS) is a mechanism that uses additional HTTP headers to tell the browser to let a web application running at one domain(origin) have permission to access selected resources from a server at a different domain.
•Amazon S3 supports CORSmaking it able to build web applications that use JavaScript and HTML 5 interact directly with resources in Amazon S3 without the need for a proxy server.
•Cross-origin requests are made using the standard HTTP request methods. Most servers will allow GET requests, meaning they will allow resources from external origins (say, a web page) to read their assets.
•For example, say you are hosting a static website in a bucket named TestWebsite , users load the website at endpoint http://TestWebsite.s3-website-us-east-1.amazonaws.com.Now you want to use JavaScript on the web pages that are stored in this bucket to be able to make authenticated GET and PUT requests against the same bucket. A web browser will block any JavaScript from allowing these requests but with CORS you can configure your bucket to explicitly enable cross-origin requests from TestWebsite.s3-website-us-east-1.amazonaws.com .

Configuring CORS on a Bucket in S3


A CORS configuration is an XML file with one or more rules that identify the origins that you will allow to access your bucket, the operations (HTTP methods) that will support for each origin, and other operation-specific information.

The following XML is a CORS configuration:
<CORSConfiguration>

 <CORSRule>
   <AllowedOrigin>http://www.example1.com</AllowedOrigin>
   <AllowedMethod>PUT</AllowedMethod>
   <AllowedMethod>POST</AllowedMethod>
   <AllowedMethod>DELETE</AllowedMethod>
   <AllowedHeader>*</AllowedHeader>
 </CORSRule>

 <CORSRule>
   <AllowedOrigin>http://www.example2.com</AllowedOrigin>
   <AllowedMethod>PUT</AllowedMethod>
   <AllowedMethod>POST</AllowedMethod>
   <AllowedMethod>DELETE</AllowedMethod>
   <AllowedHeader>*</AllowedHeader>
 </CORSRule>

 <CORSRule>
   <AllowedOrigin>*</AllowedOrigin>
   <AllowedMethod>GET</AllowedMethod>
 </CORSRule>

</CORSConfiguration>

The configuration has three rules :

•The first rule allows cross-origin PUT, POST, and DELETE requests from the http://www.example1.com origin. The rule also allows all headers in a preflight OPTIONS request through the Access-Control-Request-Headers header. In response to preflight OPTIONS requests, Amazon S3 returns requested headers.


•The second rule allows the same cross-origin requests as the first rule, but the rule applies to another origin, http://www.example2.com.


•The third rule allows cross-origin GET requests from all origins. The * wildcard character refers to all origins.

NOTE:
The CORS issue is faced while trying/running the application on local environment. Reason is you are not running your local instance on tomcat web server. So you need to explicitly install CORS plugin in browser to allow the javascript to run.
While on DEV/PROD envt, your code is deployed as war files on tomcat which already has CORS enabled which allows execution of javascript/script to run from any client machine.


Static Website Hosting on S3

A Static website can be hosted on S3 but S3 doesn't support server side scripting so dynamic websites can't be hosted, There are other AWS services which host these websites.

The website available is AWS region specific and its endpoint is in the following format
<bucket-name>.s3-website-<AWS-region>.amazonaws.com

If you want to host your own domain, not the S3 provided endpoint Amazon Route 53 helps you in hosting a website at its root domain

Steps:
1) Go to aws mgmt console, go to S3 section
2) Create new bucket , give name same as the name you want your website name to be.
3) Click 'Static website hosting', enable it, and provide index page and error page. Give page name ex: index.html, error.html.
4) AWS will give a exposed url like www.testwebsite.com.s3-website-eu-west-1.amazonaws.com
5) Hit it in browser n u may get 403 Forbidden error. Reason-Your files are not yet uploaded on S3.
6) Select your bucket. Click Upload files to upload all the website files
7) Still you will get 403 Forbidden error. Reason- All files u uploaded r not publicly accessible.
8) So you can either go to individual files in s3 website, and go to its Properties and make it publicly accessible OR you can select your s3 bucket itself (where you have hosted your static website), go to its Properties , and apply a Bucket Policy on it
to allow everyone to read/get the page.
9) Check in browser, it should be accessible now.

Using CLI to host a website in a Bucket


It is easy to set up a Bucket for website hosting using CLI after creating a bucket using the following command makes it configure for hosting a website:
aws s3 website s3://my-bucket/ --index-document index.html --error-document error.html

All files in the bucket that appear on the static site must be configured to allow visitors to open them.

-------------------------------------------------------------------------------------------------------

Using AWS Lambda with S3


We know that AWS Lambda functions are event-driven, S3 can publish events to Lambda and invoke lambda functions. This enables you to write lambda functions that process S3 events.

In S3, you add bucket notification configuration that identifies the type of event that you want S3 to publish and the Lambda function that you want to invoke

S3 and Lambda integration are of two types:

Non-stream based (async) model :

In this S3 monitor's a bucket whenever an event occurs(object creation, deletion etc)it invokes a Lambda function by passing event data as the parameter.

In a push model, you maintain event source mapping within Amazon S3 using the bucket notification configuration in this you tell S3 to monitor event type which you want to invoke Lambda.

Asynchronous Invocation:

In this, a lambda function is invocated using the event asynchronously

Steps:
0) Based on the language you are comfortable with, write a code (javascript,java, php). Assuming we write code in javascript to push the file (uploaded in s3) to get pushed in our github.
Create a folder in eclipse as container of the js file. Write the biz logic in the js file. You can write related biz code in
other js files also (like utils) and call it in main js file. But the code should be able to connect to s3, github, and should be able to read-write on s3,github with AWS SDK (i.e using aws APIs). After making it ready, zip the folder.
1) Go to aws mgmt console, go to AWS Lambda
2) Give name of lambda function as per your requirement, (ex:github-pusher1 which will push the file to github once it arrives/gets uploaded in s3.)
3) You will be given option to write code inline OR upload zip, we will upload zip file.
4) Select the main file which u want to run.
5) Click 'Create Lambda function'.
6) You can test the lambda function by clicking on test/invoke. You can also runtime change the inline js code of your lambda function and test/invoke again.
7) To see the output in console of your lambda code, you can click 'Cloudwatch' which will show the logs in console - all details/steps with SOPs shown like getting code from s3, getting ref, creating tree in git, creating commit, updating reference...code pushed to git. successful....request id ...duration...memory used.

Points to remember in AWS lambda


There are few crucial aspects important for using lambda functions,they are:

•There are few dependencies to be installed or imported into the functions


•AWS SDK for javascript in Node.js(similarly for other languages also).


•gm, Graphics magick for Node.js.


•Async utility module.


•Must create an IAM user with execution role make sure policy type is AWSlambdaexecute.


•Policy ARN can be saved because it may be used in further steps.


•Test the lambda function before deploying.


-----------------------------------------------------------------------------------------------
SNS-Simple Notification Service

Its an aws service which allows u to automate the sending of email or text msg notifications based on events that happen in your aws account.
SNS is a webservice that coordinates n manages the delivery or sending of msgs to subscribing endpoints or clients. In SNS, thr r 2 types of clients - Publishers and subscribers (refered also as Producers adn Consumers)
Publishers (which generate the message) communicate asynchronously with subscribersby producing n sending a msg.
Subscribers (i.e webservers, email ids/DLs, SQS queues, lambda functions) consume/receive the msg if they are subscribed to the message.

Example: If EC2 instance (ex: PROD) stops/crashes all of a sudden and the sys.admin is not notified then it's a prob.
Here You can configure CloudWatch to look after the health/availability of the EC2 instance and set an alarm which will be triggered if instance is down.
So if Ec2 instance goes down, CloudWatch (CW) will trigger an alarm which in turn calls the SNS which sends a txt msg/notification to sys.admin about the crash and on being notified, sys admin fixes the issue.

SNS Components: 1) Topics 2) Subscriptions 3) Publishers
Topics- How u label n group diff endpoints that u send msgs to. Example topic - EC2 failure
Subscriptions : Endpoints that a topic sends msgs to (i.e email/DL or phone no. of sys admin). Example - whoever we want the msg to be sent to
Publishers: Human (we) OR /CW Alarm/Event that gives SNS the msg that needs to be sent. Example - msg that get sent, it can be me tying a msg on computer or CW Alarm which sends the msg to the topic and then the topic sends it to the subscribers.


REDSHIFT:

RedShift:
Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. You can start with just a few hundred gigabytes of data and scale to a petabyte or more. This enables you to use your data to acquire new insights for your business and customers.
Amazon Redshift is a fast, scalable data warehouse that makes it simple and cost-effective to analyze all your data across your data warehouse and data lake. Redshift delivers ten times faster performance than other data warehouses by using machine learning, massively parallel query execution, and columnar storage on high-performance disk. You can setup and deploy a new data warehouse in minutes, and run queries across petabytes of data in your Redshift data warehouse, and exabytes of data in your data lake built on Amazon S3. You can start small for just $0.25 per hour and scale to $250 per terabyte per year, less than one-tenth the cost of other solutions.

The first step to create a data warehouse is to launch a set of nodes, called an Amazon Redshift cluster. After you provision your cluster, you can upload your data set and then perform data analysis queries. Regardless of the size of the data set, Amazon Redshift offers fast query performance using the same SQL-based tools and business intelligence applications that you use today.

Setup RedShift:
1) Sign in to your AWS account in AWS Mgmt console

2) Determine firewall rules: As part of this tutorial, you will specify a port when you launch your Amazon Redshift cluster. You will also create an inbound ingress rule in a security group to allow access through the port to your cluster.

If your client computer is behind a firewall, you need to know an open port that you can use so you can connect to the cluster from a SQL client tool and run queries. If you do not know this, you should work with someone who understands your network firewall rules to determine an open port in your firewall. Though Amazon Redshift uses port 5439 by default, the connection will not work if that port is not open in your firewall. Because you cannot change the port number for your Amazon Redshift cluster after it is created, make sure that you specify an open port that will work in your environment during the launch process.

3) Create an IAM Role:
For any operation that accesses data on another AWS resource, such as using a COPY command to load data from Amazon S3, your cluster needs permission to access the resource and the data on the resource on your behalf. You provide those permissions by using AWS Identity and Access Management, either through an IAM role that is attached to your cluster or by providing the AWS access key for an IAM user that has the necessary permissions.
Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/. And Create a role; and choose AmazonS3ReadOnlyAccess.
Now that you have created the new role, your next step is to attach it to your cluster. You can attach the role when you launch a new cluster or you can attach it to an existing cluster.

4) Launch Amazon RedShift cluster:
NOTE: The cluster that you are about to launch will be live (and not running in a sandbox). You will incur the standard Amazon Redshift usage fees for the cluster until you delete it.
Sign in to the AWS Management Console and open the Amazon Redshift console at https://console.aws.amazon.com/redshift/.
Select the region in which you want to create the cluster. (ex: U.S West -Oregon)
On the Amazon Redshift Dashboard, choose Quick launch cluster.
On the Cluster specifications page, enter the following values and then choose Launch cluster:
Node type: Choose dc2.large.
Number of compute nodes: Keep the default value of 2.
Master user name: Keep the default value of awsuser.
Master user password and Confirm password: Enter a password for the master user account.
Database port: Accept the default value of 5439.
Available IAM roles: Choose myRedshiftRole (role that you created in previous step).
On the Clusters page, choose the cluster that you just launched and review the Cluster Status information. Make sure that the Cluster Status is available and the Database Health is healthy before you try to connect to the database later

5) Authorize access to cluster:
you launched your Amazon Redshift cluster. Before you can connect to the cluster, you need to configure a security group to authorize access:
In the Amazon Redshift console, in the navigation pane, choose Clusters.
Choose examplecluster to open it, and make sure you are on the Configuration tab.
Under Cluster Properties, for VPC Security Groups, choose your security group.
After your security group opens in the Amazon EC2 console, choose the Inbound tab.
Choose Edit, and enter the following, then choose Save:
Type: Custom TCP Rule.
Protocol: TCP.
Port Range: type the same port number that you used when you launched the cluster. The default port for Amazon Redshift is 5439, but your port might be different.
Source: select Custom IP, then type 0.0.0.0/0.
Important
Using 0.0.0.0/0 is not recommended for anything other than demonstration purposes because it allows access from any computer on the internet. In a real environment, you would create inbound rules based on your own network settings.

6) Connect to cluster:
To query databases hosted by your Amazon Redshift cluster, you have two options:
Connect to your cluster and run queries on the AWS Management Console with the Query Editor.
If you use the Query Editor, you don't have to download and set up a SQL client application.
Connect to your cluster through a SQL client tool, such as SQL Workbench/j.

Run the queries after connecting to either query editor or SQL Workbench.

7) Load sample data from S3:
At this point you have a database called dev and you are connected to it. Now you will create some tables in the database, upload data to the tables, and try a query. For your convenience, the sample data you will load is available in an Amazon S3 bucket.
If you have already created tables in the dev database, load the sample data from S3 using COPY command.
NOTE: We recommend using the COPY command to load large datasets into Amazon Redshift from Amazon S3 or DynamoDB.
To load the sample data, you must provide authentication for your cluster to access Amazon S3 on your behalf.
Sample copy command:
copy users from 's3://awssampledbuswest2/tickit/allusers_pipe.txt'
credentials 'aws_iam_role=<iam-role-arn>'
delimiter '|' region 'us-west-2';

copy sales from 's3://awssampledbuswest2/tickit/sales_tab.txt'
credentials 'aws_iam_role=<iam-role-arn>'
delimiter '\t' timeformat 'MM/DD/YYYY HH:MI:SS' region 'us-west-2';

After data is copied, try select queries to verify the data that got copied:
SELECT *    FROM pg_table_def    WHERE tablename = 'sales';
SELECT firstname, lastname, total_quantity FROM (SELECT buyerid, sum(qtysold) total_quantity FROM  sales GROUP BY buyerid
ORDER BY total_quantity desc limit 10) Q, users WHERE Q.buyerid = userid ORDER BY Q.total_quantity desc;

You can optionally go the Amazon Redshift console to review the queries you executed. The Queries tab shows a list of queries that you executed over a time period you specify.

NOTE: You should revoke access to the cluster and delete it when you no longer need it so that you stop incurring charges.


Queries and answers:


What is Bastion server ? Purpose ?
As the number of EC2 instances in your AWS environment grows, so too does the number of administrative access points to those instances. ... A bastion is a special purpose server instance that is designed to be the primary access point from the Internet and acts as a proxy to your other EC2 instances
A bastion host is a server whose purpose is to provide access to a private network from an external network, such as the Internet. Because of its exposure to potential attack, a bastion host must minimize the chances of penetration.
A bastion host is a specialized computer that is deliberately exposed on a public network. From a secured network perspective, it is the only node exposed to the outside world and is therefore very prone to attack. ... Firewalls and routers can also become bastion hosts.
A bastion host is a special purpose computer on a network specifically designed and configured to withstand attacks. The computer generally hosts a single application, for example a proxy server, and all other services are removed or limited to reduce the threat to the computer. It is hardened in this manner primarily due to its location and purpose, which is either on the outside of a firewall or in a demilitarized zone (DMZ) and usually involves access from untrusted networks or computers.

In an Amazon Web Services (AWS) context, a bastion host is defined as "a server whose purpose is to provide access to a private network from an external network, such as the Internet. Because of its exposure to potential attack, a bastion host must minimize the chances of penetration."[3]. A further AWS related definition is that bastion hosts are "instances that sit within your public subnet and are typically accessed using SSH or RDP. Once remote connectivity has been established with the bastion host, it then acts as a ‘jump’ server, allowing you to use SSH or RDP to log in to other instances (within private subnets) deeper within your VPC. When properly configured through the use of security groups and Network ACLs (NACLs), the bastion essentially acts as a bridge to your private instances via the internet.
------------------------------------------------------------------------------------------------------------
Understanding Region,AZ,Data centers,Edge location, Regional Edge Cache.
ref: https://cloudacademy.com/blog/aws-global-infrastructure/

AZs are essentially the physical data centers of AWS. This is where the actual compute, storage, network, and database resources are hosted that we as consumers provision within our Virtual Private Clouds (VPCs). A common misconception is that a single availability zone is equal to a single data center. This is not the case. In fact, it’s likely that multiple data centers located close together form a single availability zone.

Each AZ will always have at least one other AZ that is geographically located within the same area, usually a city, linked by highly resilient and very low latency private fiber optic connections. However, each AZ will be isolated from the others using separate power and network connectivity that minimizes impact to other AZs should a single AZ fail.
Making use of at least two AZs in a region helps you maintain high availability of your infrastructure and its always a recommended best practice

As we now know, a Region is a collection of availability zones that are geographically located close to one other. This is generally indicated by AZs within the same city. AWS has deployed them across the globe to allow its worldwide customer base to take advantage of low latency connections. Every Region will act independently of the others, and each will contain at least two Availability Zones.

Interestingly, not all AWS services are available in every region. This is a consideration that must be taken into account when architecting your infrastructure. Some services are classed as global services, such as AWS Identity & Access Management (IAM) or Amazon CloudFront, which means that these services are not tied to a specific region.  However, most services are region-specific

Edge Locations are AWS sites deployed in major cities and highly populated areas across the globe. They far outnumber the number of availability zones available.
While Edge Locations are not used to deploy your main infrastructures such as EC2 instances, EBS storage, VPCs, or RDS resources like AZs, they are used by AWS services such as AWS CloudFront and AWS Lambda@Edge (currently in Preview) to cache data and reduce latency for end user access by using the Edge Locations as a global Content Delivery Network (CDN).
As a result, Edge Locations are primarily used by end users who are accessing and using your services.

In November 2016, AWS announced a new type of Edge Location, called a Regional Edge Cache. These sit between your CloudFront Origin servers and the Edge Locations.  A Regional Edge Cache has a larger cache-width than each of the individual Edge Locations, and because data expires from the cache at the Edge Locations, the data is retained at the Regional Edge Caches.
Therefore, when data is requested at the Edge Location that is no longer available, the Edge Location can retrieve the cached data from the Regional Edge Cache instead of the Origin servers, which would have a higher latency.


------------------------------------------------------------------------------------------------------------
CloudFront & Edge location:
Amazon CloudFront is a web service that speeds up distribution of your static and dynamic web content, such as .html, .css, .js, and image files, to your users. CloudFront delivers your content through a worldwide network of data centers called edge locations. When a user requests content that you're serving with CloudFront, the user is routed to the edge location that provides the lowest latency (time delay), so that content is delivered with the best possible performance.
If the content is already in the edge location with the lowest latency, CloudFront delivers it immediately.

If the content is not in that edge location, CloudFront retrieves it from an origin that you've defined—such as an Amazon S3 bucket, a MediaPackage channel, or an HTTP server (for example, a web server) that you have identified as the source for the definitive version of your content.


<INSERT_IMAGE> - https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/images/how-you-configure-cf.png

How You Configure CloudFront to Deliver Your Content

1) You specify origin servers, like an Amazon S3 bucket or your own HTTP server, from which CloudFront gets your files which will then be distributed from CloudFront edge locations all over the world.

An origin server stores the original, definitive version of your objects. If you're serving content over HTTP, your origin server is either an Amazon S3 bucket or an HTTP server, such as a web server. Your HTTP server can run on an Amazon Elastic Compute Cloud (Amazon EC2) instance or on a server that you manage; these servers are also known as custom origins.

If you use the Adobe Media Server RTMP protocol to distribute media files on demand, your origin server is always an Amazon S3 bucket.

2) You upload your files to your origin servers. Your files, also known as objects, typically include web pages, images, and media files, but can be anything that can be served over HTTP or a supported version of Adobe RTMP, the protocol used by Adobe Flash Media Server.

If you're using an Amazon S3 bucket as an origin server, you can make the objects in your bucket publicly readable, so that anyone who knows the CloudFront URLs for your objects can access them. You also have the option of keeping objects private and controlling who accesses them. See Serving Private Content with Signed URLs and Signed Cookies.

3) You create a CloudFront distribution, which tells CloudFront which origin servers to get your files from when users request the files through your web site or application. At the same time, you specify details such as whether you want CloudFront to log all requests and whether you want the distribution to be enabled as soon as it's created.

4) CloudFront assigns a domain name to your new distribution that you can see in the CloudFront console, or that is returned in the response to a programmatic request, for example, an API request.

5) CloudFront sends your distribution's configuration (but not your content) to all of its edge locations—collections of servers in geographically dispersed data centers where CloudFront caches copies of your objects.

As you develop your website or application, you use the domain name that CloudFront provides for your URLs. For example, if CloudFront returns d111111abcdef8.cloudfront.net as the domain name for your distribution, the URL for logo.jpg in your Amazon S3 bucket (or in the root directory on an HTTP server) will be http://d111111abcdef8.cloudfront.net/logo.jpg.

Or you can configure your CloudFront distribution so you can use your own domain name. In that case, the URL might be http://www.example.com/logo.jpg.

Optionally, you can configure your origin server to add headers to the files, to indicate how long you want the files to stay in the cache in CloudFront edge locations. By default, each object stays in an edge location for 24 hours before it expires. The minimum expiration time is 0 seconds; there isn't a maximum expiration time limit.

Ref: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Introduction.html

------------------------------------------------------------------------------------------------------------
difference between amazon cloudfront and amazon s3 transfer acceleration?

CloudFront is for content delivery. S3 Transfer Acceleration is for faster transfers and higher throughput to S3 buckets (mainly uploads).

Amazon S3 Transfer Acceleration is an S3 feature that accelerates uploads to S3 buckets using AWS Edge locations - the same Edge locations as in AWS CloudFront service.

However, (a) creating a CloudFront distribution with an origin pointing to your S3 bucket and (b) enabling S3 Transfer acceleration for your bucket - are two different things serving two different purposes.

When you create a CloudFront distribution with an origin pointing to your S3 bucket, you enable caching on Edge locations. Consequent requests to the same objects will be served from the Edge cache which is faster for the end user and also reduces the load on your origin. CloudFront is primarily used as a content delivery service.

When you enable S3 Transfer Acceleration for your S3 bucket and use <bucket>.s3-accelerate.amazonaws.com instead of the default S3 endpoint, the transfers are performed via the same Edge locations, but the network path is optimized for long-distance large-object uploads. Extra resources and optimizations are used to achieve higher throughput. No caching on Edge locations.

S3 TA actually takes advantage of Amazon CloudFront’s globally distributed edge locations to deliver/transfer fast, easy, and secure way of files over long distances between your client and an S3 bucket.

Ref: https://stackoverflow.com/questions/36882595/are-there-any-difference-between-amazon-cloudfront-and-amazon-s3-transfer-accele

Explanation 2:
Amazon S3 Transfer Acceleration enables fast, easy, and secure transfers of files over long distances between your client and an S3 bucket. Transfer Acceleration takes advantage of Amazon CloudFront’s globally distributed edge locations. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path.

When using Transfer Acceleration, additional data transfer charges may apply.

You might want to use Transfer Acceleration on a bucket for various reasons, including the following:
You have customers that upload to a centralized bucket from all over the world.
You transfer gigabytes to terabytes of data on a regular basis across continents.
You are unable to utilize all of your available bandwidth over the Internet when uploading to Amazon S3.

You can use the Amazon S3 Transfer Acceleration Speed Comparison tool to compare accelerated and non-accelerated upload speeds across Amazon S3 regions. The Speed Comparison tool uses multipart uploads to transfer a file from your browser to various Amazon S3 regions with and without using Transfer Acceleration.

To get started using Amazon S3 Transfer Acceleration, perform the following steps:

Enable Transfer Acceleration on a bucket – For your bucket to work with transfer acceleration, the bucket name must conform to DNS naming requirements and must not contain periods (".").

You can enable Transfer Acceleration on a bucket any of the following ways:

Use the Amazon S3 console. For more information, see Enabling Transfer Acceleration in the Amazon Simple Storage Service Console User Guide.

Use the REST API PUT Bucket accelerate operation.

Use the AWS CLI and AWS SDKs. For more information, see Using the AWS SDKs, CLI, and Explorers.

Transfer data to and from the acceleration-enabled bucket by using one of the following s3-accelerate endpoint domain names:

bucketname.s3-accelerate.amazonaws.com – to access an acceleration-enabled bucket.

bucketname.s3-accelerate.dualstack.amazonaws.com – to access an acceleration-enabled bucket over IPv6. Amazon S3 dual-stack endpoints support requests to S3 buckets over IPv6 and IPv4. The Transfer Acceleration dual-stack endpoint only uses the virtual hosted-style type of endpoint name. For more information, see Getting Started Making Requests over IPv6 and Using Amazon S3 Dual-Stack Endpoints.

Important

Support for the dual-stack accelerated endpoint currently is only available from the AWS Java SDK. Support for the AWS CLI and other AWS SDKs is coming soon.

Note

You can continue to use the regular endpoint in addition to the accelerate endpoints.

You can point your Amazon S3 PUT object and GET object requests to the s3-accelerate endpoint domain name after you enable Transfer Acceleration. For example, let's say you currently have a REST API application using PUT Object that uses the host name mybucket.s3.amazonaws.com in the PUT request. To accelerate the PUT you simply change the host name in your request to mybucket.s3-accelerate.amazonaws.com. To go back to using the standard upload speed, simply change the name back to mybucket.s3.amazonaws.com.

After Transfer Acceleration is enabled, it can take up to 20 minutes for you to realize the performance benefit. However, the accelerate endpoint will be available as soon as you enable Transfer Acceleration.

You can use the accelerate endpoint in the AWS CLI, AWS SDKs, and other tools that transfer data to and from Amazon S3. If you are using the AWS SDKs, some of the supported languages use an accelerate endpoint client configuration flag so you don't need to explicitly set the endpoint for Transfer Acceleration to bucketname.s3-accelerate.amazonaws.com.

You can use all of the Amazon S3 operations through the transaction acceleration endpoints, except for the following the operations: GET Service (list buckets), PUT Bucket (create bucket), and DELETE Bucket. Also, Amazon S3 Transfer Acceleration does not support cross region copies using PUT Object - Copy.

Requirements for Using Amazon S3 Transfer Acceleration
The following are the requirements for using Transfer Acceleration on an S3 bucket:

1.Transfer Acceleration is only supported on virtual style requests. For more information about virtual style requests, see Making Requests Using the REST API.

2.The name of the bucket used for Transfer Acceleration must be DNS-compliant and must not contain periods (".").

3.Transfer Acceleration must be enabled on the bucket. After enabling Transfer Acceleration on a bucket it might take up to thirty minutes before the data transfer speed to the bucket increases.

4.To access the bucket that is enabled for Transfer Acceleration, you must use the endpoint bucketname.s3-accelerate.amazonaws.com. or the dual-stack endpoint bucketname.s3-accelerate.dualstack.amazonaws.com to connect to the enabled bucket over IPv6.

5.You must be the bucket owner to set the transfer acceleration state. The bucket owner can assign permissions to other users to allow them to set the acceleration state on a bucket. The s3:PutAccelerateConfiguration permission permits users to enable or disable Transfer Acceleration on a bucket. The s3:GetAccelerateConfiguration permission permits users to return the Transfer Acceleration state of a bucket, which is either Enabled or Suspended.

Ref: https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html

------------------------------------------------------------------------------------------------------------------
What situation gives rise to use/create another bucket or a new bucket for existing account or application ?
------------------------------------------------------------------------------------------------------------------
Diff between normal Load Balancer and ELB
------------------------------------------------------------------------------------------------------------------
Diff between EBS and S3
------------------------------------------------------------------------------------------------------------------
Diff between ELB and Auto scaling

------------------------------------------------------------------------------------------------------------------
Use of DynamoDB
------------------------------------------------------------------------------------------------------------------
Use of Redshift
Ans:
Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. You can start with just a few hundred gigabytes of data and scale to a petabyte or more. This enables you to use your data to acquire new insights for your business and customers.
Amazon Redshift is a fast, scalable data warehouse that makes it simple and cost-effective to analyze all your data across your data warehouse and data lake. Redshift delivers ten times faster performance than other data warehouses by using machine learning, massively parallel query execution, and columnar storage on high-performance disk. You can setup and deploy a new data warehouse in minutes, and run queries across petabytes of data in your Redshift data warehouse, and exabytes of data in your data lake built on Amazon S3. You can start small for just $0.25 per hour and scale to $250 per terabyte per year, less than one-tenth the cost of other solutions.

The first step to create a data warehouse is to launch a set of nodes, called an Amazon Redshift cluster. After you provision your cluster, you can upload your data set and then perform data analysis queries. Regardless of the size of the data set, Amazon Redshift offers fast query performance using the same SQL-based tools and business intelligence applications that you use today.
------------------------------------------------------------------------------------------------------------------
Use of CloudFormation
------------------------------------------------------------------------------------------------------------------
Use of Redis, MemCache:
They are Caching tools which store data as key-value pairs in memory and renders it to UI.
------------------------------------------------------------------------------------------------------------------
Default encryption that S3 uses out of : SSE-S3, SSE-KMS, SSE-C (customer provided keys), CSE (Client side encryption)
------------------------------------------------------------------------------------------------------------------
Default and current storage classes - obj lifecycle mgmt configured
------------------------------------------------------------------------------------------------------------------
Versioning and MFA-Delete enabled or not in current and default aws
------------------------------------------------------------------------------------------------------------------
Cross-Region replicated on or off - default and current aws
------------------------------------------------------------------------------------------------------------------
What instance family is used by default/current aws - Reserved, on-demand, spot
------------------------------------------------------------------------------------------------------------------
Is Elastic IP configured with current AWS envt
Ans: Not sure, but private IP is assigned to each instance, no public IP.
------------------------------------------------------------------------------------------------------------------


Answered:


what is AWS?
Ans:AWS stands for Amazon Web Service; it is a collection of remote computing services also known as cloud computing platform.  This new realm of cloud computing is also known as IaaS or Infrastructure as a Service.
------------------------------------------------------------------------------------------------------------
Mention what is the difference between Amazon S3 and EC2?
Ans:
The difference between EC2 and Amazon S3 is that

EC2 is a cloud web service used for hosting your application
S3 is a data storage system where any amount of data can be stored
EC2 is like a huge computer machine which can run either Linux or Windows and can handle application like PHP, Python, Apache or any databases
S3 has a REST interface and uses secure HMAC-SHA1 authentication keys

How can you send request to Amazon S3?
Ans:
Amazon S3 is a REST service, you can send request by using the REST API or the AWS SDK wrapper libraries that wrap the underlying Amazon S3 REST API.

What does an AMI include?
Ans:
An AMI includes the following things

A template for the root volume for the instance
Launch permissions decide which AWS accounts can avail the AMI to launch instances
A block device mapping that determines the volumes to attach to the instance when it is launched

Mention what is the relation between an instance and AMI?
Ans:
From a single AMI, you can launch multiple types of instances.  An instance type defines the hardware of the host computer used for your instance. Each instance type provides different compute and memory capabilities.  Once you launch an instance, it looks like a traditional host, and we can interact with it as we would with any computer.

Explain what is AMI?
Ans:
AMI stands for Amazon Machine Image.  It’s a template that provides the information (an operating system, an application server and applications) required to launch an instance, which is a copy of the AMI running as a virtual server in the cloud.  You can launch instances from as many different AMIs as you need.

Explain what is S3?
Ans: S3 stands for Simple Storage Service. You can use S3 interface to store and retrieve any amount of data, at any time and from anywhere on the web.  For S3, the payment model is “pay as you go”.

key components of AWS?
Ans:
Route 53: A DNS web service
Simple E-mail Service: It allows sending e-mail using RESTFUL API call or via regular SMTP
Identity and Access Management: It provides enhanced security and identity management for your AWS account
Simple Storage Device or (S3): It is a storage device and the most widely used AWS service
Elastic Compute Cloud (EC2): It provides on-demand computing resources for hosting applications. It is very useful in case of unpredictable workloads
Elastic Block Store (EBS): It provides persistent storage volumes that attach to EC2 to allow you to persist data past the lifespan of a single EC2
CloudWatch: To monitor AWS resources, It allows administrators to view and collect key Also, one can set a notification alarm in case of trouble


security best practices for Amazon EC2
Ans:
Use AWS identity and access management to control access to your AWS resources
Restrict access by allowing only trusted hosts or networks to access ports on your instance
Review the rules in your security groups regularly
Only open up permissions that you require
Disable password-based login, for instance, launched from your AMI

what is T2 instances?
Ans:
T2 instances are designed to provide moderate baseline performance and the capability to burst to higher performance as required by workload.

Explain can you vertically scale an Amazon instance? How?
Ans:
Yes, you can vertically scale on Amazon instance. For that

Spin up a new larger instance than the one you are currently running
Pause that instance and detach the root webs volume from the server and discard
Then stop your live instance and detach its root volume
Note the unique device ID and attach that root volume to your new server
And start it again


------------------------------------------------------------------------------------------------------------
which facility I can use to reduce Internode latency between multiple clusters
------------------------------------------------------------------------------------------------------------
what is SQS
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that makes it easy to decouple and scale microservices, distributed systems, and serverless applications. Building applications from individual components that each perform a discrete function improves scalability and reliability, and is best practice design for modern applications. SQS makes it simple and cost-effective to decouple and coordinate the components of a cloud application. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be always available.

------------------------------------------------------------------------------------------------------------
Which suites most for multi AZ deployments:
Amazon RDS Multi-AZ deployments provide enhanced availability and durability for Database (DB) Instances, making them a natural fit for production database workloads. When you provision a Multi-AZ DB Instance, Amazon RDS automatically creates a primary DB Instance and synchronously replicates the data to a standby instance in a different Availability Zone (AZ). Each AZ runs on its own physically distinct, independent infrastructure, and is engineered to be highly reliable. In case of an infrastructure failure, Amazon RDS performs an automatic failover to the standby (or to a read replica in the case of Amazon Aurora), so that you can resume database operations as soon as the failover is complete. Since the endpoint for your DB Instance remains the same after a failover, your application can resume database operation without the need for manual administrative intervention.

------------------------------------------------------------------------------------------------------------
What is EC2 compute unit - A single EC2 instance ?
The amount of CPU that is allocated to a particular instance is expressed in terms of these EC2 Compute Units. We use several benchmarks and tests to manage the consistency and predictability of the performance of an EC2 Compute Unit

------------------------------------------------------------------------------------------------------------
When EC2 instance terminates, what happens to these - All volumes/root volume of EBS AMIs, instance store
Data on Instance store volume data does not persist
Data on EBS root volumes, have the DeleteOnTermination flag set to true, would be deleted by default
Additional EBS volumes attached have the DeleteOnTermination flag set to false are not deleted but just dettached from the instance
http://jayendrapatil.com/aws-ebs-vs-instance-store/
------------------------------------------------------------------------------------------------------------
How do I gain maximum availability


------------------------------------------------------------------------------------------------------------
Photo sharing website. Somebody access the files from your S3 bucket using a url.
How u can prevent ? - Using bucket policies,ACL ?
------------------------------------------------------------------------------------------------------------
When u restart EC2 instance, what happens to public as well as private IP:
Once an EC2 instance is launched, it's assigned a public and private IP address at boot time. An instance's private IP address will never change during the lifetime of that instance. However, its public IP address can change under certain circumstances.
Stop and Restart - When you stop a server, the associated instance is actually terminated. Therefore, when you restart the server, another virtual machine instance will be provisioned to you, so it will have new and different public and private IP addresses
Reboot - When you perform a reboot, the same virtual machine instance is rebooted. The original virtual machine instance that was provisioned to you is never returned back to Amazon. The public IP address will not change.

When you stop/start your instance, the IP address will change. Unfortunately there is no way to avoid this scenario in EC2. If you reboot the instance (within the Operating System, or via the console), it will keep the same IP addresses. Unfortunately it is not possible for us to reassign the address to your instance as that address would have been released back into the pool used by other EC2 instances.

If you want to avoid this issue in future, depending on your needs:
If you only need a fixed public IP address, you can assign an Elastic IP address to your instance. Further information on Elastic IP addresses is available from here.
If you need both public and private IP addresses to remain the same throughout the lifetime of the instance, you can launch your instance in VPC instead. The private IP address assigned to an instance in VPC remains with the instance through to termination.
------------------------------------------------------------------------------------------------------------

Use of Amazon CloudWatch:
Elastic Load Balancing publishes data points to Amazon CloudWatch for your load balancers and your back-end instances. CloudWatch enables you to retrieve statistics about those data points as an ordered set of time-series data, known as metrics. Think of a metric as a variable to monitor, and the data points as the values of that variable over time. For example, you can monitor the total number of healthy EC2 instances for a load balancer over a specified time period.
You can use metrics to verify that your system is performing as expected. For example, you can create a CloudWatch alarm to monitor a specified metric and initiate an action (such as sending a notification to an email address) if the metric goes outside what you consider an acceptable range.
Elastic Load Balancing reports metrics to CloudWatch only when requests are flowing through the load balancer. If there are requests flowing through the load balancer, Elastic Load Balancing measures and sends its metrics in 60-second intervals. If there are no requests flowing through the load balancer or no data for a metric, the metric is not reported.
------------------------------------------------------------------------------------------------------------

Placement Groups Overview

A Placement Group is a logical grouping of instances within a single Availability Zone and are recommended for applications that benefits from low network latency, high network throughput, or both.
Placement group don’t span across Availability Zones
Placement group is only available within a single Availability Zone either in the same VPC or peered VPCs
Placement group is more of an hint to AWS that the instances need to be launched physically close to each together
Using placement groups enables applications to participate in a low-latency, 10 Gbps network
Should have unique name within AWS account
Existing EC2 instance can’t be moved into a Placement Group as it might be physically located away from the existing instances
For moving an instance into the placement group,
create an AMI from the existing instance,
and then launch a new instance from the AMI into a placement group.
AWS recommends using the same homogenous instance type for all instances in a placement group
Placement Groups Best Practices

Use homogenous instance types
Launch all the placement group instances at the same time
Not a best fit for horizontally scalable web services
Ensure there is enough capacity
Jumbo frames help to achieve the maximum network throughput possible, and they are recommended for instances collocated inside a placement group

-------------------------------------------------------------------------------------------------------
Does S3 support read after write consistency ?
---------------------------------------------------------------------------------------------------------

Elastic Beanstalk:

With Elastic Beanstalk, you can quickly deploy and manage applications in the AWS Cloud without worrying about the infrastructure that runs those applications. AWS Elastic Beanstalk reduces management complexity without restricting choice or control. You simply upload your application, and Elastic Beanstalk automatically handles the details of capacity provisioning, load balancing, scaling, and application health monitoring.
Elastic Beanstalk supports applications developed in Java, PHP, .NET, Node.js, Python, and Ruby, as well as different container types for each language.
When you deploy your application, Elastic Beanstalk provisions one or more AWS resources, such as Amazon EC2 instances. The software stack that runs on your Amazon EC2 instances depends on the container type. For example, Elastic Beanstalk supports two container types for Node.js: a 32-bit Amazon Linux image and a 64-bit Amazon Linux image. Each runs a software stack tailored to hosting a Node.js application. You can interact with Elastic Beanstalk by using the AWS Management Console, the AWS Command Line Interface (AWS CLI), or eb, a high-level CLI designed specifically for Elastic Beanstalk.

You can also perform most deployment tasks, such as changing the size of your fleet of Amazon EC2 instances or monitoring your application, directly from the Elastic Beanstalk web interface (console).
To use Elastic Beanstalk, you create an application, upload an application version in the form of an application source bundle (for example, a Java .war file) to Elastic Beanstalk, and then provide some information about the application. Elastic Beanstalk automatically launches an environment and creates and configures the AWS resources needed to run your code. After your environment is launched, you can then manage your environment and deploy new application versions.

Elastic Beanstalk provides developers and systems administrators an easy, fast way to deploy and manage their applications without having to worry about AWS infrastructure. If you already know the AWS resources you want to use and how they work, you might prefer AWS CloudFormation to create your AWS resources by creating a template. You can then use this template to launch new AWS resources in the exact same way without having to recustomize your AWS resources. Once your resources are deployed, you can modify and update the AWS resources in a controlled and predictable way, providing the same sort of version control over your AWS infrastructure that you exercise over your software.
------------------------------------------------------------------------------------------------------------------
The minimum message size is 1 byte (1 character). The maximum is 262,144 bytes (256 KB). To send messages larger than 256 KB, you can use the Amazon SQS Extended Client Library for Java.

How is Amazon SQS different from Amazon MQ?

Amazon MQ, Amazon SQS, and Amazon SNS are messaging services that are suitable for anyone from startups to enterprises. If you're using messaging with existing applications, and want to move your messaging to the cloud quickly and easily, we recommend you consider Amazon MQ. It supports industry-standard APIs and protocols so you can switch from any standards-based message broker to Amazon MQ without rewriting the messaging code in your applications. If you are building brand new applications in the cloud, we recommend you consider Amazon SQS and Amazon SNS. Amazon SQS and SNS are lightweight, fully managed message queue and topic services that scale almost infinitely and provide simple, easy-to-use APIs. You can use Amazon SQS and SNS to decouple and scale microservices, distributed systems, and serverless applications, and improve reliability.
------------------------------------------------------------------------------------------------------------------
Does Amazon SQS provide message ordering?
Yes. FIFO (first-in-first-out) queues preserve the exact order in which messages are sent and received.
------------------------------------------------------------------------------------------------------------------
Does Amazon use Amazon SQS for its own applications?
Yes.
-----------------------------------------------------------------------------------------------------------------
Who can perform operations on a message queue?

Only an AWS account owner (or an AWS account that the account owner has delegated rights to) can perform operations on an Amazon SQS message queue.
-----------------------------------------------------------------------------------------------------------------
How many buckets can you create in AWS by default?
By default, you can create upto 100 buckets in each of your AWS accounts.

-----------------------------------------------------------------------------------------------------------------
What does an AMI include?

Answer:

An AMI includes the following things

A template for the root volume for the instance
Launch permissions decide which AWS accounts can avail the AMI to launch instances
A block device mapping that determines the volumes to attach to the instance when it is launched
-----------------------------------------------------------------------------------------------------------------
Mention what is the relation between an instance and AMI?

Answer:

From a single AMI, you can launch multiple types of instances.  An instance type defines the hardware of the host computer used for your instance. Each instance type provides different compute and memory capabilities.  Once you launch an instance, it looks like a traditional host, and we can interact with it as we would with any computer.
-----------------------------------------------------------------------------------------------------------------
In CloudFront what happens when content is NOT present at an Edge location and a request is made to it?
Answer:CloudFront delivers the content directly from the origin server and stores it in the cache of the edge location
-----------------------------------------------------------------------------------------------------------------

You have written a CloudFormation template that creates 1 elastic load balancer fronting 2 EC2 instances. Which section of the template should you edit so that the DNS of the load balancer is returned upon creation of the stack.
Answer: Outputs
-----------------------------------------------------------------------------------------------------------------

Which of the following is true with respect to serving private content through CloudFront?
Ans:Signed URLs can be created to access objects from CloudFront edge locations
Direct access to S3 URLs can be removed therefore allowing access only through CloudFront URLs
Mark the S3 bucket private and and create an Origin Access Identity to access the objects
-----------------------------------------------------------------------------------------------------------------

Which of the following can be used as an origin server in CloudFront?
Ans:
A webserver running on EC2
A webserver running in your own datacenter
An Amazon S3 bucket
-----------------------------------------------------------------------------------------------------------------
An instance is launched into the public subnet of a VPC. Which of the following must be done in order for it to be accessible FROM the Internet?
Ans: Attach an Elastic IP to the instance
-----------------------------------------------------------------------------------------------------------------
An instance is connected to an ENI (Elastic Network Interface) in one subnet. What happens when you attach an ENI of a different subnet to this instance?
Ans: The instance follows the rules of both the subnets
-----------------------------------------------------------------------------------------------------------------
If you craete a default VPC, or custom VPC, how many default subnets get created or u get how many subnets into ur VPC created?




-----------------------------------------------------------------------------------------------------------------
Which of the following SECURITY certifications does AWS have?
Ans:
SOC1 Type 2 (formerly SAS-70)
ISO 270001
PCI DSS
HIPAA & ITAR Compliant
-----------------------------------------------------------------------------------------------------------------
Which one of the following is not an application service on AWS
Ans: S3
-----------------------------------------------------------------------------------------------------------------
DynomoDB  is NoSQL database
-----------------------------------------------------------------------------------------------------------------
Which of the following databases can be run on RDS?
Ans: Oracle, MS SQL, MySQL
-----------------------------------------------------------------------------------------------------------------
Clients cannot load balance between multiple AWS regions
Regions are global and AZ are local within each region
-----------------------------------------------------------------------------------------------------------------
How many AWS clouds are the there in the world
Ans: 7
-----------------------------------------------------------------------------------------------------------------
What SLA do AWS provide for their EC2 service?
Ans: 99.95%
-----------------------------------------------------------------------------------------------------------------
EBS: Fast, withing single data center, not accessible from internet
----------------------------------------------------------------------------------------------------------------
You have an environment that consists of a public subnet using Amazon VPC and 3 instances that are running in this subnet. These three instances can successfully communicate with other hosts on the Internet. You launch a fourth instance in the same subnet, using the same AMI and security group configuration you used for the others, but find that this instance cannot be accessed from the Internet. What should you do to enable internet access?

Ans: Assign an elastic IP address to the 4th instance
----------------------------------------------------------------------------------------------------------------
If I want to run a database in an Amazon instance, which is the most recommended Amazon storage option?
Ans: Amazon EBS
----------------------------------------------------------------------------------------------------------------
____ also known as master-slave deployments, are a good option for applications with a high read to write ratio. In this architecture, a single master database is responsible for handling all writes. The writes are then replicated to the slaves which are designated to handle just reads.
Ans: Read replicas
----------------------------------------------------------------------------------------------------------------

Which of the following services natively encrypts data at rest within an AWS region?
Ans: Amazon Glacier, AWS Storage Gateway
----------------------------------------------------------------------------------------------------------------
A customer is hosting their company website on a cluster of web servers that are behind a public-facing load balancer. The customer also uses Amazon Route 53 to manage their public DNS. How should the customer configure the DNS zone apex record to point to the load balancer?
Ans: Create an A record aliased to the load balancer DNS name.
----------------------------------------------------------------------------------------------------------------
You are deploying an application to collect votes for a very popular television show. Millions of users will submit votes using mobile devices. The votes must be collected into a durable, scalable, and highly available data store for real-time public tabulation. Which service should you use?
Ans: Amazon Kinesis
----------------------------------------------------------------------------------------------------------------
When will you incur costs with an Elastic IP address (EIP)?
Ans; When it is allocated and associated with a stopped instance
----------------------------------------------------------------------------------------------------------------.
A company is deploying a new two-tier web application in AWS. The company has limited staff and requires high availability, and the application requires complex queries and table joins. Which configuration provides the solution for the company’s requirements?
Ans: Amazon RDS for MySQL with Multi-AZ
----------------------------------------------------------------------------------------------------------------.
Which of the following statements are true about Amazon Route 53 resource records?
Ans:
An Alias record can map one DNS name to another Amazon Route 53 DNS name
An Amazon Route 53 CNAME record can point to any DNS record hosted anywhere.
----------------------------------------------------------------------------------------------------------------.
Which of the following approaches provides the lowest cost for Amazon Elastic Block Store snapshots while giving you the ability to fully restore data?
Ans: Maintain a single snapshot the latest snapshot is both incremental and complete
----------------------------------------------------------------------------------------------------------------.
Which of the following are true regarding AWS CloudTrail?
Ans:
Logs can be delivered to a single Amazon S3 bucket for aggregation
CloudTrail is enabled on a per-region basis
CloudTrail is enabled globally

A company has an AWS account that contains three VPCs (Dev, Test, and Prod) in the same region. Test is peered to both Prod and Dev. All VPCs have non overlapping CIDR blocks. The company wants to push minor code releases from Dev to Prod to speed up time to market. Which of the following options helps the company accomplish this?
Ans; Create a new peering connection Between Prod and Dev along with appropriate routes

Company A has a S3 bucket containing premier content that they intend to make available to only paid subscribers of their website. The S3 bucket currently has default permissions of all objects being private to prevent inadvertent exposure of the premier content to non-paying website visitors. How can Company A provide only paid subscribers the ability to download a premier content file in the S3 bucket?
Ans:  Generate a pre-signed object URL for the premier content file when a paid subscriber requests a download

If a message is retrieved from a queue in Amazon SQS, how long is the message inaccessible to other users by default?
Ans: 30 seconds

You have written an application that uses the Elastic Load Balancing service to spread traffic to several web servers. Your users complain that they are sometimes forced to login again in the middle of using your application, after they have already logged in. This is not behaviour you have designed. What is a possible solution to prevent this happening?
Ans: Use ElastiCache to save session state.

Regarding the attaching of ENI to an instance, what does ‘hot attach’ refer to?
Ans: Attaching an ENI to an instance when it is running

Which of the following are correct statements with policy evaluation logic in AWS Identity and Access Management?
Ans:  An explicit allow overrides default deny
By default, all requests are denied

Which of the following services are included at no additional cost with the use of the AWS platform?
Ans: CloudFormation , Auto Scaling

How can software determine the public and private IP addresses of the Amazon EC2 instance that it is running on?
Ans: Query the local instance metadata

How is Amazon SNS different from Amazon SQS? choose 2 answers
Ans: Amazon SNS allows applications to send time-critical messages to multiple subscribers through a “push” mechanism

 Amazon SQS is a message queue service used by distributed applications to exchange messages through a polling model, and can be used to decouple sending and receiving components

 Which of the following are valid SNS delivery transports?
 SMS, Email


 Which of the following statements about SWF are true?
 Ans: SWF workflow executions can last up to a year
 SWF uses deciders and workers to complete tasks
 SWF tasks are assigned once and never duplicated

 Which of the following statements about SQS is true?
 Messages will be delivered one or more times and message delivery order is indeterminate

 What is the maximum number of S3 Buckets available per AWS account?
 100 per account


 What happens, by default, when one of the resources in a CloudFormation stack cannot be created?
 Previously-created resources are deleted and the stack creation terminates.

 Which statements about DynamoDB are true?
 DynamoDB uses optimistic concurrency control
 DynamoDB uses conditional writes for consistency

 EC2 instances are launched from Amazon Machine images (AMIS). A given public AMI can:
 only be used to launch EC2 instances in the same AWS region as the AMI is stored

 What is the type of monitoring data (for Amazon EBS volumes) available automatically in 5-minute periods at no charge called?
 Basic

 How can you secure data at rest on an EBS volume?
 Use an encrypted file system on top of the EBS volume.

 The user just started an instance at 3 PM. Between 3 PM to 5 PM, he stopped and started the instance twice. During the same period, he has run the linux reboot command by ssh once and triggered reboot from AWS console once. For how many instance hours will AWS charge this user?
 Ans: 4

 Which of the below mentioned steps will not be performed while creating the AMI of instance stored-backend?
 Define the AMI launch permissions.

 A startup company hired you to help them build a mobile application, that will ultimately store billions of images and videos in Amazon Simple Storage Service (S3). The company is lean on funding, and wants to minimize operational costs, however, they have an aggressive marketing plan, and expect to double their current installation base every six months. Due to the nature of their business, they are expecting sudden and large increases in traffic to and from S3, and need to ensure that it can handle the performance needs of their application. What other information must you gather from this customer in order to determine whether S3 is the right option?
 Ans: You must find out the total number of requests per second at peak usage

 EC2 role:
 Answer: Launch an instance with an AWS Identity and Aceess Management (IAM) role to restrict AWS API access for the instance.

 You have assigned one Elastic IP to your EC2 instance. Now we need to restart the VM without EIP changed. Which of below you should not do?
 Ans: Reboot and stop/start both works.

 About the charge of Elastic IP Address, which of the following is true?
 Ans: You can have one Elastic IP (EIP) address associated with a running instance at no charge.

 Placement Groups: enables applications to participate in a low-latency, 10 Gbps network. Which of below statements is false.
 Ans: You can move an existing instance into a placement group by specify parameter of placement group.

 What about below is false for AWS SLA
 Ans: S3 availability is guarantee to 99.95%.

 Select the correct set of options. These are the initial settings for the default security group
 Ans: Allow no inbound traffic, Allow all outbound traffic and Allow instances associated with this security group to talk to each other.

 An IAM user is trying to perform an action on an object belonging to some other root account’s bucket. Which of the below mentioned options will AWS S3 not verify?
 Ans: Permission provided by the parent of the IAM user on the bucket

 Which statements are true about AWS ElastiCache?
 Ans:
 Simplifies and offloads the management and monitoring
 Makes it easy to deploy, operate, and scale an in-memory cache

 From the list below which are not AWS regions?
 Ans: Sydney, Moscow

 Which statement is true about S3?
 Ans: Write once, read many

 What SLA do AWS provide for their S3 service?
 Ans: 99.999999999%

 Which one of the following is not an Amazon Buisness Unit?
 Ans: Bookstore Business

 Fact: Clients cannot load balance between multiple AWS Regions
 Fact: Regions are global and Availability Zones are local within each region

 Which of the following networking options is not available on AWS?
 Ans: Leased Line

 Edge Locations form part of Cloud Front an AWS service, these help to deliver content across global locations?
 Ans. True

 Can you ship portable hard drives to AWS for them to import?
 Ans: True

 Which statements are true about ELB?
 Ans:
 Is an Elastic Load Balancer
 Automatically distributes incoming application traffic across multiple Amazon EC2 instances
 Detects unhealthy instances within a pool and automatically reroutes traffic
 Provide high availability
 Cannot be used across AWS Regions

Which three statements describe EBS?
Ans:
Very very fast
Within single data centre
Not accessible from the net

What is RDS Automatic Host Replacement?
Ans: Automatically replace the compute instance powering your deployment in the event of a hardware failure.

What cli tools does AWS provide
Ans: All three - AWS CLI, AWS Tools for Windows PowerShell, Amazon EC2 CLI



The annual cost of a high utilization reserved usage depends on the number of hours used?
Ans : false

AES Encrpts to what level?
Ans: 128,192,256

Which three statements describe S3?
Ans:
Very fast
Within multiple data centres
Accessible from the net

Explain how the buffer is used in Amazon web services
Ans: The buffer is used to make the system more robust to manage traffic or load by synchronizing different component.  Usually, components receive and process the requests in an unbalanced way, With the help of buffer, the components will be balanced and will work at the same speed to provide faster services

--------------------------------------------------------------------------------------------------

Concepts from udemy:

• If you are using putty from your windows machine to connect to AWS EC2 instance, you will need to login with normal username but not password. Instead, you need a key-pair file (ppk file)
• Amazon EC2 uses public key cryptography to encrypt n decrypt login information. Public key - to encrypt a piece of data, then the recepient uses private key to decrypt the data. The public and private keys are called key-pair.
• By default, in windows while using putty tool, when you export a key, it exports in a .pem format. Use putty key generator to convert pem to ppk format.
• You can use WinSCP or MobaxTerm to SSH (Secure Shell) to EC2 instance.
• Regions means data centers in a particular geographical location of the world.
• Please note that Key-pair generated by you is only applicable for a particular region. For ex: If you generated a key-pair when selected region in N.Virginia, then u cannot use same key-pair when u r working with another region.
• So, let's generate a key-pair first:
• login to aws mgmt console, select a region (ex: N.Virginia (default)), Go to Services > Key-Pair > Create key-pair and give a name (ex: XaaS). So when you do that, u will notice a XaaS.pem file is downloaded automatically on your local machine.
• Now, to convert this pem to ppk file, go to PuttyGen > Load existing private key (.pem file) > Save private key > PuttyGen asks the location where to save. You can give the same name 'Xaas'. So a new file XaaS.ppk is created.
• So in order to login to an EC2 instance now, u will need to use this ppk (putty private key) file if you are going to use putty.
• 
  
• FYI - Storage - HDD/SSD. HDD - Legacy version, SSD (Solid State Drives) are the current running hard drives.
• 
  
• AZ represents one or more data centers (DC). Each DC with redundant power, networking and connectivity, housed in separate facilities.
• And inside a DC, you will find a lot of servers, networking, storage , security, LB equipments , etc. So lot of stuff
• Running ur apps in multiple AZs - gives client HA, Fault tolerance and scalability.
• Region - is a physical location in the world which contains of multiple AZs.
• All regions r completely isolated from each other.
• Now, regions r isolated from one another, AZs are also isolated from one another BUT the AZs lying in same region are connected with low latency networks/links (high bandwidth links). And inside AWS region, u will have two or more AZs.
• Amazon CloudFront is a web service that speeds up distribution of your static and dynamic web content. CloudFront delivers your content through a worldwide network of data centers called edge locations. When a user requests content that you're serving with CloudFront, the user is routed to the edge location that provides the lowest latency (time delay), so that content is delivered with the best possible performance.
• 
  
• Regional edge caches are CloudFront locations that are deployed globally, close to your viewers. They're located between your origin server and the POPs—global edge locations that serve content directly to viewers. As objects become less popular, individual POPs might remove those objects to make room for more popular content. Regional edge caches have a larger cache than an individual POP, so objects remain in the cache longer at the nearest regional edge cache location.
• 
  
• Set AWS Billing Alarm
• To allow/enable AWS to send you bills of your monthly usage on your email, go to AWS Mgmt console > My Account > Bills. Check the 'Receive billing alerts' checkbox and click 'Save Preferences'.
• Go to CloudWatch > Alarms > Create Alarm.
• Select Metric as USD (US Dollars)
• Enter the amount (in USD) if exceeds above: __ . Enter for example 10 dollars.
• Enter email on which notification is to be sent: ____ 
• Click 'Create Alarm'.
• 
  
• Each API call done by AWS is logged in AWS CloudTrail.
• IAM is a global AWS service, irrespective of the region you select or belong to.
• 
  
• IAM Configuration:
• It's a five step process:
• 1) Login with your root account/credentials to your AWS account
• 2) Enable MFA (MultiFactor Authentication) by enabling RSA token (hw or s/w) based authentication through code/OTP
• 3) Create users by selecting following options:
• Defining user name/id 
• Auto Generate pwd for them or custom pwd
• Enable access key/secret key generation for them for programmatic access
• Allow AWS Mgmt console access to them or not.
• Reset pwd on first time login for user, etc
• Then Assign permissions to them - by either adding them to group OR attaching existing policy (json) to the user OR copy permissions from existing user
• 
  
• 4) Use groups to assign permissions: (continuation from steps of point#3)
• If no group is defined, Create group for example: AdminAccessGroup, and add user to it. When you enter 'AdminAccessGroup', AWS will show the list of existing group permissions (JSONs) which you want to inherit for the group u r creating. Select 'AdministrativeAccess'. So all admin privileges belonging to that group will be inherited by ur custom group. And users added to this gorup will have admin access.
• Once user is created, u will be shown access key id and secret key (encrypted) and password for the user1 you created.
• 5) Apply IAM password policy
• It will ask you to define your password policy like min 1 special char, min 1 uppercase, min 1 number, enable pwd expiration after __ , allow users to change their own password , prevent password reuse, min pwd length, etc
• So, once these 5 steps are done, you will see in IAM page 'Security Status', all steps shown in green.
• 
  

-----------------------------------------------------------------------------------------------

2021 Learning - through other blogs, tutorials and videos

----------------------------------------------------------------------------------------------

IaaS – U don’t own any hardware.  Ex: AWS, Azure, GCP. If u install MS OS, u have to upgrade it or manage it manually.

Paas – Everything is done by service provider. I am owning my data and the application. Here, u just order it ex: give me Oracle server. It can be on Linux, MS os, etc but u r not responsible for managing it.

SaaS – U don’t own anything. Ex: Gmail.

 

All above services are free for 1^st year.

 

AWS Ground Station – Amazon has collaborated with NASA. That is, u can have ur own satellite service developed (may be flying ur own drone).

CloudTrail – audit trail for ur cloud usage.

CloudWatch – logs

CloudFormation – template

 

GCP – Google Cloud Platform

Register for exam at:

https://www.aws.training/SignIn

 

Study reference: 1) Udemy course 2) Whitepapers (optional).  3) Sample questions and dumps

Passing score – 700 out of 1000

Exam Time – 90 mins

 

Course:

·         Cloud concepts

·         Billing and report

Multi-choice and mult-select answers are possible

 

AWS charges u for mostly 3 things:

·         Compute (EC2 services) – CPU, RAM and Duration

·         Storage – quantity of data stored or allocated.

·         Outbound data transfer (not for uploading in their cloud like S3, but charges will be incurred if u pull/fetch data from AWS (mostly S3) using AWS services) i.e data leaving the AWS region

Total regions as per 2020 – 25 regions

 

AWS Global Infrastructure:

·         Made up of Regions (geographical area)

·         Each region has 2 or more AZ

·         AZs are isolated from other regions

·         AZs are physically separate and isolated from each other

·         AZs are basically one or more data centers.

·         Each AZ is designed as an independent failure zone.

·         Local zones – it’s a way to bring resources closer to users. Local zones place compute, storage, db, and other services closer to end users. Its an extension to a region where u can run your latency sensitive applications.

·         Edge locations – are used by AWS CloudFront services for CDN (Content Delivery Network). So they cache data in diff parts of the world. There are more edge locations than regions.

·         Regional Edge Caches sit between your CloudFront origin servers and Edge locations. REC has larger cache-width than each of the individual edge locations.

 

AWS Shared Responsibility Model:

 

 

 

 

AWS Public and Private Services:

Public services are those services which can be connected directly thru internet, and have specific endpoints.

 

Private services are not exposed to internet. But u can choose to give them a public IP address for exposing.

 

6 advantages of Cloud:

1.      Trade capital expense for variable expense

2.      Benefit from massive economies of scale

3.      Stop guessing about capacity

4.      Increase speed and agility – quickly scale in and out the resource as per need

5.      Stop spending money running and maintaining data centers

6.      Go global in minutes

 

Global services: IAM, S3 (not specific to region)

Pricing model:

1.      Dedicated instances

2.      Spot instances

3.      On-demand instances

4.      Reservations

On-demand is the best option when you need the most flexibility. There are no long-term commitments or upfront payments

 

AWS region is composed of atleast 2 or more AZ

 

Which AWS policy defines the prohibited uses of the AWS Cloud? – AWS Acceptable Use Policy

CaaS à Computing as a Service is not a cloud service model. However, Containers as a Service (CaaS) is a subset of PaaS with a focus on containers

 

 

AWS Wavelength

AWS Wavelength enables developers to build applications that deliver single-digit millisecond latencies to mobile devices and end-users.

AWS developers can deploy their applications to Wavelength Zones, AWS infrastructure deployments that embed AWS compute and storage services within the telecommunications providers’ datacenters at the edge of the 5G networks, and seamlessly access the breadth of AWS services in the region.

AWS Wavelength brings AWS services to the edge of the 5G network, minimizing the latency to connect to an application from a mobile device

 

 

AWS Outposts

AWS Outposts bring native AWS services, infrastructure, and operating models to virtually any data center, co-location space, or on-premises facility.

You can use the same AWS APIs, tools, and infrastructure across on-premises and the AWS cloud to deliver a truly consistent hybrid experience.

AWS Outposts is designed for connected environments and can be used to support workloads that need to remain on-premises due to low latency or local data processing needs.

 

Pricing & Billing

Free tier includes offers that expire after 12 months and offers that never expire.

Free services include:

• Amazon VPC.
• Elastic Beanstalk (but not the resources created).
• CloudFormation (but not the resources created).
• Identity Access Management (IAM).
• Auto Scaling (but not the resources created).
• OpsWorks.
• Consolidated Billing.

Fundamentally charges include:

1.   Compute.

2.   Storage.

3.   Data out.

Amazon EC2 pricing

EC2 pricing is based on:

• Clock hours of server uptime.
• Instance configuration.
• Instance type.
• Number of instances.
• Load balancing.
• Detailed monitoring.
• Auto Scaling (resources created).
• Elastic IP addresses (charged if allocated but not used).
• Operating systems and software packages.

 

 

There are several pricing model for AWS services, these include:

On Demand:

• Means you pay for compute or database capacity with no long-term commitments of upfront payments.
• You pay for the computer capacity per hour or per second (Linux only, and applies to On-Demand, Reserved and Spot instances).
• Recommended for users who prefer low cost and flexibility without upfront payment or long-term commitments.
• Good for applications with short-term, spiky, or unpredictable workloads that cannot be interrupted.

Dedicated Hosts:

• A dedicated host is an EC2 servers dedicated to a single customer.
• Runs in your VPC.
• Good for when you want to leverage existing server-bound software licences such as Windows Server, SQL Server, and SUSE Linux Enterprise Server.
• Also good for meeting compliance requirements.

Dedicated Instances:

• Dedicated Instances are Amazon EC2 instances that run in a VPC on hardware that’s dedicated to a single customer.
• Dedicated instances are physically isolated at the host hardware level from instances that belong to other AWS accounts.
• Dedicated instances may share hardware with other instances from the same AWS account that are not Dedicated instances.

Spot Instances:

• Purchase spare computing capacity with no upfront commitment at discounted hourly rates.
• Provides up to 90% off the On-Demand price.
• Recommended for applications that have flexible start and end times, applications that are only feasible at very low compute prices, and users with urgent computing needs for a lot of additional capacity.
• In the old model Spot instances were terminated because of higher competing bids, in the new model this does not happen but instances still may be terminated (with a 2 minute warning) when EC2 needs the capacity back – note: the exam may not be updated to reflect this yet.

 

Savings Plans:

• Commitment to a consistent amount of usage (EC2 + Fargate + Lambda); Pay by $/hour; 1 or 3-year commitment.

Reservations:

• Reserved instances provide significant discounts, up to 75% compared to On-Demand pricing, by paying for capacity ahead of time.
• Provide a capacity reservation when applied to a specific Availability Zone.
• Good for applications that have predictable usage, that need reserved capacity, and for customers who can commit to a 1 or 3-year term.

Reservations apply to various services, including:

• Amazon EC2 Reserved Instances.
• Amazon DynamoDB Reserved Capacity.
• Amazon ElastiCache Reserved Nodes.
• Amazon RDS Reserved Instances.
• Amazon RedShift Reserved Instances.

Reservation options include no upfront, partial upfront and all upfront.

Reservation terms are 1 or 3 years.

Amazon Simple Storage Service (S3) Pricing

Storage pricing is determined by:

• Storage class – e.g. Standard or IA.
• Storage quantity – data volume stored in your buckets on a per GB basis.
• Number of requests – the number and type of requests, e.g. GET, PUT, POST, LIST, COPY.
• Lifecycle transitions requests – moving data between storage classes.
• Data transfer – data transferred out of an S3 region is charged.

Amazon Glacier pricing

• Extremely low cost and you pay only for what you need with no commitments of upfront fees.
• Charged for requests and data transferred out of Glacier.
• “Amazon Glacier Select” pricing allows queries to run directly on data stored on Glacier without having to retrieve the archive. Priced on amount of data scanned, returned, and number of requests initiated.
• Three options for access to archives, listed in the table below:

 

 

 

AWS Snowball Pricing

Pay a service fee per data transfer job and the cost of shipping the appliance.

Each job allows use of Snowball appliance for 10 days onsite for free.

Data transfer in to AWS is free and outbound is charged (per region pricing).

Amazon Relational Database Service (RDS) Pricing

RDS pricing is determined by:

• Clock hours of server uptime – amount of time the DB instance is running.
• Database characteristics – e.g. database engine, size and memory class.
• Database purchase type – e.g. On-Demand, Reserved.
• Number of database instances.
• Provisioned storage – backup is included up to 100% of the size of the DB. After the DB is terminated backup storage is charged per GB per month.
• Additional storage – the amount of storage in addition to the provisioned storage is charged per GB per month.
• Requests – the number of input and output requests to the DB.
• Deployment type – single AZ or multi-AZ.
• Data transfer – inbound is free, outbound data transfer costs are tiered.
• Reserved Instances – RDS RIs can be purchased with No Upfront, Partial Upfront, or All Upfront terms. Available for Aurora, MySQL, MariaDB, Oracle and SQL Server.

 

Amazon CloudFront Pricing

CloudFront pricing is determined by:

• Traffic distribution – data transfer and request pricing, varies across regions, and is based on the edge location from which the content is served.
• Requests – the number and type of requests (HTTP or HTTPS) and the geographic region in which they are made.
• Data transfer out – quantity of data transferred out of CloudFront edge locations.
• There are additional chargeable items such as invalidation requests, field-level encryption requests, and custom SSL certificates.

AWS Lambda Pricing

Pay only for what you use and charged based on the number of requests for functions and the time it takes to execute the code.

Price is dependent on the amount of memory allocated to the function.

Amazon Elastic Block Store (EBS) Pricing

Pricing is based on three factors:

• Volumes – volume storage for all EBS volumes type is charged by the amount of GB provisioned per month.
• Snapshots – based on the amount of space consumed by snapshots in S3. Copying snapshots is charged on the amount of data copied across regions.
• Data transfer – inbound data transfer is free, outbound data transfer charges are tiered.

 

Amazon DynamoDB Pricing

Charged based on:

• Provisioned throughput (write).
• Provisioned throughput (read).
• Indexed data storage.
• Data transfer – no charge for data transfer between DynamoDB and other AWS services within the same region, across regions is charged on both sides of the transfer.
• Global tables – charged based on the resources associated with each replica of the table (replicated write capacity units, or rWCUs).
• Reserved Capacity – option available for a one-time upfront fee and commitment to paying a minimum usage level at specific hourly rates for the duration of the term. Additional throughput is charged at standard rates.

On-demand capacity mode:

• Charged for reads and writes
• No need to specify how much capacity is required
• Good for unpredictable workloads

Provisioned capacity mode:

• Specify number of reads and writes per second
• Can use Auto Scaling
• Good for predictable workloads
• Consistent traffic or gradual changes

AWS Support Plans

There are four AWS support plans available:

• Basic – billing and account support only (access to forums only).
• Developer – business hours support via email.
• Business – 24×7 email, chat and phone support.
• Enterprise – 24×7 email, chat and phone support.

Enterprise support comes with a Technical Account Manager (TAM).

Developer allows one person to open unlimited cases.

Business and Enterprise allow unlimited contacts to open unlimited cases.

Resource Groups and Tagging

Tags are key / value pairs that can be attached to AWS resources.

Tags contain metadata (data about data).

Tags can sometimes be inherited – e.g. resources created by Auto Scaling, CloudFormation or Elastic Beanstalk.

Resource groups make it easy to group resources using the tags that are assigned to them. You can group resources that share one or more tags.

Resource groups contain general information, such as:

• Region.
• Name.
• Health Checks.

And also specific information, such as:

• Public & private IP addresses (for EC2).
• Port configurations (for ELB).
• Database engine (for RDS).

AWS Organizations and Consolidated Billing

AWS organizations allows you to consolidate multiple AWS accounts into an organization that you create and centrally manage.

Available in two feature sets:

• Consolidated Billing.
• All features.

Includes root accounts and organizational units.

Policies are applied to root accounts or OUs.

Consolidated billing includes:

• Paying Account – independent and cannot access resources of other accounts.
• Linked Accounts – all linked accounts are independent.

Consolidated billing has the following benefits:

• One bill – You get one bill for multiple accounts.
• Easy tracking – You can track the charges across multiple accounts and download the combined cost and usage data.
• Combined usage – You can combine the usage across all accounts in the organization to share the volume pricing discounts and Reserved Instance discounts. This can result in a lower charge for your project, department, or company than with individual standalone accounts.
• No extra fee – Consolidated billing is offered at no additional cost.

Limit of 20 linked accounts (by default).

One bill for multiple AWS accounts.

Easy to track charges and allocate costs.

Volume pricing discounts can be applied to resources.

Billing alerts enabled on the Paying account include data for all Linked accounts (or can be created per Linked account).

Consolidated billing allows you to get volume discounts on all of your accounts.

Unused reserved instances (RIs) for EC2 are applied across the group.

CloudTrail is on a per account basis and per region basis but can be aggregated into a single bucket in the paying account.

Best practices:

• Always enable multi-factor authentication (MFA) on the root account.
• Always use a strong and complex password on the root account.
• The Paying account should be used for billing purposes only. Do not deploy resources into the Paying account

 

AWS Quick Starts à Similar to CloudFormation, but faster than that, and at the core, it uses CloudFormation only.

 

AWS Cost Calculators and Tools

• AWS Cost Explorer – enables you to visualize your usage patterns over time and to identify your underlying cost drivers.
• AWS Pricing Calculator – create cost estimates to suit your AWS use cases.

AWS Cost Explorer

The AWS Cost Explorer is a free tool that allows you to view charts of your costs.

You can view cost data for the past 13 months and forecast how much you are likely to spend over the next three months.

Cost Explorer can be used to discover patterns in how much you spend on AWS resources over time and to identify cost problem areas.

Cost Explorer can help you to identify service usage statistics such as:

• Which services you use the most.
• View metrics for which AZ has the most traffic.
• Which linked account is used the most.

AWS Pricing Calculator

AWS Pricing Calculator is a web based service that you can use to create cost estimates to suit your AWS use cases.

AWS Pricing Calculator is useful both for people who have never used AWS and for those who want to reorganize or expand their usage.

AWS Pricing Calculator allows you to explore AWS services based on your use cases and create a cost estimate.

AWS Cost & Usage Report

Publish AWS billing reports to an Amazon S3 bucket.

Reports break down costs by:

• Hour, day, month, product, product resource, tags.

Can update the report up to three times a day.

Create, retrieve, and delete your reports using the AWS CUR API Reference.

AWS Price List API

Query the prices of AWS services.

Price List Service API (AKA the Query API) – query with JSON.

AWS Price List API (AKA the Bulk API) – query with HTML.

Alerts via Amazon SNS when prices change.

AWS Budgets

Used to track cost, usage, or coverage and utilization for your Reserved Instances and Savings Plans, across multiple dimensions, such as service, or Cost Categories.

Alerting through event-driven alert notifications for when actual or forecasted cost or usage exceeds your budget limit, or when your RI and Savings Plans’ coverage or utilization drops below your threshold.

Create annual, quarterly, monthly, or even daily budgets depending on your business needs.

 

IAM:

 

 

Users, groups, Roles and Policies:

 

Root user – full permissions

Individual users – initially don’t have any permissions, need to give them permissions. After that they can login using user/pwd from AWS mgmt. console or access/secret key from CLI.

Users have ARN created with a longer id and type (:user here in snapshot below)

 

 

IAM Role:

IAM Policies:

 

 

 

MFA: Physical (Chargeable) or Virtual (mostly free)

 

 

NOTE:

Max 5 VPC Elastic IP addresses are allowed to be created for each AWS account

Max 100 S3 buckets can be created per each AWS account

Is the property of broadcast/multicast supported by VPC – No

Diff types of instances à

·         General Purpose

·         Computer Optimized

·         Memory Optimized

·         Storage Optimized

·         Accelerated Computing

Issues one can face while connecting to an EC2 instance à

·         Connection timed out

·         User key not recognized by server

·         Host key not found, permission denied

·         Server refused our key

·         Error using MindTerm on Safari browser

·         Error using Max OS X RDP Client.

Default storage class in S3 à Standard frequently accessed

Snowball à It is a data transfer option. Its used source appliances to a large amount of data into and out of AWS. With snowball, u can transfer massive amount of data from one place to another. It helps u to reduce networking costs.

 

Redshift à It’s a big data warehouse product. Its fast, powerful and fully managed data warehouse in the cloud.

Horizontal and vertical scaling à

How many subnets can u have per VPC à 200

Ec2 officially launched in 2006

Types of AMIs provided by AWS:

·         Instance store backed

·         EBS backed

Lambda – it’s a amazon compute service which allows u to run code in AWS cloud without managing servers.

ElasticCache – web service which makes it easy to deploy, scale and store data in cloud.

SimpleDB –

Name the AWS service exists only to redundantly cache data & images à Edge locations

GeoRestriction in CloudFront à helps u to prevent users of specific geographic locations from accessing content which u are distributing thru CloudFront web distribution.

Amazon EMR – EMR is a survived cluster stage which helps u to interpret the working of data structures before the intimation. Apache Hadoop and Apache Spark on AWS helps u to investigate a large amount of data. U can prepare data for the analytics goals and marketing intellect workloads using Apache Hive and using other relevant open source designs.

Boot time taken for an instance stored backed AMI à  < 5 mins

Do u need Internet Gateway to use peering connections – Yes

How to connect EBS volume to multiple instances – No, we cannot connect EBS volume to multiple instances. But, u can connect multiple EBS volumes to a single instance.

Types of cloud services – SaaS, PaaS, IaaS, DaaS (Data as a Service), CaaS (Container as a Service)

Diff between AMI and EC2 –

Diff types of LB –

·         Classic LB

·         Application LB

In which situation u will select provisioned IOPS over Std RDS storage – if u want to perform batch-related workloads.

Imp features of Amazon cloud search

·         Boolean searches

·         Prefix searches

·         Range searches

·         Entire text searches

·         AutoComplete advice

Can vertically scaling allowed in Ec2 à Yes

What is the use of lifecyle hooks in autoscaling à Lifecycle hooks are used for autoscaling to put an additional wait time to a scale in or scale out event.

Storage classes avl in S3

·         Standard

·         Standard Infrequent Access

·         Standard Redundancy Storage

·         Glacier

DB Engines used in AWS RDS:

·         MS-SQL DB

·         MariaDB

·         MySQL DB

·         Oracle DB

·         PostGres DB

 

 

AWS Organizations:

It has a management account below the root user.

SCP (Service Control Policies) control the maximum available permissions for the entire account.

Users in management account are not restricted. They can launch instances.

In below example, we have restricted dev users (belonging to an OU), to launch only t2.micro instances so that they don’t spend much on high level instances.

 

NOTE: SCPs do not grant ANY permissions, they control the available permissions i.e they define what is allowed in that particular account.

 

Password policy:

·         AWS gives u option to apply ur custom pwd policy.

·         Default pwd policy is:

o   Min 8 chars

o   Min 3 of the following à uppercase, lowercase, numbers, few special chars

o   Must not be identical to ur aws a/c name or email address.

 

IAM Revision:

 

Exam q:

An access key ID and secret access key is associated with which IAM entity? à User

 

What is the main credential for an AWS root account? à Email address used to create account

ARN à Amazon Resource Name

Which principle should be used when assigning permissions to users or groups? – Least priviledge

Which IAM entity can be used to delegate permissions?

            User/Group/Role/policy . Answer is Role.

By default, users are created with what permissions?

            No permissions/Minimal permissions/Full permissions/No access to AMC – Answer is No permissions.

 

Cheat Sheet points:

IAM is not used for application-level authentication

 

It is a best practice to avoid using the root account for anything other than billing

Power user access allows all permissions except the management of groups and users in IAM

Temporary security credentials consist of the AWS access key ID, secret access key, and security token

IAM can assign temporary security credentials to provide users with temporary access to services/resources.

Authentication Methods

Console password:

• A password that the user can enter to sign in to interactive sessions such as the AWS Management Console.
• You can allow users to change their own passwords.
• You can allow selected IAM users to change their passwords by disabling the option for all users and using an IAM policy to grant permissions for the selected users.

Access Keys:

• A combination of an access key ID and a secret access key.
• You can assign two active access keys to a user at a time.
• These can be used to make programmatic calls to AWS when using the API in program code or at a command prompt when using the AWS CLI or the AWS PowerShell tools.
• You can create, modify, view or rotate access keys.
• When created IAM returns the access key ID and secret access key.
• The secret access is returned only at creation time and if lost a new key must be created.
• Ensure access keys and secret access keys are stored securely.
• Users can be given access to change their own keys through IAM policy (not from the console).
• You can disable a user’s access key which prevents it from being used for API calls.

Server certificates:

• SSL/TLS certificates that you can use to authenticate with some AWS services.
• AWS recommends that you use the AWS Certificate Manager (ACM) to provision, manage and deploy your server certificates.
• Use IAM only when you must support HTTPS connections in a region that is not supported by ACM.

IAM Users

An IAM user is an entity that represents a person or service.

Can be assigned:

• An access key ID and secret access key for programmatic access to the AWS API, CLI, SDK, and other development tools.
• A password for access to the management console.

By default, users cannot access anything in your account.

The account root user credentials are the email address used to create the account and a password.

The root account has full administrative permissions and these cannot be restricted.

Best practice for root accounts:

• Don’t use the root user credentials.
• Don’t share the root user credentials.
• Create an IAM user and assign administrative permissions as required.
• Enable MFA.

IAM users can be created to represent applications and these are known as “service accounts”.

You can have up to 5000 users per AWS account.

Each user account has a friendly name and an ARN which uniquely identifies the user across AWS.

A unique ID is also created which is returned only when you create the user using the API, Tools for Windows PowerShell or the AWS CLI.

The Access Key ID and Secret Access Key are not the same as a password and cannot be used to login to the AWS console

The Access Key ID and Secret Access Key can only be used once and must be regenerated if lost.

Groups - You cannot nest groups (groups within groups).

Roles are created and then “assumed” by trusted entities and define a set of permissions for making AWS service requests.

With IAM Roles you can delegate permissions to resources for users and services without using permanent credentials (e.g. user name and password).

IAM users or AWS services can assume a role to obtain temporary security credentials that can be used to make AWS API calls.

You can delegate using roles.

There are no credentials associated with a role (password or access keys).

IAM users can temporarily assume a role to take on permissions for a specific task.

A role can be assigned to a federated user who signs in using an external identity provider.

Temporary credentials are primarily used with IAM roles and automatically expire.

Roles can be assumed temporarily through the console or programmatically with the AWS CLI, Tools for Windows PowerShell or API.

 

IAM roles with EC2 instances:

• IAM roles can be used for granting applications running on EC2 instances permissions to AWS API requests using instance profiles.
• Only one role can be assigned to an EC2 instance at a time.
• A role can be assigned at the EC2 instance creation time or at any time afterwards.
• When using the AWS CLI or API instance profiles must be created manually (it’s automatic and transparent through the console).
• Applications retrieve temporary security credentials from the instance metadata.

Role Delegation:

• Create an IAM role with two policies:
• Permissions policy – grants the user of the role the required permissions on a resource.
• Trust policy – specifies the trusted accounts that are allowed to assume the role.
• Wildcards (*) cannot be specified as a principal.
• A permissions policy must also be attached to the user in the trusted account.

Policies

Policies are documents that define permissions and can be applied to users, groups and roles.

Policy documents are written in JSON (key value pair that consists of an attribute and a value).

All permissions are implicitly denied by default.

The most restrictive policy is applied.

The IAM policy simulator is a tool to help you understand, test, and validate the effects of access control policies.

The Condition element can be used to apply further conditional logic.

STS

The AWS Security Token Service (STS) is a web service that enables you to request temporary, limited-privilege credentials for IAM users or for users that you authenticate (federated users).

Temporary security credentials work almost identically to long-term access key credentials that IAM users can use, with the following differences:

• Temporary security credentials are short-term.
• They can be configured to last anywhere from a few minutes to several hours.
• After the credentials expire, AWS no longer recognizes them or allows any kind of access to API requests made with them.
• Temporary security credentials are not stored with the user but are generated dynamically and provided to the user when requested.
• When (or even before) the temporary security credentials expire, the user can request new credentials, as long as the user requesting them still has permission to do so.

Advantages of STS are:

• You do not have to distribute or embed long-term AWS security credentials with an application.
• You can provide access to your AWS resources to users without having to define an AWS identity for them (temporary security credentials are the basis for IAM Roles and ID Federation).
• The temporary security credentials have a limited lifetime, so you do not have to rotate them or explicitly revoke them when they’re no longer needed.
• After temporary security credentials expire, they cannot be reused (you can specify how long the credentials are valid for, up to a maximum limit)

ederation (typically AD):

• Uses SAML 2.0.
• Grants temporary access based on the users AD credentials.
• Does not need to be a user in IAM.
• Single sign-on allows users to login to the AWS console without assigning IAM credentials.

Federation with Mobile Apps:

• Use Facebook/Amazon/Google or other OpenID providers to login.

Cross Account Access:

• Lets users from one AWS account access resources in another.
• To make a request in a different account the resource in that account must have an attached resource-based policy with the permissions you need.
• Or you must assume a role (identity-based policy) within that account with the permissions you need.

 

Server Virtualization:

 

In case of failure of physical hardware/server, you can also move the VM to another physical server/machine:

 

EC2:

 

EC2 in a public subnet:

 

 

Launch 2 instances – 1 Linux and 1 Windows (both free versions with AMI type also as t2.micro which is free, to avoid costs).

Once u have selected the Linux EC2 instance, and u click Connect, u can connect to it with following options:

 

For connecting to Windows EC2 instance:

After copying the above details, open RDP from your windows machine and connect with above details.

 

Created instances – can be terminated to prevent costs. EC2 instance with All data (along with EBS) will be deleted.

You can also ‘STOP’ the instances instead of ‘TERMINATE’, but stopping will just stop them. Although stopped instances won’t charge u , but the EBS related to it, and the data (for ex: S3 if any binded to it) will add up to the cost. So its recommended that if u created the ec2 instance just for practicing, terminate it (permanently delete) instead of stopping it.

 

Amazon EC2 User Data:

It shows (screenshot below) what code is executed when u launch EC2 instance.

User data gives u the capability to run commands when u start ur instance.

 

EC2 Metadata:

Metadata is information about ur EC2 instance. U can find additional info related to ur instance.

 

Accessing S3 from EC2 with IAM role:

1.      Create an EC2 instance.

2.      Create an S3 bucket.

3.      Connect to the EC2 instance using EC2 instance connect, and type aws s3 ls

4.      It will not allow, as ‘aws configure’ is not run previously

5.      Now, for ‘aws configure’, u need to supply access key and secret key of the user u are trying to login with.

6.      Go to the user in AMC. Click ‘Get Access Key’ button, which will give u access key, secret key (only one time read), and allow u to download the csv containing it.

7.      Now run ‘aws configure’ from the EC2 instance connect window, enter the access key, scret key, region, etc and then run ‘aws s3 ls’ which will show ur s3 bucket name

8.      Also ‘aws s3 ls s3://bucket-name’ will list the files that u uploaded in ur bucket.

9.      But this approach has a disadvantage. If u go to “~/.aws” and run ls cmd, u will see a config and credentials file here. The credentials file has ur access key and scret key stored in plain txt which is not secure. And if u delete this credentials file, then again ur “aws s3 ls” commands will not run. That is, the details which u entered with “aws configure” are gone.

10.  So better approach is to use IAM Role (assume Role) approach with IAM, which is an AWS best practice.

11.  So, go to Roles in AMC. Create role, and select ‘AmazonS3ReadOnlyAccess’ role (least permission) considering that initially readonly role is sufficient. And then try connecting S3 from your Ec2.

Then go to actions for the selected Ec2:

 

Modify IAM Role:

Select S3Readonly role, and Save.

Now, go to the Ec2 instance connect console, and try again:

12.  You will see that u r able to connect to ur s3 bucket and also able to see the files here.

13.  So this approach is using the assumeRole policy of AWS and is much secure compared to the access key/secret key approach.

14.   

AWS Batch:

 

 

Amazon LightSail:

·         It’s a compute service similar to EC2.

·         Difference –

o   LightSail has a much simpler interface/UI for those who don’t have experience with technology.

o   Has less features than EC2

o   Cost is less than EC2.

·        

 

Docker containers and microservices:

AWS ECS (Amazon Elastic Container Services) is the AWS service that allows u to run docker containers.

Diff between VM (with hypervisor above the physical server) and containerization:

 

Container does not have its own OS. It uses OS of the underlying host.

 

Microservice architecture instead of Monolithic architecture:

Microservice with containers:

Here, the UI ms can run in an nginx container, shipping ms can run in a python container and order ms can run in a java container. (screenshot below).

Also, microservices can be spread across multiple hosts (servers) with containers for redundancy and fault tolerance. And easily movable from one server to another.

 

ECS (Elastic Container Service) – Service that enables us to run docker containers on AWS.

Main points:

·         It can run across multiple AZs.

·         An ECS cluster is a logical grouping of tasks or services

·         Here, a task is a docker container (in ECS, we call it a task rather than a container)

·        

·         Task has its task definition where it has the info of image name, ports, etc.

·         Other task u see has the image registry (Amazon ECR-Elastic Container Registry), its similar to dockerhub. The docker images can be stored in Amazon ECR.

·         So when one task is run, it pulls the images from this ECR and the application is run.

·          

·         Now there is an ECS service too, which is used to maintain the desired count of tasks which shud be running (here 4).

·        

There are 2 types of ECS:

1.      EC2 launch type

2.      Fargate launch type – u don’t have to worry about managing ec2 instances or scaling.

 

Exam Cram:

AWS EC2 User data and metadata are not encrypted.

 

 

Cheat sheets:

An Amazon Machine Image (AMI) is a special type of virtual appliance that is used to create a virtual machine within the Amazon Elastic Compute Cloud (“EC2”).

An AMI includes the following:

• One or more EBS snapshots, or, for instance-store-backed AMIs, a template for the root volume of the instance (for example, an operating system, an application server, and applications).
• Launch permissions that control which AWS accounts can use the AMI to launch instances.
• A block device mapping that specifies the volumes to attach to the instance when it’s launched.

AMIs come in three main categories:

• Community AMIs – free to use, generally you just select the operating system you want.
• AWS Marketplace AMIs – pay to use, generally come packaged with additional, licensed software.
• My AMIs – AMIs that you create yourself.

 

 

Metadata and User Data:

• User data is data that is supplied by the user at instance launch in the form of a script.
• Instance metadata is data about your instance that you can use to configure or manage the running instance.
• User data is limited to 16KB.
• User data and metadata are not encrypted.
• Instance metadata is available at http://169.254.169.254/latest/meta-data.

The Instance Metadata Query tool allows you to query the instance metadata without having to type out the full URI or category names.

Pricing

On-demand:

• Good for users that want the low cost and flexibility of EC2 without any up-front payment or long term commitment.
• Applications with short term, spiky, or unpredictable workloads that cannot be interrupted.
• Applications being developed or tested on EC2 for the first time.

Reserved:

• Applications with steady state or predictable usage.
• Applications that require reserved capacity.
• Users can make up-front payments to reduce their total computing costs even further.
• Standard Reserved Instances (RIs) provide up to 75% off on-demand price.
• Convertible RIs provide up to 54% off on-demand price – provides the capability to change the attributes of the RI as long as the exchange results in the creation of RIs of equal or greater value.
• Scheduled RIs are available to launch within the time window you reserve. This option allows you to match your capacity reservation to a predictable recurring schedule that only requires a fraction of a day, a week, or a month.

Spot:

• Applications that have flexible start and end times.
• Applications that are only feasible at very low compute prices.
• Users with an urgent need for a large amount of additional compute capacity.
• If Amazon terminate your instances you do not pay, if you terminate you pay for the hour.

Dedicated hosts:

• Physical servers dedicated just for your use.
• You then have control over which instances are deployed on that host.
• Available as On-Demand or with Dedicated Host Reservation.
• Useful if you have server-bound software licences that use metrics like per-core, per-socket, or per-VM.
• Each dedicated host can only run one EC2 instance size and type.
• Good for regulatory compliance or licensing requirements.
• Predictable performance.
• Complete isolation.
• Most expensive option.
• Billing is per host.

Dedicated instances:

• Virtualized instances on hardware just for you.
• Also uses physically dedicated EC2 servers.
• Does not provide the additional visibility and controls of dedicated hosts (e.g. how instance are placed on a server).
• Billing is per instance.
• May share hardware with other non-dedicated instances in the same account.
• Available as On-Demand, Reserved Instances, and Spot Instances.
• Cost additional $2 per hour per region.

Savings Plans:

• Savings Plans is a flexible pricing model that provides savings of up to 72% on your AWS compute usage.
• This pricing model offers lower prices on Amazon EC2 instances usage, regardless of instance family, size, OS, tenancy or AWS Region.
• Also applies to AWS Fargate and AWS Lambda usage.

Instance Types

Amazon EC2 provides a wide selection of instance types optimized to fit different use cases.

Instance types comprise varying combinations of CPU, memory, storage, and networking capacity and give you the flexibility to choose the appropriate mix of resources for your applications.

Each instance type includes one or more instance sizes, allowing you to scale your resources to the requirements of your target workload.

The table below provides an overview of the different EC2 instance types:

 

Amazon Elastic Container Service (ECS)

Amazon Elastic Container Service (ECS) is another product in the AWS Compute category. It provides a highly scalable, high performance container management service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances.

Amazon ECS eliminates the need for you to install, operate, and scale your own cluster management infrastructure

Using API calls you can launch and stop container-enabled applications, query the complete state of clusters, and access many familiar features like security groups, Elastic Load Balancing, EBS volumes and IAM roles.

Amazon ECS can be used to schedule the placement of containers across clusters based on resource needs and availability requirements.

An Amazon ECS launch type determines the type of infrastructure on which your tasks and services are hosted.

There are two launch types and the table below describes some of the differences between the two launch types:

 

The Elastic container registry (ECR) is a managed AWS Docker registry service for storing, managing and deploying Docker images.

There is no additional charge for Amazon ECS. You pay for AWS resources (e.g. EC2 instances or EBS volumes) you create to store and run your application.

Amazon ECR is integrated with Amazon EC2 Container Service (ECS).

With Amazon ECR, there are no upfront fees or commitments. You pay only for the amount of data you store in your repositories and data transferred to the Internet.

 

AWS Lambda

AWS Lambda is a serverless computing technology that allows you to run code without provisioning or managing servers.

AWS Lambda executes code only when needed and scales automatically.

You pay only for the compute time you consume (you pay nothing when your code is not running).

Benefits of AWS Lambda:

• No servers to manage.
• Continuous scaling.
• Subsecond metering.
• Integrates with almost all other AWS services.

Primary use cases for AWS Lambda:

• Data processing.
• Real-time file processing.
• Real-time stream processing.
• Build serverless backends for web, mobile, IOT, and 3rd party API requests.

 

 

 

·         Amazon Lightsail provides developers compute, storage, and networking capacity and capabilities to deploy and manage websites, web applications, and databases in the cloud.

·         Amazon Lightsail includes everything you need to launch your project quickly – a virtual machine, SSD-based storage, data transfer, DNS management, and a static IP.

·         Amazon Lightsail provides preconfigured virtual private servers (instances) that include everything required to deploy and application or create a database.

·         The underlying infrastructure and operating system is managed by Amazon Lightsail.

·         Best suited to projects that require a few dozen instances or fewer.

·         Provides a simple management interface.

·         Good for blogs, websites, web applications, e-commerce etc.

·         Can deploy load balancers and attach block storage

·         Public API.

·         Limited to 20 Amazon Lightsail instances, 5 static IPs, 3 DNS zones, 20 TB block storage, 40 databases, and 5 load balancers per account.

·         Up to 20 certificates per calendar year.

·         Can connect to each other and other AWS resources through public Internet and private (VPC peering) networking.

·         Application templates include WordPress, WordPress Multisite, Drupal, Joomla!, Magento, Redmine, LAMP, Nginx (LEMP), MEAN, Node.js, and more.

·         Amazon Lightsail currently supports 6 Linux or Unix-like distributions: Amazon Linux, CentOS, Debian, FreeBSD, OpenSUSE, and Ubuntu, as well as 2 Windows Server versions: 2012 R2 and 2016

 

Amazon LightSail Databases

Amazon Lightsail databases are instances that are dedicated to running databases.

An Amazon Lightsail database can contain multiple user-created databases, and you can access it by using the same tools and applications that you use with a stand-alone database.

Amazon Lightsail managed databases provide an easy, low maintenance way to store your data in the cloud.

Amazon Lightsail manages a range of maintenance activities and security for your database and its underlying infrastructure.

Amazon Lightsail automatically backs up your database and allows point in time restore from the past 7 days using the database restore tool.

Amazon Lightsail databases support the latest major versions of MySQL. Currently, these versions are 5.6, 5.7, and 8.0 for MySQL.

Amazon Lightsail databases are available in Standard and High Availability plans.

High Availability plans add redundancy and durability to your database, by automatically creating standby database in a separate Availability Zone.

Amazon Lightsail is very affordable.

Amazon Lightsail plans are billed on an on-demand hourly rate, so you pay only for what you use.

For every Amazon Lightsail plan you use, we charge you the fixed hourly price, up to the maximum monthly plan cost

 

 

 

AWS Elastic Beanstalk

AWS Elastic Beanstalk is the fastest and simplest way to get web applications up and running on AWS.

Developers simply upload their application code and the service automatically handles all the details such as resource provisioning, load balancing, auto-scaling, and monitoring.

Elastic Beanstalk is ideal if you have a PHP, Java, Python, Ruby, Node.js, .NET, Go, or Docker web application.

Elastic Beanstalk uses core AWS services such as Amazon EC2, Amazon Elastic Container Service (Amazon ECS), Auto Scaling, and Elastic Load Balancing to easily support applications that need to scale to serve millions of users.

AWS Batch

AWS Batch enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS.

AWS Batch dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted

 

Storage Services:

AWS Storage services are classified as – S3 (Object based storage), EBS(Elastic Block Storage , attached to EC2 instances), EFS (Elastic File System – which u can also mount to EC2 instances)

Block based storage:

Your PC’s hard disk drive is a good example of it.

 

Network Attached Storage (NAS):

 

Object based storage:

 

Comparison of all 3 storage systems:

From AWS perspective, we have:

For block storage – we have Amazon Elastic Block Store (EBS)

For file storage – we have Amazon Elastic File System (EFS)

For object storage – we have Amazon Simple Storage Service (S3)

 

When u launch an EC2 instance, u get a default volume attached to it called EBS. It’s the place where the OS and the other default data is stored persistently (until u terminate the EC2 instance).

EC2 instance can have one or more EBS volumes attached to it. But u cannot attach multiple EC2 instances to a single EBS volume.

Also, u cannot attach an EC2 instance to an EBS volume lying in diff AZ.

 

EBS SSD-Backed Volumes (Solid State Drives):

Gp2 and io1 are commonly used. (When we launch EC2 instance, gp2 is the default type of volume attached to that EC2).

Gp3 and io2 are new in market as of now (Oct 2021).

This SSDs are faster than HDDs, and SSD’s performance is higher compared to HDDs. Also cost is higher compared to HDDs.

Few common HDD volumes are as follows:

 

Facts:

 

How to take backups of your EBS volumes:

U can do it by taking snapshots. And u can also automate it (taking snapshot at regular intervals automatically – which is called DLM – Data Lifecycle Manager)

When u take a snapshot, it does not reside in that AZ, it is stored in S3.

Snapshots are incremental – i.e SnapA has dataX, SnapB has the difference/delta compared to SnapA, etc. (snapshot below for reference)

U can create a volume out of a snapshot, u can also create an AMI out of a snapshot.

 

 

 

DLM:

Now, in addition to EBS volumes, we can have another type of volumes that we can attach to our EC2 intances . It is EC2 instance store volumes.

Instance store volumes are very high performant due to locally available. But are not recommended as on instance shutdown, data wud be lost. So, in case of storing such data which is a replica of original, or which does not matter to us (in case if lost during instance shutdown) , shud only be stored here.

 

Amazon Machine Image (AMIs):

You can create snapshot of a volume, create an AMI from it, and then spin-up an EC2 intance from that AMI

You can also create a volume, create AMI from it, then spinup EC2 intance from that AMI – and u can spinup the new EC2 instance in another AZ also (but same region).

But u can also create copy of a volume to diff region, create AMI from it, and launch ec2 instance out of it in that region as well.

 

Now, to cleanup: 1) Delete EC2 instance first that u created from the AMI  2) Go to AMI and click Deregister   3) Go to snapshot, delete it (u have to deregister the AMI first, and then only delete the snapshot).  4) Go to Volumes, and see if u have any volume left (or created for testing purpose). The volume attached to the ec2 instances wud have been deleted on its own  when u terminated the ec2 instance.

 

Amazon EFS:

S3:

Additional features:

 

Storage classes in S3:

S3 Intelligent Tiering class moves ur data intelligently between diff storage classes – based on your data access pattern.

S3 Standard-IA – shud be used only if u have data retrieval very low (as the cost of it is per GB). If ur data retrieval is huge or very high, then no meaning of using this class. Better to use S3 standard storage class

S3 OneZone-IA class will not give u data replication across multiple AZs, infact only 1 AZ.

Glacier and Glacier-deep archive classes are cheapest storage classes which shud be used only for archived data (like maintaining old data for compliance).

 

Storage class is set at object level i.e at the time of uploading a doc/docs in S3, u have the option to specify the storage class. Default is “Standard”.

 

Amazon S3 versioning:

S3 replication:

It is replicating the data from 1 bucket to another.

It is of 2 types – 1) Cross-Region replication (CRR)  2) Same-Region replication (SRR)

You can also replicate data between diff accounts if u want to.

NOTE: You have to enable versioning before u use replication

 

 

When u enable replication, the newly uploaded objects get replicated. But existing objects stay as it is.

Once u enable versioning, u cannot remove/disable it. Just u can suspend it for new incoming objects.

 

Configure S3 Static Website:

You can upload files to your S3 bucket. Select those files – example – index.html and error.html and mark them as public.

Also go to S3 bucket actions > make it static website public. After that it will ask for specifying home page (mention index.html) and error page (mention error.html), that’s it.

Once u save and go to this static website hosting property in the s3 bucket properties, it will give u a public url for ur website. Hit it in browser and u see the contents u mentioned in index.html

 

Bucket policies and bucket policy generator

Bucket policies are json files which give u option to control who can access the files. Bucket policy generator prompts u to enter a few details and at the end generates a json for you. So u don’t need to write the json for it manually.

Glacier and Glacier deep archive:

Glacier deep archive is much cheaper but u need to consider the retrieval time.

 

 

AWS Storage Gateway :

It’s a service to connect your on-premise apps into cloud storage. Its called hybrid cloud storage service as it connects ur on-prem cloud

 

WE have 3 types of storage gateways:

These can be virtual appliances running in ur on-premise datacenter.

File gateway is file based storage gateway. It can use NFS or SMB protocol.

Volume gateway is a block based storage system. That is , ur on-prem servers are mounting a block based volume, and then they get access to s3 storage.

Backup gateway – here, ur application service can use either block or file protocol to access s3.

 

Exam Cram:

Quiz:

EBS volumes are stored at which contruct à Region/AZ ? Ans - > AZ

Which AMI can be used to mount an EFS file system à Linux/MS windows server 2019/Windows 2016/All.  Ans à Linux

Which storage service is used by EC2 for root volume à EBS/S3/EFS/Storage gateway. Ans -> EBS

Cheat sheet:

S3 is designed to deliver 99.999999999% durability, and stores data for millions of applications used by market leaders in every industry

Files can be anywhere from 0 bytes to 5 TB.

There is unlimited storage available.

Files are stored in buckets.

Buckets are root level folders.

Any subfolder within a bucket is known as a “folder”.

S3 is a universal namespace so bucket names must be unique globally.

S3 is persistent storage.

Persistent data stores are non-volatile storage systems that retain data when powered off.

This is in contrast to transient data stores and ephemeral data stores which lose the data when powered off.

The following table provides a description of persistent, transient and ephemeral data stores and which AWS service to use:

Bucket names must follow a set of rules:

• Names must be unique across all of AWS.
• Names must be 3 to 63 characters in length.
• Names can only contain lowercase letters, numbers and hyphens.
• Names cannot be formatted as an IP address.

Objects consist of:

• Key (name of the object).
• Value (data made up of a sequence of bytes).
• Version ID (used for versioning).
• Metadata (data about the data that is stored).

Lifecycle management is a feature of S3 – set rules to transfer objects between storage classes at defined time intervals.

Tiers:

• S3 standard.
• S3-IA.
• S3 One Zone – IA.
• Glacier.

 

When you create a bucket you need to select the region where it will be created.

It is a best practice to create buckets in regions that are physically closest to your users to reduce latency.

AWS Snowball

With AWS Snowball (Snowball), you can transfer hundreds of terabytes or petabytes of data between your on-premises data centers and Amazon Simple Storage Service (Amazon S3).

Uses a secure storage device for physical transportation.

AWS Snowball Client is software that is installed on a local computer and is used to identify, compress, encrypt, and transfer data.

Uses 256-bit encryption (managed with the AWS KMS) and tamper-resistant enclosures with TPM.

Snowball (80TB) (50TB model available only in the USA).

Snowball Edge (100TB) comes with onboard storage and compute capabilities.

Snowmobile – exabyte scale with up to 100PB per Snowmobile.

Snowcone is a small device used for edge computing, storage and data transfer.

Snowball can import to S3 or export from S3.

Import/export is when you send your own disks into AWS – this is being deprecated in favour of Snowball.

Snowball must be ordered from and returned to the same region.

To speed up data transfer it is recommended to run simultaneous instances of the AWS Snowball Client in multiple terminals and transfer small files as batches.

Amazon Elastic Block Store (EBS)

Amazon Elastic Block Store (Amazon EBS) provides persistent block storage volumes for use with Amazon EC2 instances in the AWS Cloud.

Each Amazon EBS volume is automatically replicated within its Availability Zone to protect you from component failure, offering high availability and durability.

Amazon EBS volumes offer the consistent and low-latency performance needed to run your workloads. With Amazon EBS, you can scale your usage up or down within minutes – all while paying a low price for only what you provision.

The following table shows a comparison of a few EBS volume types:

EBS volume data persists independently of the life of the instance.

EBS volumes do not need to be attached to an instance.

You can attach multiple EBS volumes to an instance.

You cannot attach an EBS volume to multiple instances (use Elastic File Store instead).

EBS volumes must be in the same AZ as the instances they are attached to.

Termination protection is turned off by default and must be manually enabled (keeps the volume/data when the instance is terminated).

Root EBS volumes are deleted on termination by default.

Extra non-boot volumes are not deleted on termination by default.

The behavior can be changed by altering the “DeleteOnTermination” attribute.

EBS Snapshots:

• Snapshots capture a point-in-time state of an instance.
• Snapshots are stored on S3.
• Does not provide granular backup (not a replacement for backup software).
• If you make periodic snapshots of a volume, the snapshots are incremental, which means that only the blocks on the device that have changed after your last snapshot are saved in the new snapshot.
• Even though snapshots are saved incrementally, the snapshot deletion process is designed so that you need to retain only the most recent snapshot in order to restore the volume.
• Snapshots can only be accessed through the EC2 APIs.
• EBS volumes are AZ specific but snapshots are region specific.

INSTANCE STORES

Instance store volumes are high performance local disks that are physically attached to the host computer on which an EC2 instance runs.

Instance stores are ephemeral which means the data is lost when powered off (non-persistent).

Instances stores are ideal for temporary storage of information that changes frequently, such as buffers, caches, or  scratch data.

Instance store volume root devices are created from AMI templates stored on S3.

Instance store volumes cannot be detached/reattached.

 

Amazon Elastic File System (EFS)

EFS is a fully-managed service that makes it easy to set up and scale file storage in the Amazon Cloud.

Good for big data and analytics, media processing workflows, content management, web serving, home directories etc.

EFS uses the NFS protocol.

Pay for what you use (no pre-provisioning required).

Can scale up to petabytes.

EFS is elastic and grows and shrinks as you add and remove data.

Can concurrently connect 1 to 1000s of EC2 instances, from multiple AZs.

A file system can be accessed concurrently from all AZs in the region where it is located.

By default you can create up to 10 file systems per account.

On-premises access can be enabled via Direct Connect or AWS VPN.

Can choose General Purpose or Max I/O (both SSD).

The VPC of the connecting instance must have DNS hostnames enabled.

EFS provides a file system interface, file system access semantics (such as strong consistency and file locking).

Data is stored across multiple AZ’s within a region.

Read after write consistency.

Need to create mount targets and choose AZ’s to include (recommended to include all AZ’s).

Instances can be behind an ELB.

There are two performance modes:

• “General Purpose” performance mode is appropriate for most file systems.
• “Max I/O” performance mode is optimized for applications where tens, hundreds, or thousands of EC2 instances are accessing the file system.

Amazon EFS is designed to burst to allow high throughput levels for periods of time.

AWS Storage Gateway

AWS Storage Gateway is a hybrid cloud storage service that gives you on-premises access to virtually unlimited cloud storage.

Customers use Storage Gateway to simplify storage management and reduce costs for key hybrid cloud storage use cases.

These include moving backups to the cloud, using on-premises file shares backed by cloud storage, and providing low latency access to data in AWS for on-premises applications.

To support these use cases, Storage Gateway offers three different types of gateways:

• File Gateway – provides file system interfaces to on-premises servers.
• Volume Gateway – provides block-based access for on-premises servers.
• Tape Gateway – provides a virtual tape library that is compatible with common backup software (block and file interfaces).

 

Additional AWS Services and Tools:

Amazon Elastic Container Service for Kubernetes (EKS):

• Amazon Elastic Container Service for Kubernetes (EKS) is a managed Kubernetes service that makes it easy for you to run Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes control plane.
• EKS is certified Kubernetes conformant, so existing applications running on upstream Kubernetes are compatible with Amazon EKS.
• EKS automatically manages the availability and scalability of the Kubernetes control plane nodes that are responsible for starting and stopping containers, scheduling containers on virtual machines, storing cluster data, and other tasks.
• EKS automatically detects and replaces unhealthy control plane nodes for each cluster.
• Generally available but only in limited regions currently.

AWS Batch:

• With AWS Batch, you simply package the code for your batch jobs, specify their dependencies, and submit your batch job using the AWS Management Console, CLIs, or SDKs.
• AWS Batch allows you to specify execution parameters and job dependencies, and facilitates integration with a broad range of popular batch computing workflow engines and languages (e.g., Pegasus WMS, Luigi, and AWS Step Functions).
• AWS Batch efficiently and dynamically provisions and scales Amazon EC2 and Spot Instances based on the requirements of your jobs. AWS Batch provides default job queues and compute environment definitions that enable you to get started quickly.

Database

Amazon Elasticache:

• Amazon ElastiCache offers fully managed Redis and Memcached.
• Seamlessly deploy, run, and scale popular open source compatible in-memory data stores.
• Amazon ElastiCache can be used to significantly improve latency and throughput for many read-heavy application workloads (such as social networking, gaming, media sharing and Q&A portals) or compute-intensive workloads (such as a recommendation engine) by allowing you to store the objects that are often read in cache.
• Amazon ElastiCache simplifies and offloads the management, monitoring, and operation of in-memory cache environments, enabling you to focus on the differentiating parts of your applications.
• Pay only for the resources you consume based on node hours used.

Amazon Neptune:

• Amazon Neptune is a fast, reliable, fully-managed graph database service that makes it easy to build and run applications that work with highly connected datasets.
• With Amazon Neptune, you can create sophisticated, interactive graph applications that can query billions of relationships in milliseconds.
• SQL queries for highly connected data are complex and hard to tune for performance. Instead, Amazon Neptune allows you to use the popular graph query languages Apache TinkerPop Gremlin and W3C’s SPARQL to execute powerful queries that are easy to write and perform well on connected data.

Migration

AWS Migration Hub:

• AWS Migration Hub provides a single location to track the progress of application migrations across multiple AWS and partner solutions.
• Using Migration Hub allows you to choose the AWS and partner migration tools that best fit your needs, while providing visibility into the status of migrations across your portfolio of applications.
• For example, you might use AWS Database Migration Service, AWS Server Migration Service, and partner migration tools such as ATADATA ATAmotion, CloudEndure Live Migration, or RiverMeadow Server Migration SaaS to migrate an application comprised of a database, virtualized web servers, and a bare metal server.
• Using Migration Hub, you can view the migration progress of all the resources in the application.

AWS Database Migration Service:

• AWS Database Migration Service helps you migrate databases to AWS quickly and securely.
• The source database remains fully operational during the migration, minimizing downtime to applications that rely on the database.
• The AWS Database Migration Service can migrate your data to and from most widely used commercial and open-source databases.
• AWS Database Migration Service supports homogenous migrations such as Oracle to Oracle, as well as heterogeneous migrations between different database platforms, such as Oracle or Microsoft SQL Server to Amazon Aurora.
• With AWS Database Migration Service, you can continuously replicate your data with high availability and consolidate databases into a petabyte-scale data warehouse by streaming data to Amazon Redshift and Amazon S3.

AWS Server Migration Service:

• AWS Server Migration Service (SMS) is an agentless service which makes it easier and faster for you to migrate thousands of on-premises workloads to AWS
• AWS SMS allows you to automate, schedule, and track incremental replications of live server volumes, making it easier for you to coordinate large-scale server migrations

Networking & Content Delivery

Amazon API Gateway:

• Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
• With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services.
• Back-end services may include Amazon Elastic Compute Cloud (Amazon EC2), code running on AWS Lambda, or any web application.

AWS Direct Connect:

• AWS Direct Connect is a cloud service solution that makes it easy to establish a dedicated network connection from your premises to AWS.
• Using AWS Direct Connect, you can establish private connectivity between AWS and your datacenter, office, or colocation environment, which in many cases can reduce your network costs, increase bandwidth throughput, and provide a more consistent network experience than Internet-based connections.
• AWS Direct Connect lets you establish a dedicated network connection between your network and one of the AWS Direct Connect locations.
• Using industry standard 802.1q VLANs, this dedicated connection can be partitioned into multiple virtual interfaces.
• This allows you to use the same connection to access public resources such as objects stored in Amazon S3 using public IP address space, and private resources such as Amazon EC2 instances running within an Amazon Virtual Private Cloud (VPC) using private IP space, while maintaining network separation between the public and private environments.

Developer Tools

AWS CodeStar:

• AWS CodeStar enables you to quickly develop, build, and deploy applications on AWS. AWS CodeStar provides a unified user interface, enabling you to easily manage your software development activities in one place.
• With AWS CodeStar, you can set up your entire continuous delivery toolchain in minutes, allowing you to start releasing code faster. AWS CodeStar makes it easy for your whole team to work together securely, allowing you to easily manage access and add owners, contributors, and viewers to your projects.
• With AWS CodeStar, you can use a variety of project templates to start developing applications on Amazon EC2, AWS Lambda, and AWS Elastic Beanstalk.
• AWS CodeStar projects support many popular programming languages including Java, JavaScript, PHP, Ruby, and Python.

 

AWS CodeCommit:

• AWS CodeCommit is a fully-managed source control service that hosts secure Git-based repositiories.
• It makes it easy for teams to collaborate on code in a secure and highly scalable ecosystem.
• CodeCommit eliminates the need to operate your own source control system or worry about scaling its infrastructure.
• You can use CodeCommit to securely store anything from source code to binaries, and it works seamlessly with your existing Git tools.

AWS CodeBuild:

• AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy.
• With CodeBuild, you don’t need to provision, manage, and scale your own build servers. CodeBuild scales continuously and processes multiple builds concurrently, so your builds are not left waiting in a queue.
• You can get started quickly by using prepackaged build environments, or you can create custom build environments that use your own build tools.
• With CodeBuild, you are charged by the minute for the compute resources you use

 

AWS CodeDeploy:

• AWS CodeDeploy is a fully managed deployment service that automates software deployments to a variety of compute services such as Amazon EC2, AWS Lambda, and your on-premises servers.
• AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications.
• You can use AWS CodeDeploy to automate software deployments, eliminating the need for error-prone manual operations. The service scales to match your deployment needs, from a single Lambda function to thousands of EC2 instances.

AWS CodePipeline:

• AWS CodePipeline is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates.
• CodePipeline automates the build, test, and deploy phases of your release process every time there is a code change, based on the release model you define.
• This enables you to rapidly and reliably deliver features and updates.
• You can easily integrate AWS CodePipeline with third-party services such as GitHub or with your own custom plugin.
• https://aws.amazon.com/codepipeline/features/

AWS X-Ray:

• AWS X-Ray helps developers analyze and debug production, distributed applications, such as those built using a microservices architecture.
• With X-Ray, you can understand how your application and its underlying services are performing to identify and troubleshoot the root cause of performance issues and errors.
• X-Ray provides an end-to-end view of requests as they travel through your application, and shows a map of your application’s underlying components.
• You can use X-Ray to analyze both applications in development and in production, from simple three-tier applications to complex microservices applications consisting of thousands of service.

Management Tools

https://digitalcloud.training/certification-training/aws-certified-cloud-practitioner/aws-cloud-management/

AWS CloudFormation:

• AWS CloudFormation provides a common language for you to describe and provision all the infrastructure resources in your cloud environment.
• CloudFormation allows you to use a simple text file to model and provision, in an automated and secure manner, all the resources needed for your applications across all regions and accounts.
• This file serves as the single source of truth for your cloud environment.
• You can use JSON or YAML to describe what AWS resources you want to create and configure.

AWS Config:

• AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources.
• Config continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of recorded configurations against desired configurations.
• With Config, you can review changes in configurations and relationships between AWS resources, dive into detailed resource configuration histories, and determine your overall compliance against the configurations specified in your internal guidelines.
• This enables you to simplify compliance auditing, security analysis, change management, and operational troubleshooting.

AWS OpsWorks:

• AWS OpsWorks is a configuration management service that provides managed instances of Chef and Puppet.
• Chef and Puppet are automation platforms that allow you to use code to automate the configurations of your servers.
• OpsWorks lets you use Chef and Puppet to automate how servers are configured, deployed, and managed across your Amazon EC2 instances or on-premises compute environments.
• OpsWorks has three offerings, AWS Opsworks for Chef Automate, AWS OpsWorks for Puppet Enterprise, and AWS OpsWorks Stacks

AWS Service Catalog:

• AWS Service Catalog allows organizations to create and manage catalogs of IT services that are approved for use on AWS.
• These IT services can include everything from virtual machine images, servers, software, and databases to complete multi-tier application architectures.
• AWS Service Catalog allows you to centrally manage commonly deployed IT services, and helps you achieve consistent governance and meet your compliance requirements, while enabling users to quickly deploy only the approved IT services they need
• Uses CloudFormation templates.

AWS Systems Manager:

• AWS Systems Manager gives you visibility and control of your infrastructure on AWS
• Systems Manager provides a unified user interface so you can view operational data from multiple AWS services and allows you to automate operational tasks across your AWS resources.
• With Systems Manager, you can group resources, like Amazon EC2 instances, Amazon S3 buckets, or Amazon RDS instances, by application, view operational data for monitoring and troubleshooting, and take action on your groups of resources.
• Systems Manager simplifies resource and application management, shortens the time to detect and resolve operational problems, and makes it easy to operate and manage your infrastructure securely at scale.

AWS Managed Services:

• AWS Managed Services provides ongoing management of your AWS infrastructure so you can focus on your applications.
• By implementing best practices to maintain your infrastructure, AWS Managed Services helps to reduce your operational overhead and risk.
• AWS Managed Services automates common activities such as change requests, monitoring, patch management, security, and backup services, and provides full-lifecycle services to provision, run, and support your infrastructure.
• AWS Managed Services delivers consistent operations management and predictable results by following ITIL® best practices, and provides tooling and automation to increase efficiency, and reduce your operational overhead and risk.

Amazon Athena:

• Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL.
• Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.
• With a few clicks in the AWS Management Console, customers can point Athena at their data stored in S3 and begin using standard SQL to run ad-hoc queries and get results in seconds.
• You can use Athena to process logs, perform ad-hoc analysis, and run interactive queries
• Athena scales automatically – executing queries in parallel – so results are fast, even with large datasets and complex queries.

Amazon EMR:

• Amazon Elastic Map Reduce (EMR) provides a managed Hadoop framework that makes it easy, fast, and cost-effective to process vast amounts of data across dynamically scalable Amazon EC2 instances.
• You can also run other popular distributed frameworks such as Apache Spark, HBase, Presto, and Flink in Amazon EMR, and interact with data in other AWS data stores such as Amazon S3 and Amazon DynamoDB.
• Amazon EMR securely and reliably handles a broad set of big data use cases, including log analysis, web indexing, data transformations (ETL), machine learning, financial analysis, scientific simulation, and bioinformatic.

Amazon CloudSearch:

• Amazon CloudSearch is a managed service in the AWS Cloud that makes it simple and cost-effective to set up, manage, and scale a search solution for your website or application.
• Amazon CloudSearch supports 34 languages and popular search features such as highlighting, autocomplete, and geospatial search.

Amazon Elasticsearch:

• Amazon Elasticsearch Service, is a fully managed service that makes it easy for you to deploy, secure, operate, and scale Elasticsearch to search, analyze, and visualize data in real-time.
• With Amazon Elasticsearch Service you get easy-to-use APIs and real-time analytics capabilities to power use-cases such as log analytics, full-text search, application monitoring, and clickstream analytics, with enterprise-grade availability, scalability, and security.

Amazon Kinesis:

• Amazon Kinesis makes it easy to collect, process, and analyze real-time, streaming data so you can get timely insights and react quickly to new information.
• There are four types of Kinesis service:
• Kinesis Video Streams makes it easy to securely stream video from connected .devices to AWS for analytics, machine learning (ML), and other processing.
• Kinesis Data Streams enables you to build custom applications that process or analyze streaming data for specialized needs.
• Kinesis Data Firehose is the easiest way to load streaming data into data stores and analytics tools.
• Amazon Kinesis Data Analytics is the easiest way to process and analyze real-time, streaming data.

AWS Data Pipeline:

• AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals.
• With AWS Data Pipeline, you can regularly access your data where it’s stored, transform and process it at scale, and efficiently transfer the results to AWS services such as Amazon S3, Amazon RDS, Amazon DynamoDB, and Amazon EMR.
• AWS Data Pipeline helps you easily create complex data processing workloads that are fault tolerant, repeatable, and highly available.

AWS Glue:

• AWS Glue is a fully managed extract, transform, and load (ETL) service that makes it easy for customers to prepare and load their data for analytics.
• You can create and run an ETL job with a few clicks in the AWS Management Console.
• You simply point AWS Glue to your data stored on AWS, and AWS Glue discovers your data and stores the associated metadata (e.g. table definition and schema) in the AWS Glue Data Catalog.
• Once cataloged, your data is immediately searchable, queryable, and available for ETL.
• AWS Glue generates the code to execute your data transformations and data loading processes.

Media Services

Amazon Elastic Transcoder:

• Amazon Elastic Transcoder is media transcoding in the cloud.
• It is designed to be a highly scalable, easy to use and a cost effective way for developers and businesses to convert (or “transcode”) media files from their source format into versions that will playback on devices like smartphones, tablets and PCs.

Security, Identity and Compliance

Amazon Cognito:

• Amazon Cognito lets you add user sign-up, sign-in, and access control to your web and mobile apps quickly and easily.
• Amazon Cognito scales to millions of users and supports sign-in with social identity providers, such as Facebook, Google, and Amazon, and enterprise identity providers via SAML 2.0

AWS Certificate Manager:

• AWS Certificate Manager is a service that lets you easily provision, manage, and deploy public and private Secure Sockets Layer/Transport Layer Security (SSL/TLS) certificates for use with AWS services and your internal connected resources.
• SSL/TLS certificates are used to secure network communications and establish the identity of websites over the Internet as well as resources on private networks.
• AWS Certificate Manager removes the time-consuming manual process of purchasing, uploading, and renewing SSL/TLS certificates.

AWS CloudHSM:

• AWS CloudHSM is a cloud-based hardware security module (HSM) that enables you to easily generate and use your own encryption keys on the AWS Cloud.
• With CloudHSM, you can manage your own encryption keys using FIPS 140-2 Level 3 validated HSMs.
• CloudHSM offers you the flexibility to integrate with your applications using industry-standard APIs, such as PKCS#11, Java Cryptography Extensions (JCE), and Microsoft CryptoNG (CNG) libraries.

AWS Directory Service:

• AWS Directory Service for Microsoft Active Directory, also known as AWS Managed Microsoft AD, enables your directory-aware workloads and AWS resources to use managed Active Directory in the AWS Cloud.
• AWS Managed Microsoft AD is built on actual Microsoft Active Directory and does not require you to synchronize or replicate data from your existing Active Directory to the cloud.
• You can use standard Active Directory administration tools and take advantage of built-in Active Directory features, such as Group Policy and single sign-on (SSO).
• With AWS Managed Microsoft AD, you can easily join Amazon EC2 and Amazon RDS for SQL Server instances to your domain, and use AWS Enterprise IT applications such as Amazon WorkSpaces with Active Directory users and groups.

AWS Artifact:

• AWS Artifact is your go-to, central resource for compliance-related information that matters to you.
• It provides on-demand access to AWS’ security and compliance reports and select online agreements.
• Reports available in AWS Artifact include our Service Organization Control (SOC) reports, Payment Card Industry (PCI) reports, and certifications from accreditation bodies across geographies and compliance verticals that validate the implementation and operating effectiveness of AWS security controls.
• Agreements available in AWS Artifact include the Business Associate Addendum (BAA) and the Nondisclosure Agreement (NDA).

Machine Learning

Amazon Rekognition:

• Amazon Rekognition makes it easy to add image and video analysis to your applications.
• You just provide an image or video to the Rekognition API, and the service can identify the objects, people, text, scenes, and activities, as well as detect any inappropriate content.
• Amazon Rekognition also provides highly accurate facial analysis and facial recognition on images and video that you provide.
• You can detect, analyze, and compare faces for a wide variety of user verification, people counting, and public safety use cases.

Amazon SageMaker:

• Amazon SageMaker is a fully-managed platform that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale.
• Amazon SageMaker removes all the barriers that typically slow down developers who want to use machine learning.
•  

Amazon Comprehend:

• Amazon Comprehend is a natural language processing (NLP) service that uses machine learning to find insights and relationships in text.
• The service identifies the language of the text; extracts key phrases, places, people, brands, or events; understands how positive or negative the text is; analyzes text using tokenization and parts of speech; and automatically organizes a collection of text files by topic.
• Using these APIs, you can analyze text and apply the results in a wide range of applications including voice of customer analysis, intelligent document search, and content personalization for web applications.

Amazon Transcribe:

• Amazon Transcribe is an automatic speech recognition (ASR) service that makes it easy for developers to add speech-to-text capability to their applications.
• Using the Amazon Transcribe API, you can analyze audio files stored in Amazon S3 and have the service return a text file of the transcribed speech.
• Amazon Transcribe can be used for lots of common applications, including the transcription of customer service calls and generating subtitles on audio and video content.
• The service can transcribe audio files stored in common formats, like WAV and MP3, with time stamps for every word so that you can easily locate the audio in the original source by searching for the text

Mobile Services

AWS AppSync:

• AWS AppSync makes it easy to build data-driven mobile and browser-based apps that deliver responsive, collaborative experiences by keeping the data updated when devices are connected, enabling the app to use local data when offline, and synchronizing the data when the devices reconnect.
• AWS AppSync uses the open standard GraphQL query language so you can request, change, and subscribe to the exact data you need with just a few lines of code.

AWS Device Farm:

• AWS Device Farm is an app testing service that lets you test and interact with your Android, iOS, and web apps on many devices at once, or reproduce issues on a device in real time.
• View video, screenshots, logs, and performance data to pinpoint and fix issues and increase quality before shipping your app.

Application Integration

AWS Step Functions:

• AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly.
• Using Step Functions, you can design and run workflows that stitch together services such as AWS Lambda and Amazon ECS into feature-rich applications.
• Workflows are made up of a series of steps, with the output of one step acting as input into the next.

Amazon MQ:

• Amazon MQ is a managed message broker service for Apache ActiveMQ that makes it easy to set up and operate message brokers in the cloud.
• Message brokers allow different software systems–often using different programming languages, and on different platforms–to communicate and exchange information.
• Messaging is the communications backbone that connects and integrates the components of distributed applications, such as order processing, inventory management, and order fulfillment for e-commerce.

Amazon SQS:

• Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
• SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware, and empowers developers to focus on differentiating work.
• Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.

Amazon SWF:

• Amazon SWF helps developers build, run, and scale background jobs that have parallel or sequential steps.
• You can think of Amazon SWF as a fully-managed state tracker and task coordinator in the Cloud.

Internet of Things

AWS IoT Core:

• AWS IoT Core is a managed cloud service that lets connected devices easily and securely interact with cloud applications and other devices.
• AWS IoT Core can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely.
• With AWS IoT Core, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected.

Desktop & App Streaming

Amazon Workspaces:

• Amazon WorkSpaces is a managed, secure cloud desktop service. You can use Amazon WorkSpaces to provision either Windows or Linux desktops in just a few minutes and quickly scale to provide thousands of desktops to workers across the globe.
• Amazon WorkSpaces offers you an easy way to provide a secure, managed, cloud-based virtual desktop experience to your end-users.
• Unlike traditional on-premises Virtual Desktop Infrastructure (VDI) solutions, you don’t have to worry about procuring, deploying, and managing a complex environment – Amazon WorkSpaces takes care of the heavy lifting and provides a fully managed service.

 

DNS, Elastic load balancing, and Auto Scaling:

You can create your domain through this DNS service (Route 53) in AWS Mgmt console as follows:

It will charge u $12 per year as mentioned in above snapshot.

Scaling:

Vertical scaling (Scaling Up): We increase the resources (hardware) , but machine is the same. So we have single point of failure here.

In AWS, it happens as follows: (small instance type t2 is updated to higher instance type c5. i.e the instance remains the same, but the instance type is changed/upgraded).

Horizontal scaling (Scaling Out): When ur application is accessed by many users, this is preferred approach. If one instance fails, other instance can take over.

In AWS, it happens as follows: (we just add more instances as and when required)

 

 

 

 

Amazon EC2 Auto Scaling:  It scales our EC2 instances automatically horizontally (i.e scaling out) by launching n terminating ec2 instances. Example below:

Here, an autoscaling group has launched the instances.

When EC2 status check fails for one instance, a msg goes the autoscaling group that one ec2 is not functioning. So Autoscaling group launches a new ec2 instance.

 

Another scenario which can happen is - When EC2 metric reports say that CPU usage overall is > 80, this msg goes to CloudWatch (which is a performance monitoring service) that the aggregate cpu is more than 80%. and CloudWatch intimates the autoscaling group. So Autoscaling group launches a new ec2 instance.

So autoscaling helps us from failure plus also provides additional no. of instances while usage is high, scale up when demand increases and also scale down when demand decreases.

For creating in AWS mgmt. console,

1.      Create a new EC2 launch template

a.       Provide name

b.      Select AMI as Amazon Linux 2 AMI (HVM) , SSD Volume type

c.       Instance type t2.micro

d.      Select ur key-pair u created

e.       Select ur security group u created.

f.        In advanced details, go to bottom User data field. Paste the script which created the web server.(what it does is it will use the metadata to find the AZ where ec2 instance is running

g.      Click ‘Create launch template’

2.      Create an auto scaling group.

a.       Name it like MyASG

b.      Select launch template as the MyWebServers (which u created in first step).Next

c.       Select multiple subnets (atleast 3),which makes sure that when auto scaling group is launching the instances, it will do it in multiple AZs as It spreads the resources across so to achieve higher fault tolerance and HA. Click next

d.      Click next

e.       When it asks for group size, enter 2 in all fields . it means u r statically telling the auto scaling group to make sure I have 2 instances at all times. Click next till end.

f.        Click ‘create auto scaling group’.

3.      So this will launch 2 instances into two diff AZs

Load balancing:

Fault tolerance means ability to recover in case of something like a component failure.

To overcome, this, u can do the following:

Now, combining LB and fault tolerance. AWS built in provides fault tolerance and spreads the load between diff AZs

Adding autoscaling here, provides a better solution as the faulty/stopped web server can be replaced by auto scaling group automatically as shown below.

 

So we regularly use autoscaling and Load balancing together to ensure we have right no. of ec2 instances that we can recover from failure and we can direct the incoming connections across all our instances.

 

Amazon Elastic Load Balancer (ELB):

Types of ELBs:

Application Load balancer (ALB) – most commonly used. And modern/New generation LB.

Network load balancer – less used but comes under New generation LB. But used in case of high performance needed and very low latency required.

 

Old generation LB: Classic LB

 

Attaching an Application Load Balancer to our auto scaling group so we can load balance some connections coming into our instances:

Steps:

1.      Create a target group

2.      Select “instances” , specify a name for target grp as ‘TG1’. Http – port 80 keep as it is

3.      VPC – default selected. Click next

4.      U will see ur 2 instances, select both and click ‘Include as pending below’.

5.      Click ‘Create target group’.

6.      This will connect our instances to our target grp

7.      Click TG1 (ur target grp) to see the details.

8.      Now, click Load Balancer on the left.

9.      Create load balancer > select Application LB > give the name as ‘MyALB’.

10.  Select the AZs which u selected while configuring your auto scaling group. If u selected all AZs while creating ur auto scaling group, here also select all AZs.

11.  It means that if the target is in AZ, then LB will be able to forward the connection to it.

12.  Click Next., again Next

13.  Select ur SG (WebAccess) which has port 80 open.

14.  In Target group, select ‘Existing target group’ TG1. Next, next.

15.  Click ‘Create’. So ur LB is created. Will take a few mins to reflect in AMC.

16.  Your ‘MyALB’ will be shown in ‘provisioning’ state as of now, in 1-2 mins, it becomes ‘active’.

17.  Go to Target groups – to check that we have healthy instances. See the ‘Details’ and ‘Targets’ panel to confirm that its in healthy state.

18.  Go to Load balancers on left. Select ur LB. See details below where your DNS name is provided. U can access it from browser (if u had pasted that user data / script previously).

a.      If u refresh ur browser page every few secs, u will notice that the AZ name displayed is changing. It means ur request is going to diff AZs and routed by LB dynamically.

19.   

Elastically scale the application:

Steps in AMC:

1)      Go to Auto Scaling groups , select MyASG

2)      Attach the target group to ur auto scaling grp

3)      Edit the Load balancing panel

4)      In Load balancers, select your target group TG1 and click update.

5)      Go to MyASG > Automatic scaling > Add policy

6)      Create scaling policy , in metric type – select ALB request per target , select target group TG1 and target value as 5. So here, if each target receives 5 or more requests, it will scale automatically.

7)      Go to load balancers on left, in your LB details, click DNS name. Hit in browser.

8)      Now, go to Route 53 (if you did Route 53 to register your domain name). U will find ur domain name in the hosted zones.

9)      Go in ur hosted zone and ‘Create record’.

·         Enable alias, and in ‘Route traffic to’ dropdown select – Alias to application and classic Load balancer (here u r choosing ur load balancer)

·         Select your region ex: us-east-1, and your load balancer which automatically populates.

·         Click ‘Create records’.

10)  Above steps will do the following:

·         In the connection attempts which u make to ur hostname, the requests will be fwd to ur LB:

·        

·         If u have not registered ur dns name,  use/access the LB url

11)  Hit the dnsname or LB url as per above config which u have done.

12)  Keep sending requests from browser atleast for 3 mins just to make sure the auto scaling works/happens in background.

13)  U will find that u have now 6 ec2 instances – which are scaled based on the no. of connection attempts.

14)  Go to auto scaling groups > MyASG, and see the desired capacity which should be 6 now. We set it to 2 initially. The autoscaling grp has changed it based on demand. U can see the activity tab in the MyASG.

15)  Also u can see the details in CloudWatch service > Alarms > filter on ‘In Alarm’  state.  U will find your application ELB in alarm state here.

16)  Now, if u leave the MyASG is same state for around 15 mins, it will determine that the no. of ec2 instances is not required now due to less load/incoming requests, and so it will reduce the desired capacity back to 2.

17)  Now, terminate the resources to avoid paying:

·         Delete your MyASG, it will also terminate ur instances (u can go n chk inside ur ec2 instances list)

·         U don’t need to worry about the ‘launch templates’, it doesn’t cost u any money

·         U don’t need to worry about the ‘target groups’ also.

·         Go to load balancers > MyALB > delete it.

 

Scaling Policies

We apply them to our auto scaling groups – as per we want them to behave and for terminating instances

There are few types of policies:

Target tracking – for example – u want that ur ec2 instances are utilizing approx. 70% of CPU each, then u know that u might run into perf. issues. So it will make sure that if utilization is too low than 70%, it will start terminating instances, and if its much high than 70% then it will launch new instances.

Simple scaling – just adjusts the group size based on metric. If u say after 80% cpu util, u want to launch another inst, it does it.

Step scaling – It will adjust differently based on the alarm breach happened. Example – if ur metric shows that u r just above ur threshold then it will launch one more instance. But if its much more above the threshold, then it will launch 2 more inst or may b 4 inst (this config also u can specify)

Scheduled scaling – example – people come to work at 9 AM in morning, so this type of scaling can be scheduled at 8.45 AM for example so that in 15 mins, the scaling of instances is done and capacity is increased for users load to be taken up.

 

Exam cram:

Here – weighted means send 80% traffic to one address and 20% to another address

 

ALB functions at layer 7 i.e it routes the connections based on the information in the request for ex: domain name or path in the url

Network LB – routes connections based on the IP protocol data (at layer 4 of the OSI model).

Questions:

How can a company enable elasticity for an application running on Amazon EC2?

Auto scaling / Elastic load balancing / multi-AZ / enabling failver in ec2

Ans: Auto scaling

 

How does Elastic Load Balancing (ELB) assist with fault tolerance? à

            By distributing connections to multiple back-end instances /

            By directing traffic according to latency /

            By caching content closer to users /

            By automatically launching instances.

Ans: By distributing connections to multiple back-end instances

 

Which of the following statements is INCORRECT about Elastic Load Balancing?

ELB can distribute connections across AZs

ELB can distribute connections across regions

ELB can be internet facing

ELB enables HA and fault tolerance

Ans: ELB can distribute connections across regions

 

NOTE: To direct traffic across regions use Amazon Route 53

 

What does Elastic Load Balancing use to ensure instances are available?

EC2 status checks / CloudWatch metrics / Scaling plans / Health checks

Ans: Heath checks

 

Which type of Elastic Load Balancer routes connections based on IP protocol data at layer 4 only?

Application LB / Network LB / Classic LB

Ans: Network LB

 

 

What type of template is used by Amazon EC2 Auto Scaling to define instance family, AMI key pair, and security groups?

Scaling plan / Scaling policy / Launch configuration / Autoscaling group

Ans: Launch configuration

A launch configuration is the template used to create new EC2 instances and includes parameters such as instance family, instance type, AMI, key pair and security groups

NOTE: Scaling policies determine when, if, and how the ASG scales and shrinks (on-demand/dynamic scaling, cyclic/scheduled scaling)

 

Which services does Amazon Route 53 provide?

Domain registration, DNS, firewall protection /

Health checking, DNS, domain registration

Health checking, DNS, IP Routing

Domain registration, DNS, content distribution

Ans: Health checking, DNS, domain registration

 

In Amazon Route 53, what is the name for the configuration item that holds a collection of records belonging to a domain?

DNS Record / Alias / Hosted zone / Routing policy

Ans: Hosted zone

 

NOTE: A hosted zone represents a set of records belonging to a domain

 

Cheat sheet:

Amazon EC2 Auto Scaling

Amazon EC2 Auto Scaling automates the process of launching (scaling out) and terminating (scaling in) Amazon EC2 instances based on the traffic demand for your application.

Auto Scaling helps to ensure that you have the correct number of EC2 instances available to handle the application load.

Amazon EC2 Auto Scaling provides elasticity and scalability.

You create collections of EC2 instances, called an Auto Scaling group (ASG).

You can specify the minimum number of instances in each ASG, and AWS Auto Scaling will ensure the group never goes beneath this size.

You can also specify the maximum number of instances in each ASG and the group will never go above this size.

A desired capacity can be configured and AWS Auto Scaling will ensure the group has this number of instances.

You can also specify scaling policies that control when Auto Scaling launches or terminates instances.

Scaling policies determine when, if, and how the ASG scales and shrinks (on-demand/dynamic scaling, cyclic/scheduled scaling).

 

Scaling Plans define the triggers and when instances should be provisioned/de-provisioned.

A launch configuration is the template used to create new EC2 instances and includes parameters such as instance family, instance type, AMI, key pair and security groups.

 

Amazon Elastic Load Balancing (ELB)

ELB automatically distributes incoming application traffic across multiple targets, such as Amazon EC2 instances, containers, and IP addresses.

ELB can handle the varying load of your application traffic in a single Availability Zone or across multiple Availability Zones.

ELB features high availability, automatic scaling, and robust security necessary to make your applications fault tolerant.

There are four types of Elastic Load Balancer (ELB) on AWS:

• Application Load Balancer (ALB) – layer 7 load balancer that routes connections based on the content of the request.
• Network Load Balancer (NLB) – layer 4 load balancer that routes connections based on IP protocol data.
• Classic Load Balancer (CLB) – this is the oldest of the three and provides basic load balancing at both layer 4 and layer 7 (not on the exam anymore).
• Gateway Load Balancer (GLB) – distributes connections to virtual appliances and scales them up or down (not on the exam).

Application Load Balancer (ALB)

ALB is best suited for load balancing of HTTP and HTTPS traffic and provides advanced request routing targeted at the delivery of modern application architectures, including microservices and containers.

Operating at the individual request level (Layer 7), Application Load Balancer routes traffic to targets within Amazon Virtual Private Cloud (Amazon VPC) based on the content of the request.

 

Network Load Balancer (NLB)

NLB is best suited for load balancing of TCP traffic where extreme performance is required.

Operating at the connection level (Layer 4), Network Load Balancer routes traffic to targets within Amazon Virtual Private Cloud (Amazon VPC) and is capable of handling millions of requests per second while maintaining ultra-low latencies.

Network Load Balancer is also optimized to handle sudden and volatile traffic patterns.

 

 

Amazon Route 53

Route 53 is the AWS Domain Name Service.

Route 53 performs three main functions:

• Domain registration – Route 53 allows you to register domain names.
• Domain Name Service (DNS) – Route 53 translates name to IP addresses using a global network of authoritative DNS servers.
• Health checking – Route 53 sends automated requests to your application to verify that it’s reachable, available and functional.

You can use any combination of these functions.

Route 53 benefits:

• Domain registration.
• DNS service.
• Traffic Flow (send users to the best endpoint).
• Health checking.
• DNS failover (automatically change domain endpoint if system fails).
• Integrates with ELB, S3, and CloudFront as endpoints.

Routing policies determine how Route 53 DNS responds to queries.

The following table highlights the key function of each type of routing policy:

Amazon CloudFront

CloudFront is a content delivery network (CDN) that allows you to store (cache) your content at “edge locations” located around the world.

This allows customers to access content more quickly and provides security against DDoS attacks.

CloudFront can be used for data, videos, applications, and APIs.

 

CloudFront benefits:

• Cache content at Edge Location for fast distribution to customers.
• Built-in Distributed Denial of Service (DDoS) attack protection.
• Integrates with many AWS services (S3, EC2, ELB, Route 53, Lambda)

Origins and Distributions:

• An origin is the origin of the files that the CDN will distribute.
• Origins can be either an S3 bucket, an EC2 instance, an Elastic Load Balancer, or Route 53 – can also be external (non-AWS).
• To distribute content with CloudFront you need to create a distribution.
• There are two types of distribution: Web Distribution and RTMP Distribution.

CloudFront uses Edge Locations and Regional Edge Caches:

• An edge location is the location where content is cached (separate to AWS regions/AZs).
• Requests are automatically routed to the nearest edge location.
• Regional Edge Caches are located between origin web servers and global edge locations and have a larger cache.
• Regional Edge caches aim to get content closer to users.

The diagram below shows where Regional Edge Caches and Edge Locations are placed in relation to end users:

 

Application Services

First is Serverless Services. We don’t have server or OS here to manage. They automatically scale and they charge u for what u use.

So the serverless services have become very popular with many modern cloud practices.

S3 bucket is a serverless service as u don’t manage it, just make calls to upload/retrieve data.

AWS Lambda is a serverless compute service which allows us to run code. Here we have stored some code in Lambda, when it receives an event notification from S3 static website, it executes:

In this case, the Lambda might have various functions to do. Like storing the file in another S3, or might put a msg into a queue. The SQS (simple queue service) will store the msg for being processed by another component.

The queue can notify another lambda function and then lambda can execute the code to store the data in dynamodb, etc. Any steps or workflow can be executed.

All components above are serverless, none of them have Ec2 instances to manage.

 

Fargate is used for docker containers.

AWS Lambda function:

Flow below :

You only pay for the time when the Lambda function executes. U don’t pay for the time during which lambda function/code is stored on AWS.

Steps:

1)      Create lambda function from AMC

2)      Test it

3)      When u execute, AWS will just show ‘Hello from Lambda’ msg and the detailed logs r captured in CloudWatch log events.

 

Application Integration Services

1.      SQS – Simple Queue Service – its used for decoupling. Example below:

If u have a load on ur application (ex: Diwali sales) where large no of requests/orders come at a time and ur app is not able to handle. So in below diagram, the direct integration shows a scenario without SQS where ur app suffers a huge load. But with decoupled integration, SQS will keep/store the requests/messages with it and the application will poll the SQS to pull the messages, so in this case the app is not loaded.

 

 

2.      Amazon MQ (similar to SQS, but not an AWS service)

3.      AWS SNS – Simple Notification Service – Used for sending notifications

4.      Step functions – service used for orchestrating the parts of ur application.

5.      AWS SWF (Simple Workflow Service)

 

Summary:

 

Amazon EventBridge / CloudWatch Events:

 

Creating Event driven application

It is where events in one resource leads to events in another resource.

Steps (we will work on to achieve the above flow shown in diagram above – one by one by creating a lambda fn first, then create a msg in sqs and trigger lambda to see if the logs are written in CW. Then we will submit notif to SNS topic and check the whole flow).:

1. Go to Lambda > Create function “WriteToCWLogs”, and click ‘Create function’.

2. In the Lambda source code window, edit the index.js, and overwrite the below code:

exports.handler = async function(event, context){

  event.Records.forEach(record => {

    const { body } = record;

    console.log(body);

  });

  return {};

}

 

When u click ‘Deploy’ , your code will be deployed. So ur lambda function is ready.

Next – go to SQS queue, and create a queue named “MySimpleQueue”.

Once u create the queue, u will see LambdaTriggers as follows:

 

Now, we want the queue to trigger the lambda function.

If u click on ‘Configure Lambda fc trigger’ and select your lambda then it will give error saying that the role does not have permissions to call ‘ReceiveMessage’ on SQS. So u will need to give permissions. Go to your lambda function > permissions > execution role . Click the link as below

It will take u to the IAM page for editing the permissions where u can see that lambda has just basic rights to write to CW logs, not more than that.

Click Attach policies button and select ‘AWSLambdaSQSQueueExecutionRole’,  and attach it.

If u see the JSON of it, u will find (snapshot below), that it has access to what we want.

 

 

 

Go back to SQS, and save:

On saving, It will give a message that  - Lambda function arn:aws:lambda:us-east-1:839668878731:function:WriteToCWLogs is triggered when a message arrives in this queue

 

Once saved, on the same page click ‘Send and receive msgs’

Type a msg “Test message” in msg body and click ‘Send message’.

Now, we have submitted a msg to our queue (SQS), it shud trigger our lambda function. And then we shud see something in CW logs.

 

Go to Lambda function and click ’Monitor’ panel. U may or maynot see the CloudWatch metric at this place, but click ‘View logs in CloudWatch’ and u will see the logs there with ur msg:

 

So this is our msg from SQS queue.

Now, we’ll go to SNS and setup a notification for a topic.

Go to SNS > Topics > Create topic. Select Standard option and give name “MySNSTopic”. Create Topic

We also need a subscription. Create a subscription

After selecting your topic ARN, Protocol and Endpoint as above - Click ‘Create subscription’

Now, go to SQS “MySimpleQueue” . In SNS subscriptions, u will see your MySNSTopic listed as below.

But still u need to click ‘Subscribe to Amazon SNS topic’ button, select your MySNSTopic and click Save.

 

Go to SNS > Topics > MySNSTopic > click Publish Message.

Enter msg subject and msg body, and click ‘Publish message’.

That should now go all the way thru to the CW logs.

So, we did the following:

We just submitted an SNS topic.

SNS queue is subscribed to the topic so msg shud come into the queue.

SNS queue triggers a lambda fn which processes that info and puts it in CW logs.

Check CW logs to see the new logs captured.

Now, to end up paying charges, stop the following:

SQS > MySimpleQueue > delete

If u want  u can delete  your SNS topic and lambda function but they don’t cost u until something is triggered or notifications are going thru ur SNS topics

 

Amazon API Gateway:

It acts as a gateway for your APIs. It sits inside a region. We create an API gateway, and then we can connect to it from internet using either mobile client, website or service on the internet, etc.

At the backend, the API Gateway could be talking to ur Lambda OR to your ec2 instances within a vpc (private or public). Also API gateway can talk to any other aws service.

So it acts as a front-end of ur application and then internally it will be able to communication with various diff APIs to the backend services within AWS or external to AWS as well.

 

Exam cram:

 

Questions:
An application needs to send SMS text messages to customers to notify them of product updates. Which service can be used?

AWS step fn / SQS/ SNS/ Lambda

 

 

A company needs to orchestrate several batch processes on AWS. Which serverless service can assist?

Amazon SWF / Amazon SQS / Amazon EventBridge / AWS Step functions

 

 

How can a company decouple an application which uses a message-oriented API to communicate data between application components?

SQS/ SNS / Step Fn / VPC route table

 

 

How can an application be configured to send a notification to multiple Amazon SQS queues?

FIFO queue / Use SNS topic / Create AWS step fn state machine

 

Multiple SQS queues can be subscribed to a single SNS topic

 

 

What is a defining feature of serverless technology?

High avl / can be very cheap / automatic scaling / no instances to manage

 

 

 

 

 

Cheat sheet:

 

AWS Organizations

AWS organizations allows you to consolidate multiple AWS accounts into an organization that you create and centrally manage.

Available in two feature sets:

• Consolidated Billing.
• All features.

Includes root accounts and organizational units.

Policies are applied to root accounts or OUs.

Consolidated billing includes:

• Paying Account – independent and cannot access resources of other accounts.
• Linked Accounts – all linked accounts are independent.

 

AWS Control Tower

Simplifies the process of creating multi-account environments.

Sets up governance, compliance, and security guardrails for you.

Integrates with other services and features to setup the environment for you including:

• AWS Organizations, SCPs, OUs, AWS Config, AWS CloudTrail, Amazon S3, Amazon SNS, AWS CloudFormation, AWS Service Catalog, AWS Single Sign-On (SSO).

Examples of guardrails AWS Control Tower can configure for you include:

• Disallowing public write access to Amazon Simple Storage Service (Amazon S3) buckets.
• Disallowing access as a root user without multi-factor authentication.
• Enabling encryption for Amazon EBS volumes attached to Amazon EC2 instances

AWS Config

AWS Config is a fully-managed service that provides you with an AWS resource inventory, configuration history, and configuration change notifications to enable security and regulatory compliance.

With AWS Config, you can discover existing and deleted AWS resources, determine your overall compliance against rules, and dive into configuration details of a resource at any point in time. AWS Config enables compliance auditing, security analysis, resource change tracking, and troubleshooting.

AWS Service Catalog

AWS Service Catalog allows organizations to create and manage catalogs of IT services that are approved for use on AWS.

AWS Service Catalog allows you to centrally manage commonly deployed IT services.

IT services can include virtual machine images, servers, software, and databases and multi-tier application architectures.

Enables users to quickly deploy only the approved IT services they need.

AWS Systems Manager

Manages many AWS resources including Amazon EC2, Amazon S3, Amazon RDS etc.

Systems Manager Components:

• Automation.
• Run Command.
• Inventory.
• Patch Manager.
• Session Manager.
• Parameter Store.

AWS Personal Health Dashboard

AWS Personal Health Dashboard provides alerts and remediation guidance when AWS is experiencing events that may impact you.

Personal Health Dashboard gives you a personalized view into the performance and availability of the AWS services underlying your AWS resources.

The dashboard displays relevant and timely information to help you manage events in progress.

Also provides proactive notification to help you plan for scheduled activities.

Alerts are triggered by changes in the health of AWS resources, giving you event visibility, and guidance to help quickly diagnose and resolve issues.

You get a personalized view of the status of the AWS services that power your applications, enabling you to quickly see when AWS is experiencing issues that may impact you.

Also provides forward looking notifications, and you can set up alerts across multiple channels, including email and mobile notifications, so you receive timely and relevant information to help plan for scheduled changes that may affect you.

Alerts include remediation details and specific guidance to enable you to take immediate action to address AWS events impacting your resources.

Can integrate with Amazon CloudWatch Events, enabling you to build custom rules and select targets such as AWS Lambda functions to define automated remediation actions.

The AWS Health API allows you to integrate health data and notifications with your existing in-house or third-party IT Management tools.

Service Health Dashboard

AWS publishes up-to-the-minute information on service availability.

This information is not personalized to you (unlike Personal Health Dashboard).

AWS OpsWorks

AWS OpsWorks is a configuration management service that provides managed instances of Chef and Puppet.

Updates include patching, updating, backup, configuration and compliance management.

AWS Trusted Advisor

AWS Trusted Advisor is an online tool that provides you real time guidance to help you provision your resources following AWS best practices.

Trusted Advisor checks help optimize your AWS infrastructure, improve security and performance, reduce your overall costs, and monitor service limits.

AWS Basic Support and AWS Developer Support customers get access to 6 security checks (S3 Bucket Permissions, Security Groups – Specific Ports Unrestricted, IAM Use, MFA on Root Account, EBS Public Snapshots, RDS Public Snapshots) and 50 service limit checks.

AWS Business Support and AWS Enterprise Support customers get access to all 115 Trusted Advisor checks (14 cost optimization, 17 security, 24 fault tolerance, 10 performance, and 50 service limits) and recommendations.