Smart Overview Of AWS

These AWS tutorial are suitable for both freshers and experienced professionals at any level. These tutorials are for intermediate to somewhat advanced AWS professionals, but even if you are just a beginner or fresher you should be able to understand the tutorials by examples and explanations here we give.

What is Cloud Computing?

---

Cloud computing is the on-demand delivery of compute power, database storage, applications, and other IT resources through a cloud services platform via the internet with pay-as-you-go pricing.

How Does Cloud Computing Work?

---

Cloud computing provides a simple way to access servers, storage, databases and a broad set of application services over the Internet. A Cloud services platform such as Amazon Web Services owns and maintains the network-connected hardware required for these application services, while you provision and use what you need via a web application.

How Does Cloud Computing Work?

• Six Advantages and Benefits of Cloud Computing
• Trade capital expense for variable expense
• Benefit from massive economies of scale
• Stop guessing capacity
• Increase speed and agility

• Stop spending money on running and maintaining data center
• Go global in minutes
• AWS Global Infrastructure

The AWS Cloud operates 42 Availability Zones within 16 geographic Regions around the world, with announced plans for eight more Availability Zones and three more Regions.

Regions and Availability Zones

---

• AWS GovCloud (2)
• US West—-Oregon (3), Northern California (3)
• US East—-Northern Virginia (5), Ohio (3)
• Canada—-Central (2)
• South America—-São Paulo (3)
• Europe—-Ireland (3), Frankfurt (2), London (2)
• Asia Pacific—-Singapore (2), Sydney (3), Tokyo (3), Seoul (2), Mumbai (2)
• China—-Beijing (2)
• Upcoming Regions—-Paris,Ningxia,Stockholm

Why AWS

• Fastest Growing cloud computing platform on the planet
• Largest public cloud computing platform on the planet
• More and more organizations are outsourcing their IT to AWS • The AWS certifications are the most popular IT certifications now.
• Top paid IT certification according to Forbes.

Regions and Availability Zones

Compute

Networking & CDN

Databases

Migration Services

Analytics Services

Security & Identity

Management Tools

Application Services

Developer Tools

IOT

Storage

IAM

---

AWS Identity and Access Management (IAM) is a web service that helps you securely control access to AWS resources for your users. You use IAM to control who can use your AWS resources
(authentication) and what resources they can use and in what ways (authorization).

IAM Features

• Centralized control of your AWS account
• Shared access to AWS account
• Granular Permissions
• Identity Federation (Active directory)
• Multifactor Authentication
• Provide temporary access to users/devices where necessary
• Allowed you to setup your own password rotation policy
• Integerates with many AWS services
• Supports PCI DSS compliance
• Free AWS service and a Global Service

IAM components

• Users — Think of a person
• Groups — One or more users with similar permissions
• Roles — Roles can be assigned to AWS resources
• Policies — A document that defines one or more permissions

Simple Storage Service (S3)

---

• Simple, durable, massively scalable object storage.
• Amazon Simple Storage Service (Amazon S3) is object storage with a simple web service interface to store and retrieve any amount of data from anywhere on the web.
• Data is spread across multiple device and facilities
• Files size can be from 0 byte to 5TB
• Single upload file size limit is 5GB
• Unlimited Storage
• Files are stored in Buckets
• Bucket Namespace is unique globally
• Upload successful code is HTTP 200
• https://s3-ap-south-1.amazonaws.com/bucketname
• Read after write consistency for PUTS of new objects
• Eventual consistency for Overwrite puts and deletes

S3 Properties

• Key value store, key is a name and value is data.
• Version ID
• Metadata (Data about data)
• Access control list
• Bucket policies

S3 Featues

• Amazon Guarantees 99.99% availability
• 99.999999999% durability (11* 9’s)
• Multiple Tiers available
• Lifecycle Management
• Versioning
• Encryption
• Data security using Access control list and bucket policies

S3 Classes

S3 Pricing

Charged for,

• Storage
• Requests
• Storage Management
• Data Transfer

S3 Storage Management

• Versioning
• Cross region Replication • Static Website hosting
• S3 Transfer Acceleration
• Life cycle Management

S3-Versioning

• Stores all versions of an object
• Once enabled versioning cannot be disabled but can be suspended
• Integrates with Lifecycle rules
• MFA authentication can be enabled for object delete to provide additional level of security

Cross region replication

---

• Versioning must be enabled on both source and destination buckets.
• Files in an existing bucket are not replicated automatically.
• cannot replicate to multiple buckets.
• Delete Markers are also replicated.

Life cycle Management

---

• Can be used in conjunction with Versioning
• can be applied to current and previous versions
• Transition to s3-IA class after 30 days of creation and 128kb size.
• 60 days after IA, object can be transferred to Glacier.
• Can also be permanently delete.
• By Default, All Buckets Are Private
• We can Implement security at Bucket level by Bucket Polices.
• Object level security by implementing Access control lists.
• S3 Buckets can be configured to create access logs which logs all requests made to the S3 Bucket. This can be done to another Bucket.

S3 Encryption

---

• In Transit,
• SSL/TLS
• At Rest,
• Server Side Encryption
• S3 Managed Keys–S3-SSE
• AWS Key Management Service,Mananged Keys–SSE-KMS
• Server Side Encryption with Customer Provided Keys–SSE-C
• Client Side Encryption

AWS Import/Export

---

AWS import/export accelerates large amount of data into and out of the AWS cloud using portable devices for transport. AWS import/export disk transfers our data directly onto and off of storage devices using Amazon’s high speed internal network and bypassing the internet.

Snowball Types

---

• Snowball
• Snowball Edge
• Snowmobile

Snowball Features

• Snowball is a petabyte-scale data transort solution that uses secure appliances to transfer large amounts of data into and out of AWS.
• Transferrig data with snowball is simple,fast,secure and one fifth the cost of high speed internet.
• 80TB snowball in all regions.snowball uses multiple layers of security including tamper-resistent enclosures,256-bit encryption,and an industry standard TPM for full chain of security.
• Once data transfer completed and verified, AWS performs a software erasure of the snowball appliance.
• Snowball Edge is a 100TB Data tranfer device.

Snowmobile Features

• Exabite-scale data tranfer device.
• 100PB per snowmobile.
• 45-foot long ruggedized shipping container pulled by semi-trailer
• Move massive amounts of data including video libraries,image repositories or complete data center migration.

Snowmobile

Cloudfront(CDN)

---

• Amazon CloudFront is a web service that speeds up distribution of your static and dynamic web content, such as .html, .css, .php, and image files, to your users
• CloudFront delivers your content through a worldwide network of data centers called edge locations. When a user requests content that you’re serving with CloudFront, the user is routed to the edge location that provides the lowest latency (time delay), so that content is delivered with the best possible performance

Cloudfront Components

---

• Edge Location — This is the location where content will be cached. Not read only, can be used for uploading files.
  content will be cached for TTL. can be cleared with additional fee.
• Origin — The Source for the files, can be s3 bucket, Http Server,ELB or Route53
• Distribution — This is the name given in CDN which consists of Collection of Edge locations

Cloudfront Distributions

• Web Distribution — used for Websites
• RTMP — used for Media Streaming

Cloudfront work flow

EC2(Elastic compute cloud)

---

• Amazon Elastic compute cloud is a webservice that provides resizable compute capacity in the cloud.
• Amazon Ec2 Reduces the time to obtain and boot new server instances into minutes.
• Allowing us to quickly scale capacity up and down as our computing requirements change.

EC2 Features

• Amazon Ec2 changes the Economics of computing by allowing you to pay only for capacity that we actually use.
• Amazon Ec2 provides developers the tool to build failiure resilient applications and isolate themselves from common failure scenarios

EC2 Purachasing Options

• On Demand
• Reserved Instances
• Scheduled Instances
• Spot Instances
• Dedicated Hosts
• Dedicated Instances

Ec2 Instance Purchase Options

• On-Demand instances — Pay, by the hour, for the instances that you launch.
• Reserved Instances — Purchase, at a significant discount, instances that are always available, for a term from one to three years.
• Scheduled Instances — Purchase instances that are always available on the specified recurring schedule, for a one-year term.
• Spot instances — Bid on unused instances, which can run as long as they are available and your bid is above the Spot price, at a significant discount.
• Dedicated hosts — Pay for a physical host that is fully dedicated to running your instances, and bring your existing persocket, per-core, or per-VM software licenses to reduce costs.
• Dedicated instances — Pay, by the hour, for instances that run on singletenant hardware.

Spot Instance Pricing

• If the Spot instance was terminated by Amazon Ec2, you will not be charged for a partial hour of usage.
• However if we terminate the instances ourself, we will be charged for any hour in which the instance ran.

Ec2 Instance Types Current Generation

---

Types of EC2

---

Ec2 Placement Groups

• A Placement group is a logical grouping of instances wthin a single Availability Zone. Using placement groups enables applications to participate in a low latency,10Gbps network.
• Placement groups are recommended for applications that require low latency , High netwotk throughput or both.

Placement Groups Restrictios

• A Placement group can’t span multiple AZ’s
• Name for placement group must be unique within AWS account
• Only certain types of instances(compute,GPU,Memory,Storage optimized) can be launched into placement groups.
• AWS recommends homogenos instances in placement groups.
• can’t merge placement groups.
• can’t move an existing instance into a placement group

EC2 MetaData

• Data about data. to get all the information about the particular instance from the command line.
• Curl http://169.254.169.254/latest/meta-data/

Security Groups

All inbound traffic is blocked by default

All outbound traffic is allowed by default Changes are applied immediately one SG can be applied to many Ec2 Instances one Ec2 instance can have many SG.

Security groups are stateful. can not block specific ip address can specify allow rules not deny rules.

Volumes & Snapshots

• Volumes exists on EBS(Elastic Block Store)
• Snapshots exists in S3
• Snapshots are point in time copies of volumes and incremental.
• First snapshot will take some time to create.

Raid Array Snapshot

• Due to Interdependancies of RAID array, We need to take Application Consistent snapshot.

Freeze the File system

Unmount the RAID Array

Shutting down the associated instance and take snapshot.

AMI(Amazon Machine Image)

---

• Amzon Machine image provides the information to launch the virtual servers in the cloud.
• can launch multiple servers from one AMI

AMI-components

-A Template for the root volume for the instance(OS/App/)

-Launch Permissions that control which AWS accounts can use the AMI

-Block Device Mapping specifies which volumes would attach to the       instance when it launches.

AMI

• AMI’s are Regional, you can only launch an AMI from the region in which it stored.
• But we can copy AMI’s to other regions through console and API.

EBS (Elastic Block Store)

• Amazon EBS volumes allow us to create storage volumes and attach them to Amazon Ec2 instances.once attached,you can create a file system on top of thses volumes.run a database.
• EBS volumes are placed in Specific AZ, where they are automatically replicated to protect Single point of failiure.

EBS Types

• General Purpose — SSD(GP2)Ratio of 3 iops per gb with upto 10,000

iops.Ratio of 3 iops per gb with upto 10,000 iops

• Provisioned IOPS — Use if you need more than 10,000 IOPS upto

20,000 IOPS

• HDD, Throughput Optimized(ST1)-freuently accessed workloads
• HDD,cold-)SC1) — less frequently accessed data
• HDD, MAgnetic — standard-cheap,infrequently accessed storage we cannot mount 1 EBS volume to multiple Ec2 instances.

EBS and Instance store volumes

• Instance store volumes are called Ephermal storage
• Instance store volumes cannot be stopped.if the host fails all data wiil be lost
• EBS backed instance can be stopped
• We can reboot both and will not loose any data.
• By Default both ROOT volumes will be deleted on Termination, however with EBS volumes we can use “delete on termination” unchecked and keep the volume.

EBS vs Instance Store

• AMI’s are catogorized either EBS backed or Instance store backed.
• EBS Backed : The root device for an instance launched from the AMI is an AMAzon EBS volume created from an Amazon EBS snapshot.

Instance Store:  The root device for an instance launched from an AMI is an instance store volume created from a template stored in Amazon S3.

Cloudwatch

---

• Cloud watch is a monitoring service
• Default monitoring is 5 Minutes and Detailed monitoring is 1 Minute intervals.
• Dashboards — Monitoring
• Alarms — To notify when threshold breaches
• Events   — Trigger events when AWS resource state changes
• Logs — To store, Monitor,Aggregate logs.

AWS Lambda

---

• AWS Lambda is a computer service where you can upload your code and create a lambda function. AWS Lambda takes care of provisioning and managing the servers for the code we use to run.
• Operating systems, Patching, Scaling everything taken care by AWS.

AWS Roles

• Roles are more secure than storing your access key and secrate access key on inside the Ec2 instance.
• Roles are easier to manage
• Roles can be assighned to an Ec2 instance after it is being created both through CLI and through Management console.
• Roles are Universal, can be used in any region

ELB

---

• Elastic Load balancers are distributing the incoming traffic across the registered instances.
• Check the health of instances, and route traffic only to the healthy instances. stops sending traffic to unhealthy instances.
• Appliction load balancers and classic load balancers available.
• SSL offload can be done on both the load balancers.
• ELB can be either Internet facing or internal.
• Dont have public IP Address. only public dns names used for access.

Autoscaling

• Auto Scaling helps you maintain application availability and allows you to scale your Amazon EC2 capacity up or down automatically according to conditions you define.
• Auto Scaling to help ensure that you are running your desired number of Amazon EC2 instances.
• Auto Scaling can also automatically increase the number of Amazon EC2 instances during demand spikes to maintain performance and decrease capacity during lulls to reduce costs.

EFS(Elastic File System)

---

• Amazon EFS file systems are distributed across an unconstrained number of storage servers, enabling file systems to grow elastically to petabyte-scale and allowing massively parallel access from Amazon EC2 instances to your data.
• Amazon EFS data is distributed across multiple Availability Zones, providing a high level of durability and availability.
• Charged for Amount of Data we put into the Volume.

EBS Vs EFS

Database

---

Relational Database Service(RDS)

• Amazon Relational Database Service (Amazon RDS) is a web service that makes it easier to set up, operate, and scale a relational database in the cloud.
• It provides cost-efficient, resizable capacity for an industry-standard relational database and manages common database administration tasks.
• Amazon RDS manages backups, software patching, automatic failure detection, and recovery.
• You can have automated backups performed when you need them, or create your own backup snapshot. These backups can be used to restore a database, and the Amazon RDS restore process works reliably and efficiently.

RDS

• AWS RDS Supports Following DB Engines,
• SQL Server
• Oracle
• MySQL Server
• PostgreSQL
• Aurora
• MariaDB

Non Relational Databases(NoSQL)

---

DynamoDB

• Amazon DynamoDB is a fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale. It is a fully managed cloud database and supports both document and key-value store models.
• Its flexible data model, reliable performance, and automatic scaling of throughput capacity, makes it a great fit for mobile, web, gaming, ad tech, IoT, and many other applications.
• Amazon DynamoDB delivers seamless, automatic scaling of throughput and storage scaling via API and the AWS Management Console. There is virtually no limit on how much throughput or storage that you can dial up at a time.
• Amazon DynamoDB is highly available, with automatic and synchronous data replication across three facilities in a Region. This helps protect your data against individual machine, or even facility level failures.

DataWareHousing(Redshift)

---

• Amazon Redshift is a fast, fully managed data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL and your existing Business Intelligence (BI) tools.
• It allows you to run complex analytic queries against petabytes of structured data, using sophisticated query optimization, columnar storage on high-performance local disks, and massively parallel query execution.
• With Amazon Redshift, you can start small for just $0.25 per hour with no commitments and scale out to petabytes of data for $1,000 per terabyte per year, less than a tenth the cost of traditional solutions.

OLAP and OLTP

---

• OLAP – Amazon Redshift is specifically designed for online analytic processing (OLAP) and business intelligence (BI) applications, which require complex queries against large datasets.
• OLTP – Online Transaction Processing is a class of software programs capable of supporting transaction-oriented applications on the Internet. Typically, OLTP systems are used for order entry, financial transactions, customer relationship management (CRM) and retail sales.(SQL Databases)

ElasticCache

• Amazon ElastiCache is a web service that makes it easy to deploy, operate, and scale an in-memory data store or cache in the cloud.
• The service improves the performance of web applications by allowing you to retrieve information from fast, managed, in-memory data stores, instead of relying entirely on slower disk-based databases.
• Amazon ElastiCache supports two open-source in-memory engines,
• MemcaheD
• Redis

Elastic cache

---

Aurora

---

• MySQL Compatible Relational Database.
• Five times better performance than MySQL
• Giving strong performance and availability
• Starts with 10GB and increments in 10GB upto 64TB
• Compute Resources can scale upto 32 Vcpu’s and 244GB memory 2 copies of data in each AZ with minimum of 3 AZ so 6 copies of data.
• can loss upto 2 copies without affecting write availability
• can loss upto 3 copies wihout affecting read availability
• Storage is self healing, data blocks are continously scanned for errors and repaired automatically.

Read Replica’s

---

• 2 Types of Replica’s available.
• Aurora Replica (currently 15)
• MySQL read replica ( Currently 5)
• Failover occurs automatically.

Simple Que Service(SQS)

---

• Amazon SQS is a web service that gives you access a message que that can be used to store messages while waiting for a computer to process them.
• SQS is a Pull based Message que system.
• Messages are 256KB in size.
• Messages can be kept in the que from 1 minute to 14 days. default is 4 days.
• Visibility timeout: Amount of time that the message is invisible in the que after a reader picks up the message processed by the instance.

SQS

• The message will be deleted from the que once the message is processed with in the visibility timeout expires.
• if not finished within visibility timeout period, the message will be visible again in the que.
• Visibility time out maximum is 12 hours.
• SQS guarantees that the message will be processed at least once.
• Short Polling- Returns immediately even if the que empty.
• Long Polling – Doesn’t reply a responce until a message arrives in the que or the long poll times out.
• Que’s are either standard and FIFO.

Simple Workflow Service(SWF)

---

SWF

---

• Workflow Starters – The Application that initiates a workflow
• Deciders – Control the flow of activity tasks ina workflow Execution. if a job fails, decider will decide what to do next.
• Activity Workers – Carry out the activity tasks.

SWF Diagram

SQS vs SWF

---

SQS	SWF
14 Day Retention Period	Upto 1 Year for workfow executions
Message oriented API	Task oriented API
We need to manage duplicated messages	SWF ensures that task is assigned only once and never duplicated
Application level tracking is user responsibility	Keeps track of all events and tasks in an application

Simple Notification Service(SNS)

---

• Amazon Simple Notification Service (SNS) is a flexible, fully managed pub/sub messaging and mobile notifications service for coordinating the delivery of messages to subscribing endpoints and clients.
• SNS eliminates the complexity and overhead associated with managing and operating dedicated messaging software and infrastructure.

SNS

• can send messages to following endpoints.
• HTTP,
• HTTPS
• Email
• Email-Json
• SMS
• Application
• Lamda

SNS and SQS are messaging services in AWS.

SNS is Push based, SQS is Pull Based

Elastic Transcoder

• Converts media files from original source format to different formats that will play on smartphones, tablets, pc’s etc..
• It converts the best possible performance according to the device it plays.

API Gateway

---

• Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
• Amazon API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, authorization and access control, monitoring, and API version management
• Amazon API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, authorization and access control, monitoring, and API version management

API Gateway Diagram

API Gateway

---

• Benifits of using API Gateway
• Low cost & Efficient
• Scales effortlessly
• Caching available
• connect to cloudwatch to log all the API calls.

Kinesis

---

• Amazon Kinesis makes it easy to collect, process, and analyze realtime, streaming data so you can get timely insights and react quickly to new information.
• With Amazon Kinesis, you can ingest real-time data such as application logs, website clickstreams, IoT telemetry data, and more into your databases, data lakes and data warehouses, or build your own real-time applications using this data.
• Amazon Kinesis enables you to process and analyze data as it arrives and respond in real-time instead of having to wait until all your data is collected before the processing can begin.

Kinesis Types

• Kinesis Streams – Data are stored in shards 1 MB per shard.
• Kinesis Firehose – Can Scale Automatically.
• Kinesis Analytics – Can do Analytics on top of it.

Kinesis workflow