AWS Storage Gateway

AWS Storage Gateway
Amazon S3
Amazon Glacier
Amazon EBS Snapshots
File Storage
Volume Storage
Tape Storage

3 Interfaces in Storage Gateway –
File Interface
Volume Interface
Tape Interface

Block Storage – Amazon EBS
File Storage – Amazon EFS
Object Storage – Amazon S3

Create a Bucket using:

1. SDKs
2. AWS CLI
3. Amazon S3 Mangement Console

Bucket restrictions & limitations

1. Owned by creator & ownership is not transferable
2. No limit on number of objects
3. No nesting of buckets
4. By default you can create 100 buckets. if you want to increase you need to contact AWS support

Naming a bucket
https://my-s3bucket.s3.amazonaws.com
my-s3bucket –> must be Unique & DNS-Compliant

1. Must be between 3 to 63 Characters long
2. Can contain lowercase letters, numbers, and hyphens & must start and end with a lowercase letter or a number
3. Must not be formatted as an IP address (eg.,198.51.100.1)
   Note: AWS recommends not to use Period symbol in the bucket name

Safe key name characters
Alphanumeric: 0-9, a-z, A-Z
Special Characters: !-_.*'()

Object elements of S3
Bucket Name
Version ID
Value – Objects can be 0-5TB
Metadata
– Creation time & date
– Content type
– Storage class
Access control information
– ACLs
– Resource-based policies
– User-based policies

Object Metadata –

1. System-defined metadata
2. User-defined metadata

how to check object metadata in S3 from AWS CLI (system-defined metadata)
command: aws s3api head-object –bucket mytestbucket –key img013.png -output json

how to modify the object metadata (user-defined metadata)
you need to change the value for the Key: x-amz-meta-mymetadata

S3 Object Storage Classes:
1.S3 Standard (default)
– Active (Frequent) data
– Miliseconds access

2. S3 Intelligent-Tiering
   • Automated tiering
   • Active & infrequent accessed data
   • Miliseconds access
3. S3 Standard-IA
   • Infrequently accessed data
   • Miliseconds access
   • use case – Backup Storage, Disaster Recovery and where data doesn’t change frequently
4. S3 One Zone-IA [important note: with S3 One Zone-IA your data is stored in only 1 Availability Zone. All other storage classes store data in at least 3 Availability Zones]
   • Infrequently accessed data
   • Miliseconds access
   • This cost less compare to S3 Standard-IA
   • use case – For Secondary backup copies of On-premises data, easily recreation data for cross region amazon S3 from an other amazon S3 region
5. S3 Glacier
   • Archive data
   • Minutes to hours access
   • 3 types of retrival option
     1. Expedite: allow you to access within 1 to 5 Minutes for the object within 150 MB
     2. Standard: allow you to access within 3 to 5 Hours
     3. Bulk: allow you to access within 5 to 12 Hours
6. S3 Glacier Deep Archive
   • Long-term archive data & archive preservation
   • Hours access
   • 2 Retrival Options
     1. Standard: allow you to access within 12 Hours
     2. Bulk: allow you to access within 48 Hours
   • use case – electronic medical record, media archive, security camera footage

S3 Intelligent-Tiering
There is 2 access tier option (frequent access tier& infrequent access tier) in this storage class. whenever you have new object that will be going to frequent access tier automatically and if that object not access for 30 days then object will be moved to Infrequent access tier. Again when the same object accessed, that object will be moved to frequent access tier. This process is automated.

[important note: Objects less than 128 KB will not be auto moved, so if you have a data set with lots of small objects (<128 KB), or short lived data this may not be the best storage class to choose.]

AWS S3 Integration with other major AWS Services:

1. Amazon Elastic Compute Cloud (Amazon EC2)
2. Amazon EMR
3. Amazon CloudFront
4. AWS Snowball
5. AWS Snowball Edge
6. AWS Snowmobile

Access Control Overview
By default, bucket access is Private (Block all public access)

2 types of access

1. Resource-based policy
2. User-based policy

Access Control Lists (ACLs)

Server-side encryption
S3 encrypt the object before storing & decrypt while download
3 types Manage encryption keys
1. Amazon S3 Managed Keys (SSE-S3)
2. AWS KMS-Managed Keys (SSE-KMS)
3. Customer Provided Keys (SSE-C)
Default Encrytion with SSE-S3 or SSE-KMS.

Client-side encryption
Data is encrypted before upload
2 Options Client-side encryption
1. AWS KMS-managed customer master key
2. Client-side master key

Knowledge Check-

1. What Amazon S3 feature can be used to prevent someone from granting public access to your data stored in S3 buckets ?
   a. AWS Identity and Access Management (IAM)
   b. Amazon S3 Block Public Access –> Answer
   c. Amazon S3 Select
   d. AWS Trusted Advisor
2. What is different about S3 One Zone-IA and the other object storage classes ?
   a. It stores in only 1 Availability Zone –> Answer
   b. It can store objects in only 1 region
   c. It can store objects in only 1 bucket
   d. It stores objects in only 2 Availability Zones
3. What type of resource policy can be used to control access to an Amazon S3 bucket ?
   a. Lifecycle policy
   b. Bucket policy –> Answer
   c. IAM policy
   d. User policy
4. Which of the following is part of an object ?
   a. IAM policy
   b. Key name –> Answer
   c. Bucket
   d. Bucket policy
5. What are six storage classes available in Amazon S3 ? (Select SIX)
   a. Amazon S3 One Zone-Infrequent Access
   b. Amazon S3 Access Control List
   c. Amazon S3 Standard-Infrequent Access
   d. Amazon S3 Select
   e. Amazon S3 Glacier
   f. Amazon S3 Standard
   g. Amazon S3 Intelligent-Tiering
   h. Amazon S3 Glacier Deep Archive
   Answer: a,c,e,f,g,h

AWS (Amazon Web Services) Acronyms & Abbreviations

While going through AWS cloud. I came across many acronyms to know for better understanding. To remember acronyms / abbreviations I do list of notes. I don’t say I have noted everything, I have noted whichever I have gone through. I thought to share it & I will keep on add it. I am sorry to say that I didn’t sorted alphabetically.

IAM – Identity Access Management
AMI – Amazon Machine Image
S3 – Simple Storage Service
ACL – Access Control List
ACP – Access Control Permissions
RRS – Reduced Redundancy Storage
JSON – Javascript Object Notation
CLI – Command Line Interface
EC2 – Amazon Elastic Compute Cloud
SSH – Secure Socket Shell
SSL – Secure Sockets Layer
TLS – Transport Layer Security
ACM – AWS Certificate Manager
SNI – Server Name Indication
CLB – Classic Load Balancer
ELB – Elastic Load Balancer
ASG – Auto Scaling Group
EBS – Elastic Block Storage
EFS – Elastic File System
RDS – Relational Database Service
DR – Disaster Recovery
TDE – Transparent Data Encryption
DNS – Domain Name System
CORS – Cross Origin Resource Sharing
CRR – Cross Region Replication
SRR – Same Region Replication
WORM – Write Once Read Many
CDN – Content Delivery Network
OAI – Origin Access Identity
SMB – Server Message Block
iSCSI – Internet Small Computer Systems Interface
SQS – Simple Queue Service
SNS – Simple Notification Service
KDS – Kinesis Data Streams
SES – Simple Email Service
DLQ – Dead Letter Queue
SSE – Server-Side Encryption
CMK – Customer Master Key
RCU – Read Capacity Units
WCU – Write Capacity Units
DAX – DynamoDB Accelerator
TTL – Time To Live
DMS – Database Migration Service
CUP – Cognito User Pools
JWT – JSON Web Tokens
FIP – Federated Identity Pool
SAM – Serverless Application Model
STS – Security Token Service
ELB – Elastic Load Balancer
POPs – Points Of Presence (CloudFront)
NFS – Network File System
OLTP – OnLine Transaction Processing
OLAP – Online Analytical Processing
MPP – Massively Parellel Query Execution
IoT – Internet of Things
S3 – Simple Storage Service
ASG – Auto Scaling Group
ECS – Elastic Container Service
ECR – Elastic Container Registry
HSM – Hardware Security Module
OIDC – OpenID Connect [OAuth authentication layer]
PII – Personally Identifiable Information
RBAC – Role Based Access Control
KPI – Key Performance Indicator
ISP – Internet Service Provider
NVMe – Non-Volatile Memory Express
ENI – Elastic Network Interfaces
ENA – Elastic Network Adapter
EFA – Elastic Fabric Adapter
MPI – Message Passing Interface
HPC – High Performance Computing
MPI – Message Passing Interface
NCCL – NVIDIA Collective Communications Library
ML – Machine Learning
ARN – Amazon Resource Names
PFS – Perfect Forward Secrecy
DDoS – Distributed Denial of Service
RAM – AWS Resource Access Manager
HSM – AWS Cloud-based Hardware Security Module (CloudHSM)
EMR – Elastic Map Reduce
SPF – Sender Policy Framework
DKIM – Domainkeys Identified Mail
EDA – Electronic Design Automation
DFSR – Distributed File System Replication
CIDR – Classless Inter-Domain Routing
NAT – Network Address Translation (Gateway)
NACL – Network Access Control Lists
SAML – Security Assertion Markup Language
KPI – Key Performance Indicator
ROA – Route Origin Authorization
BYOIP – Bring Your Own IP Addresses
RPO – Recovery Point Objective
RTO – Recovery Time Objective
ABAC – Attribute-Based Access Control
RBAC – Role-Based Access Control
RIR – Regional internet registry
ARIN – American Registry for Internet Numbers
RIPE – Réseaux IP Européens Network Coordination Centre
TCO – Total Cost of Ownership

BIND – Berkeley Internet Name Domain