Skip to content
Go back

03 - Unlocking AWS Storage - The Complete 2024 Guide

Published:  at  05:00 PM

AWS provides a comprehensive portfolio of storage services, each optimized for specific use cases: object, block, file, backup, migration, and hybrid. Below are the latest details and features for each major storage service.


Amazon S3 (Simple Storage Service)

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Buckets & ObjectsS3 stores data as objects in buckets. Each object has data, metadata, and a unique key. Buckets are globally unique.
Storage ClassesMultiple classes for cost and performance:
- S3 Standard (frequent access)
- S3 Intelligent-Tiering (auto cost optimization)
- S3 Standard-IA (infrequent access)
- S3 One Zone-IA (single AZ)
- S3 Glacier Instant Retrieval (low-cost, fast access)
- S3 Glacier Flexible Retrieval
- S3 Glacier Deep Archive (lowest cost, long-term)
- S3 Express One Zone (high-performance, single AZ, 2023+).
VersioningMaintains multiple versions of an object for protection against deletion/overwrites.
Replication- Cross-Region Replication (CRR): For DR and compliance.
- Same-Region Replication (SRR): For backup within a region.
- Multi-Destination Replication (2022+): Replicate to multiple buckets/regions.
Encryption- Server-Side Encryption (SSE-S3, SSE-KMS, SSE-C).
- Client-Side Encryption.
- Default encryption can be enforced at bucket level.
Performance & AccessS3 scales automatically. Supports parallelization, multipart uploads, high concurrency.
- S3 Access Points for managing access at scale.
- S3 Object Ownership for simplifying ACLs.
S3 Select & Object Lambda- S3 Select: Retrieve subsets of data using SQL-like queries.
- S3 Object Lambda: Transform data on the fly using Lambda functions (e.g., redact, reformat).
Event NotificationsNative integration with Lambda, SNS, SQS for event-driven architectures.
Strong ConsistencyRead-after-write consistency for all operations.
Lifecycle ManagementAutomate transition and expiration of objects across storage classes.
Requester PaysOption to have data download costs paid by requester.

Amazon EBS (Elastic Block Store)

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Persistent Block StorageProvides block-level storage for EC2, like virtual hard drives.
Volume Types- gp3: General Purpose SSD (default, cost-effective, customizable IOPS/throughput).
- io2/io2 Block Express: High IOPS SSD for critical workloads.
- st1: Throughput-optimized HDD.
- sc1: Cold HDD.
Elastic ResizingVolumes can be resized and performance adjusted without downtime.
AZ-BoundVolumes are tied to a single Availability Zone. Snapshots can be copied across AZs/regions.
SnapshotsIncremental, stored in S3, can be shared across accounts/regions.
- Fast Snapshot Restore for low-latency restores.
EncryptionEncryption at rest and in transit, integrated with AWS KMS.
Multi-Attachio1/io2 volumes can be attached to multiple EC2 instances (clustered applications).
RecyclingRecycle Bin for EBS Snapshots (2022+): Retain deleted snapshots for a retention period.

Amazon EFS (Elastic File System)

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Scalable NFSFully managed NFS file system, scales automatically.
For EC2 & ContainersMountable across multiple EC2, ECS, EKS, and Lambda (via EFS Access Points).
High AvailabilityData stored across multiple AZs (Regional).
EFS One ZoneLower-cost, single-AZ option for non-critical data.
Performance Modes- General Purpose (default)
- Max I/O (for highly parallel workloads)
Storage Classes- Standard
- Infrequent Access (IA): Automatic lifecycle management moves files to IA for cost savings.
EncryptionAt rest and in transit, integrated with KMS.
Use CasesShared file storage, analytics, content management, lift-and-shift, serverless.

Amazon FSx Family

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
FSx for LustreHigh-performance file system for HPC, ML, analytics. Integrates with S3.
FSx for Windows File ServerFully managed Windows file system with SMB protocol, AD integration.
FSx for NetApp ONTAPEnterprise features: snapshots, cloning, multi-protocol (NFS, SMB), data tiering to S3.
FSx for OpenZFSOpen-source ZFS file system, snapshots, clones, data compression.

AWS Backup

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Centralized BackupsManages and automates backups for EBS, RDS, DynamoDB, EFS, FSx, EC2, S3 (2023+), and more.
PITR (Point-in-Time Recovery)Restore resources to a specific point in time.
Vault LockImmutability for backup data (WORM).
Cross-Region & Cross-AccountBackup copies for compliance and DR.
Backup PoliciesAutomated backup scheduling, lifecycle management, and compliance reporting.
Application-Consistent BackupsIntegrates with AWS services for consistent backups (e.g., RDS, EC2).

AWS Storage Gateway

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Hybrid StorageConnects on-premises environments with AWS Cloud storage.
Gateway Types- File Gateway: NFS/SMB interface to S3.
- Volume Gateway: Block storage with cloud-backed snapshots.
- Tape Gateway: Virtual tape library for backup workflows.
Use CasesBackup, disaster recovery, hybrid cloud storage, migration.

AWS DataSync

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Automated Data TransferFast, secure, automated data movement between on-premises storage and AWS, or between AWS services.
Supported Sources/DestinationsS3, EFS, FSx, NFS, SMB, HDFS, and more.
Use CasesMigration, backup, replication, analytics.

AWS Snow Family

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Edge & Migration DevicesPhysical devices for edge computing, data migration, and disconnected environments.
Products- Snowcone: Small, portable edge device.
- Snowball Edge: Storage and compute at the edge.
- Snowmobile: Exabyte-scale data transfer.
Use CasesLarge-scale migration, edge analytics, disaster recovery, remote operations.

AWS Transfer Family

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Managed File TransfersFully managed SFTP, FTPS, and FTP service for transferring files directly into and out of S3 or EFS.
Use CasesSecure data exchange, partner integrations, legacy migration.

Amazon S3 Glacier

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
Archival StorageLow-cost storage for data archiving and long-term backup.
Retrieval Options- Instant Retrieval
- Flexible Retrieval
- Deep Archive (lowest cost, hours retrieval)
Use CasesCompliance, digital preservation, backup, cold data storage.

Amazon Elastic File Cache (EFC) (Preview/Region Limited)

Official docs

Cues (Keywords/Questions)Notes (Detailed Explanations)
File System CachingHigh-speed cache for NFS file systems, accelerates access to frequently used data.
Use CasesHPC, analytics, media workflows.

Summary

Abstractions:

New in 2023-2024:

For the latest details, always refer to the AWS Storage Documentation at https://aws.amazon.com/products/storage/


Suggest Changes

Previous Post
04 - AWS Database Ecosystem - Complete Service Breakdown
Next Post
02 - AWS Data Engineering Fundamentals