Learn how cloud warehouses and Delta Lake power the modern stack: separate compute, time travel, ACID tables, MERGE, and the Medallion layers.
You have already landed acme-shop's data in an S3 data lake. Now the questions get sharper.
An AWS lakehouse is four layers working together: S3 stores the files, a table format organises them, the Glue Data Catalog registers them, and query...
Athena, Redshift, and Spark all read the same lake, but each is built for a different workload and billed differently.
Plain Parquet files on S3 are cheap and fast to read, but they break in three predictable ways once several jobs write to them.
Athena can create and modify Iceberg tables with ordinary SQL, including UPDATE, DELETE, and MERGE.
A slowly changing dimension keeps history when an attribute such as a customer's city changes, so an old order still joins to the city the customer...
You have already landed acme-shop's data in an S3 data lake. Now the questions get sharper.
An AWS lakehouse is four layers working together: S3 stores the files, a table format organises them, the Glue Data...
Athena, Redshift, and Spark all read the same lake, but each is built for a different workload and billed differently.
Plain Parquet files on S3 are cheap and fast to read, but they break in three predictable ways once several jobs write...
Athena can create and modify Iceberg tables with ordinary SQL, including UPDATE, DELETE, and MERGE.
A slowly changing dimension keeps history when an attribute such as a customer's city changes, so an old order still...
The Medallion Architecture organises a lakehouse into three layers of increasing quality: Bronze, Silver, and Gold.
You do not need hands-on mastery of these today.
⚠️ Security: Use a sandbox AWS account or a personal IAM user with least privilege for S3, Athena, and Glue.
Concepts and services Athena Iceberg syntax seen in this module
Cost and safety mistakes Treating Iceberg snapshots as a backup is the most dangerous one, because it feels like safety.
Roles that use the skills in this module.
Data Engineer
Platform Engineer
Cloud Engineer
DevOps Engineer
Not a software engineer sheet. Every problem comes from real DevOps, SRE, Platform and Cloud interviews, from your first script to a system you build yourself.
Open the Coding SheetAn AWS lakehouse keeps data as files in Amazon S3, as a lake does, and adds a table format such as Apache Iceberg on top. The table format gives warehouse-style guarantees: transactions, schema enforcement, and time travel. Engines like Athena, Redshift, and Spark then query the same tables through the Glue Data Catalog.
Often not at small scale. Athena is enough for ad hoc SQL and light dashboards when tables are partitioned and stored as Parquet. Redshift earns its place when many users run heavy, repeated BI queries and you want predictable performance, caching, and concurrency scaling.
No. Snapshots are kept only until they are expired by maintenance such as VACUUM, and they live in the same bucket as the data. Use time travel to undo a recent mistake, and keep versioning, replication, or a separate backup for anything that must survive longer.
Athena bills by the amount of data a query scans. Queries over unpartitioned CSV or JSON, SELECT star on wide tables, and dashboards that refresh every few minutes can multiply that number quickly. Partitioned Parquet or Iceberg tables and workgroup scan limits are the standard fixes.
On AWS, Iceberg has the deepest native support: Athena, Redshift, Glue, EMR, and S3 Tables all work with it. Delta Lake is a strong choice if your team already runs Spark or Databricks. The better choice is the one your engines already support well.