This is a high-level overview that the top services on AWS a data engineer should know in order to solve their data engineering challenges. I explain it by using an example of integrating 2 different data sources to create a central data repository to enable our hypothetical analytics team to perform their own self-service analytics. This video is broken down into Data Ingestion, Data Lake, Transformation, Data Warehouse, Data Analytics, Application Integration, Data Pipeline Orchestration, and Monitoring.


Timeline

00:00 Introduction
01:05 Data Ingestion
04:24 Storage - S3
05:10 Transformation
05:47 Data Catalog
06:44 Data Warehouse
07:13 Data Analytics
09:08 Application Integration
10:28 Orchestration
11:57 Monitoring

buy me a coffee: https://www.buymeacoffee.com/dataengu

useful links:

AWS Serverless Data Lake Architecture:    • AWS Serverless Data Lake Architecture  
Optimize Data Lake:    • 3 Tips To Optimize Your Data Lake On ...  
SNS vs SQS:    • SNS vs SQS Comparison? Whats the diff...  

#AWS
#dataengineering