Data Engineering Concepts

Last updated Dec 9, 2022 - Edit Source

• Data engineering
• Concept

Some core concepts we are going to explore:

• Data Warehouse vs Data Lake vs Data Lakehouse
• Storage Layer vs Data Lake File Format vs Data Lake Table Format
• Data Catalog
• Modern Data Stack vs Open Data Stack
• Data Engineering Lifecycle
• ELT vs ETL vs EtLT
• Functional Data Engineering vs Software-Defined Assets
• Metrics Layer vs Semantic Warehouse vs Data Virtualization
  • Metrics, Key Performance Indicator (KPI)
  • Push-Downs vs Rollup
  • Data Modeling vs Dimensional Modeling
  • Data Contract
• OLAP vs OLTP
• MapReduce and Apache Hadoop
• Declarative vs Imperative
• Notebooks

See also What is Data Engineering.