# KodeKloud: Beta Courses Google Cloud Professional Data Engineer Certification

## Google Cloud Professional Data Engineer Certification

### Introduction

- [Course Overview](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Introduction/Course-Overview/page.md): Overview of a hands-on course preparing learners for the Google Cloud Professional Data Engineer certification, covering GCP data services, ETL, orchestration, security, ML integration, and exam labs.

### GCP Networking

- [Cloud VPC Introduction](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Networking/Cloud-VPC-Introduction/page.md): Introduction to Cloud VPC explaining logical isolation, IP addressing, subnets, routing, firewall control, connectivity options, and best practices for designing secure cloud networks.
- [Cloud VPC IP Addresses and Subnets](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Networking/Cloud-VPC-IP-Addresses-and-Subnets/page.md): Overview of Google Cloud VPCs, IP addressing and subnets, public versus private networking, Cloud NAT, firewall rules, and CIDR planning best practices for scalable and secure deployments.
- [Demo Creating a VPC and Subnet in GCP](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Networking/Demo-Creating-a-VPC-and-Subnet-in-GCP/page.md): Step-by-step tutorial showing how to create and manage a custom VPC and subnet in Google Cloud, including CIDR sizing, firewall rules, verification, and cleanup.
- [Cloud VPC Private Service Connect](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Networking/Cloud-VPC-Private-Service-Connect/page.md): Explains Google Cloud Private Service Connect enabling private internal IP access to Google and third party services, using service attachments, restricted endpoints and static routes to avoid public internet
- [Cloud VPC Connecting to other networks](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Networking/Cloud-VPC-Connecting-to-other-networks/page.md): Overview of Google Cloud VPC connectivity options for on-prem, VPC-to-VPC, interconnect, and multi-cloud scenarios comparing VPN, Dedicated and Partner Interconnect and peering
- [Cloud VPC Project Isolation Techniques](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Networking/Cloud-VPC-Project-Isolation-Techniques/page.md): Guide to GCP project isolation strategies comparing Shared VPC and separate VPCs, and explaining IAM, firewall controls and connectivity options to secure multi-project networks.
- [Cloud NAT Network Address Translation](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Networking/Cloud-NAT-Network-Address-Translation/page.md): Explains Google Cloud NAT, a managed service that enables outbound internet access for private VMs without external IPs, preserving privacy, scalability, and centralized egress control.
- [Cloud Firewall Rules](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Networking/Cloud-Firewall-Rules/page.md): Overview of Google Cloud VPC firewall rules, their components, configuration examples, best practices and troubleshooting guidance for data engineers securing and managing network access to instances

### Identity and Access Management (IAM) in GCP

- [Cloud IAM Introduction](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Cloud-IAM-Introduction/page.md): Introduction to Google Cloud IAM explaining identities, roles, policies, least privilege best practices, service accounts, policy inheritance, auditing, and permission troubleshooting
- [Cloud IAM Principles of Least Privilege](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Cloud-IAM-Principles-of-Least-Privilege/page.md): Guidelines for applying the Principle of Least Privilege in cloud IAM to minimize permissions, use scoped and custom roles, conduct audits, and secure service accounts
- [Cloud IAM Permissions and Roles](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Cloud-IAM-Permissions-and-Roles/page.md): Explains Google Cloud IAM permissions, roles, policy structure, best practices and tooling to enforce least privilege across BigQuery, Cloud Storage, Compute Engine and other resources.
- [Cloud IAM Role Type Details](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Cloud-IAM-Role-Type-Details/page.md): Explains Google Cloud IAM role types—basic, predefined, and custom—and best practices for granting least privilege to reduce blast radius and manage permissions
- [Cloud IAM Types of Service Accounts](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Cloud-IAM-Types-of-Service-Accounts/page.md): Explains Google Cloud service accounts, their types, usage differences, and production best practices for secure least privileged identity management.
- [Cloud IAM Groups](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Cloud-IAM-Groups/page.md): Explains using IAM groups to manage Google Cloud permissions by assigning roles to groups for scalable, consistent access control, simplifying onboarding, offboarding, and enforcing least privilege
- [Cloud IAM Access Approval](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Cloud-IAM-Access-Approval/page.md): Guide to Cloud IAM Access Approval explaining manual review workflow for sensitive roles, time-bound permissions, approval best practices, approvers' checks, auditing and automated revocation.
- [Org Level Roles](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Org-Level-Roles/page.md): Explains Google Cloud organization-level IAM roles, inheritance across org folder project, and guidance on Viewer Editor Owner permissions and least privilege
- [Demo Cloud IAM Roles and Permissions](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Demo-Cloud-IAM-Roles-and-Permissions/page.md): Guide to creating a custom Google Cloud IAM role, assigning it to a service account, generating JSON keys, and verifying Cloud Storage read permissions.
- [Demo Using Service account in local editor](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Identity-and-Access-Management-IAM-in-GCP/Demo-Using-Service-account-in-local-editor/page.md): Guide to using a downloaded Google Cloud service account JSON key locally with gcloud CLI to authenticate, verify permissions, adjust roles, and clean up credentials.

### Cloud Observability

- [Cloud Logging](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Cloud-Observability/Cloud-Logging/page.md): Overview of Google Cloud Logging, its components, workflow, configuration patterns, and guidance for ingesting, routing, storing, querying, exporting, and monitoring logs for data engineers.
- [Demo Exploring Logs Explorer](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Cloud-Observability/Demo-Exploring-Logs-Explorer/page.md): Hands-on guide to creating and locating Cloud Logging entries in GCP using Cloud Shell and Logs Explorer, with resource and time filters, full-text and structured JSON queries for troubleshooting
- [Cloud Monitoring](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Cloud-Observability/Cloud-Monitoring/page.md): Overview of Google Cloud Monitoring, covering telemetry sources, agents, dashboards, alerts, custom metrics, and integration with logging and tracing for observability and troubleshooting.

### Development & CI/CD

- [Google Cloud SDK Components gcloud CLI](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Development-CICD/Google-Cloud-SDK-Components-gcloud-CLI/page.md): Guide to Google Cloud SDK and gcloud CLI covering components, authentication, configuration, example commands, and best practices for automation, security, and multi project workflows
- [Demo Hands on with gcloud CLI](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Development-CICD/Demo-Hands-on-with-gcloud-CLI/page.md): Hands-on guide demonstrating gcloud CLI usage in Cloud Shell, covering common commands, configuration, authentication, and interacting with Compute Engine, Cloud Storage, IAM, and GKE.
- [Demo Hands on creating complete infrastructure stack using gcloud CLI](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Development-CICD/Demo-Hands-on-creating-complete-infrastructure-stack-using-gcloud-CLI/page.md): Step-by-step gcloud CLI walkthrough to provision a GCP infrastructure stack including service account and IAM, VPC, subnet, firewall rules, a private Compute Engine VM, and cleanup.
- [Cloud Build serverless CICD platform](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Development-CICD/Cloud-Build-serverless-CICD-platform/page.md): Google Cloud Build serverless CI/CD service automating building testing and deploying artifacts via containerized build steps
- [Cloud Build YAMLs](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Development-CICD/Cloud-Build-YAMLs/page.md): Explains a Cloud Build YAML example to build a Docker image, push to Google Container Registry, and deploy to Cloud Run as a simple CI/CD pipeline with best practices.
- [Development and CICD Summary](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Development-CICD/Development-and-CICD-Summary/page.md): Guide summarizing using Google Cloud SDK and gcloud CLI, practicing commands, and using Cloud Build for automated CI/CD pipelines to build, test, and deploy applications to Cloud Run

### Data Security & Encryption

- [Data Loss Prevention DLP API](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Security-Encryption/Data-Loss-Prevention-DLP-API/page.md): Overview of Google Cloud Data Loss Prevention API for discovering, classifying, and de‑identifying sensitive data across storage, databases, logs, and images with monitoring and integration best practices.
- [DLP API Integration with GCP Services](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Security-Encryption/DLP-API-Integration-with-GCP-Services/page.md): Overview of Google Cloud DLP API integrations with GCP services for discovering and de-identifying sensitive data, usage patterns, workflows, and best practices.
- [Cloud Key Management Service KMS](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Security-Encryption/Cloud-Key-Management-Service-KMS/page.md): Overview of Google Cloud Key Management Service managing cryptographic key lifecycle, protection levels, hierarchy, and common data engineering use cases for CMEK, Cloud HSM, and EKM

### Data Ingestion Options

- [Data Ingestion Introduction](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Data-Ingestion-Introduction/page.md): Overview of data ingestion patterns, contrasting batch and streaming approaches, pipeline stages, trade-offs, and GCP services for building reliable ingestion pipelines
- [Batch vs Streaming Ingestion Patterns](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Batch-vs-Streaming-Ingestion-Patterns/page.md): Comparison of batch versus streaming data ingestion patterns, covering latency, volume, complexity, cost, use cases, and guidance for choosing or combining approaches.
- [What is a Coupled System](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/What-is-a-Coupled-System/page.md): Explains differences between tightly coupled and loosely coupled microservice designs using login and fraud services, comparing trade offs, use cases, and message bus options.
- [Cloud PubSub Intro Topics Subscriptions](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Cloud-PubSub-Intro-Topics-Subscriptions/page.md): Overview of Google Cloud Pub/Sub focusing on topics, subscriptions, delivery modes, architecture patterns and exam-relevant concepts for building scalable reliable event-driven ingestion systems
- [Connecting to Cloud PubSub](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Connecting-to-Cloud-PubSub/page.md): Overview of connecting applications to Google Cloud Pub/Sub, covering client libraries, authentication with Application Default Credentials, publish and subscribe patterns, code examples, and operational best practices
- [Cloud PubSub PullPush methods](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Cloud-PubSub-PullPush-methods/page.md): Compares Cloud Pub/Sub push and pull delivery methods, explaining mechanisms, acknowledgements, use cases, scaling, and guidance for choosing between low-latency push and batch-oriented pull
- [Cloud PubSub Acknowledgement](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Cloud-PubSub-Acknowledgement/page.md): Explains Google Cloud Pub/Sub acknowledgement mechanisms, deadlines, ack/nack behaviors, and Python examples for reliable at least once message processing
- [Cloud PubSub Delivery Guarantees](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Cloud-PubSub-Delivery-Guarantees/page.md): Explains Cloud Pub/Sub delivery guarantees, comparing at‑least‑once and exactly‑once semantics, how acknowledgements affect duplicates, and guidance on idempotency, deduplication, and use cases.
- [Cloud PubSub Dead Letter Queue DLQ](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Cloud-PubSub-Dead-Letter-Queue-DLQ/page.md): Overview of Google Cloud Pub/Sub dead letter queues, configuration, causes, monitoring, and reprocessing strategies for handling messages that repeatedly fail processing
- [Demo Working with PubSub](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Demo-Working-with-PubSub/page.md): Guide demonstrating how to create Pub/Sub topics and subscriptions in GCP, publish JSON messages with attributes using gcloud, and pull messages while following best practices
- [Demo Hands on with PubSub Subscription and its integration](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Demo-Hands-on-with-PubSub-Subscription-and-its-integration/page.md): Hands on demo connecting Pub/Sub topic to Cloud Storage so published messages are written as files for zero infrastructure ingestion and downstream processing.
- [Datastream For Change Data Capture CDC](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Ingestion-Options/Datastream-For-Change-Data-Capture-CDC/page.md): Explains Google Cloud Datastream CDC service that captures database change events and streams them to BigQuery, Cloud Storage, or Pub/Sub for near real time analytics and event driven pipelines.

### Data Storage Options

- [Structured Unstructured and Semi structured Data](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Storage-Options/Structured-Unstructured-and-Semi-structured-Data/page.md): Explains structured, semi-structured, and unstructured data using an e-commerce example and cloud storage guidance, mapping data types to storage patterns and practical tips for Google Cloud
- [Google Cloud Storage GCS Overview and Bucket Setup](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Storage-Options/Google-Cloud-Storage-GCS-Overview-and-Bucket-Setup/page.md): Guide to Google Cloud Storage and how to create, configure, and manage production ready buckets for data engineering including storage classes, location, access control, versioning, and lifecycle policies.
- [Demo Setting up GCS Bucket](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Storage-Options/Demo-Setting-up-GCS-Bucket/page.md): Guide to creating and configuring Google Cloud Storage buckets, including lifecycle rules, access control, storage classes, uploading objects, and gsutil CLI examples
- [GCS Storage Class Lifecycle Rules Versioning and Retention](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Storage-Options/GCS-Storage-Class-Lifecycle-Rules-Versioning-and-Retention/page.md): Overview of Google Cloud Storage features including storage classes, lifecycle rules, object versioning, retention policies and retention lock to optimize costs and protect data.
- [GCS Availability and Failover Strategies](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Storage-Options/GCS-Availability-and-Failover-Strategies/page.md): Explains GCS durability, location choices, automatic failover, strong consistency, and disaster recovery strategies including versioning and Storage Transfer Service
- [GCS Quick Summary](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Storage-Options/GCS-Quick-Summary/page.md): Concise guide to Google Cloud Storage fundamentals including objects and buckets, storage classes, locations, lifecycle management, versioning and retention, security controls, and common commands

### Database (SQL, NoSQL and memory)

- [Databases in GCP SQL NoSQL and Memory](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Databases-in-GCP-SQL-NoSQL-and-Memory/page.md): Comparison of GCP database families and services, their use cases, tradeoffs, and guidance for choosing SQL, NoSQL, and in memory solutions like Spanner BigQuery Firestore Bigtable Memorystore
- [Basics on Databases OLTP and OLAP](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Basics-on-Databases-OLTP-and-OLAP/page.md): Explains differences between OLTP and OLAP databases, their use cases, pros and cons, integration patterns like ETL or CDC, and corresponding Google Cloud services
- [Structured Datastore Options in GCP](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Structured-Datastore-Options-in-GCP/page.md): Comparison of Cloud SQL and Cloud Spanner for structured GCP datastores, with guidance on choosing by scale, consistency, availability, operational effort, and cost.
- [Cloud SQL](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Cloud-SQL/page.md): Overview of Google Cloud SQL, a fully managed relational database service that automates maintenance, backups, replication, and offers high availability, engine choices, monitoring, and secure connectivity.
- [Cloud Spanner](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Cloud-Spanner/page.md): Overview of Cloud Spanner, Google Cloud’s globally scalable, strongly consistent relational database with SQL, ACID transactions, automatic sharding, Paxos replication, and managed high availability.
- [Cloud Spanner Internals](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Cloud-Spanner-Internals/page.md): Explains Google Cloud Spanner internals, covering global API gateway, regional processing, TrueTime commit semantics, Colossus-backed split storage, replication, and scalability for globally consistent transactions.
- [Demo Cloud Spanner Setup and Table Creation](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Demo-Cloud-Spanner-Setup-and-Table-Creation/page.md): Guide to provisioning Cloud Spanner, creating an interleaved flights and bookings schema, adding indexes, inserting sample data, and running queries in Spanner Studio.
- [How does Cloud Spanner achieve Scalability High Availability](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/How-does-Cloud-Spanner-achieve-Scalability-High-Availability/page.md): Explains how Cloud Spanner achieves horizontal scalability and high availability using automatic splitting, distributed replicas, leader election, and transparent rebalancing
- [TrueTime with GCP Cloud Spanner](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/TrueTime-with-GCP-Cloud-Spanner/page.md): Explains Google Cloud Spanner's TrueTime, an interval-based time service that bounds clock uncertainty to enable globally consistent, externally ordered ACID transactions.
- [Cloud Spanner Interleaved Tables](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Cloud-Spanner-Interleaved-Tables/page.md): Cloud Spanner interleaved tables co-locate parent and child rows to reduce disk seeks and speed parent-child queries, with primary key prefix requirement and cascade delete behavior.
- [Demo Cloud Spanner Interleaved Tables Cascading Delete](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Demo-Cloud-Spanner-Interleaved-Tables-Cascading-Delete/page.md): Demonstrates Cloud Spanner interleaved tables and cascading deletes showing performance benefits, example queries, and how deleting a parent atomically removes its interleaved child rows.
- [Cloud SQL vs Cloud Spanner](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Cloud-SQL-vs-Cloud-Spanner/page.md): Comparison of Google Cloud SQL and Cloud Spanner, highlighting differences in scalability, consistency, availability, cost, use cases, and operational complexity
- [Semi Structured Data Options in GCP](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Semi-Structured-Data-Options-in-GCP/page.md): Overview of semi-structured data formats and GCP services for storing, querying, and processing JSON, XML, Avro, Parquet with guidance on use cases and analytics workflows
- [Understanding Bigtable](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Understanding-Bigtable/page.md): Overview of Google Cloud Bigtable, a highly scalable wide-column NoSQL database optimized for low-latency, petabyte-scale time-series and high-throughput workloads, not suited for multi-row transactional relational use.
- [BigTable Data Model and Column Families](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/BigTable-Data-Model-and-Column-Families/page.md): Overview of Bigtable row keys, storage order, and column family design best practices to optimize performance, avoid hotspotting, and reduce I/O and cost
- [BigTable Row Key Design and Principles](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/BigTable-Row-Key-Design-and-Principles/page.md): Guidelines for Bigtable row key and schema design to optimize data distribution, query performance, avoid hotspots, manage timestamps and versions, use column families, denormalization, and sparse columns
- [BigTable Quick Summary](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/BigTable-Quick-Summary/page.md): Compact overview of Google Cloud Bigtable focusing on schema design, row-key strategies, column families, and access-pattern driven performance and scalability best practices.
- [Firestore Intro Data Modeling Datastore Mode](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Firestore-Intro-Data-Modeling-Datastore-Mode/page.md): Explains Google Cloud Firestore features, data models, and differences between Native and Datastore modes for real-time client apps versus backend workloads
- [Demo GCP Firestore Demo](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Demo-GCP-Firestore-Demo/page.md): Walkthrough of Google Cloud Firestore usage covering database creation, adding and querying documents, exporting to Cloud Storage, and preparing data for BigQuery analytics.
- [Cloud Memorystore Intro and Use Cases](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Cloud-Memorystore-Intro-and-Use-Cases/page.md): Overview of Google Cloud Memorystore, caching benefits, Redis versus Memcached tradeoffs, cache-aside patterns, TTL and invalidation guidance to improve read latency and reduce database load.
- [Quick Summary on Database SQL NoSQL and memory](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Database-SQL-NoSQL-and-memory/Quick-Summary-on-Database-SQL-NoSQL-and-memory/page.md): Compact guide to Google Cloud database options (SQL, NoSQL, in-memory) with use cases, scaling, and decision tips for selecting the right service.

### Data Orchestration Options

- [Cloud Composer Orchestrating Data Workflows](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Composer-Orchestrating-Data-Workflows/page.md): Introduction to Cloud Composer, Google’s managed Apache Airflow for orchestrating, scheduling, and monitoring data pipelines across GCP services.
- [Cloud Composer Orchestration Patterns Triggering Methods](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Composer-Orchestration-Patterns-Triggering-Methods/page.md): Explains Cloud Composer DAG triggering methods and orchestration patterns for building reliable, decoupled, and scalable Airflow workflows on Google Cloud.
- [Cloud Composer Monitoring Security and IAM](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Composer-Monitoring-Security-and-IAM/page.md): Guide to Cloud Composer monitoring, security, IAM roles, service accounts, network controls, and cost optimization for running and maintaining production Airflow environments on Google Cloud
- [Cloud Composer Production Best Practices](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Composer-Production-Best-Practices/page.md): Best practices for running Apache Airflow on Google Cloud Composer, covering idempotent tasks, atomic operations, timeouts, retries, secret management, environment separation, and CI/CD for reliable production pipelines.
- [Cloud Composer GCP Integration and Real World Use Cases](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Composer-GCP-Integration-and-Real-World-Use-Cases/page.md): Overview of using Cloud Composer to orchestrate Google Cloud services like Cloud Functions, Pub/Sub, GCS, BigQuery, Dataflow and Dataproc for production data pipelines
- [Cloud Workflows Orchestration](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Workflows-Orchestration/page.md): Explains Google Cloud Workflows, a serverless orchestration service for sequencing API and microservice calls, when to use it and comparisons with Composer and Cloud Functions
- [Cloud Functions Serverless Fundamentals](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Functions-Serverless-Fundamentals/page.md): Overview of Google Cloud Functions and serverless fundamentals, covering event-driven execution, supported runtimes, triggers, examples, billing model, and best uses for short lived stateless tasks.
- [Cloud FunctionsPerformance Characteristics and Common Data Processing Patterns](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-FunctionsPerformance-Characteristics-and-Common-Data-Processing-Patterns/page.md): Overview of Google Cloud Functions performance, cold starts, timeouts, invocations, and common event-driven data processing patterns with practical tips and an example CSV to BigQuery workflow
- [Cloud Functions Real World Integration Examples](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Functions-Real-World-Integration-Examples/page.md): Practical integration patterns using Google Cloud Functions to implement event-driven ETL, database streaming, IoT telemetry processing, and serverless APIs with operational guidance for production.
- [Cloud Functions Best Practices and Optimization](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Functions-Best-Practices-and-Optimization/page.md): Best practices for optimizing Cloud Functions performance, reliability, security including cold-start reduction, resource reuse, retries, observability, secrets and least privilege
- [Cloud Functions vs Cloud Workflows](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Cloud-Functions-vs-Cloud-Workflows/page.md): Comparison of Google Cloud Functions and Cloud Workflows to help choose between short-lived event handlers and long-running, stateful orchestrations coordinating multiple services with retries and branching
- [Quick Summary Data Orchestration Options](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Orchestration-Options/Quick-Summary-Data-Orchestration-Options/page.md): Comparison of Google Cloud orchestration services—Composer, Functions, and Workflows—with use cases, triggers, best fits, examples, and a decision checklist for data pipeline and API orchestration choices.

### Data Processing 

- [Dataflow Introduction](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Dataflow-Introduction/page.md): Overview of Google Cloud Dataflow, a managed serverless service running Apache Beam pipelines for unified batch and streaming data processing, autoscaling, and exporting results to BigQuery or Cloud Storage
- [Data Transformation Pipeline](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Data-Transformation-Pipeline/page.md): Explains a Cloud Dataflow Apache Beam pipeline that ingests Pub/Sub events, applies ParDo transforms and GroupByKey aggregates, and writes results to sinks like Cloud Storage or BigQuery
- [Windowing in Dataflow](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Windowing-in-Dataflow/page.md): Explains windowing in Google Cloud Dataflow, covering fixed sliding session and global windows, watermarks triggers allowed lateness and Apache Beam examples for streaming aggregation.
- [Watermarks Triggers](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Watermarks-Triggers/page.md): Explains watermarks, triggers, and allowed lateness in Dataflow and Apache Beam streaming, showing trigger types, handling late events, and balancing latency versus correctness.
- [Pipeline Design Optimization](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Pipeline-Design-Optimization/page.md): Guide to designing and optimizing Google Cloud Dataflow pipelines, focusing on fusion, shuffles, parallelization, autoscaling, handling data skew, and when to break fusion for better throughput and latency
- [Performance Scaling Cost Optimization](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Performance-Scaling-Cost-Optimization/page.md): Guidance for improving Google Cloud Dataflow pipeline performance, autoscaling, shuffle strategies, handling data skew, Streaming Engine benefits, and cost optimization best practices.
- [Dataflow Exam Summary](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Dataflow-Exam-Summary/page.md): Summary of Apache Beam and Google Dataflow concepts for Data Engineer exam covering unified batch and streaming, windowing, watermarks, triggers, state, autoscaling, templates, integration, and monitoring
- [Dataproc Introduction](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Dataproc-Introduction/page.md): Overview of Google Cloud Dataproc, a managed Hadoop and Spark service for running open source big data workloads with fast provisioning, autoscaling, and tight GCP integration.
- [Dataproc Autoscaling Features](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Dataproc-Autoscaling-Features/page.md): Overview of Google Cloud Dataproc autoscaling, comparing ephemeral and long-lived clusters, autoscaling triggers and settings, graceful decommissioning, and practical best practices for cost and performance
- [Running Spark Jobs on Dataproc](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Running-Spark-Jobs-on-Dataproc/page.md): Guide to running Spark jobs on Google Cloud Dataproc covering preparation, submission, monitoring, storage, and cluster lifecycle for scalable ETL and analytics
- [Preemptible VMs on Dataproc](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Preemptible-VMs-on-Dataproc/page.md): How to use preemptible VMs on Google Cloud Dataproc to cut costs, when they suit workloads, and architecture and reliability best practices for safe deployment.
- [Dataproc Cluster Optimization and Sizing](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Dataproc-Cluster-Optimization-and-Sizing/page.md): Guidance for right‑sizing and optimizing Google Dataproc clusters by matching machine types to workloads, measuring bottlenecks, and using autoscaling, preemptible and ephemeral clusters to reduce cost.
- [Demo Hands on with Dataproc](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Demo-Hands-on-with-Dataproc/page.md): Hands-on demo showing how to run a PySpark job on Google Cloud Dataproc to aggregate CSV orders from Cloud Storage and write results back to Cloud Storage
- [Storage Options and Migration to Dataproc](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Storage-Options-and-Migration-to-Dataproc/page.md): Guide comparing Dataproc storage options, costs, performance, and durability, with practical four‑phase migration steps from Hadoop or Spark to Dataproc
- [Dataproc Summary](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Processing/Dataproc-Summary/page.md): Overview of Google Cloud Dataproc, a managed Apache Spark and Hadoop service, covering features, storage options, cost and scaling strategies, use cases, and integration with GCP.

### Data Integration & Transformation Tools

- [Data Fusion Architecture Core Components](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Integration-Transformation-Tools/Data-Fusion-Architecture-Core-Components/page.md): Overview of Google Cloud Data Fusion, a managed visual, code-free ETL service that builds, runs, and monitors data integration pipelines with connectors, autoscaling, batch and streaming support
- [Data Fusion Studio Use Cases and Data Wrangler](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Integration-Transformation-Tools/Data-Fusion-Studio-Use-Cases-and-Data-Wrangler/page.md): Overview of Google Cloud Data Fusion, focusing on Pipeline Studio and Data Wrangler for building, cleaning, and operationalizing ETL/ELT pipelines to migrate and load data into BigQuery.
- [Data Fusion Best Practices Optimization](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Integration-Transformation-Tools/Data-Fusion-Best-Practices-Optimization/page.md): Guide to Google Cloud Data Fusion best practices for performance, security, cost optimization, and operational DevOps practices, with comparisons to Dataflow and Dataproc for exam and production use
- [Cloud Dataprep by Trifacta](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Integration-Transformation-Tools/Cloud-Dataprep-by-Trifacta/page.md): Serverless, ML-guided, visual data preparation tool on Google Cloud for interactive profiling, cleaning, and transforming datasets with connectors to BigQuery and Cloud Storage
- [Data Fusion vs Dataprep by Trifacta](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Integration-Transformation-Tools/Data-Fusion-vs-Dataprep-by-Trifacta/page.md): Comparison of Google Cloud Data Fusion and Cloud Dataprep by Trifacta, outlining strengths, use cases, and trade offs for enterprise ETL orchestration versus self service data preparation.

###  Data Warehouse & Analytics Options

- [BigQuery Introduction](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Introduction/page.md): Overview of Google BigQuery, its serverless data warehouse architecture, features, columnar storage and query engine, benefits, and common analytics use cases.
- [BigQuery Core Components](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Core-Components/page.md): Overview of BigQuery core components, supported data types, query execution lifecycle, and practical tips for organizing data, controlling access, optimizing queries, and reducing costs.
- [BigQuery Anatomy of BigQuery](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Anatomy-of-BigQuery/page.md): Explains BigQuery architecture and core systems showing how Colossus, Capacitor, Dremel, Borg, Jupiter enable scalable, fast, reliable serverless analytics
- [BigQuery Data Structures Datasets Tables Views](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Data-Structures-Datasets-Tables-Views/page.md): Explains BigQuery data structures and best practices for organizing datasets, tables, views and materialized views, including access control, partitioning, clustering, and DDL examples.
- [Demo Dataset Creation Data Uploads Tables](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/Demo-Dataset-Creation-Data-Uploads-Tables/page.md): Guide to using the BigQuery console to create datasets, tables, insert and query data, build views, inspect job details, and follow best practices
- [Demo Materialized Views](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/Demo-Materialized-Views/page.md): Guide demonstrating BigQuery materialized views including creation, limitations, refresh options, joins, performance benefits, and SQL examples using employees and departments tables
- [BigQuery Table Partitioning and Clustering](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Table-Partitioning-and-Clustering/page.md): Explains BigQuery table partitioning and clustering benefits, differences, examples, best practices, and sample DDL to improve query performance and reduce cost.
- [BigQuery Storage Types and Cost Management](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Storage-Types-and-Cost-Management/page.md): Explains BigQuery storage tiers, physical vs logical storage, slot-based compute pricing, and practical cost management techniques like partitioning, clustering, and choosing on-demand or flat-rate/flex slots.
- [BigQuery Backing up Data](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Backing-up-Data/page.md): Guide to BigQuery Time Travel for querying and restoring historical table snapshots, configuring retention, using bq CLI, and best practices to prevent accidental data loss
- [BigQuery Spark BigQuery Connector](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Spark-BigQuery-Connector/page.md): Explains the Spark BigQuery connector, its Dataproc integration, usage patterns, code examples, performance and IAM considerations for reading and writing BigQuery from Spark jobs
- [BigQuery Federated Queries](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/BigQuery-Federated-Queries/page.md): Explains BigQuery federated queries, how to query external data sources at runtime, use cases, performance and cost considerations, examples and best practices for ad hoc and prototyping analysis.
- [Looker Studio for Visualization](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Warehouse-Analytics-Options/Looker-Studio-for-Visualization/page.md): Overview of Looker Studio integration with BigQuery and Google Cloud, covering dashboards, real-time connectors, performance and cost best practices for scalable, shareable reporting.

### Machine Learning Options

- [Why Machine Learning in GCP Data Professional](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Machine-Learning-Options/Why-Machine-Learning-in-GCP-Data-Professional/page.md): Explains why and how data engineers use machine learning on GCP, highlighting BigQuery ML, SQL-first workflows, pipeline automation, model lifecycle, serving, and production integration.
- [BigQuery ML for Data Engineers](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Machine-Learning-Options/BigQuery-ML-for-Data-Engineers/page.md): Explains how data engineers can build, evaluate, and serve machine learning models inside BigQuery using SQL to reduce data movement, streamline pipelines, and integrate with Google Cloud services
- [BigQuery ML Model Types](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Machine-Learning-Options/BigQuery-ML-Model-Types/page.md): Guide to BigQuery ML workflows using SQL to create, train, evaluate, and predict with models, plus production practices for automating, monitoring, and deploying machine learning at scale.
- [BigQuery ML Workflow](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Machine-Learning-Options/BigQuery-ML-Workflow/page.md): Describes a BigQuery ML SQL workflow to create train evaluate and predict models with hyperparameter tuning metrics and production deployment guidance
- [Achieving Optimal BigQuery ML](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Machine-Learning-Options/Achieving-Optimal-BigQuery-ML/page.md): A practical roadmap for optimizing BigQuery ML covering data preparation, model selection, cost management, and production pipelines to maximize model quality, efficiency, and operational reliability

### Multi-Cloud & Lakehouse Solutions

- [BigQuery Omni Cross Cloud Analytics Revolution](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Multi-Cloud-Lakehouse-Solutions/BigQuery-Omni-Cross-Cloud-Analytics-Revolution/page.md): Explains BigQuery Omni, a managed service that runs BigQuery compute in AWS and Azure to query S3 and Blob Storage in place without moving data.
- [BigQuery Omni vs Data Movement Tools](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Multi-Cloud-Lakehouse-Solutions/BigQuery-Omni-vs-Data-Movement-Tools/page.md): Comparison of BigQuery Omni, Storage Transfer Service, and BigQuery Data Transfer Service to help data engineers decide whether to query data in place or move it into Google Cloud
- [BigLake Intro Unified Security](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Multi-Cloud-Lakehouse-Solutions/BigLake-Intro-Unified-Security/page.md): Introduction to BigLake, a unified storage layer enabling cross‑cloud analytics, unified governance, fine‑grained security, and multi‑engine access to open-format data without data duplication
- [BigLake vs External Tables](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Multi-Cloud-Lakehouse-Solutions/BigLake-vs-External-Tables/page.md): Compare Google Cloud BigLake and BigQuery external tables, highlighting differences in security, governance, performance, and best use cases for production versus ad hoc queries.
- [Quick Summary on Multi Cloud Lakehouse Solutions](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Multi-Cloud-Lakehouse-Solutions/Quick-Summary-on-Multi-Cloud-Lakehouse-Solutions/page.md): Exam-focused guide comparing BigQuery Omni, BigLake, and Google transfer services, explaining use cases, benefits, and tips for choosing the right multi cloud lakehouse solution.

### Data Management and Governance

- [Data Management and Governance](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Management-and-Governance/Data-Management-and-Governance/page.md): Explains centralized data management and governance benefits, capabilities, and practical steps to ensure secure, discoverable, high quality enterprise data using tools like Dataplex.
- [Dataplex The Unified Governance Layer](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Management-and-Governance/Dataplex-The-Unified-Governance-Layer/page.md): Overview of Google Cloud Dataplex, a centralized governance layer that organizes, catalogs, and enforces policies and data quality across distributed assets without moving underlying data.
- [Data Catalog The Discovery and Classification Engine](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Management-and-Governance/Data-Catalog-The-Discovery-and-Classification-Engine/page.md): Overview of data catalogs focusing on discovery, metadata management, tagging and classification to enable governance, lineage, ownership, and faster discovery of datasets with automation and best practices
- [Analytics Hub The Data Sharing Platform](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Management-and-Governance/Analytics-Hub-The-Data-Sharing-Platform/page.md): Explains Google Cloud Analytics Hub for governed secure BigQuery data sharing, enabling publishers to create exchanges and listings while subscribers query linked datasets without copying data.
- [Quick Summary on Data Catalog Dataplex](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/Data-Management-and-Governance/Quick-Summary-on-Data-Catalog-Dataplex/page.md): Comparison of Google Cloud Data Catalog and Dataplex, contrasting metadata discovery and tagging with unified data governance, monitoring, and enforcement across cloud data platforms

### GCP Data Engineering Architecture and Landscape

- [GCP Data Engineering Landscape](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Data-Engineering-Architecture-and-Landscape/GCP-Data-Engineering-Landscape/page.md): Overview of Google Cloud data engineering landscape, mapping services to ingestion, storage, processing, orchestration, and monitoring with guidance on choosing tools for production data platforms.
- [Architecture Real Time Stream Processing IoT](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Data-Engineering-Architecture-and-Landscape/Architecture-Real-Time-Stream-Processing-IoT/page.md): Production-ready architecture for real-time IoT telemetry ingestion, stream processing, storage, analytics, and presentation on Google Cloud Platform using Pub/Sub, Dataflow, BigQuery, Bigtable, and Firestore.
- [Architecture Gaming Analytics](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Data-Engineering-Architecture-and-Landscape/Architecture-Gaming-Analytics/page.md): Scalable GCP architecture for gaming analytics combining batch and streaming ingestion with Cloud Storage, Pub/Sub, Dataflow, BigQuery and low-latency stores for real-time insights and ML.
- [Architecture Publisher side analysis](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Data-Engineering-Architecture-and-Landscape/Architecture-Publisher-side-analysis/page.md): A Google Cloud publisher-side analytics architecture for ingesting, processing, storing, analyzing, and visualizing event data to measure ad performance, attribute conversions, and compute campaign and publisher ROI.
- [Architecture Time Series Analysis](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Data-Engineering-Architecture-and-Landscape/Architecture-Time-Series-Analysis/page.md): A GCP architecture guide for ingesting, processing, storing, and analyzing time series data using Pub/Sub, Cloud Storage, Dataflow, Bigtable, and BigQuery
- [Architecture Complex Event Processing](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Data-Engineering-Architecture-and-Landscape/Architecture-Complex-Event-Processing/page.md): Overview of Complex Event Processing architecture on Google Cloud, detailing streaming and batch pipelines, service roles like Pub/Sub, Dataflow, Bigtable, BigQuery, storage, and downstream integrations

### GCP Core Fundamentals & Understanding

- [Googles View of the Professional Data Engineer](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Core-Fundamentals-Understanding/Googles-View-of-the-Professional-Data-Engineer/page.md): Overview of the Google Cloud Professional Data Engineer certification covering required skills, core GCP technologies, role responsibilities, exam logistics, and practical preparation guidance
- [Lifecycle of data in a company](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Core-Fundamentals-Understanding/Lifecycle-of-data-in-a-company/page.md): Overview of the data lifecycle in businesses covering ingestion, storage, processing, governance, monitoring, and delivery for analytics and machine learning with practical best practices.
- [GCPs Global Infrastructure](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Core-Fundamentals-Understanding/GCPs-Global-Infrastructure/page.md): Explains Google Cloud’s global infrastructure, including regions, zones, and edge PoPs, and guidance on latency, high availability, data residency, and deployment strategies.
- [Legacy Use Cases](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Core-Fundamentals-Understanding/Legacy-Use-Cases/page.md): Discusses modernizing decades-old insurance data systems, challenges of legacy on-premises stacks, and migration strategies to cloud-native streaming, governance, and observability.
- [GCP Resource Hierarchy](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Core-Fundamentals-Understanding/GCP-Resource-Hierarchy/page.md): Explains Google Cloud's resource hierarchy and how organization, folders, projects, and resources determine IAM, policies, inheritance, and billing for centralized governance.
- [GCP Resource Hierarchy Project Identification Framework](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Core-Fundamentals-Understanding/GCP-Resource-Hierarchy-Project-Identification-Framework/page.md): Describes GCP project identifiers (name, project ID, project number), their rules, uses, best practices, and gcloud commands for correct identification and automation.
- [Demo GCP Console Overview](https://notes.kodekloud.com/docs/Google-Cloud-Professional-Data-Engineer-Certification/GCP-Core-Fundamentals-Understanding/Demo-GCP-Console-Overview/page.md): Concise tour of Google Cloud Console for data engineers covering project selection, IDs, navigation, services, and built‑in AI assistants like Gemini