GCP Data Fusion & Big Query training course

Hands-On Lab: Build a Batch Pipeline

• Create a batch pipeline from Cloud Storage to BigQuery

• Apply transformations and data validation

• Test end-to-end data flow

Transformations & Data Quality

• Use Wrangler for data cleaning and preparation

• Perform joins, aggregations, and error handling

• Ensure data quality and consistency

Operationalizing Pipelines

• Schedule and monitor pipelines for production use

• Troubleshoot common issues

• Implement best practices for operational pipelines

End-of-Day Lab

• Complete an end-to-end batch pipeline implementation

• Reinforce learning through practical application

---

Day 2 — Analytics & Optimization with BigQuery

 

BigQuery Architecture & Concepts

• Explore the serverless model and separation of storage vs compute

• Understand BigQuery pricing considerations

• Learn how BigQuery supports analytics at scale

BigQuery Data Modelling

• Design schemas for performance and scalability

• Use partitioning and clustering effectively

• Optimise data layout for queries and storage

BigQuery SQL Fundamentals

• Work with joins, aggregations, and window functions

• Write queries for analytics and reporting

• Apply best practices for efficient SQL

Performance & Cost Optimization

• Tune queries and use materialized views

• Leverage BI Engine for faster analytics

• Apply techniques to reduce query costs

Integrating Data Fusion with BigQuery

• Implement ELT patterns and incremental loads

• Connect Data Fusion pipelines to BigQuery for analytics

• Automate data integration workflows

Security & Governance

• Apply IAM roles and permissions

• Implement row-level and column-level security

• Ensure data governance best practices are followed

Freestyle Lab

• Build a production-style pipeline

• Optimise queries and pipeline performance

• Review and reinforce learning from Day 1 and Day 2