Resource hierarchy
At the top of the hierarchy are projects, then datasets, and finally tables/views. The table below summarizes each level and its purpose.- Projects: A project is the top-level container. Every BigQuery dataset, table, and job belongs to a project. Billing is associated with the project and IAM permissions/quotas are evaluated at this level.
- Datasets: Datasets group related tables and views. Think of a dataset as a folder that lets you manage access and organize artifacts by purpose, application, or team.
- Tables: Tables store the actual data. BigQuery uses columnar storage and is optimized for analytical queries (scans across columns rather than rows).
- Views: Views are saved SQL statements. They don’t store data — when you query a view, BigQuery executes the underlying query and returns results. Use views to hide complexity, expose curated projections, or implement row/column-level filters.
Data types supported in BigQuery
BigQuery supports primitive and complex data types so you can represent traditional relational data and modern nested/semi-structured formats.
These complex types (
ARRAY, STRUCT, JSON) let BigQuery natively store nested and semi-structured data such as logs, events, and clickstreams without prior normalization.
Using columnar storage with nested types allows BigQuery to read only the required columns (and nested fields), which reduces query I/O and cost when you query selectively.
How queries run in BigQuery
Understanding the query execution lifecycle helps you write efficient and cost-effective queries. Typical steps:- Parsing and validation
- BigQuery parses SQL, checks syntax, validates table/column references, and resolves data types.
- Query planning and optimization
- The planner converts SQL into an execution plan and applies optimizations (predicate pushdown, projection pruning, join ordering, etc.).
- Resource allocation
- BigQuery assigns compute resources called slots. For on-demand customers, the service manages slots automatically; flat-rate customers consume slots from their purchased pool.
- Data reading (scanning)
- BigQuery reads the minimum data required. Columnar storage means only referenced columns and relevant partitions/clusters are scanned. Billing is by bytes processed (unless using flat-rate).
- Execution and processing
- Distributed workers perform join, aggregation, window functions, and UDF execution.
- Results delivery
- Results are returned to the console, exported, or written to a destination table.
SELECT * FROM table can be expensive on wide or large tables because they scan all columns. Always select only the columns you need and leverage partitioning and clustering to limit scanned data.
Example — prefer selecting specific columns:
Practical tips to reduce cost and improve performance
- Partition tables by date (or ingestion time) to limit scanned partitions for time-range queries.
- Cluster on frequently filtered columns to co-locate similar values and reduce read I/O.
- Project only the columns you need—avoid
SELECT *. - Use approximate functions (e.g.,
APPROX_COUNT_DISTINCT) when exact precision is not required. - Leverage cached query results when the same query is run and underlying data hasn’t changed.
- For large predictable workloads, consider flat-rate slots to stabilize cost and performance.
Remember: on-demand billing charges by bytes processed for queries and by bytes stored for tables. Partitioning, clustering, and projecting columns are the most effective levers to reduce query cost.
Summary
This lesson covered BigQuery’s core components: projects, datasets, tables, and views. We reviewed primary data types (primitive and complex) and walked through the query execution lifecycle. With this foundation you can better organize data, control access and billing, and write queries that are both performant and cost-efficient. Further lessons will dive deeper into partitioning, clustering, IAM best practices, and advanced query optimization.Links and references
- BigQuery overview (Google Cloud)
- Partitioned tables
- Clustering tables
- BigQuery pricing
- Query optimization best practices