Objects and Buckets
- Objects are the basic data unit in GCS. Each object contains file data plus metadata such as content type, size, creation time, generation, and ACLs/attributes.
- Buckets are top-level containers for objects. Bucket names are globally unique across Google Cloud — two buckets in different projects cannot share the same name.
- Each bucket has two important settings that affect cost, latency, and durability:
- Location (regional, dual-region, multi-region)
- Storage class (Standard, Nearline, Coldline, Archive)
Bucket names are globally unique. Choose names intentionally (for example, include a project or org prefix) to avoid collisions.
- Use project or organization prefixes in bucket names to avoid naming collisions and make ownership obvious.
- Keep metadata and meaningful object naming conventions to simplify lifecycle rules and analytics.
Storage Classes
GCS provides four primary storage classes optimized for different access patterns and cost trade-offs. The table below summarizes use cases, cost characteristics, and minimum storage durations.
Exam tip: for datasets accessed ~once per year, Archive is usually the best fit.
Location and Availability
Choose bucket location according to latency, availability, and compliance requirements:
Decision factors:
- Latency: pick the region(s) closest to your consumers.
- Durability/Availability: dual-region or multi-region increases resilience.
- Cost/compliance: regional storage can reduce cost and help satisfy data residency requirements.
Lifecycle Management
Lifecycle rules automate object transitions (to cheaper storage classes) and deletions based on conditions like object age, storage class, or custom time. Use lifecycle policies to optimize long-term storage cost without manual intervention. Example lifecycle rule: move objects to Nearline after 30 days- Transition objects to cheaper storage classes (
SetStorageClass). - Delete objects automatically after a condition is met (
Delete). - Use
matchesStorageClass,isLive,createdBefore,numNewerVersions, orcustomTimeBeforeconditions to target objects precisely.
- Apply lifecycle via the Cloud Console,
gsutilwith bucket configuration, or the Storage API.
Versioning and Retention
- Versioning: Enable object versioning on a bucket to retain older object generations after overwrite or deletion. This protects against accidental deletes or overwrites.
- Example: enable versioning with gsutil
- Example: enable versioning with gsutil
- Retention policies: Apply an immutable retention period to a bucket to prevent object deletion or modification for a specified duration.
- Example: set a 365-day retention policy
- Example: set a 365-day retention policy
- Bucket Lock: After you lock a retention policy, it becomes permanent and cannot be shortened — enforcing WORM (write once, read many) behavior for compliance.
Be careful: enabling Bucket Lock to permanently enforce retention policies is irreversible. Confirm retention requirements before locking.
Security and Access Control
Encryption:- Data is encrypted in transit and at rest by default.
- Customer-Managed Encryption Keys (CMEK): Use Cloud KMS keys you control to encrypt buckets or objects while Google manages key wrapping.
- Customer-Supplied Encryption Keys (CSEK): You supply encryption keys with each request; GCS does not store them. Use CSEK only if you can securely manage and supply the keys.
- IAM (recommended): Use role-based IAM policies to manage bucket and object access at scale.
- Uniform bucket-level access (recommended): Disable object ACLs and rely exclusively on IAM for simpler, auditable access policies.
- Fine-grained ACLs: Still available for legacy scenarios but are more complex and error-prone.
- Signed URLs and Signed Policy Documents let you grant time-limited access to objects without modifying IAM policies.
Quick Commands and Examples
Note: Replace
my-project-bucket and us-central1 with your actual bucket name and region. Wrap placeholders like <BUCKET_NAME> or <KEY> in backticks when using them inline.
Summary
Key takeaways for Google Cloud Storage:- Objects are the data units; buckets are globally unique containers with configurable location and storage class.
- Choose the right storage class (Standard, Nearline, Coldline, Archive) based on access patterns and cost.
- Select the appropriate location type (regional, dual-region, multi-region) for latency, availability, and compliance.
- Use lifecycle rules to automate storage class transitions and deletions to reduce costs.
- Combine versioning, retention policies, and Bucket Lock to protect data and meet compliance requirements.
- Secure data with encryption (CMEK/CSEK options), IAM-based access controls, uniform bucket-level access, and temporary signed URLs.
Links and References
- Google Cloud Storage documentation
- Cloud KMS documentation (CMEK)
- Signed URLs guide
- Lifecycle management guide