Learn Google Cloud from scratch - projects, IAM, compute, storage, networking, and operations - to confidently manage GCP and clear the Associate Cloud Engineer exam.
### The problem this module actually solves A team migrating from AWS to Google Cloud creates a new GCP project the same way they'd create an AWS account, expecting IAM permissions, billing, and resource organization to work the same way underneath. Within a week they've hit three genuinely confusing moments: why does every resource need a Project, not just a Resource Group; why does a Service Account look like both an identity and a resource at the same time; and why does deleting a project not actually delete it for 30 days. None of this is GCP being needlessly different - it reflects a genuinely different set of design decisions, and understanding those decisions early prevents weeks of confusion later. This module builds Google Cloud understanding from zero, using the same broad structure as the Associate Cloud Engineer (ACE) exam's official five domains - environment setup, planning resources, deploying resources, operating them, and securing access - but the actual goal is competence running a real GCP environment, with the certification as a natural result of that understanding, not the other way around. ### How this module is organized The module moves through Foundations (projects, billing, IAM), Compute (Compute Engine, GKE, Cloud Run, Cloud Functions), Storage and Databases (Cloud Storage, Cloud SQL, Firestore, BigQuery), Networking (VPC, firewall rules, load balancing), and Operations (Cloud Monitoring, Cloud Logging) - each section builds on the previous one, since nearly every later concept assumes you already understand the Project and IAM model covered first. > 📌 **Remember:** In GCP, almost everything traces back to a Project. A Project is the unit of billing, the unit of IAM permission scoping, and the container every resource lives inside - understanding this one concept correctly makes the rest of GCP fall into place faster. ---
### The four-level hierarchy that organizes everything Google Cloud organizes every resource inside a strict hierarchy, and unlike a flat list of accounts, permissions and policies set at a higher level automatically apply to everything beneath it. +------------------------------------------+ | Organization (tied to a Google Workspace | | or Cloud Identity domain) | +------------------------------------------+ | v +------------------------------------------+ | Folders (optional - group projects by team,| | department, or environment) | +------------------------------------------+ | v +------------------------------------------+ | Projects (the actual billing and IAM | | scoping boundary) | +------------------------------------------+ | v +------------------------------------------+ | Resources (VMs, buckets, databases, etc.) | +------------------------------------------+ A **Project** is the fundamental unit almost everything else in GCP is scoped to - every resource belongs to exactly one project, every project has its own unique Project ID (globally unique, chosen once, never changeable), and billing is configured per project by linking it to a Billing Account. > 🔴 **Common Mistake:** Assuming a GCP Project works like an AWS Account or an Azure Subscription in every respect. It is the closest equivalent for IAM and billing scoping, but a single Google Cloud Organization commonly contains many projects, often one per application or environment, rather than the coarser account-per-team pattern common elsewhere. ### Creating and organizing projects ```bash # Create a new project with a specific, permanent Project ID gcloud projects create devops-network-prod-01 \ --name="DevOps Network Production" # List all projects you have access to gcloud projects list # Set a project as the active default for subsequent commands gcloud config set project devops-network-prod-01 ``` > **Note:** The Project ID (`devops-network-prod-01` here) is globally unique across all of Google Cloud and cannot be changed after creation - choose it carefully, since a typo or a name you later regret means creating an entirely new project rather than renaming this one. ### Organizational Policies - governance rules that apply automatically **Organization Policies** let you set constraints that apply across an entire Organization, Folder, or Project - for example, restricting which regions resources can be created in, or disabling the creation of external IP addresses on VMs by default. ```bash # View the current state of a specific organization policy constraint gcloud resource-manager org-policies describe \ compute.vmExternalIpAccess \ --project=devops-network-prod-01 ``` > 💡 **Tip:** Set Organizational Policies at the Folder level when a rule should apply to every project within a specific team or environment, rather than configuring the same constraint individually on each project - policy inheritance means you only need to set it once at the right level. ### Managing users and groups with Cloud Identity **Cloud Identity** is Google's identity management service - similar in role to Azure AD or an AWS Organization's user management, it's where your organization's users and groups actually live, separate from any single project. ```bash # Grant a user a role at the project level gcloud projects add-iam-policy-binding devops-network-prod-01 \ --member="user:priya.sharma@example.com" \ --role="roles/editor" ``` ### Enabling APIs - the gatekeeper for every GCP service Unlike some clouds where every service is available the moment an account exists, GCP requires each service's API to be explicitly enabled per project before it can be used at all - a deliberate friction point that also serves as a simple audit trail of what a project actually uses. ```bash # Enable the Compute Engine API for the active project gcloud services enable compute.googleapis.com # List all currently enabled APIs for the active project gcloud services list --enabled ``` > 🔴 **Common Mistake:** Trying to create a resource and getting a confusing "API not enabled" error, then assuming something is broken with the account. This is expected default behavior - the specific service's API simply needs to be enabled for that project first, which typically takes seconds once run. ### Assessing and requesting quota increases Every project has default resource quotas - a limit on how many VM CPUs, IP addresses, or API calls can be used per project per region, designed to prevent runaway costs and protect shared infrastructure. ```bash # Check current quota usage and limits for Compute Engine in a region gcloud compute regions describe asia-south1 \ --project=devops-network-prod-01 ``` > 📌 **Remember:** Quotas are a safety mechanism, not a bug - a project hitting its quota unexpectedly during a legitimate scaling event should trigger a quota increase request through the Console, not a workaround. Requesting an increase in advance of a known future need (like a planned launch event) avoids being blocked at the worst possible moment. ---
### Understanding Billing Accounts and their relationship to Projects A **Billing Account** is a separate object from a Project - it holds the actual payment method, and one Billing Account can be linked to many different Projects, all of whose usage gets billed to that same account. +------------------------------------------+ | Billing Account (holds payment method) | +------------------------------------------+ | | v v +------------------+ +------------------+ | Project: prod-app | | Project: dev-app | +------------------+ +------------------+ This separation is deliberate - a single company can have one Billing Account funding many projects across different teams, with each project's own IAM controlling who can actually manage resources inside it, independent of who can see or manage the billing itself. ### Linking a project to a billing account ```bash # List available billing accounts gcloud billing accounts list # Link a project to a specific billing account gcloud billing projects link devops-network-prod-01 \ --billing-account=012345-6789AB-CDEF01 ``` ### Setting budgets and alerts before you need them A **Budget** defines a spending threshold for a Billing Account (or a specific project within it) and triggers alerts once actual or forecasted spend crosses a configured percentage - the same proactive safeguard covered in other cloud platforms, and just as essential here. ```text Console path: Billing -> Budgets & alerts -> Create Budget Scope: a specific project, or the whole billing account Amount: your defined monthly threshold Alert thresholds: 50%, 90%, 100% of budget (customizable) Notification: email to billing admins, or a Pub/Sub topic for automation ``` > 💡 **Tip:** Route a budget alert to a Pub/Sub topic rather than only an email, and pair it with a Cloud Function that can automatically take action - like disabling billing on a runaway test project - for a genuinely automated safety net rather than a notification someone might miss. ### Exporting billing data for detailed analysis **Billing exports** send detailed, itemized billing data to BigQuery automatically, letting you run actual SQL queries against your cost data - answering questions like "which specific service drove last month's cost increase" far more precisely than the Console's built-in reports alone. ```bash # Billing export to BigQuery is configured through the Console, # creating a dataset that daily billing data is exported into automatically ``` > 📌 **Remember:** Billing exports to BigQuery are the foundation for any serious cost analysis or chargeback reporting across multiple teams - set this up early, since historical data only starts accumulating from the point the export is enabled, not retroactively. ---
### Understanding the three role types GCP IAM controls who (a user, group, or Service Account) can do what (a Role) to which resource (a Project, Folder, or specific resource) - the same three-part model as other clouds, but GCP's roles specifically come in three distinct types worth telling apart clearly. +------------------------------------------+ | Basic Roles (legacy, broad) | | Owner, Editor, Viewer | | Apply broadly across an entire project | | Generally too broad for production use | +------------------------------------------+ | Predefined Roles (Google-curated, granular) | | e.g. roles/compute.instanceAdmin | | Scoped to a specific service's specific needs | +------------------------------------------+ | Custom Roles (you define exactly) | | Built from individual permissions | | Used when predefined roles are too broad | | or too narrow for a specific need | +------------------------------------------+ > 🔴 **Common Mistake:** Defaulting to the Editor basic role because it "definitely won't cause a permissions error." Editor grants broad create/modify access across nearly every resource type in the project - a predefined role scoped to the specific service actually being used (like `roles/compute.instanceAdmin.v1` for someone who only manages VMs) is almost always the more appropriate choice. ### Creating IAM policy bindings ```bash # Grant a predefined role scoped to Compute Engine only gcloud projects add-iam-policy-binding devops-network-prod-01 \ --member="user:rahul.verma@example.com" \ --role="roles/compute.instanceAdmin.v1" # View the current IAM policy for a project gcloud projects get-iam-policy devops-network-prod-01 ``` ### Creating a custom role ```bash # Define a custom role with only the exact permissions needed gcloud iam roles create customVmViewer \ --project=devops-network-prod-01 \ --title="Custom VM Viewer" \ --description="Can view VM details but not start, stop, or delete them" \ --permissions="compute.instances.get,compute.instances.list" ``` ### Understanding Service Accounts - identities for applications, not humans A **Service Account** is an identity used by an application, a VM, or an automated process to authenticate to GCP APIs - conceptually similar to a Managed Identity in Azure or an IAM Role in AWS, but in GCP a Service Account is itself also a resource with its own email-like identifier and its own IAM permissions that can be granted to other principals. +------------------------------------------+ | VM or application | +------------------------------------------+ | | authenticates as v +------------------------------------------+ | Service Account (its own identity, with | | specific IAM roles granted to it) | +------------------------------------------+ | v +------------------------------------------+ | GCP APIs (Cloud Storage, BigQuery, etc.) | +------------------------------------------+ ```bash # Create a Service Account gcloud iam service-accounts create app-backend-sa \ --display-name="Application Backend Service Account" # Grant that Service Account a specific role gcloud projects add-iam-policy-binding devops-network-prod-01 \ --member="serviceAccount:app-backend-sa@devops-network-prod-01.iam.gserviceaccount.com" \ --role="roles/storage.objectViewer" ``` ### Service Account impersonation and short-lived credentials **Service Account impersonation** lets an already-authenticated user or another Service Account temporarily act as a different Service Account, without ever downloading that Service Account's long-lived key file - a meaningfully safer pattern than distributing static JSON key files. ```bash # Generate a short-lived access token by impersonating a Service Account, # rather than downloading and using its permanent key file gcloud auth print-access-token \ --impersonate-service-account=app-backend-sa@devops-network-prod-01.iam.gserviceaccount.com ``` > ⚠️ **Security:** Avoid creating and downloading Service Account key files whenever a safer alternative exists - impersonation, or attaching a Service Account directly to a Compute Engine VM (so the VM authenticates as that identity automatically, with no key file at all). A downloaded key file is a long-lived credential that can leak exactly like an AWS access key or an Azure service principal secret. ---
### Choosing the right compute service for a workload GCP offers four primary compute options, and the ACE exam specifically tests the judgment of matching a workload to the right one, not just knowing each service exists. +------------------------------------------+ | Compute Engine | | Full VMs, full OS control | | Best for: workloads needing OS-level access,| | custom software, or specific configurations | +------------------------------------------+ | Google Kubernetes Engine (GKE) | | Managed Kubernetes for many coordinated | | containerized services | +------------------------------------------+ | Cloud Run | | Fully managed, serverless containers | | Scales to zero, pay only when invoked | +------------------------------------------+ | Cloud Functions | | Event-driven, single-purpose functions | | Shortest-lived, most granular compute unit | +------------------------------------------+ ### Launching a Compute Engine instance ```bash gcloud compute instances create vm-web-prod-01 \ --zone=asia-south1-a \ --machine-type=e2-medium \ --image-family=debian-12 \ --image-project=debian-cloud \ --boot-disk-size=50GB ``` > **Note:** `--image-project=debian-cloud` specifies which project the OS image itself belongs to - Google publishes standard OS images under dedicated public projects like `debian-cloud`, `ubuntu-os-cloud`, and `windows-cloud`, rather than requiring you to build an image from scratch. ### Using Spot VMs and custom machine types for cost efficiency **Spot VMs** are GCP's equivalent of AWS Spot Instances or Azure Spot VMs - spare capacity at a steep discount, with the trade-off that Google can reclaim the instance with short notice, appropriate only for fault-tolerant, interruptible workloads. ```bash # Create a Spot VM for a batch processing job gcloud compute instances create vm-batch-spot \ --zone=asia-south1-a \ --machine-type=e2-standard-4 \ --image-family=debian-12 \ --image-project=debian-cloud \ --provisioning-model=SPOT \ --instance-termination-action=STOP ``` **Custom machine types** let you define an exact vCPU and memory combination rather than picking from Google's predefined sizes, useful when a workload's actual needs fall between two standard sizes. ```bash # Create a VM with a custom 6 vCPU, 20GB memory configuration gcloud compute instances create vm-custom-shape \ --zone=asia-south1-a \ --custom-cpu=6 \ --custom-memory=20GB \ --image-family=debian-12 \ --image-project=debian-cloud ``` ### Configuring OS Login for centralized SSH access management **OS Login** ties SSH access to a user's actual Google Identity and IAM permissions, rather than manually managing individual SSH public keys per VM - meaning revoking a departing employee's GCP access also immediately revokes their SSH access to every VM, with no separate key cleanup needed. ```bash # Enable OS Login at the project level, applying to all VMs by default gcloud compute project-info add-metadata \ --metadata enable-oslogin=TRUE ``` ### Building Managed Instance Groups for autoscaling A **Managed Instance Group (MIG)** is GCP's equivalent of an AWS Auto Scaling Group or Azure VM Scale Set - a group of identical VMs created from an Instance Template, scaled automatically based on load. ```bash # Create an Instance Template defining the VM configuration gcloud compute instance-templates create web-server-template \ --machine-type=e2-medium \ --image-family=debian-12 \ --image-project=debian-cloud # Create a Managed Instance Group using that template gcloud compute instance-groups managed create web-server-mig \ --template=web-server-template \ --size=2 \ --zone=asia-south1-a # Configure autoscaling based on CPU utilization gcloud compute instance-groups managed set-autoscaling web-server-mig \ --zone=asia-south1-a \ --max-num-replicas=10 \ --min-num-replicas=2 \ --target-cpu-utilization=0.7 ``` ### Deploying to Google Kubernetes Engine ```bash # Configure kubectl for cluster interaction gcloud container clusters get-credentials gke-lab-cluster \ --zone=asia-south1-a # Create an Autopilot cluster - Google manages node provisioning entirely gcloud container clusters create-auto gke-lab-cluster \ --region=asia-south1 ``` > 📌 **Remember:** GKE Autopilot mode removes node management entirely - you define workloads, and Google provisions and manages the underlying nodes automatically. Standard mode gives you direct control over node pools and machine types, at the cost of managing that infrastructure yourself. ### Deploying to Cloud Run ```bash # Deploy a containerized application directly to Cloud Run gcloud run deploy inventory-service \ --image=asia-south1-docker.pkg.dev/devops-network-prod-01/app-images/inventory-service:v1 \ --region=asia-south1 \ --platform=managed \ --allow-unauthenticated ``` > 💡 **Tip:** Cloud Run is the right default for a stateless, containerized web service that needs to scale to zero when idle - it combines the operational simplicity of a fully managed platform with the flexibility of running any container, without needing a Kubernetes cluster at all. ---
A team migrates three years of transaction logs into a single Cloud Storage bucket, leaves everything in the Standard class because "we might need to look at any of it," and the storage bill for data nobody has opened in over a year quietly becomes one of the largest line items on the account. None of this data needed deleting - it needed a cheaper storage class, and a Lifecycle rule that moves it there automatically as it ages. ### Choosing the right storage class for Cloud Storage **Cloud Storage** is GCP's object storage service, equivalent to AWS S3 or Azure Blob Storage. Like both of those, it offers multiple storage classes trading cost against retrieval frequency and minimum storage duration - but GCP's classes carry a specific detail the other clouds handle slightly differently: an explicit **minimum storage duration** attached to each colder tier. +------------------------------------------+ | Standard | | Frequently accessed data | | No minimum storage duration | +------------------------------------------+ | Nearline | | Accessed roughly once a month or less | | 30-day minimum storage duration | +------------------------------------------+ | Coldline | | Accessed roughly once a quarter or less | | 90-day minimum storage duration | +------------------------------------------+ | Archive | | Accessed less than once a year | | 365-day minimum storage duration | +------------------------------------------+ The minimum storage duration is not just a soft guideline - deleting or overwriting an object before that window elapses still bills as if it had been stored the full minimum period. This changes the actual math behind choosing a class: an object that might get deleted or replaced in 45 days is a worse fit for Coldline (90-day minimum) than it first appears, even though Coldline's per-GB rate looks cheaper. ```bash # Create a bucket with the Standard storage class gcloud storage buckets create gs://app-assets-prod \ --location=asia-south1 \ --default-storage-class=STANDARD # Upload a file gcloud storage cp ./logo.png gs://app-assets-prod/ # Set a Lifecycle rule moving objects to Coldline after 90 days, # defined in a separate JSON configuration file gcloud storage buckets update gs://app-assets-prod \ --lifecycle-file=lifecycle-config.json ``` > **Note:** The Lifecycle configuration itself is a JSON policy document defining conditions (like object age) and the action to take once that condition is met (like transitioning to a colder class, or deleting the object outright). This is the GCP equivalent of an S3 Lifecycle policy or an Azure Blob Lifecycle Management rule - the same underlying idea, expressed through a GCP-specific configuration format. > 🔴 **Common Mistake:** Choosing a colder storage class purely for its lower per-GB storage price, without checking the minimum storage duration against how long the data will actually be kept before being deleted or overwritten. A dataset cycled out every 60 days is a poor fit for Coldline's 90-day minimum, even though Coldline's advertised rate looks like the cheaper option on paper. ### Choosing between zonal and regional Persistent Disks A **Persistent Disk** is the block storage attached to a Compute Engine VM, equivalent to an AWS EBS volume or an Azure Managed Disk. The choice between zonal and regional replication is really a question about how much downtime a single zone failure is allowed to cause. +------------------------------------------+ | Zonal Persistent Disk | | Data replicated within a single zone | | Lower cost | | If the zone fails, the disk is unavailable | | until that zone recovers | +------------------------------------------+ | Regional Persistent Disk | | Data synchronously replicated across two | | zones in the same region | | Higher cost, survives a single zone failure | | Can be attached to a VM in either zone | +------------------------------------------+ ```bash # Create a zonal Persistent Disk - the default, cheaper option gcloud compute disks create disk-app-data \ --zone=asia-south1-a \ --size=100GB \ --type=pd-ssd # Create a regional Persistent Disk for higher availability - # note this specifies a region and a pair of replica zones, not a single zone gcloud compute disks create disk-app-data-regional \ --region=asia-south1 \ --replica-zones=asia-south1-a,asia-south1-b \ --size=100GB \ --type=pd-ssd ``` > 💡 **Tip:** Regional Persistent Disks matter most specifically for stateful workloads that cannot simply be recreated elsewhere on demand - a database's own data disk is a strong candidate; a stateless application server's boot disk usually is not, since that VM can just be recreated from its instance template if its zone fails. ### Choosing the right database product for a workload The ACE exam specifically tests matching a data workload to the correct managed database product, and this is also one of the most consequential real architecture decisions a team makes early - migrating from the wrong database months into a project, once application code already assumes a specific query pattern, is far more expensive than choosing correctly at the start. +------------------------------------------+ | Cloud SQL | | Managed MySQL, PostgreSQL, SQL Server | | Best for: traditional relational workloads | | at moderate scale, single-region | +------------------------------------------+ | Firestore | | Serverless NoSQL document database | | Best for: mobile/web app data, flexible | | schemas, real-time client sync | +------------------------------------------+ | Spanner | | Globally distributed, strongly consistent | | relational database | | Best for: massive scale needing both SQL | | semantics and horizontal scalability | +------------------------------------------+ | Bigtable | | Wide-column NoSQL for very high throughput | | Best for: time-series data, IoT telemetry, | | workloads needing extremely low latency | | at very high write volume | +------------------------------------------+ | BigQuery | | Serverless data warehouse for analytics | | Best for: running SQL queries across massive | | datasets for reporting and analysis, not for | | transactional application reads and writes | +------------------------------------------+ ```bash # Create a Cloud SQL instance running PostgreSQL gcloud sql instances create pg-orders-prod \ --database-version=POSTGRES_15 \ --tier=db-custom-2-7680 \ --region=asia-south1 # Query data directly in BigQuery bq query --use_legacy_sql=false \ 'SELECT region, SUM(revenue) FROM `prod-project.sales.orders` GROUP BY region' ``` > **Note:** `db-custom-2-7680` in the Cloud SQL command specifies a custom machine configuration - 2 vCPUs and 7680 MB of memory - rather than picking from a predefined tier, the same custom-shape flexibility Compute Engine offers for VMs. > 📌 **Remember:** Cloud SQL is the right default for a standard relational workload with moderate scale needs. Reach for Spanner specifically when a workload needs both SQL semantics and horizontal scale beyond what a single Cloud SQL instance can provide - not simply because a workload feels "important" or "large." Using BigQuery as a transactional database for an application's live reads and writes is a common and costly mismatch - it is built and priced for analytical queries over large datasets, not frequent small reads and writes. > 🔴 **Common Mistake:** Defaulting to Firestore for a workload that actually needs complex relational joins and multi-row transactions across unrelated entities. Firestore's document model handles hierarchical, per-document data extremely well, but a workload that genuinely needs relational joins across many tables is a better fit for Cloud SQL or Spanner from the start. ---
The problem this module actually solves A team migrating from AWS to Google Cloud creates a new GCP project the same way...
The four-level hierarchy that organizes everything Google Cloud organizes every resource inside a strict hierarchy, and ...
Understanding Billing Accounts and their relationship to Projects A Billing Account is a separate object from a Project ...
Understanding the three role types GCP IAM controls who (a user, group, or Service Account) can do what (a Role) to whic...
Choosing the right compute service for a workload GCP offers four primary compute options, and the ACE exam specifically...
A team migrates three years of transaction logs into a single Cloud Storage bucket, leaves everything in the Standard cl...
Understanding VPCs and the global nature of GCP networking A Virtual Private Cloud (VPC) in GCP is conceptually similar ...
A support engineer discovers a production outage only because a customer complained, not because any alert fired - the m...
This lab builds a small, connected environment touching every major area of this module - project setup, IAM, compute, s...
Concept Key fact Project The core billing and IAM scoping unit - every resource belongs to exactly one Service Account A...
Aligns directly with DevOps, Site Reliability (SRE), and Platform Engineering job descriptions.