Multi-Cloud Architecture: Lessons from Running Workloads on AWS and GCP

Multi-cloud gets pitched in vendor slide decks as a strategy — resilience against a single provider's outage, negotiating leverage, "best of breed" service selection. In practice, most multi-cloud setups we've inherited from clients weren't a strategy at all; they were the accumulated result of an acquisition that came with its own AWS account, a data science team that picked GCP for BigQuery, and nobody ever consolidating. That's a different starting point than a deliberate multi-cloud design, and it changes what "lessons learned" actually means here — this is less "why you should go multi-cloud" and more "what we've learned keeping workloads on both alive without the egress bill or the on-call rotation becoming unmanageable."

The lesson that costs the most money: egress

Cross-cloud data transfer is the single most underestimated cost in a multi-cloud setup, and it's underestimated specifically because it doesn't show up until the architecture is already chatty across the boundary. Both AWS and GCP charge in the range of $0.08–$0.12 per GB for egress to the public internet or to another cloud (exact rates vary by volume tier and destination), and a service on AWS calling a service on GCP thousands of times a minute racks that up fast — we've seen a client's monthly cross-cloud data transfer bill exceed their compute spend on one of the two clouds entirely, simply because a synchronous API call pattern crossed the cloud boundary on every request instead of once per batch.

The fix isn't "don't use two clouds" — it's designing the boundary deliberately. Put chatty, latency-sensitive services in the same cloud as whatever they talk to most. Reserve the cross-cloud boundary for infrequent, batchable transfers (a nightly data sync into BigQuery, not a per-request lookup), and if a genuinely high-frequency cross-cloud path is unavoidable, budget for it explicitly rather than discovering it in a bill.

Latency: the boundary you can't optimize away

Even with fast interconnects, a call that crosses from an AWS region to a GCP region adds real latency — typically an extra 20–60ms round-trip on top of whatever the call itself takes, depending on region pairing and whether you're going over the public internet or a direct interconnect (AWS Direct Connect / GCP Cloud Interconnect, or a third-party network provider that peers with both). That's invisible in a demo and very visible in a p99 latency graph once it's on a request path a user is waiting on.

Our rule of thumb: anything in a user-facing request path should resolve within one cloud. Cross-cloud calls belong in background jobs, event-driven pipelines, or scheduled batch processes where an extra 40ms is noise, not in the synchronous path between a user clicking a button and a response rendering.

When to actually use which cloud

This is where "best of breed" earns its reputation, selectively. GCP's BigQuery remains genuinely ahead for large-scale analytical workloads — the separation of storage and compute, and the pricing model built around it, makes ad hoc queries over large datasets both fast and predictably priced in a way we haven't found an equivalent for on AWS without more manual tuning (Redshift can get there, but it takes more operational investment to tune well). GCP's Kubernetes engine (GKE) also still has an edge in operational polish over EKS for teams that are Kubernetes-native — node auto-provisioning and upgrade management require less hand-holding.

AWS wins on service breadth and maturity almost everywhere else — if a workload needs a managed service for something reasonably specific (a particular queueing pattern, a specialized database engine, an ML inference service with a particular hardware profile), AWS is more likely to have a first-party managed option, and its IAM model, while more complex, gives finer-grained control that matters for larger organizations with dedicated security teams.

The "best of breed" argument for multi-cloud is real for maybe two or three specific services per client, not for the architecture as a whole — everything else should live wherever reduces operational surface area.

Operational cost: the one that doesn't show up in a cost report

Beyond egress, the real ongoing cost of multi-cloud is operational: two IAM models to reason about, two networking models, two sets of Terraform providers, two monitoring integrations, and an on-call team that needs enough fluency in both to debug an incident at 2 a.m. without reaching for the wrong cloud's documentation. We size this cost by asking a blunt question during planning: is the benefit of running this specific workload on this specific cloud worth doubling the number of platforms someone on this team needs to hold in their head? For most workloads, the honest answer is no — consolidate onto one cloud and reserve the second one for the two or three services where the "best of breed" gap is real and measurable, not aspirational.

A concrete pattern that's worked

For clients we've helped stand up deliberate (not accidental) multi-cloud architectures, the pattern that's held up best: primary application infrastructure — API services, the primary transactional database, user-facing compute — lives entirely on one cloud, chosen once and not relitigated per-service. Analytics and data warehousing live on GCP regardless of where the primary infrastructure sits, fed by a batched, scheduled export (not a live query path) across the cloud boundary. Infrastructure as code is written cloud-agnostically where the abstraction genuinely holds (Terraform modules per cloud, not a single fictional cross-cloud abstraction layer that inevitably leaks), and nobody adds a third cloud without the same egress-and-operational-cost conversation happening again from scratch.

Multi-cloud isn't free redundancy, and it isn't free flexibility. It's a real, ongoing cost paid in engineering attention and egress bills, worth it for a specific handful of workloads where one cloud is genuinely better — and a liability everywhere else it accumulates by accident.