Deploy on Google Cloud (GKE)

This page is the Google Cloud companion to Deploy from scratch, for chart 4.60.0 or newer. Follow that guide step by step. Where a step has a GCP-specific answer, this page gives it. The goal is an install where no static cloud credential exists anywhere: object storage and the model providers authenticate as Google service accounts through GKE Workload Identity.

The chart ships these answers as one example values file, values-gke.yaml: Cloud SQL, keyless GCS, the GKE Ingress, Google’s load-balancer ranges in the NetworkPolicy, and secrets from Google Secret Manager through External Secrets, sized for about 4 vCPU and 10 GiB. Deploy on GKE with Argo CD installs it as one Argo CD Application, with every precondition and its command.

From-scratch step	On GCP
Prerequisites	GKE cluster with Dataplane V2 and Workload Identity
Step 2: PostgreSQL	Cloud SQL for PostgreSQL 16+, private IP
Step 3: Object storage	GCS bucket, Workload Identity, no Secret
Step 7: The edge	The GKE Ingress the chart renders, or a Gateway of your own
Step 8: Models	Vertex AI or AI DIAL with a service account, configured in the admin console

The cluster

Create a GKE Standard or Autopilot cluster with:

• Dataplane V2 (--enable-dataplane-v2). It enforces the chart’s NetworkPolicies, including ipBlock exceptions. Autopilot has it on by default.
• Workload Identity Federation for GKE (--workload-pool=<PROJECT_ID>.svc.id.goog). Every keyless path on this page depends on it.
• A private cluster, with Cloud NAT for egress to model providers and to ghcr.io, where the images are published.
• A node pool with at least one node that has 2 free CPUs for the worker during a rolling update.
• Optional: a GKE Sandbox node pool (--sandbox type=gvisor) if you will turn on the cloud executor. GKE installs the gvisor RuntimeClass itself, so leave the chart’s sandbox-runtime installer (sandboxRuntime.enabled) off. See Cloud executor on Kubernetes.

Install cert-manager as the from-scratch guide says.

NodeLocal DNSCache

If NodeLocal DNSCache is on, pods resolve through 169.254.20.10. That is a link-local address, inside the deny ranges of the egress proxy and of the sandbox workspace policies, which only admit kube-dns pods. Leave it off, or add the address to those policies, before you turn on the cloud executor.

Service accounts

Create one Google service account (GSA) per component that calls a Google API, and link it to the component’s Kubernetes service account (KSA). The chart names the KSAs after the release, so with the release name elitea:

Component	KSA	GSA (example)	Needs
elitea-main	elitea-main	elitea-main@<PROJECT_ID>.iam.gserviceaccount.com	the GCS bucket
elitea-llm-gateway	elitea-llm-gateway	elitea-llm@<PROJECT_ID>.iam.gserviceaccount.com	Vertex AI and/or the AI DIAL audience

PROJECT_ID=<your project>
for gsa in elitea-main elitea-llm; do
  gcloud iam service-accounts create "$gsa" --project "$PROJECT_ID"
done

# Each KSA may act as its GSA. Nothing else may.
gcloud iam service-accounts add-iam-policy-binding \
  "elitea-main@$PROJECT_ID.iam.gserviceaccount.com" \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:$PROJECT_ID.svc.id.goog[elitea/elitea-main]"
gcloud iam service-accounts add-iam-policy-binding \
  "elitea-llm@$PROJECT_ID.iam.gserviceaccount.com" \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:$PROJECT_ID.svc.id.goog[elitea/elitea-llm-gateway]"

Use linked GSAs rather than granting roles to the KSA principal directly. AI DIAL authentication needs a Google-signed ID token, and GKE’s metadata server issues one only for a linked GSA.

Cloud SQL for PostgreSQL

Create a PostgreSQL 16 or newer instance with a private IP in the cluster’s VPC. No extension is needed: the platform keeps its vectors in Qdrant through elitea-vector.

• Connections. Set the max_connections database flag deliberately and copy the same number into postgresql.maxConnections. The chart checks every component’s pools at their replica ceilings against it; see the connection budget.
• TLS. Turn on “Allow only SSL connections” and use sslmode=require in both DSNs. The chart has no mount for a server CA, so verify-full is not available.
• No Cloud SQL Auth Proxy. The chart renders no sidecars. Connect over private IP. If your policy requires the proxy, run it as its own Deployment and Service, and point the DSNs at that Service.

Cloud SQL has no superuser, so use the DBA tab of the from-scratch Step 2 (dbInit.enabled: false, dbInit.externallyManaged: true). Create the login role with gcloud, then run the few statements below once, connected as postgres. This is the only SQL in the install. After it, the migration Job builds everything else.

gcloud sql users create elitea_v2 --instance <instance> \
  --password "$(cat elitea-material/db-password)"

-- as postgres; it must be able to act as the owner it assigns
GRANT elitea_v2 TO postgres;
CREATE DATABASE elitea_v2 OWNER elitea_v2;
CREATE DATABASE elitea_v2_agentstate OWNER elitea_v2;

Set DB_HOST to the instance’s private IP when you run the mint script, so both DSNs point at it. With secrets in Secret Manager, use deploy/scripts/mint-secrets.sh instead: it writes the one flat JSON secret that values-gke.yaml reads, and you pass the two DSNs to it (Deploy on GKE with Argo CD).

To let the chart’s dbInit hook create the agent-state database instead (from-scratch Step 2, the other tab), give it an admin DSN whose role has the CREATEDB attribute and membership in elitea_v2 (Cloud SQL’s postgres user has CREATEDB; grant the membership as above). Leave dbInit.appRole and dbInit.agentStateDatabase empty: the Job reads them from the DSNs.

Certificate authorities. values-gke.yaml sets certManager.issuers.create: true: the chart makes the elitea-internal-ca chain and the elitea-runtime-ca Issuer itself, so you do not apply from-scratch Step 1’s internal-ca.yaml. You still provide the runtime CA keypair, as the Secret elitea-runtime-ca (from Secret Manager, runtime-ca.crt and runtime-ca.key).

GCS bucket with no keys

BUCKET=<project>-elitea-artifacts
gcloud storage buckets create "gs://$BUCKET" --location <region> \
  --uniform-bucket-level-access --public-access-prevention

# Scoped to this one bucket, never to the project.
gcloud storage buckets add-iam-policy-binding "gs://$BUCKET" \
  --member "serviceAccount:elitea-main@$PROJECT_ID.iam.gserviceaccount.com" \
  --role roles/storage.admin

elitea-main needs more than object access on its bucket. At boot it replaces the bucket’s lifecycle configuration (storage.buckets.update) and reads its encryption settings. roles/storage.admin granted on the bucket alone covers both. Use a dedicated bucket.

In the values file, replace the S3 block of the from-scratch Step 6 with:

main:
  serviceAccount:
    annotations:
      iam.gke.io/gcp-service-account: elitea-main@<PROJECT_ID>.iam.gserviceaccount.com
  env:
    STORAGE_BACKEND: "gcs"
    STORAGE_CONTAINER: "<project>-elitea-artifacts"
    GCS_ENDPOINT: ""        # empty for real GCS

Also remove S3_ACCESS_KEY and S3_SECRET_KEY from main.secrets, and do not create elitea-main-storage-secrets. With Application Default Credentials there is no signing key, so the platform does not hand out presigned URLs: downloads and workspace transfers stream through elitea-main instead. Nothing needs configuring for that.

Set the annotation, or the pod runs as the node

With no Secret and no annotation, the Google client library falls back to the node’s own service account. If that account can reach the bucket, elitea-main starts and works under the wrong identity. Always set the annotation.

The edge

The public edge is one route, / to elitea-main:8080: elitea-main relays /app and /docs to elitea-web, serves its MCP endpoint itself, and deletes client-supplied identity headers before any route runs. From-scratch Step 7 lists what the edge must still do: keep server-sent event streams open for minutes and accept 150 MiB uploads.

GKE Ingress (recommended)

The chart renders it: the standard Google Cloud external Application Load Balancer, with container-native load balancing straight to the elitea-main pods.

main:
  ingress:
    enabled: true
    className: gce
    annotations:
      kubernetes.io/ingress.global-static-ip-name: elitea-ip   # gcloud compute addresses create elitea-ip --global
    hosts: [elitea.example.com]
    tls:
      - secretName: elitea-tls
        hosts: [elitea.example.com]
    certManager:
      clusterIssuer: <your ClusterIssuer>
    gke:
      enabled: true
      backendConfig:
        securityPolicy: ""     # a Cloud Armor policy, optional
      frontendConfig:
        sslPolicy: ""          # an SSL policy, optional

networkPolicies:
  main:
    ingressFrom:               # no ingress pods: Google's front ends connect directly
      - ipBlock: {cidr: 35.191.0.0/16}
      - ipBlock: {cidr: 130.211.0.0/22}

main.ingress.gke adds a BackendConfig (timeoutSec: 3600, so a streamed answer outlives the load balancer’s 30-second default; an HTTP health check on /healthz port 8080; connection draining; the optional Cloud Armor policy) and a FrontendConfig (HTTP to HTTPS redirect, the optional SSL policy). main.ingress.certManager has cert-manager issue the tls Secret the load balancer serves. Add Google’s ranges to global.trustedProxyCidrs too (it fills ELITEA_TRUSTED_PROXY_CIDRS and the authentication document’s list). global.publicUrl fills the Ingress host and the TLS host when you leave main.ingress.hosts and tls[].hosts empty.

Behind a CDN or proxy, issue the origin certificate through that provider’s cert-manager issuer (certManager.issuerKind, issuerGroup) and attach a Cloud Armor policy that admits only the proxy’s ranges; Deploy on GKE with Argo CD has the commands. elitea-main writes a comment into every idle stream after 25 seconds, inside a proxy’s idle timeout of about 100 seconds.

A Gateway

Any Gateway API implementation works with the one route, the managed GKE Gateway (gke-l7-global-external-managed) included: render it with main.ingress.gatewayApi: true, or apply deploy/gateway-api/httproute.yaml.

• The managed GKE Gateway has the same 30-second backend default: attach a GCPBackendPolicy to the elitea-main Service with timeoutSec of, for example, 3600, and a HealthCheckPolicy on /healthz port 8080. It has no gateway pods, so networkPolicies.main.ingressFrom is the two ipBlocks above.
• Envoy Gateway in the cluster, exposed by a Service of type LoadBalancer: select its data-plane pods in ingressFrom (app.kubernetes.io/name: envoy in envoy-gateway-system).

Run every check of the from-scratch Step 9 before go-live.

Single sign-on

Install with OIDC configured in the values file (OIDC_* in from-scratch Step 6). That is what makes the sign-in routes exist. Any identity provider with OIDC works, including Google Workspace (OIDC_ISSUER_URL: https://accounts.google.com), Entra ID, Okta and Keycloak. Name the first administrator as email:<address>. That form matches when the provider marks the address as verified.

After the first sign-in, everything else is in the admin console, under Configuration › Authentication, with no restart:

• SAML. Add a SAML 2.0 provider. Give the identity provider this service’s metadata URL, https://<host>/auth/saml/metadata, and copy its entity ID, SSO URL and certificate back into the form. Signing authentication requests is optional.
• SCIM. Create a SCIM client, put its Tenant URL and token into the identity provider, then bind directory groups to a project and a role.

Authentication describes every field.

Platform models with a service account

Platform models are the shared models every project can use. An administrator adds them in the admin console. On GCP, authenticate them as the gateway’s service account rather than with a pasted key.

1. Allow workload identity on the gateway

It needs chart 4.60.0 or newer. Older charts have no llmGateway.workloadIdentity and render no annotations on the gateway’s service account. Workload identity is off by default, and it is honoured only for platform credentials, the ones an administrator adds in the admin console. A project’s own credential can never borrow the gateway’s identity.

llmGateway:
  serviceAccount:
    annotations:
      iam.gke.io/gcp-service-account: elitea-llm@<PROJECT_ID>.iam.gserviceaccount.com
  workloadIdentity:
    enabled: true
  env:
    # Add the provider hosts you use.
    GATEWAY_EGRESS_ALLOWLIST: >-
      <region>-aiplatform.googleapis.com,aiplatform.googleapis.com,dial.example.com

main:
  env:
    # Recommended: projects use shared platform models only.
    ELITEA_ALLOW_PROJECT_OWN_LLMS: "false"

A private AI DIAL address needs its IP or CIDR in the allowlist as well; see model egress.

2. Grant the service account access

Vertex AI

gcloud projects add-iam-policy-binding "$PROJECT_ID" \
  --member "serviceAccount:elitea-llm@$PROJECT_ID.iam.gserviceaccount.com" \
  --role roles/aiplatform.user

Enable the Vertex AI API in that project. To use models from another project, grant the role there and enter that project in the provider form.

AI DIAL

The gateway signs in to DIAL with a Google-signed ID token for elitea-llm@<PROJECT_ID>.iam.gserviceaccount.com, sent as Authorization: Bearer. The token’s audience is the DIAL endpoint’s origin, unless you set another one in the provider form.

Your DIAL administrators configure DIAL core to accept that token:

• an identity provider with issuer https://accounts.google.com and JWKS https://www.googleapis.com/oauth2/v3/certs
• accept only the audience above and this service account’s e-mail (or sub)
• give that principal a role with access to the models you will publish

Until that is in place, the connection check in the admin console fails with DIAL’s 401.

3. Add the provider and publish models

1. Open the LLM Proxy

   In the admin console, open Configuration › LLM Proxy › Providers & models.

2. Add a provider

   Choose Vertex AI or EPAM DIAL, and set Authentication to Workload identity (GKE service account).

   • Vertex AI: the Google Cloud project and location. There is no key field.
   • EPAM DIAL: the endpoint and API version, plus a token audience if your DIAL expects one other than the endpoint’s origin.

   Save the provider, then run Test connection. A keyless provider can be tested only after it is saved. A warning under the Workload identity option means llmGateway.workloadIdentity.enabled is not on.

3. Publish models

   Add a platform model for each model you share: its name as the provider knows it, its context window, and, for DIAL, its DIAL protocol. Default fits most models. Choose OpenAI Responses API for gpt models that need it, and Anthropic Messages API for Claude with extended thinking.

4. Choose the platform default model

   Set it on the same tab. New conversations use it.

Then open a chat as any user, pick a shared model and send a message.

Checklist for the values file

On top of the from-scratch values file (the chart’s values-gke.yaml has all of this, plus the edge, externalSecrets and certManager.issuers.create):

global:
  publicUrl: https://elitea.example.com
  trustedProxyCidrs: ["<pod CIDR>", 35.191.0.0/16, 130.211.0.0/22]

networkPolicies:
  main:
    ingressFrom:            # the GKE Ingress; see "The edge" for a Gateway
      - ipBlock: {cidr: 35.191.0.0/16}
      - ipBlock: {cidr: 130.211.0.0/22}

main:
  serviceAccount:
    annotations:
      iam.gke.io/gcp-service-account: elitea-main@<PROJECT_ID>.iam.gserviceaccount.com
  env:
    STORAGE_BACKEND: "gcs"
    STORAGE_CONTAINER: "<project>-elitea-artifacts"
    ELITEA_ALLOW_PROJECT_OWN_LLMS: "false"
  # remove S3_ACCESS_KEY / S3_SECRET_KEY from main.secrets

llmGateway:
  serviceAccount:
    annotations:
      iam.gke.io/gcp-service-account: elitea-llm@<PROJECT_ID>.iam.gserviceaccount.com
  workloadIdentity:
    enabled: true

After the install, helm get notes elitea -n elitea prints the gcloud Workload Identity bindings for the two service accounts above, with this release’s namespace and Kubernetes service account names.