Google Cloud Storage

Write files to a Google Cloud Storage bucket in CSV or JSONL format. Use Zeotap to export model results, audience lists, or enriched data as files in GCS for downstream processing.

Prerequisites

• A Google Cloud project with a Cloud Storage bucket
• A service account granted storage.buckets.get, storage.objects.create, and storage.objects.delete on the target bucket (see Required Permissions)
• A service account JSON key file
• The bucket must not have Requester Pays  enabled

Authentication

GCS uses Service Account authentication.

1. In the Google Cloud Console, go to IAM & Admin > Service Accounts
2. Create a service account (or use an existing one)
3. Grant the permissions listed under Required Permissions on the target bucket
4. Create a JSON key for the service account
5. Upload or paste the JSON key in Zeotap

Required Permissions

The service account needs the following IAM permissions on the target bucket:

On buckets with hierarchical namespace  enabled, one more permission is required: storage.folders.create. Creating an object automatically creates any missing parent folders, and Zeotap’s date-partitioned file paths create a new folder each day. The predefined role options below already include this permission; add it to a custom role explicitly.

No predefined roles/storage.object* role includes storage.buckets.get, so grant one of the following on the bucket:

• Recommended — a custom role with exactly the permissions above:

  gcloud iam roles create signalsmithGcsWriter \
    --project=YOUR_PROJECT_ID \
    --title="Zeotap GCS Writer" \
    --permissions=storage.buckets.get,storage.objects.create,storage.objects.delete
   
  gcloud storage buckets add-iam-policy-binding gs://YOUR_BUCKET \
    --member="serviceAccount:YOUR_SERVICE_ACCOUNT_EMAIL" \
    --role="projects/YOUR_PROJECT_ID/roles/signalsmithGcsWriter"

• Predefined roles — Storage Object User (roles/storage.objectUser) plus Storage Legacy Bucket Reader (roles/storage.legacyBucketReader), both granted on the bucket

• Storage Admin (roles/storage.admin) on the bucket also works, but grants far more than Zeotap needs, including control of the bucket’s IAM policy

The Storage Object Creator (roles/storage.objectCreator) role alone is not sufficient: it does not include storage.buckets.get, so the connection test and every sync run fail with cannot access bucket. A grant that adds bucket access but omits storage.objects.delete also fails at the start of every sync run: the permission check writes a temporary zero-byte marker (a unique name per attempt) and then must delete it, and a denied delete fails the run with an error naming storage.objects.delete and the full path of the leftover marker. Leftover markers are zero bytes and safe to delete manually.

Configuration

Target Settings

These are set per destination as a default and can be overridden per sync — so multiple syncs can share one bucket while each writes to its own path.

Supported Operations

Sync Modes: Insert, Mirror, Snapshot

Audience Sync Modes: Add, Mirror, Snapshot

Features

• Field Mapping: No — GCS writes all mapped fields as file columns
• Schema Introspection: No — GCS is a file-based destination without schema discovery

How It Works

Zeotap writes data as files to the configured GCS bucket:

1. Data is serialized into the selected format (CSV or JSONL)
2. If gzip compression is enabled, the file is compressed before upload
3. Files are uploaded to gs://{bucket}/{path_prefix}/{YYYY-MM-DD}/{batch_id}.{ext}[.gz], where {batch_id} has the form batch_<timestamp>_<sequence>_<random> — collision-proof, so concurrent uploads within a run can never overwrite each other. When a File Path Pattern is configured, it replaces the default {path_prefix}/{YYYY-MM-DD}/{batch_id} layout.
4. Each sync run produces a new set of date-partitioned files; previously written data files are never modified or deleted

File path patterns

By default files land at {path_prefix}/{YYYY-MM-DD}/{batch_id}.{ext}[.gz]. Set File Path Pattern to control the exact object path and filename using macros. This is useful when a downstream system expects files in a specific folder structure or naming convention.

Example: exports/{audience_name}/{date}/{batch_id} produces objects like exports/VIP Users/2026-07-21/batch_1784624703_1_a1b2c3d4.csv.gz.

Available macros

All date/time macros are UTC and are based on the run’s start time, not each file’s write time — so a run that crosses a date or hour boundary keeps all its files in a single {date}/{hour} partition.

The available identifiers depend on how the destination is written:

• Model sync — sync_id, sync_name, run_id (no audience/journey macros).
• Audience sync — the above plus audience_id, audience_name.
• Journey send-to-destination tile — run_id, journey_id, journey_run_id, tile_id, and audience_id when the tile targets an audience (no sync_id/sync_name).

A macro that doesn’t apply to the current context renders empty and its path segment collapses. If a pattern renders entirely empty, Zeotap falls back to the default date-partitioned layout.

Notes:

• The file extension is appended automatically (.csv/.jsonl, plus .gz when compression is on) — do not include it in the pattern.
• Uniqueness is guaranteed. A run is delivered as many batches (one file each), written up to 4 at a time. If your pattern has no {batch_id}, a unique batch id is appended automatically so files never overwrite each other. Include {batch_id} explicitly to control where it appears.
• Slashes and control characters inside a name macro (e.g. an audience named A/B) are replaced with _ so they don’t create unintended folders; the pattern’s own / separators are preserved. A name that is only dots (./..) is likewise neutralized so it can never form a traversing segment.
• Leading/trailing slashes are trimmed and empty path segments collapsed (so an unset macro like {audience_id} on a plain model sync doesn’t leave a blank folder).
• The pattern is validated on save: unknown macros and .. path-traversal segments are rejected.

Rows per file

By default Zeotap sizes batches adaptively (starting at 10,000 rows/file and adjusting to destination performance). Set Rows Per File to write a fixed number of rows per output file instead — useful when a downstream loader expects files of a predictable size. The value is a hard per-file row count (not adaptively grown or shrunk). It must be between 1 and 1,000,000 rows; values outside that range are rejected when you save the destination.

Sync mode semantics

• Insert writes the full current dataset as new files on every run.
• Mirror writes the full dataset on the first run, then only the rows that changed on later runs. Every row in a mirror-mode file carries an _operation column — added, changed, or removed — so downstream consumers can apply updates and deletions. On the first run all rows are marked added. In CSV files _operation is always the last column; treat the name as reserved in mirror mode.
• Add (audience syncs) writes only new and updated audience members. Members who left the audience since the previous run are excluded from the files.
• Snapshot writes the complete current dataset as new files on every run and never diffs. There is no _operation column (the files are the whole dataset) and unchanged rows are included every time. Records that left are simply absent from the next drop rather than delivered tagged removed, so treat each run’s output as authoritative and replace your prior copy rather than merging into it. Unlike Insert, snapshot is guaranteed to read the full model output on every warehouse: it never adopts native change tracking (Snowflake streams, Databricks CDF), which can only surface deltas.

Troubleshooting

Permission denied

Verify the service account holds every permission listed under Required Permissions on the bucket, and that the bucket-level IAM policy grants them to the service account:

• cannot access bucket from the connection test or at the start of a sync run means storage.buckets.get is missing.
• write permission test failed at the start of a sync run means the service account cannot create the write-check marker — storage.objects.create is missing.
• cleanup of permission-test marker ... failed at the start of a sync run means storage.objects.delete is missing. The error includes the full path of the leftover zero-byte marker, which is safe to delete manually.
• On buckets with hierarchical namespace enabled, failures at the start of a sync run can also mean storage.folders.create is missing — creating an object in a new date-partitioned path creates a folder.

Bucket not found

Ensure the bucket name is correct. GCS bucket names are globally unique and case-sensitive.

Invalid service account JSON

The JSON key file must be a valid service account key with "type": "service_account". Download a fresh key from the Google Cloud Console if the current one is corrupted or expired.

Quota exceeded

GCS enforces request rate limits per bucket. For high-frequency syncs, consider using a dedicated bucket or reducing sync frequency.