Firestore

Google Cloud Firestore is a serverless, NoSQL document database built for automatic scaling, high performance, and ease of application development. Use Zeotap to write customer profiles, enrichment data, and computed attributes directly into Firestore collections for real-time access by your applications.

Prerequisites

A Google Cloud project with Firestore enabled (Native mode or Datastore mode)
A GCP service account with the Cloud Datastore User role (roles/datastore.user) or equivalent permissions
A service account JSON key file downloaded from the Google Cloud Console
A Firestore database created in your project (the default database or a named database)

Permissions

The service account must have the following IAM permissions on the target Firestore database:

datastore.entities.create — Create new documents
datastore.entities.update — Update existing documents
datastore.entities.get — Read documents (used for connection testing)
datastore.entities.list — List documents (used for connection testing)

The simplest approach is to grant the Cloud Datastore User role (roles/datastore.user) which includes all required permissions.

Authentication

Service Account

Navigate to Destinations in Zeotap and click Add Destination
Select Firestore from the destination list
In the GCP Project ID field, enter your Google Cloud project ID
Paste the full contents of your service account JSON key file into the Service Account JSON field
Click Test Connection to verify that the service account can access the Firestore database

To create a service account key:

Go to the Google Cloud Console IAM & Admin page
Select or create a service account
Grant it the Cloud Datastore User role
Navigate to Keys > Add Key > Create new key > JSON
Download the JSON key file

Configuration

Field	Type	Required	Default	Description
GCP Project ID	Text	Yes	—	The Google Cloud project ID where the Firestore database is located
Database ID	Text	No	`(default)`	The Firestore database ID. Use `(default)` for the default database, or enter the ID of a named database

Target Settings

Field	Type	Required	Default	Description
Collection	Text	Yes	—	The Firestore collection to write documents to. Supports nested collections using slash notation (e.g., `users/profiles`)
Document ID Field	Text	No	—	The mapped field whose value is used as the Firestore document ID. If left blank, the sync primary key is used as the document ID

Supported Operations

Sync Modes

Mode	Supported	Description
Upsert	Yes	Create new documents or overwrite existing documents matching the document ID
Insert	Yes	Create new documents only. Writes to an existing document return a `FAILED_PRECONDITION` row error
Update	Yes	Update existing documents only. Writes to a missing document return a `FAILED_PRECONDITION` row error
Mirror	—	Not supported. Firestore does not provide a bulk truncate API for collections

Audience Sync Modes

Mode	Supported	Description
Add	—	Not supported — Firestore has no native audience or list membership concept
Remove	—	Not supported
Mirror	—	Not supported
Upsert	—	Not supported

Features

Field Mapping: Yes — map source fields to Firestore document fields with automatic type conversion
Schema Introspection: No — Firestore is schemaless and does not enforce a fixed document structure

Default Destination Fields

Field	Type	Description
`id`	string	Unique identifier for the document
`email`	string	Email address
`first_name`	string	First name
`last_name`	string	Last name
`created_at`	timestamp	Creation timestamp
`updated_at`	timestamp	Last update timestamp

How It Works

Zeotap writes data to Firestore using the REST API batchWrite endpoint:

Each row in the sync batch is converted into a Firestore document with typed field values (strings, integers, doubles, booleans, maps, arrays)
The document ID is determined by the Document ID Field setting, or falls back to the sync primary key
Rows are chunked into groups of 500 (the maximum per batchWrite request)
Each chunk is sent as a POST request to https://firestore.googleapis.com/v1/projects/{project}/databases/{database}/documents:batchWrite
Authentication uses a short-lived OAuth2 access token obtained via the service account JWT bearer assertion flow
The response contains a per-write status array — each write succeeds or fails independently (non-atomic)
Failed writes are reported as row-level errors with the Firestore error code and message

For insert mode, each write includes a precondition that the document must not already exist. For update mode, the precondition requires the document to already exist. For upsert mode, no precondition is applied — documents are created or overwritten unconditionally.

Rate Limits

Batch size: Up to 500 write operations per batchWrite request
Document writes: 10,000 writes per second per database (can be increased via Google Cloud support)
Document size: Maximum 1 MiB (1,048,576 bytes) per document
Field depth: Maximum 20 levels of nested fields
Concurrent connections: No hard limit, but Google Cloud may throttle at very high request rates (HTTP 429)

Zeotap automatically retries requests that receive HTTP 429 (Too Many Requests) or 5xx responses with exponential backoff (up to 4 attempts).

Best Practices

Use meaningful document IDs: Set the Document ID Field to a stable, unique identifier (e.g., user_id, customer_id) rather than relying on auto-generated primary keys. This enables efficient upsert operations and predictable document paths.
Keep documents small: Firestore charges per document read/write. Avoid syncing large JSON blobs or arrays with thousands of elements. Consider splitting wide data across sub-collections.
Choose the right sync mode: Use upsert for most use cases. Use insert only when you want to avoid overwriting existing documents (e.g., event logging). Use update when you want to enrich existing documents without creating new ones.
Use named databases for isolation: If you have multiple environments or tenants, create separate Firestore databases and configure each destination with the appropriate Database ID.
Grant least-privilege permissions: Use the Cloud Datastore User role rather than broader roles like Editor or Owner. Restrict the service account to only the project and database it needs.
Monitor write quotas: Check the Firestore Usage dashboard in Google Cloud Console to monitor document write rates and ensure you stay within quota limits.

Troubleshooting

PERMISSION_DENIED error

The service account does not have sufficient permissions on the Firestore database. Verify that the service account has the Cloud Datastore User role (roles/datastore.user) on the project. Check in the IAM console that the role is granted at the project level, not just a specific resource.

NOT_FOUND error on batchWrite

The specified Firestore database does not exist. Verify the Database ID is correct. If using the default database, ensure the value is (default). If using a named database, confirm it has been created in the Firestore console .

Invalid service account JSON

The service account JSON key is malformed or incomplete. Ensure you are pasting the complete JSON content (not just the private key). The JSON must contain client_email, private_key, and token_uri fields. Re-download the key from the Google Cloud Console if needed.

Documents not appearing in Firestore

Verify the Collection name is correct and matches the collection path you are querying in the Firestore console. Remember that collection names are case-sensitive. Also check that the sync mode is appropriate — if using update mode, documents will only be written if they already exist.

Token exchange failed

The service account JSON key may have been revoked or the private key is corrupted. Generate a new key from the Google Cloud Console. Also verify that the service account has not been disabled or deleted.

ALREADY_EXISTS error in insert mode

When using insert sync mode, writes will fail for documents that already exist. This is expected behavior. Switch to upsert mode if you want to create or overwrite documents regardless of whether they already exist.

Request timeout or network error

Firestore requests may time out if the network connection is slow or the Firestore service is temporarily unavailable. Zeotap retries transient errors automatically with exponential backoff. If timeouts persist, check your network connectivity to firestore.googleapis.com and verify there are no active Google Cloud incidents .

Rate limit exceeded (HTTP 429)

Your write rate exceeds the Firestore database quota. The default limit is 10,000 writes per second. Zeotap retries 429 responses automatically, but sustained high throughput may require requesting a quota increase from Google Cloud support. Consider reducing sync frequency or batch size.