> ## Documentation Index
> Fetch the complete documentation index at: https://support.entegrata.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Collection Settings
> Configure schedules, load types, unique keys, and filters for resources
## Overview
Resource collection settings control how and when individual resources collect data. These settings can override connection-level defaults, allowing fine-grained control over each table, view, or endpoint.
Click the **Config** button (three dots) in the last column for the resource
Select **Collection Settings** from the dropdown menu
The Collection Settings modal has multiple tabs:
### Schedule Tab
**Interval-Based Schedule**
Set a specific collection interval:
* Time-based intervals (e.g., "Every 6 hours")
* Cron expressions for custom schedules
* Specific times (e.g., "Daily at 3:00 AM")
**Timing (Static vs Rolling)**
For sub-daily intervals, choose **Rolling** (default — the next run starts one interval after the previous collection finishes) or **Static** (runs at a consistent clock time anchored to a start time you set). See [Timing: Static vs Rolling](/data/collector/connections/collection-settings#timing-static-vs-rolling) for details.
**Active Status Toggle**
Enable or disable collection for this resource
### Load Type
Choose how data is collected:
**Full Load**
* Copies all data every time
* Best for small datasets or when you need complete snapshots
* No special source requirements
**Incremental Load**
* Copies only new or changed data
* Requires an incremental load field, also called a cursor field (like `modified_date` or `updated_at`)
* Much faster for large datasets
**Time-Window Incremental Load**
* Like Incremental Load, but re-scans a bounded time window on the cursor field every run instead of picking up where the last run left off
* Best when the cursor field exists but isn't fully reliable (e.g., `modified_date` doesn't always update) — re-scanning the window catches records the cursor would otherwise miss
* Requires both a cursor field and a configured time window (see below)
**Change Data Load** (SQL sources only)
* Detects and collects only records that have truly changed by comparing source and Entegrata
* Used when a table does not have a reliable cursor field for incremental loads but has a stable date field (e.g., `created_date`, `transaction_date`)
* More expensive than Incremental Load — choose Incremental whenever a reliable cursor field is available
* See [Sync Operation](/data/collector/resources/sync-operations) for a full explanation
### Incremental Load Field
If you select Incremental or Time-Window Incremental load type, choose the field that tracks when records were last updated:
The incremental load field must:
* Exist in the source data
* Update whenever a record changes
* Be a date, datetime, or timestamp type
### Time Window
If you select Time-Window Incremental load type, configure the window that bounds each collection run:
**Relative window**
A sliding window anchored to "now", such as the last 3 months or last 30 days. The window moves forward with each run, so collection always covers the most recent period.
**Absolute window**
A fixed start and end date (e.g., `2025-01-01` to `2025-06-30`). The same range is scanned on every run — useful for backfilling a specific historical period.
Use a relative window for ongoing collection of recent activity; use an absolute window when you're targeting a defined past period.
### Unique Key(s)
Specify which field(s) uniquely identify each record:
**Source-Managed Keys**
If the source system manages primary keys (e.g., database tables with PRIMARY KEY constraints), Entegrata uses those automatically and they cannot be changed.
You'll see: "Primary keys are managed by the source and cannot be changed"
**User-Defined Keys**
For sources without explicit primary keys (e.g., views, API endpoints, files), select one or more fields that uniquely identify records:
* Single field (e.g., `id`, `email`)
* Multiple fields (composite key, e.g., `customer_id` + `order_date`)
Unique keys are used to:
* Detect duplicate records
* Merge updates in incremental loads
* Ensure data quality
Add filters to limit what data is collected from this resource.
### Creating Filters
1. Select a field from the resource
2. Choose an operator (equals, greater than, contains, etc.)
3. Provide a value to filter on
4. Combine multiple filters with AND/OR logic
### Example Filters
* Collect only active records: `status = 'active'`
* Collect recent data: `created_date >= '2024-01-01'`
* Exclude test data: `email NOT LIKE '%@test.com'`
Filters are applied at the source before data is transferred, reducing collection time and storage costs.
Click **Save** to apply the resource collection settings
Turn on data collection for specific resources
Manually start collection for testing
Track collection execution and performance
Resolve resource collection issues