Poll Based (Hourly)
Set up automated polling so Chipper checks your S3 bucket for new log files on a regular schedule (every hour by default). This is the simplest setup method but there will be up to an hour delay before you see logs in Chipper.
We recommend using Event Driven file uploads as this will give you real-time logs in Chipper.
Configuration
Before configuring this method, make sure you've completed:
- ✅ Chipper Access - IAM credentials created
The following steps are all through Chipper.
1. Navigate to Log Sources
In Chipper, go to Settings → Log Sources → Add Log Source
2. Select AWS S3
Choose AWS S3 as your log source type.
3. Fill in Required Fields
| Field | Description | Example |
|---|---|---|
| Instrument | Whatever instrument you are setting up log ingestion for | Tecan-1 |
| Name | Friendly name for this source | Production API Logs |
| AWS Access Key | IAM user access key (from access setup) | AKIAIOSFODNN7EXAMPLE |
| AWS Secret Key | IAM user secret key | wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY |
| S3 Bucket Name | Name of your S3 bucket | my-app-logs |
| S3 Region | AWS region of your bucket | us-east-1 |
4. Configure Optional Settings
| Field | Description | Default |
|---|---|---|
| Prefix | Folder path within bucket | (empty = root) |
| File Pattern | Only process files matching this pattern | *.log |
Prefix Examples:
logs/production/- Only files in this folderapp-name/2024/- Specific app and year- Leave empty to process all files in bucket
File Pattern Examples:
*.log- Only .log fileserror-*.txt- Files starting with "error-"*- All files (use with caution)
5. Test Connection
Click Test Connection to verify:
- ✅ AWS credentials are valid
- ✅ S3 bucket exists and is accessible
- ✅ Permissions allow listing and reading files
6. Save Configuration
Click Save to activate polling. Chipper will start checking for new files on the next hourly cycle.
Next Steps
- For faster ingestion, see Event Driven (Instant)