Data Sources API

Create, manage, and sync data sources that power your bot's knowledge base.

List Data Sources

GET/data-sources/

Returns all data sources accessible to the authenticated user.

Query Parameters

organizationuuid

Filter by organization ID

statusstring

Filter by status: pending, processing, completed, failed

Response

[
  {
    "id": "ds_123",
    "name": "Product Documentation",
    "sourceType": "notion",
    "status": "completed",
    "statusMessage": null,
    "progressPercentage": 100,
    "documentCount": 45,
    "agentCount": 2,
    "lastSyncAt": "2024-01-15T10:30:00Z",
    "createdAt": "2024-01-10T08:00:00Z"
  },
  {
    "id": "ds_456",
    "name": "Help Center",
    "sourceType": "web",
    "status": "processing",
    "statusMessage": "Crawling pages...",
    "progressPercentage": 60,
    "documentCount": 30,
    "agentCount": 1,
    "lastSyncAt": null,
    "createdAt": "2024-01-15T09:00:00Z"
  }
]

Create Data Source

POST/data-sources/

Create a new data source and begin initial processing.

Request Body

namestringrequired

Display name for the data source

sourceTypestringrequired

Type: pdf, web, notion, google_drive, postgres, mysql, mongodb, api

organizationIduuidrequired

Organization to create data source in

configobject

Source-specific configuration (varies by type)

Config by Source Type

PDF Upload

{
  "name": "Product Manual",
  "sourceType": "pdf",
  "organizationId": "org_123",
  "config": {}
}
// Note: Files are uploaded separately via multipart form

Web Crawl

{
  "name": "Help Center",
  "sourceType": "web",
  "organizationId": "org_123",
  "config": {
    "url": "https://help.example.com",
    "crawlDepth": 3,
    "maxPages": 100,
    "includePatterns": ["/docs/*", "/help/*"],
    "excludePatterns": ["/blog/*"]
  }
}

Notion

OAuth Required

Notion data sources require OAuth authorization. Initiate the flow from the dashboard or use the OAuth endpoints.

{
  "name": "Company Wiki",
  "sourceType": "notion",
  "organizationId": "org_123",
  "config": {
    "accessToken": "secret_xxx",
    "pageIds": ["page_id_1", "page_id_2"],
    "includeSubpages": true
  }
}

PostgreSQL

{
  "name": "Product Database",
  "sourceType": "postgres",
  "organizationId": "org_123",
  "config": {
    "host": "db.example.com",
    "port": 5432,
    "database": "products",
    "user": "readonly_user",
    "password": "xxx",
    "ssl": true,
    "tables": ["products", "categories"],
    "query": "SELECT name, description FROM products WHERE active = true"
  }
}

Get Data Source

GET/data-sources/{dataSourceId}/

Get detailed information about a specific data source.

Response

DataSource

iduuid

Unique identifier

namestring

Display name

sourceTypestring

Source type

statusstring

pending, processing, completed, failed

statusMessagestring

Current status details

progressPercentageinteger

Processing progress (0-100)

currentPhasestring

Current processing phase

documentCountinteger

Number of indexed documents

agentCountinteger

Number of connected agents

lastSyncAtdatetime

Last successful sync

createdAtdatetime

Creation timestamp

Delete Data Source

DELETE/data-sources/{dataSourceId}/

Delete a data source and remove all its documents from the knowledge base. Connected agents will no longer have access to this content.

-purple-500">curl -X DELETE https://api.ragchats.ai/api/data-sources/ds_123/ \
  -H -green-500">"Authorization: Bearer YOUR_TOKEN"

Trigger Sync

POST/data-sources/{dataSourceId}/sync/

Manually trigger a sync for connected data sources (Notion, Google Drive, etc.). This re-fetches content and updates the knowledge base.

-purple-500">curl -X POST https://api.ragchats.ai/api/data-sources/ds_123/sync/ \
  -H -green-500">"Authorization: Bearer YOUR_TOKEN"

Response

{
  "message": "Sync started",
  "taskId": "task_abc123"
}

Async Processing

Syncing is asynchronous. Poll the data source endpoint to check status, or set up webhooks to receive notifications when sync completes.

Status Values

Status

pendingstring

Queued for processing

processingstring

Currently being processed

completedstring

Successfully processed and indexed

failedstring

Processing failed. Check statusMessage for details.