API Datasource Integration Guide

This guide explains how to build an API that’s compatible with our datasource system. Follow these specifications to ensure your API works seamlessly with our platform.

API Requirements

API Endpoint

Your API endpoint must be publicly accessible with a valid domain or IP address: ✅ Valid Examples:

https://api.yoursite.com/items
https://yoursite.com/api/products
http://123.456.789.123:8080/api/data
https://subdomain.example.org/endpoint

❌ Invalid Examples:

http://localhost:3000/api (localhost not accessible)
http://127.0.0.1/api (local IP not accessible)
http://192.168.1.100/api (private network not accessible)

Note: Your API must be accessible from external servers. Local development URLs cannot be reached by our system.

HTTP Method and Caching

Your API must support GET requests with these characteristics:

Accept all parameters via query string
Handle cache-busting parameters (we add _nonce automatically)
Return consistent results for identical requests

Response Format

Your API must return JSON responses in this exact format:

{
  "total_count": 100,
  "items": [
    {
      "id": "123",
      "title": "Product Name",
      "date_modified": "2024-03-20T15:30:45.000Z",
      "description": "Detailed product description...",
      "price": 29.99,
      "regular_price": 39.99,
      "sale_price": 29.99,
      "stock_status": "instock",
      "currency": "USD",
      "variations": [
        {
          "id": "456",
          "name": "Size L",
          "price": 32.99
        }
      ]
    }
  ]
}

Field Requirements

Required Fields

Every item must include these fields:

Field	Type	Validation	Example
`id`	string \| number	Must be unique, string or number	`"123"` or `456`
`title`	string	Must be non-empty string	`"Product Name"`
`date_modified`	string	Exact ISO 8601 UTC with milliseconds: `YYYY-MM-DDTHH:mm:ss.SSSZ`	`"2024-03-20T15:30:45.000Z"`

Critical Date Format Requirements

The date_modified field is strictly validated:

✅ Correct: "2024-03-20T15:30:45.000Z"
❌ Wrong: "2024-03-20T15:30:45Z" (missing milliseconds)
❌ Wrong: "2024-03-20 15:30:45" (wrong format)
❌ Wrong: "2024-03-20T15:30:45+00:00" (wrong timezone format)

Optional Fields

These fields have special system recognition, but you can include any additional fields you need:

Field	Type	Description
`description`	string	Detailed description of the item
`price`	number	Current price
`regular_price`	number	Regular price before any discounts
`sale_price`	number	Sale price if applicable
`stock_status`	string	Stock status: `instock`, `outofstock`, `onbackorder`, or any custom value
`currency`	string	Currency code: `USD`, `EUR`, `IRR`, `ریال`, `تومان`, or any custom code
`variations`	array	Product variations with pricing and inventory
`...any_field`	any	You can add any custom fields you need

Variations Field Details

Each variation can include any fields you need. Common examples:

Field	Type	Description
`id`	string/number	Unique variation identifier
`title`	string	Variation display name
`price`	number	Variation price
`regular_price`	number	Variation regular price
`sale_price`	number	Variation sale price
`stock_status`	string	Variation stock status
`stock`	number	Available quantity
`...any_field`	any	Any custom variation attributes

Example 1 - E-commerce Product:

{
  "id": "123",
  "title": "Wireless Headphones",
  "date_modified": "2024-03-20T15:30:45.000Z",
  "description": "High-quality wireless headphones...",
  "price": 99.99,
  "regular_price": 129.99,
  "sale_price": 99.99,
  "stock_status": "instock",
  "currency": "USD",
  "category": "electronics",
  "brand": "TechBrand",
  "sku": "WH-123",
  "weight": 0.3,
  "variations": [
    {
      "id": "123-black-large",
      "title": "Wireless Headphones - Black Large",
      "price": 99.99,
      "regular_price": 129.99,
      "sale_price": 99.99,
      "stock_status": "instock",
      "stock": 25,
      "color": "black",
      "size": "large",
      "sku": "WH-123-BL",
      "weight": 0.35
    },
    {
      "id": "123-white-medium",
      "title": "Wireless Headphones - White Medium",
      "price": 104.99,
      "regular_price": 129.99,
      "sale_price": 104.99,
      "stock_status": "outofstock",
      "stock": 0,
      "color": "white",
      "size": "medium",
      "sku": "WH-123-WM",
      "expected_restock": "2024-04-15"
    },
    {
      "id": "123-red-small",
      "title": "Wireless Headphones - Red Small",
      "price": 94.99,
      "regular_price": 129.99,
      "sale_price": 94.99,
      "stock_status": "onbackorder",
      "stock": 5,
      "color": "red",
      "size": "small",
      "sku": "WH-123-RS",
      "pre_order": true
    }
  ]
}

Example 2 - Q&A Knowledge Base:

{
  "id": 1,
  "title": "What is Python?",
  "date_modified": "2024-03-15T12:00:00.000Z",
  "question": "What is Python?",
  "answer": "Python is a high-level, interpreted programming language known for its simplicity and readability."
}

Key Points:

Fields in the table above get special recognition for search and filtering
You can include unlimited custom fields for any data type (products, articles, Q&A, etc.)
All data you send will be stored and made searchable
Custom fields can be strings, numbers, objects, arrays, or any valid JSON type

Query Parameters

Your API must support all these parameters:

Pagination Parameters

Parameter	Type	Required	Description	Example
`take`	number	Yes	Number of items to return	`?take=30`
`skip`	number	Yes	Number of items to skip	`?skip=60`

Ordering Parameters

Parameter	Type	Required	Description	Example
`order`	string	Yes	Sort direction: `ASC` or `DESC`	`?order=ASC`
`order_by`	string	Yes	Fields to sort by (comma-separated)	`?order_by=date_modified,id`

Filtering Parameters

Parameter	Type	Required	Description	Example
`modified_after`	string	Yes	ISO date - returns items modified after this date (excludes exact match)	`?modified_after=2024-03-20T15:30:45.000Z`

Cache Busting

Parameter	Type	Required	Description	Example
`_nonce`	string	Auto	Automatically added by our system	`?_nonce=1679328645123`

Authentication Methods

No Authentication

Leave authentication type as “none”. Your API will receive requests without authentication headers.

API Key Authentication

Configure your datasource with:

API Key: Your actual key value
Header Name: Custom header name (e.g., X-API-Key, Authorization)

Example request your API will receive:

GET /api/items?take=10&skip=0&_nonce=1679328645123
Host: your-api.com
X-API-Key: your-api-key-here

Bearer Token Authentication

Standard OAuth 2.0 Bearer token format. Example request your API will receive:

GET /api/items?take=10&skip=0&_nonce=1679328645123
Host: your-api.com
Authorization: Bearer your-token-here

Validation Requirements

Your API will be tested for these capabilities:

1. Basic Structure

Response has total_count (number) and items (array)
Items array is not empty
Each item has required fields with correct types

2. Pagination Accuracy

Test: GET /api?take=2&order=ASC&order_by=date_modified
Then: GET /api?take=1&skip=0&order=ASC&order_by=date_modified
Then: GET /api?take=1&skip=1&order=ASC&order_by=date_modified

Requirement: skip=0 returns first item, skip=1 returns second item exactly.

3. Ordering Functionality

Test: GET /api?take=2&order=ASC&order_by=date_modified
Test: GET /api?take=2&order=DESC&order_by=date_modified
Test: GET /api?take=1&order=ASC&order_by=date_modified,id

Requirement: Both ASC/DESC work correctly, multi-field sorting supported.

4. Date Filtering Precision

Test: Filter with date before oldest item (should return that item)
Test: Filter with exact oldest item date (should NOT return that item)
Test: Filter with future date (should return empty)

Requirement: modified_after must return only items with date_modified > filter_date (greater than, not equal).

Implementation Examples

PHP Laravel

public function index(Request $request)
{
    $query = Item::query();

    // Date filtering
    if ($request->has('modified_after')) {
        $query->where('updated_at', '>', $request->modified_after);
    }

    // Ordering
    $orderBy = explode(',', $request->get('order_by', 'updated_at,id'));
    $order = $request->get('order', 'ASC');

    foreach ($orderBy as $field) {
        $query->orderBy($field, $order);
    }

    // Pagination
    $take = (int) $request->get('take', 30);
    $skip = (int) $request->get('skip', 0);

    $total = $query->count();
    $items = $query->skip($skip)->take($take)->get();

    return response()->json([
        'total_count' => $total,
        'items' => $items->map(function ($item) {
            return [
                'id' => $item->id,
                'title' => $item->title,
                'date_modified' => $item->updated_at->format('Y-m-d\TH:i:s.v\Z'),
                'description' => $item->description,
                'price' => $item->price,
                // ... other fields
            ];
        })
    ]);
}

Node.js Express

app.get('/api/items', async (req, res) => {
  const {
    take = 30,
    skip = 0,
    order = 'ASC',
    order_by = 'updated_at,id',
    modified_after,
  } = req.query;

  let query = {};
  if (modified_after) {
    query.updated_at = { $gt: new Date(modified_after) };
  }

  const orderFields = order_by.split(',').map((field) => [field, order === 'DESC' ? -1 : 1]);

  const totalCount = await Item.countDocuments(query);
  const items = await Item.find(query)
    .sort(Object.fromEntries(orderFields))
    .skip(parseInt(skip))
    .limit(parseInt(take));

  res.json({
    total_count: totalCount,
    items: items.map((item) => ({
      id: item._id,
      title: item.title,
      date_modified: item.updated_at.toISOString(),
      description: item.description,
      price: item.price,
      // ... other fields
    })),
  });
});

Python FastAPI

from fastapi import FastAPI, Query, HTTPException, Security, Depends
from fastapi.security import APIKeyHeader, HTTPBearer, HTTPAuthorizationCredentials
from typing import Optional
import json
from pathlib import Path

app = FastAPI()

# Authentication setup
api_key_header = APIKeyHeader(name="X-API-Key", auto_error=False)
bearer_auth = HTTPBearer(auto_error=False)

def verify_api_key(api_key: str = Security(api_key_header)):
    if api_key and api_key != "your-valid-api-key":
        raise HTTPException(status_code=403, detail="Invalid API key")
    return api_key

def verify_bearer_token(credentials: HTTPAuthorizationCredentials = Security(bearer_auth)):
    if credentials and credentials.credentials != "your-valid-token":
        raise HTTPException(status_code=403, detail="Invalid bearer token")
    return credentials

def load_json_file(filename: str) -> list:
    with open(filename, "r", encoding="utf-8") as f:
        return json.load(f)

@app.get("/api/items")
async def get_items(
    skip: int = Query(default=0, ge=0),
    take: int = Query(default=30, ge=1, le=100),
    order_by: str = Query(default="date_modified,id"),
    order: str = Query(default="ASC", enum=["ASC", "DESC"]),
    modified_after: Optional[str] = Query(default=None),
    # Uncomment for API key auth:
    # api_key: str = Depends(verify_api_key),
    # Uncomment for Bearer token auth:
    # bearer_token = Depends(verify_bearer_token),
):
    items = load_json_file("items.json")

    # Filter by modified_after (greater than, not equal)
    if modified_after:
        items = [item for item in items if item["date_modified"] > modified_after]

    # Handle sorting
    order_fields = order_by.split(",")
    for field in reversed(order_fields):
        field = field.strip()
        if field == "date_modified":
            items.sort(key=lambda x: x["date_modified"], reverse=(order == "DESC"))
        elif field == "id":
            items.sort(key=lambda x: x["id"], reverse=(order == "DESC"))

    # Get total count before pagination
    total_count = len(items)

    # Apply pagination
    items = items[skip:skip + take]

    return {
        "total_count": total_count,
        "items": [
            {
                "id": item["id"],
                "title": item["title"],
                "date_modified": item["date_modified"],
                "description": item.get("description", ""),
                "price": item.get("price"),
                # ... other fields
            }
            for item in items
        ]
    }

Complete working example: For a full implementation with proper date validation, error handling, and multiple endpoints, check out our GitHub example repository.

Common Issues & Solutions

Date Format Errors

Problem: Date validation fails Solution: Use exactly YYYY-MM-DDTHH:mm:ss.SSSZ format with UTC timezone

Pagination Issues

Problem: Inconsistent results between pages Solution: Ensure stable sorting with unique secondary field (like id)

Authentication Failures

Problem: Invalid auth headers Solution: Verify header names match datasource configuration exactly

Testing Your API

Before integration, test these scenarios:

Basic functionality: GET /api?take=5
Pagination: GET /api?take=5&skip=5
Ordering: GET /api?order=DESC&order_by=date_modified
Date filtering: GET /api?modified_after=2024-01-01T00:00:00.000Z
Combined parameters: All parameters together

Use tools like Postman or curl to verify your API meets all requirements before connecting to our system.

Datastore

Api

API Datasource Integration Guide

API Requirements

API Endpoint

HTTP Method and Caching

Response Format

Field Requirements

Required Fields

Critical Date Format Requirements

Optional Fields

Variations Field Details

Query Parameters

Ordering Parameters

Filtering Parameters

Cache Busting

Authentication Methods

No Authentication

API Key Authentication

Bearer Token Authentication

Validation Requirements

1. Basic Structure

3. Ordering Functionality

4. Date Filtering Precision

Implementation Examples

PHP Laravel

Node.js Express

Python FastAPI

Common Issues & Solutions

Date Format Errors

Authentication Failures

Testing Your API

Datastore

​API Datasource Integration Guide

​API Requirements

​API Endpoint

​HTTP Method and Caching

​Response Format

​Field Requirements

​Required Fields

​Critical Date Format Requirements

​Optional Fields

​Variations Field Details

​Query Parameters

​Pagination Parameters

​Ordering Parameters

​Filtering Parameters

​Cache Busting

​Authentication Methods

​No Authentication

​API Key Authentication

​Bearer Token Authentication

​Validation Requirements

​1. Basic Structure

​2. Pagination Accuracy

​3. Ordering Functionality

​4. Date Filtering Precision

​Implementation Examples

​PHP Laravel

​Node.js Express

​Python FastAPI

​Common Issues & Solutions

​Date Format Errors

​Pagination Issues

​Authentication Failures

​Testing Your API

API Datasource Integration Guide

API Requirements

API Endpoint

HTTP Method and Caching

Response Format

Field Requirements

Required Fields

Critical Date Format Requirements

Optional Fields

Variations Field Details

Query Parameters

Pagination Parameters

Ordering Parameters

Filtering Parameters

Cache Busting

Authentication Methods

No Authentication

API Key Authentication

Bearer Token Authentication

Validation Requirements

1. Basic Structure

2. Pagination Accuracy

3. Ordering Functionality

4. Date Filtering Precision

Implementation Examples

PHP Laravel

Node.js Express

Python FastAPI

Common Issues & Solutions

Date Format Errors

Pagination Issues

Authentication Failures

Testing Your API