API Datasource Integration Guide

This guide explains how to build an API that’s compatible with our datasource system. Follow these specifications to ensure your API works seamlessly with our platform.

API Requirements

API Endpoint

Your API endpoint must be publicly accessible with a valid domain or IP address:

✅ Valid Examples:

  • https://api.yoursite.com/items
  • https://yoursite.com/api/products
  • http://123.456.789.123:8080/api/data
  • https://subdomain.example.org/endpoint

❌ Invalid Examples:

  • http://localhost:3000/api (localhost not accessible)
  • http://127.0.0.1/api (local IP not accessible)
  • http://192.168.1.100/api (private network not accessible)

Note: Your API must be accessible from external servers. Local development URLs cannot be reached by our system.

HTTP Method and Caching

Your API must support GET requests with these characteristics:

  • Accept all parameters via query string
  • Handle cache-busting parameters (we add _nonce automatically)
  • Return consistent results for identical requests

Response Format

Your API must return JSON responses in this exact format:

{
  "total_count": 100,
  "items": [
    {
      "id": "123",
      "title": "Product Name",
      "date_modified": "2024-03-20T15:30:45.000Z",
      "description": "Detailed product description...",
      "price": 29.99,
      "regular_price": 39.99,
      "sale_price": 29.99,
      "stock_status": "instock",
      "currency": "USD",
      "variations": [
        {
          "id": "456",
          "name": "Size L",
          "price": 32.99
        }
      ]
    }
  ]
}

Field Requirements

Required Fields

Every item must include these fields:

FieldTypeValidationExample
idstring | numberMust be unique, string or number"123" or 456
titlestringMust be non-empty string"Product Name"
date_modifiedstringExact ISO 8601 UTC with milliseconds: YYYY-MM-DDTHH:mm:ss.SSSZ"2024-03-20T15:30:45.000Z"

Critical Date Format Requirements

The date_modified field is strictly validated:

  • ✅ Correct: "2024-03-20T15:30:45.000Z"
  • ❌ Wrong: "2024-03-20T15:30:45Z" (missing milliseconds)
  • ❌ Wrong: "2024-03-20 15:30:45" (wrong format)
  • ❌ Wrong: "2024-03-20T15:30:45+00:00" (wrong timezone format)

Optional Fields

These fields have special system recognition, but you can include any additional fields you need:

FieldTypeDescription
descriptionstringDetailed description of the item
pricenumberCurrent price
regular_pricenumberRegular price before any discounts
sale_pricenumberSale price if applicable
stock_statusstringStock status: instock, outofstock, onbackorder, or any custom value
currencystringCurrency code: USD, EUR, IRR, ریال, تومان, or any custom code
variationsarrayProduct variations with pricing and inventory
...any_fieldanyYou can add any custom fields you need

Variations Field Details

Each variation can include any fields you need. Common examples:

FieldTypeDescription
idstring/numberUnique variation identifier
titlestringVariation display name
pricenumberVariation price
regular_pricenumberVariation regular price
sale_pricenumberVariation sale price
stock_statusstringVariation stock status
stocknumberAvailable quantity
...any_fieldanyAny custom variation attributes

Example 1 - E-commerce Product:

{
  "id": "123",
  "title": "Wireless Headphones",
  "date_modified": "2024-03-20T15:30:45.000Z",
  "description": "High-quality wireless headphones...",
  "price": 99.99,
  "regular_price": 129.99,
  "sale_price": 99.99,
  "stock_status": "instock",
  "currency": "USD",
  "category": "electronics",
  "brand": "TechBrand",
  "sku": "WH-123",
  "weight": 0.3,
  "variations": [
    {
      "id": "123-black-large",
      "title": "Wireless Headphones - Black Large",
      "price": 99.99,
      "regular_price": 129.99,
      "sale_price": 99.99,
      "stock_status": "instock",
      "stock": 25,
      "color": "black",
      "size": "large",
      "sku": "WH-123-BL",
      "weight": 0.35
    },
    {
      "id": "123-white-medium",
      "title": "Wireless Headphones - White Medium",
      "price": 104.99,
      "regular_price": 129.99,
      "sale_price": 104.99,
      "stock_status": "outofstock",
      "stock": 0,
      "color": "white",
      "size": "medium",
      "sku": "WH-123-WM",
      "expected_restock": "2024-04-15"
    },
    {
      "id": "123-red-small",
      "title": "Wireless Headphones - Red Small",
      "price": 94.99,
      "regular_price": 129.99,
      "sale_price": 94.99,
      "stock_status": "onbackorder",
      "stock": 5,
      "color": "red",
      "size": "small",
      "sku": "WH-123-RS",
      "pre_order": true
    }
  ]
}

Example 2 - Q&A Knowledge Base:

{
  "id": 1,
  "title": "What is Python?",
  "date_modified": "2024-03-15T12:00:00.000Z",
  "question": "What is Python?",
  "answer": "Python is a high-level, interpreted programming language known for its simplicity and readability."
}

Key Points:

  • Fields in the table above get special recognition for search and filtering
  • You can include unlimited custom fields for any data type (products, articles, Q&A, etc.)
  • All data you send will be stored and made searchable
  • Custom fields can be strings, numbers, objects, arrays, or any valid JSON type

Query Parameters

Your API must support all these parameters:

Pagination Parameters

ParameterTypeRequiredDescriptionExample
takenumberYesNumber of items to return?take=30
skipnumberYesNumber of items to skip?skip=60

Ordering Parameters

ParameterTypeRequiredDescriptionExample
orderstringYesSort direction: ASC or DESC?order=ASC
order_bystringYesFields to sort by (comma-separated)?order_by=date_modified,id

Filtering Parameters

ParameterTypeRequiredDescriptionExample
modified_afterstringYesISO date - returns items modified after this date (excludes exact match)?modified_after=2024-03-20T15:30:45.000Z

Cache Busting

ParameterTypeRequiredDescriptionExample
_noncestringAutoAutomatically added by our system?_nonce=1679328645123

Authentication Methods

No Authentication

Leave authentication type as “none”. Your API will receive requests without authentication headers.

API Key Authentication

Configure your datasource with:

  • API Key: Your actual key value
  • Header Name: Custom header name (e.g., X-API-Key, Authorization)

Example request your API will receive:

GET /api/items?take=10&skip=0&_nonce=1679328645123
Host: your-api.com
X-API-Key: your-api-key-here

Bearer Token Authentication

Standard OAuth 2.0 Bearer token format.

Example request your API will receive:

GET /api/items?take=10&skip=0&_nonce=1679328645123
Host: your-api.com
Authorization: Bearer your-token-here

Validation Requirements

Your API will be tested for these capabilities:

1. Basic Structure

  • Response has total_count (number) and items (array)
  • Items array is not empty
  • Each item has required fields with correct types

2. Pagination Accuracy

Test: GET /api?take=2&order=ASC&order_by=date_modified
Then: GET /api?take=1&skip=0&order=ASC&order_by=date_modified
Then: GET /api?take=1&skip=1&order=ASC&order_by=date_modified

Requirement: skip=0 returns first item, skip=1 returns second item exactly.

3. Ordering Functionality

Test: GET /api?take=2&order=ASC&order_by=date_modified
Test: GET /api?take=2&order=DESC&order_by=date_modified
Test: GET /api?take=1&order=ASC&order_by=date_modified,id

Requirement: Both ASC/DESC work correctly, multi-field sorting supported.

4. Date Filtering Precision

Test: Filter with date before oldest item (should return that item)
Test: Filter with exact oldest item date (should NOT return that item)
Test: Filter with future date (should return empty)

Requirement: modified_after must return only items with date_modified > filter_date (greater than, not equal).

Implementation Examples

PHP Laravel

public function index(Request $request)
{
    $query = Item::query();

    // Date filtering
    if ($request->has('modified_after')) {
        $query->where('updated_at', '>', $request->modified_after);
    }

    // Ordering
    $orderBy = explode(',', $request->get('order_by', 'updated_at,id'));
    $order = $request->get('order', 'ASC');

    foreach ($orderBy as $field) {
        $query->orderBy($field, $order);
    }

    // Pagination
    $take = (int) $request->get('take', 30);
    $skip = (int) $request->get('skip', 0);

    $total = $query->count();
    $items = $query->skip($skip)->take($take)->get();

    return response()->json([
        'total_count' => $total,
        'items' => $items->map(function ($item) {
            return [
                'id' => $item->id,
                'title' => $item->title,
                'date_modified' => $item->updated_at->format('Y-m-d\TH:i:s.v\Z'),
                'description' => $item->description,
                'price' => $item->price,
                // ... other fields
            ];
        })
    ]);
}

Node.js Express

app.get('/api/items', async (req, res) => {
  const {
    take = 30,
    skip = 0,
    order = 'ASC',
    order_by = 'updated_at,id',
    modified_after,
  } = req.query;

  let query = {};
  if (modified_after) {
    query.updated_at = { $gt: new Date(modified_after) };
  }

  const orderFields = order_by.split(',').map((field) => [field, order === 'DESC' ? -1 : 1]);

  const totalCount = await Item.countDocuments(query);
  const items = await Item.find(query)
    .sort(Object.fromEntries(orderFields))
    .skip(parseInt(skip))
    .limit(parseInt(take));

  res.json({
    total_count: totalCount,
    items: items.map((item) => ({
      id: item._id,
      title: item.title,
      date_modified: item.updated_at.toISOString(),
      description: item.description,
      price: item.price,
      // ... other fields
    })),
  });
});

Python FastAPI

from fastapi import FastAPI, Query, HTTPException, Security, Depends
from fastapi.security import APIKeyHeader, HTTPBearer, HTTPAuthorizationCredentials
from typing import Optional
import json
from pathlib import Path

app = FastAPI()

# Authentication setup
api_key_header = APIKeyHeader(name="X-API-Key", auto_error=False)
bearer_auth = HTTPBearer(auto_error=False)

def verify_api_key(api_key: str = Security(api_key_header)):
    if api_key and api_key != "your-valid-api-key":
        raise HTTPException(status_code=403, detail="Invalid API key")
    return api_key

def verify_bearer_token(credentials: HTTPAuthorizationCredentials = Security(bearer_auth)):
    if credentials and credentials.credentials != "your-valid-token":
        raise HTTPException(status_code=403, detail="Invalid bearer token")
    return credentials

def load_json_file(filename: str) -> list:
    with open(filename, "r", encoding="utf-8") as f:
        return json.load(f)

@app.get("/api/items")
async def get_items(
    skip: int = Query(default=0, ge=0),
    take: int = Query(default=30, ge=1, le=100),
    order_by: str = Query(default="date_modified,id"),
    order: str = Query(default="ASC", enum=["ASC", "DESC"]),
    modified_after: Optional[str] = Query(default=None),
    # Uncomment for API key auth:
    # api_key: str = Depends(verify_api_key),
    # Uncomment for Bearer token auth:
    # bearer_token = Depends(verify_bearer_token),
):
    items = load_json_file("items.json")

    # Filter by modified_after (greater than, not equal)
    if modified_after:
        items = [item for item in items if item["date_modified"] > modified_after]

    # Handle sorting
    order_fields = order_by.split(",")
    for field in reversed(order_fields):
        field = field.strip()
        if field == "date_modified":
            items.sort(key=lambda x: x["date_modified"], reverse=(order == "DESC"))
        elif field == "id":
            items.sort(key=lambda x: x["id"], reverse=(order == "DESC"))

    # Get total count before pagination
    total_count = len(items)

    # Apply pagination
    items = items[skip:skip + take]

    return {
        "total_count": total_count,
        "items": [
            {
                "id": item["id"],
                "title": item["title"],
                "date_modified": item["date_modified"],
                "description": item.get("description", ""),
                "price": item.get("price"),
                # ... other fields
            }
            for item in items
        ]
    }

Complete working example: For a full implementation with proper date validation, error handling, and multiple endpoints, check out our GitHub example repository.

Common Issues & Solutions

Date Format Errors

Problem: Date validation fails Solution: Use exactly YYYY-MM-DDTHH:mm:ss.SSSZ format with UTC timezone

Pagination Issues

Problem: Inconsistent results between pages Solution: Ensure stable sorting with unique secondary field (like id)

Authentication Failures

Problem: Invalid auth headers Solution: Verify header names match datasource configuration exactly

Testing Your API

Before integration, test these scenarios:

  1. Basic functionality: GET /api?take=5
  2. Pagination: GET /api?take=5&skip=5
  3. Ordering: GET /api?order=DESC&order_by=date_modified
  4. Date filtering: GET /api?modified_after=2024-01-01T00:00:00.000Z
  5. Combined parameters: All parameters together

Use tools like Postman or curl to verify your API meets all requirements before connecting to our system.