Datasets

Store structured scraped data in tabular datasets. Each dataset holds rows of JSON objects with automatic field detection.

Creating a Dataset

import { ScrayleClient } from '@scrayle/sdk';

const client = new ScrayleClient();

const dataset = await client.storage.dataset.createDataset('product-data');
console.log(dataset.id);

Adding Items

await client.storage.dataset.addItems(dataset.id, [
  { name: 'Product A', price: 29.99, category: 'Electronics' },
  { name: 'Product B', price: 49.99, category: 'Electronics' },
  { name: 'Product C', price: 15.00, category: 'Books' },
]);

Reading Items

const items = await client.storage.dataset.getItems(dataset.id, {
  page: 1,
  pageSize: 100,
});

for (const item of items.items) {
  console.log(item.name, item.price);
}

console.log('Total items:', items.total);

Listing Datasets

const datasets = await client.storage.dataset.listDatasets({
  page: 1,
  pageSize: 20,
});

for (const ds of datasets.items) {
  console.log(ds.id, ds.name, ds.stats.count, 'items');
}

Updating and Deleting

// Rename a dataset
await client.storage.dataset.updateDataset(dataset.id, 'new-name');

// Delete a dataset and all items
await client.storage.dataset.delDataset(dataset.id);

API Reference