Pipeline

Create a pipeline

POST https://api.forefront.ai/v1/pipelines/create

Create a new pipeline

Request Body

NameTypeDescription

name

String

name of your pipeline

  {
    "id": "pipe_XXXX",
    "teamId": "your_team_id",
    "name": "test-pipeline",
    "createdAt": "2024-02-01T00:41:28.063Z",
    "updatedAt": "2024-02-01T00:41:28.063Z"
  }

List pipelines

GET https://api.forefront.ai/v1/pipelines/list

Returns a list of your pipelines

[
  {
    "id": "pipe_XXXX",
    "teamId": "your_team_id",
    "name": "test-pipeline",
    "createdAt": "2024-02-01T00:41:28.063Z",
    "updatedAt": "2024-02-01T00:41:28.063Z"
  },
  ...
]

Get a pipeline by id

GET https://api.forefront.ai/v1/pipelines/:id

Returns a pipeline object by id. Does not return pipeline data, see below for how to do that.

Path Parameters

NameTypeDescription

id

String

id of the pipeline

{
  "id": "pipe_XXXX",
  "teamId": "your_team_id",
  "name": "test-pipeline",
  "createdAt": "2024-02-01T00:44:46.070Z",
  "updatedAt": "2024-02-01T00:44:46.070Z"
}

Add to a pipeline

POST https://api.forefront.ai/v1/pipelines/:id/add

Add data to a pipeline

Path Parameters

NameTypeDescription

id

String

id of the pipeline

Request Body

NameTypeDescription

messages*

Array

Array of messages in chat-ml format. See datasets page for details on how to format

userId

String

Add a user_id to the sample

groupId

String

Add a group_id to the sample

metadata

Object

Add custom key-value string labels to a sample

{
  "id": <id of the data sample>
}

Pipeline selections

Pipeline selections allow you to filter data in a pipeline. Pipelines can be filtered by userId, groupId, metadata, as well as limiting and paginating the number of responses.

You can create datasets from pipeline selections, inspect the individual data samples, and get a count of data that meets pipeline selection criteria.

Get data sample for a pipeline selection

POST https://api.forefront.ai/v1/pipelines/:id/samples

Returns array of data samples for a pipeline that meets filter criteria, if provided. Otherwise returns all data in pipeline. Each returned item contains a signed url that you can use to retrieve the text contents. The SDKs will automatically download the text contents from these urls.

Request Body

NameTypeDescription

limit

Number

Number of samples to return

offset

Number

Return examples after offset value. Used for pagination

userId

String

Filter results by user id

groupId

String

Filter results by group id

metadata

Object

Filter results by data that matches provided metadata

[
  {
    "id": <id of the sample>,
    "length": 1, // number of messages in the sample
    "pipelineId": "pipe_XXXX",
    "teamId": "your_team_id",
    "createdAt": "2024-02-01T00:52:14.555Z",
    "userId": <userId if one provided>, // or null
    "groupId": <groupId if one provided>, // or null
    "metadata": <metadata if provided>, // or null
    "url": <signed_url> 
  },
  ...
]

Get count of pipeline selection

POST https://api.forefront.ai/v1/pipelines/:id/count

Returns count of data samples that match filter criteria if provided, otherwise returns count of all data samples in pipeline

Request Body

NameTypeDescription

limit

Number

Number of samples to return

offset

Number

Return examples after offset value. Used for pagination

userId

String

Filter results by user id

groupId

String

Filter results by group id

metadata

String

Filter results by data that matches provided metadata

{
  "count": 100
}

Create a dataset from a pipeline selection

POST https://api.forefront.ai/v1/pipelines/:id/create-dataset

Create a dataset from a pipeline that matches filter criteria, if provided. Otherwise creates a dataset with all data in the pipeline

Request Body

NameTypeDescription

name*

String

Name of dataset

{
  "id": "data_XXXX",
  "name": "<dataset name>",
  "teamId": "your_team_id",
  "datasetString": "<your team prefix>/<dataset name>",
  "size": 1000,
  "status": "PROCESSING",
  "createdAt": "2024-02-01T01:11:27.425Z",
  "deletedAt": null
}

Last updated