Pipeline

Create a pipeline

POST https://api.forefront.ai/v1/pipelines/create

Create a new pipeline

Request Body

Name	Type	Description
name	String	name of your pipeline

200: OK Returns the pipeline object

  {
    "id": "pipe_XXXX",
    "teamId": "your_team_id",
    "name": "test-pipeline",
    "createdAt": "2024-02-01T00:41:28.063Z",
    "updatedAt": "2024-02-01T00:41:28.063Z"
  }

List pipelines

GET https://api.forefront.ai/v1/pipelines/list

Returns a list of your pipelines

200: OK

[
  {
    "id": "pipe_XXXX",
    "teamId": "your_team_id",
    "name": "test-pipeline",
    "createdAt": "2024-02-01T00:41:28.063Z",
    "updatedAt": "2024-02-01T00:41:28.063Z"
  },
  ...
]

Get a pipeline by id

GET https://api.forefront.ai/v1/pipelines/:id

Returns a pipeline object by id. Does not return pipeline data, see below for how to do that.

Path Parameters

Name	Type	Description
id	String	id of the pipeline

200: OK

{
  "id": "pipe_XXXX",
  "teamId": "your_team_id",
  "name": "test-pipeline",
  "createdAt": "2024-02-01T00:44:46.070Z",
  "updatedAt": "2024-02-01T00:44:46.070Z"
}

Add to a pipeline

POST https://api.forefront.ai/v1/pipelines/:id/add

Add data to a pipeline

Path Parameters

Name	Type	Description
id	String	id of the pipeline

Request Body

Name	Type	Description
messages*	Array	Array of messages in chat-ml format. See datasets page for details on how to format
userId	String	Add a user_id to the sample
groupId	String	Add a group_id to the sample
metadata	Object	Add custom key-value string labels to a sample

200: OK

{
  "id": <id of the data sample>
}

Pipeline selections

Pipeline selections allow you to filter data in a pipeline. Pipelines can be filtered by userId, groupId, metadata, as well as limiting and paginating the number of responses.

You can create datasets from pipeline selections, inspect the individual data samples, and get a count of data that meets pipeline selection criteria.

Get data sample for a pipeline selection

POST https://api.forefront.ai/v1/pipelines/:id/samples

Returns array of data samples for a pipeline that meets filter criteria, if provided. Otherwise returns all data in pipeline. Each returned item contains a signed url that you can use to retrieve the text contents. The SDKs will automatically download the text contents from these urls.

Request Body

Name	Type	Description
limit	Number	Number of samples to return
offset	Number	Return examples after offset value. Used for pagination
userId	String	Filter results by user id
groupId	String	Filter results by group id
metadata	Object	Filter results by data that matches provided metadata

200: OK

[
  {
    "id": <id of the sample>,
    "length": 1, // number of messages in the sample
    "pipelineId": "pipe_XXXX",
    "teamId": "your_team_id",
    "createdAt": "2024-02-01T00:52:14.555Z",
    "userId": <userId if one provided>, // or null
    "groupId": <groupId if one provided>, // or null
    "metadata": <metadata if provided>, // or null
    "url": <signed_url> 
  },
  ...
]

Get count of pipeline selection

POST https://api.forefront.ai/v1/pipelines/:id/count

Returns count of data samples that match filter criteria if provided, otherwise returns count of all data samples in pipeline

Request Body

Name	Type	Description
limit	Number	Number of samples to return
offset	Number	Return examples after offset value. Used for pagination
userId	String	Filter results by user id
groupId	String	Filter results by group id
metadata	String	Filter results by data that matches provided metadata

200: OK

{
  "count": 100
}

Create a dataset from a pipeline selection

POST https://api.forefront.ai/v1/pipelines/:id/create-dataset

Create a dataset from a pipeline that matches filter criteria, if provided. Otherwise creates a dataset with all data in the pipeline

Request Body

Name	Type	Description
name*	String	Name of dataset

200: OK

{
  "id": "data_XXXX",
  "name": "<dataset name>",
  "teamId": "your_team_id",
  "datasetString": "<your team prefix>/<dataset name>",
  "size": 1000,
  "status": "PROCESSING",
  "createdAt": "2024-02-01T01:11:27.425Z",
  "deletedAt": null
}