# Pipeline

## Create a pipeline

<mark style="color:green;">`POST`</mark> `https://api.forefront.ai/v1/pipelines/create`

Create a new pipeline

#### Request Body

| Name | Type   | Description           |
| ---- | ------ | --------------------- |
| name | String | name of your pipeline |

{% tabs %}
{% tab title="200: OK Returns the pipeline object" %}

```
  {
    "id": "pipe_XXXX",
    "teamId": "your_team_id",
    "name": "test-pipeline",
    "createdAt": "2024-02-01T00:41:28.063Z",
    "updatedAt": "2024-02-01T00:41:28.063Z"
  }
```

{% endtab %}
{% endtabs %}

## List pipelines

<mark style="color:blue;">`GET`</mark> `https://api.forefront.ai/v1/pipelines/list`

Returns a list of your pipelines

{% tabs %}
{% tab title="200: OK " %}

```
[
  {
    "id": "pipe_XXXX",
    "teamId": "your_team_id",
    "name": "test-pipeline",
    "createdAt": "2024-02-01T00:41:28.063Z",
    "updatedAt": "2024-02-01T00:41:28.063Z"
  },
  ...
]
```

{% endtab %}
{% endtabs %}

## Get a pipeline by id

<mark style="color:blue;">`GET`</mark> `https://api.forefront.ai/v1/pipelines/:id`

Returns a pipeline object by id. Does not return pipeline data, see below for how to do that.

#### Path Parameters

| Name | Type   | Description        |
| ---- | ------ | ------------------ |
| id   | String | id of the pipeline |

{% tabs %}
{% tab title="200: OK " %}

```
{
  "id": "pipe_XXXX",
  "teamId": "your_team_id",
  "name": "test-pipeline",
  "createdAt": "2024-02-01T00:44:46.070Z",
  "updatedAt": "2024-02-01T00:44:46.070Z"
}
```

{% endtab %}
{% endtabs %}

## Add to a pipeline

<mark style="color:green;">`POST`</mark> `https://api.forefront.ai/v1/pipelines/:id/add`

Add data to a pipeline

#### Path Parameters

| Name | Type   | Description        |
| ---- | ------ | ------------------ |
| id   | String | id of the pipeline |

#### Request Body

| Name                                       | Type   | Description                                                                         |
| ------------------------------------------ | ------ | ----------------------------------------------------------------------------------- |
| messages<mark style="color:red;">\*</mark> | Array  | Array of messages in chat-ml format. See datasets page for details on how to format |
| userId                                     | String | Add a user\_id to the sample                                                        |
| groupId                                    | String | Add a group\_id to the sample                                                       |
| metadata                                   | Object | Add custom key-value string labels to a sample                                      |

{% tabs %}
{% tab title="200: OK " %}

```
{
  "id": <id of the data sample>
}
```

{% endtab %}
{% endtabs %}

## Pipeline selections

Pipeline selections allow you to filter data in a pipeline. Pipelines can be filtered by userId, groupId, metadata, as well as limiting and paginating the number of responses.

You can create datasets from pipeline selections, inspect the individual data samples, and get a count of data that meets pipeline selection criteria.

## Get data sample for a pipeline selection

<mark style="color:green;">`POST`</mark> `https://api.forefront.ai/v1/pipelines/:id/samples`

Returns array of data samples for a pipeline that meets filter criteria, if provided. Otherwise returns all data in pipeline. Each returned item contains a signed url that you can use to retrieve the text contents. The SDKs will automatically download the text contents from these urls.

#### Request Body

| Name     | Type   | Description                                             |
| -------- | ------ | ------------------------------------------------------- |
| limit    | Number | Number of samples to return                             |
| offset   | Number | Return examples after offset value. Used for pagination |
| userId   | String | Filter results by user id                               |
| groupId  | String | Filter results by group id                              |
| metadata | Object | Filter results by data that matches provided metadata   |

{% tabs %}
{% tab title="200: OK " %}

```json
[
  {
    "id": <id of the sample>,
    "length": 1, // number of messages in the sample
    "pipelineId": "pipe_XXXX",
    "teamId": "your_team_id",
    "createdAt": "2024-02-01T00:52:14.555Z",
    "userId": <userId if one provided>, // or null
    "groupId": <groupId if one provided>, // or null
    "metadata": <metadata if provided>, // or null
    "url": <signed_url> 
  },
  ...
]
```

{% endtab %}
{% endtabs %}

## Get count of pipeline selection

<mark style="color:green;">`POST`</mark> `https://api.forefront.ai/v1/pipelines/:id/count`

Returns count of data samples that match filter criteria if provided, otherwise returns count of all data samples in pipeline

#### Request Body

| Name     | Type   | Description                                             |
| -------- | ------ | ------------------------------------------------------- |
| limit    | Number | Number of samples to return                             |
| offset   | Number | Return examples after offset value. Used for pagination |
| userId   | String | Filter results by user id                               |
| groupId  | String | Filter results by group id                              |
| metadata | String | Filter results by data that matches provided metadata   |

{% tabs %}
{% tab title="200: OK " %}

```
{
  "count": 100
}
```

{% endtab %}
{% endtabs %}

## Create a dataset from a pipeline selection&#x20;

<mark style="color:green;">`POST`</mark> `https://api.forefront.ai/v1/pipelines/:id/create-dataset`

Create a dataset from a pipeline that matches filter criteria, if provided. Otherwise creates a dataset with all data in the pipeline

#### Request Body

| Name                                   | Type   | Description     |
| -------------------------------------- | ------ | --------------- |
| name<mark style="color:red;">\*</mark> | String | Name of dataset |

{% tabs %}
{% tab title="200: OK " %}

```
{
  "id": "data_XXXX",
  "name": "<dataset name>",
  "teamId": "your_team_id",
  "datasetString": "<your team prefix>/<dataset name>",
  "size": 1000,
  "status": "PROCESSING",
  "createdAt": "2024-02-01T01:11:27.425Z",
  "deletedAt": null
}
```

{% endtab %}
{% endtabs %}