Import from HuggingFace

Introduction

Huggingface is great for exploring open-source models, but trying out the models can be difficult without owning your own hardware and/or waiting for model weights to download.

Forefront offers the unique ability to import Huggingface models for inferencing and fine-tuning. If a model you're interested in isn't already on the platform, you can click "Import from Huggingface" on the Models page and paste in the model string. After a few minutes, the model will be available on the platform. Note this currently only works for Mistral models.

Automatic chat-ml tuning

Not all models are trained to use the familiar chat-ml syntax as this is a learned ability during fine-tuning. This can make usage of imported models difficult unless you have researched the specific syntax to use.

To improve the experience of model experimentation, Forefront automatically fine-tunes each imported model on a small dataset of chat-ml examples. We've found that this dramatically improves the the chat behavior of most models (although not all) while preserving model quality.

These models are specially labeled with the -chat-ml suffix and will appear under the Forefront team.

You can select if you'd like to use the original model or our chat-ml tuned version of the model. In both the Playground and the API, you can choose to use either chat-ml syntax or prompt-completion syntax depending on which version of the model you are using.

As an example, the imported version Mistral-7B-Instruct-v0.2 is named mistralai/Mistral-7B-Instruct-v0.2. Our chat-ml tuned version of the model is named forefront/Mistral-7B-Instruct-v0.2-chat-ml

Tips for using imported models

Start with Forefront's chat-tuned version of the model

If you don't know the specific syntax an imported model was trained with, then you will likely want to start by using Forefront's chat-ml tuned version of the model in the playground.

Research the original syntax used to train the model

If you're still not getting great results, you should figure out the syntax the model was trained with. This information is usually on the Huggingface model page or accompanying research paper.

Easily switch between chat and prompt-completion syntax in the playground

You can easily use the prompt-completion syntax in the playground by clicking the Chat dropdown in the upper left of the Playground page.