Search…
Embeddings
Learn how to use embeddings and the different use cases solved by representing words as numbers.
An embedding is a list of numbers (a vector) in high dimensional space. They are used to represent words as numbers such that the distance between two numbers in the vector space is correlated with meaning (semantic similarity). For example, you can see how closely related concepts are by looking at how close their embeddings (numbers) are.
Currently we offer the following base embedding models for functionalities like text search, text similarity, and code search:
- GPT-J-6B (4096 dimensions)
- Fairseq GPT-13B (5120 dimensions)
- GPT-NeoX-20B (6144 dimensions)
A higher embedding dimension means the model will have a greater understanding of text and its meaning.
How to get embeddings
To obtain an embedding vector for a piece of text or texts we make a request to the Embeddings endpoint as shown in the following code snippet:
Python
1
import requests
2
​
3
headers = {"Authorization": "Bearer <INSERT_YOUR_TOKEN>"}
4
​
5
body = {
6
"texts": ["hello world", "once upon a time"]
7
}
8
​
9
res = requests.post(
10
"https://search-qa.forefront.link/embedding",
11
json=body,
12
headers=headers
13
)
Copied!
Note that the maximum length of input text for our embedding endpoints are 2048 tokens. You should verify that your inputs do not exceed this limit before making a request.
Unless your are embedding code, we suggest replacing newlines
\n in your input with a single space.Example use
For this use case, we'll be classifying product titles by their product category. The principles of this use case can be applied to any text or code similarity task like classification, search, or labelling.
First, store your API key and URL. Then write a function to get embeddings for any array of text.
Python
1
import requests
2
import torch
3
​
4
# Remember to insert your own API key and url
5
api_key = "{Settings -> API Keys}"
6
api_url = "https://example.forefront.link/embedding (this is a model's url that you'd normally use for completions but added a /embedding on the URL)"
7
​
8
def get_embeddings(texts):
9
body = {
10
'texts': texts
11
}
12
headers = {
13
'content-type': 'application/json',
14
'authorization': f'Bearer {YOUR_API_KEY}'
15
}
16
res = requests.post(url, headers=headers, json=body)
17
return torch.tensor(res.json()['result'])
Copied!
​
Python
1
import math
2
​
3
def process_schema(schema):
4
out = {}
5
for k, v in schema.items():
6
​
7
embeds = []
8
# this seemingly random number 8 is to batch the embeddings (8 is batch size) from your model on Forefront
9
# this will result in much better throughput than sending 1 at a time
10
for i in range(math.ciel(len(v) / 8)):
11
e = get_embeddings(v[i * 8: (i + 1) * 8])
12
for j in e.size(0):
13
embeds.append(e[j])
14
​
15
​
16
embeds = torch.stack(embeds)
17
out[k] = embeds
18
return out
19
​
20
# this is a geometric mean
21
# https://en.wikipedia.org/wiki/Geometric_mean
22
def gmean(input_x, dim):
23
log_x = torch.log(input_x)
24
return torch.exp(torch.mean(log_x, dim=dim))
25
​
26
# this is a harmonic mean
27
# https://en.wikipedia.org/wiki/Harmonic_mean
28
def harmonic_mean(arr, dim=0):
29
n = arr.size(0)
30
return torch.reciprocal(torch.reciprocal(arr).sum(dim)) * n
Copied!
Python
1
def classify(query, processed_schema):
2
3
query_embed = get_embeddings([query])
4
classes = []
5
logits = None
6
for k, v in processed_schema.items():
7
classes.append(k)
8
items = []
9
for item in v:
10
items.append(torch.dist(item, query_embed).item())
11
​
12
# use the harmonic mean bc if there is a strong match it should bias the entire group
13
# if using a normal mean, even if there is a dead match, the average will be only somewhat affected
14
# when using the harmonic mean, if one example is a perfect match then the entire class will rank as a perfect match
15
hmean = harmonic_mean(torch.tensor(items))
16
if logits is None:
17
logits = hmean[None, ...]
18
else:
19
logits = torch.cat((logits, hmean[None, ...]), dim=0)
20
21
# doing the inverse bc you want dist to be low (close together in that high dimensional space)
22
logits = logits.max() - logits
23
# softmax to normalize as a probability distribution
24
probs = torch.nn.functional.softmax(logits, dim=0)
25
​
26
classification = {}
27
for i, c in enumerate(classes):
28
classification[c] = probs[i].contiguous()
29
​
30
return classification
Copied!
Now let's classify products according to a specified schema:
Python
1
# these are dummy examples for ecommerce titles
2
schema = {
3
'lighting & decor': ['Off White Lampshade | Best for 40 inch Table Lamps', 'Modern Wall Art - Artisenal Quality Wall Art From Paris'],
4
'clothing': ['Luxe Leather Bomber Jacket', 'Cashmere Joggers | Keep Warm This Winter', 'Everyday Skinny Jeans - Marble Washed'],
5
'sports & outdoors': ['10 Foot Regulation Basketball Goal', 'Wilson/Spalding Real Genuine Leather NBA Basketball', 'Wilson Tennis Racket | 32 Inch Frame | 12 Inch Handle']
6
}
7
​
8
# do this once. you can cache this result so it only ever happens once, and then when you run in production it is _SUPER_ fast
9
processed_schema = process_schema(schema)
10
​
11
classify('Luxe Satin Egyptian Cotten Sweater', processed_schema)
12
​
13
classify('Wall Lamp With Cast Iron Frame', processed_schema)
Copied!
This next code snippet is showing you the point of different mean:
Python
1
gmean(torch.tensor([0.1, 0.5, 0.6, 0.7, 0.001]), 0)
2
# tensor(0.1160)
3
​
4
harmonic_mean(torch.tensor([0.1, 0.5, 0.6, 0.7, 0.001]), 0)
5
# tensor(0.0049)
6
​
7
# this is the normal mean, also called tha arithmatic mean
8
torch.mean(torch.tensor([0.1, 0.5, 0.6, 0.7, 0.001]))
9
#tensor(0.3802)
Copied!
Last modified 3mo ago
Copy link