Embeddings

Meta’s ads research provides a starting point for understanding embeddings: it describes turning many pieces of information about a person into a few compact numerical profiles.3 In Meta’s research, these profiles can be shared across advertising models, allowing different models to use the same representation when ordering ads for that person.3 Meta’s research describes this reuse as a way to reduce duplicated work and infrastructure demands while supporting personalized ad rankings.3

Building on reusable profiles, Meta’s recommendation research describes embeddings tailored to a person’s interest in a particular item being considered, using a record of that person’s past activity.5 In Meta’s research, a large model produces these representations, and smaller models for different recommendation areas combine them with their own information.5 For search, Meta’s patent describes combining numerical representations of individual search words into a representation of the whole query.4 Meta’s patent then uses that combined representation to construct a scoring system that compares posts using their precomputed topic scores, helping order results by their relevance to the query.4

For uses other than relevance scoring, Meta’s ads research identifies tasks that numerical embeddings do not naturally support: human interpretation, combining sets of interests, and finding matches through an index—a lookup structure used to find matching items.2 Meta’s engineering team describes converting those embeddings into compact, unordered collections of interest concepts called Bag-of-Meaning tokens.2 In Meta’s ads research, a person’s tokens describe their interests, while an ad’s or advertiser’s tokens describe the interests they serve, giving both sides a shared vocabulary for those tasks.2

Moving from how embeddings are used to how they are maintained, Meta’s Reels research identifies a mismatch between the content representations used while a model learns and those used when it calculates recommendation scores.1 Meta’s research addresses this with a memory layer: a stored collection of item representations that is updated during training and read when calculating scores, so both stages use the same representations.1 Meta’s research reports fresher embeddings and a simpler update process, replacing three separate update paths with one.1 Meta’s research also reports that the share of items receiving a calculated recommendation score rose from 96% to 100%, with representations available even for newly created Reels not yet in the stored collection.1