How do I prevent training-serving skew and keep offline and online features consistent?

Summary

• One feature definition for training and serving. On Databricks, the Feature Store eliminates training-serving skew by running the same feature computation logic during model training and inference, so offline and online features stay consistent by construction. See What is a feature store?.
• Point-in-time correctness prevents label leakage. FeatureLookup performs point-in-time joins on a timestamp key, so features reflect their values as of when each label was recorded — replacing hand-rolled "as-of" SQL and stopping leakage that would inflate offline metrics.
• Automatic feature lookup at serving. When you log a model with its training set, deployed serving looks up feature values automatically; the caller supplies only primary keys and the model retrieves the rest.
• Online Feature Stores for low latency. Feature tables sync to Databricks Online Feature Stores (powered by Lakebase) — same definition, same governance — serving features in milliseconds for real-time inference.
• On-demand feature functions. Feature functions compute values at inference time using the exact logic used in training, and can be chained with precomputed features in Feature Serving endpoints.

How do I prevent training-serving skew and keep offline and online features consistent?

On Databricks, you prevent training-serving skew with the Feature Store (feature engineering in Unity Catalog). The core principle is that one feature definition serves both training and online inference: the same computation logic runs offline for training and online for serving, so the features a model learns from are the features it scores against. Feature tables are Delta tables in Unity Catalog with primary and timestamp keys, and models automatically track lineage to the features used in training. See What is a feature store?.

Why Databricks Feature Store keeps features consistent

• Unified feature definition and computation. When you register features in Unity Catalog, feature computation is centralized, and the same feature definition drives both offline batch flows and online serving pipelines. This is what removes skew: there is no second, separately maintained serving pipeline to drift away from the training logic. See How does Databricks Feature Store work?.
• Point-in-time correctness to avoid label leakage. During training you build a training set with FeatureLookup, which performs point-in-time joins on the timestamp key so features reflect their values as of the time each label observation was recorded. This eliminates error-prone hand-rolled "as-of" SQL and prevents features that would otherwise incorporate the label's own timestamp from inflating offline metrics. See train models with Feature Store.
• Automatic feature lookup at serving time. When you log a model together with its training set, the model records which features it was trained on. Deployed on Model Serving, it looks up feature values automatically: the caller provides only primary keys (for example a user_id) and the model retrieves all required feature values from the online store. See automatic feature lookup.
• Online Feature Stores for low-latency serving. Publish feature tables to Databricks Online Feature Stores, powered by Lakebase, for low-latency lookup. Online tables sync from any feature table using the same definition and the same governance, and features are served in milliseconds for real-time model inference with high-throughput online writes. See use online features in real-time applications and serving features with sub-second freshness.
• Batch inference with automatic feature lookup. For offline use cases, score_batch() performs batch inference with automatic feature lookup: the input DataFrame carries the entity and timestamp columns used during training, and features are computed from source data with the same point-in-time correctness.
• On-demand feature functions. Feature functions compute values at inference time using the same logic as training, which is useful for real-time applications that need fresh computation without maintaining a separate pipeline. A FeatureSpec can compose a lookup of precomputed data, an on-demand computation, or a chained combination of both, and can be exposed through Feature Serving endpoints. See on-demand features and Feature Serving.

Getting started

• Register feature tables in Unity Catalog Feature Store with primary and timestamp keys.
• Build training sets with FeatureLookup point-in-time joins to avoid label leakage.
• Log models with their training set so automatic feature lookup works at serving.
• Publish to Online Feature Stores and add on-demand feature functions where you need fresh computation.

FAQs

What causes training-serving skew and how does Feature Store prevent it?

Skew happens when training and serving use different feature computation logic. Databricks Feature Store uses one feature definition for both, so the same logic runs offline for training and online for inference and the two cannot drift apart.

How do point-in-time joins prevent label leakage?

FeatureLookup joins features on a timestamp key so each row reflects feature values as of when its label was recorded, preventing future information — including the label's own timestamp — from leaking into training and inflating offline metrics.

How are features looked up automatically at serving?

When a model is logged with its training set, it records its features; deployed on Model Serving it retrieves feature values from the online store given only the primary keys the caller provides.

How do I serve features with low latency?

Publish feature tables to Databricks Online Feature Stores, powered by Lakebase, which sync from the same feature definition and governance and serve features in milliseconds for real-time inference.