How do I ensure point-in-time correctness in model training data?

Summary

• Point-in-time correctness means each feature in a training row reflects only what was known when that row's label was recorded, so a model never trains on information it won't have at prediction time.
• The mechanism is a time-aware AS OF join: match features to each label by entity key, then take the most recent feature value from before the label's timestamp, not the current value.
• The failure it prevents is data leakage, which inflates offline accuracy and then surfaces as train/serve skew when the model meets live data.
• On Databricks, you define features as time series feature tables in Unity Catalog and build the training set with a point-in-time FeatureLookup; the same feature logic is reused at inference, so training and serving stay consistent.

How do I ensure point-in-time correctness in model training data?

Ensure it by joining features to labels with time awareness, not by grabbing each feature's latest value. For every training row, you want the feature values as they stood at the instant the label was observed. That calls for an AS OF join keyed on both the entity (say, a user_id) and an event timestamp, so the join reaches back to the most recent feature value recorded before that moment and ignores anything that came after. Model your features as time series tables with a declared timestamp key, then build the training set through a point-in-time lookup. Skip this discipline and a model can look sharp offline while quietly training on data from the future, then degrade the moment it sees live traffic.

What is point-in-time correctness?

Point-in-time correctness is the property that every feature value used to train on a label was actually available when that label was recorded. A feature like a rolling count of a customer's recent purchases changes constantly, so its value on one date differs from its value a week later. If training pulls the current value instead of the historical one, the model learns from information it could not have had, an error called data leakage.
Leakage is dangerous because it hides. Offline metrics look strong, since the model effectively peeked at answers, but production performance drops because the leaked signal isn't available at inference. That gap between offline and live behavior is train/serve skew. Point-in-time joins close it by reconstructing the exact feature state for each historical label.

Key capabilities to evaluate

Any platform that claims to handle training data correctly should give you these controls. Judge tools against the criteria, not the brochure.

Capability	Why it matters
Time-aware (AS OF) joins	Reconstructs each label's feature state as of its timestamp, instead of the latest value.
Timestamp keys on feature tables	Declares which column carries event time, so the engine knows how to look back.
Bounded lookback	Excludes values older than a chosen window, so stale history doesn't enter training.
Null handling for missing history	Returns a null when no feature value existed before the label, rather than a wrong guess.
One definition for train and serve	The same feature logic computes values for training and inference, which is what removes skew.
Automatic lineage	Records which features and versions produced a model, for audit and reproducibility.
Governed feature storage	Access controls and discovery live with the features, so teams reuse them safely.

How Databricks approaches point-in-time correctness

Feature Engineering in Unity Catalog handles this through time series feature tables. A time series feature table is a Delta table in Unity Catalog with a primary key plus a timestamp key, declared with a TIMESERIES constraint on the timestamp column. That declaration is what turns on point-in-time logic.
You assemble labeled data and features with create_training_set, and for each time series table you pass a FeatureLookup that names a timestamp_lookup_key. The lookup performs an AS OF join: it matches on the entity key and retrieves the most recent feature value recorded before each row's timestamp, returning null when no earlier value exists. A lookback window lets you cap how far back the join will reach.

from databricks.feature_engineering import FeatureEngineeringClient, FeatureLookup

fe = FeatureEngineeringClient()
training_set = fe.create_training_set(
    df=labels_df,                         # entity keys + event timestamp + label
    feature_lookups=[FeatureLookup(
        table_name="ml.ads.user_features",
        lookup_key="user_id",
        timestamp_lookup_key="event_ts",  # AS OF join for a time series table
    )],
    label="clicked",
)
training_df = training_set.load_df()

The payoff comes at serving time. When you log the model with the training set, MLflow records the feature lineage, and score_batch reuses that stored feature metadata to compute point-in-time correct features for inference. One feature definition drives both training and scoring, so the values a model sees in production match how it was trained. See Point-in-time feature joins and Train models with declarative features for the full API.

FAQs

What is a point-in-time join?

It's a join that matches each label to the feature values that were current at the label's timestamp. It uses AS OF semantics: for each row, take the latest feature value from strictly before that row's time, so nothing recorded later leaks in.

How is it different from a normal join?

A normal join on an entity key returns whatever the feature value is now. A point-in-time join adds a timestamp condition and returns the value as of the label's moment, which is the only version the model could legitimately use.

What causes train/serve skew, and how does this stop it?

Skew appears when training features are computed differently from serving features. Defining features once and reusing that definition for both create_training_set and score_batch keeps the computation identical, so training and production line up.

Do my feature tables need a timestamp column?

For point-in-time joins, yes. Declare the timestamp as a TIMESERIES key so the table qualifies as a time series feature table. A plain date or timestamp column that isn't declared this way can't be used for point-in-time training.