How do I govern and discover features across data science teams?

Summary

• Feature Views are first-class Unity Catalog objects, making them discoverable, access-controlled, and tracked with full lineage across your data science organization.
• Unity Catalog provides unified governance for features alongside data: search for existing features, understand lineage, and control access at column and row levels.
• When you log a model with MLflow, its feature dependencies are recorded automatically, creating a complete lineage from data through features to deployed models.
• A three-layer governance approach combines DataOps (Unity Catalog for features and training data), ModelOps (MLflow for experiments and models), and DevOps (version control and CI/CD) for enterprise ML governance.

How do I govern and discover features across data science teams?
Features are the lifeblood of machine learning, but teams often recreate the same features independently, leading to inconsistent logic, redundant computation, and data quality issues. Governing and discovering features across teams requires a central repository where features are documented, discoverable, lineage-tracked, and access-controlled. Databricks integrates Unity Catalog with MLflow to make features first-class citizens in your data platform.

What is feature governance, and why does it matter?

Feature governance is the practice of managing feature definitions, ownership, lineage, and access across an organization. Without it, data scientists and analysts recreate features in isolation, leading to subtle logic divergence between training and production pipelines. A feature created for one model might silently differ from a feature of the same name used elsewhere, causing prediction inconsistencies and debugging nightmares. Governance ensures a single source of truth for feature definitions and enables teams to discover and reuse features others have built.

Key capabilities to evaluate for feature governance

Capability	Why it matters
Centralized feature discovery	Data scientists can search and find existing features rather than rebuilding them. Saves time and prevents divergence.
Feature lineage and dependencies	Track which data sources, transformations, and models use each feature. Understand impact of changes before they break downstream consumers.
Unified access control	Enforce permissions at the feature level. Some teams see certain sensitive features; others don't.
Feature versioning and metadata	Know who created a feature, when it was last updated, its purpose, and which models depend on it.
Integrated model serving	At inference time, features are automatically looked up from the governance layer, not manually coded or cached.
Cross-team collaboration	One team publishes features for use by other teams, creating a shared data science infrastructure.

How Databricks approaches feature governance

Use Unity Catalog for Feature Views. Feature Views are first-class Unity Catalog objects. Define a feature in SQL or Python, register it in Unity Catalog, and it becomes discoverable across your organization with full lineage and access control. Other data scientists search Unity Catalog and find your feature. They can consume it in their training pipelines or use it for online inference without reimplementing the logic.
Log feature dependencies with MLflow. When you train a model, MLflow captures which features your model uses. At inference time, Model Serving automatically looks up those features from your governance layer, eliminating manual plumbing. This creates an end-to-end lineage: data source to feature to model to prediction.
Apply three-layer governance. Organize your ML governance into three layers: DataOps (Unity Catalog governs feature tables and training data), ModelOps (MLflow manages experiments, model registry, and serving), and DevOps (version control and CI/CD). This separation of concerns lets each team own their domain while staying integrated.
Enable feature discovery and reuse. Data scientists use workspace search to find existing features, see who created them, discover which pipelines or models use them, and check when they were last updated. If a feature fits their use case, they reuse it. If not, they create a new one and publish it for others. This culture of sharing prevents feature duplication and reduces time to market for new models.
Maintain lineage and audit trails. Unity Catalog tracks lineage automatically: which models depend on which features, which data sources feed which features, and who accessed what. This transparency is critical for debugging model issues, enforcing compliance, and understanding the impact of data changes.

Use cases: where feature governance delivers the most value

Feature governance is most valuable in organizations running multiple machine learning models on overlapping data. A financial services company might have credit risk models, fraud detection models, and pricing models, all of which need similar features like customer age, account balance, and transaction history. Without governance, each model team builds these features independently, leading to subtle differences in aggregation windows, null handling, or filter logic. With governance, the risk team publishes customer features in Unity Catalog, and fraud and pricing teams reuse them. A retail company might have demand forecasting, inventory optimization, and recommendation models that all need product popularity and customer preference features. Centralized feature governance enables cross-team reuse and ensures consistent feature definitions across models.

FAQs

How do I discover features published by other teams?

Search Unity Catalog using the workspace search interface or the SQL API. Filter by catalog and schema to find features. Each feature listing shows who created it, when it was last updated, which models or pipelines use it, and its lineage. If you find a feature that fits your needs, reference it in your model training pipeline.

Can different teams use different feature definitions for the same logical concept?

Yes, but it's not recommended. If teams define the same feature differently (different aggregation window, filter logic, or null handling), models trained on those features behave inconsistently. Unity Catalog lineage helps surface these divergences. Best practice is to establish a shared feature and enforce its use across teams using access control.

How do I ensure feature definitions stay consistent between training and serving?

MLflow captures feature dependencies when you log your model. Model Serving uses those dependencies to automatically look up features at inference time, eliminating the manual lookup code that often diverges from training. The feature logic is defined once in Unity Catalog and reused consistently.

What happens if I need to change a feature definition?

Create a new version of the feature in Unity Catalog and update models incrementally. Use lineage to identify which models depend on the old feature. Coordinate with those teams before retiring the old version. This prevents surprise breakage and gives teams time to retrain with the new logic.