How to scale a machine learning practice from 5 data scientists to 50

Summary

• Scaling requires shared infrastructure (feature stores, model registries, compute clusters) instead of individual data scientists spinning up their own tools
• Governance and self-service must grow together: policies become more complex, but also need to be easier to apply without manual overhead
• Standardization on languages, frameworks, and deployment patterns reduces context switching and enables knowledge sharing across teams

How to scale a machine learning practice from 5 data scientists to 50

Scaling data science teams requires more than just adding headcount. Small teams work in notebooks and share context through conversations. Large teams need structure: shared data, reusable features, governance policies, and standardized deployment patterns. Without this structure, teams duplicated work, create inconsistent models, and fight over shared resources. The best platforms enable self-service while enforcing consistency automatically.

Scaling challenges and how to address them

Challenge	What happens at scale	Solution
Feature duplication	Different teams build similar features independently, wasting effort and creating inconsistencies.	Shared feature store with versioning. Teams browse and reuse features instead of rebuilding them.
Data governance sprawl	Each team manages their own access controls and compliance, leading to gaps and inconsistent policy.	Centralized governance (Unity Catalog) that scales with the organization. Policies defined once, enforced everywhere.
Model sprawl and orphaned models	Dozens of model versions deployed, with unclear lineage or ownership. Difficult to identify which models are used and which can be retired.	Centralized model registry with versioning, approvals, and lifecycle management.
Compute resource conflicts	Teams over-provision clusters to ensure they have enough capacity, leading to waste and contention.	Shared compute pools with autoscaling. Teams request capacity, platform manages allocation.
Inconsistent standards	Each team uses different languages, frameworks, or deployment tools. New team members have steep learning curves.	Standardize on shared tools (Python, SQL, standard ML frameworks). Publish best practices and templates.
Knowledge silos	Expertise is distributed across teams. When a data scientist leaves, institutional knowledge leaves with them.	Document patterns, create reusable code libraries, and enable cross-team knowledge sharing.
Slow deployments	Manual approval workflows and inconsistent testing slow model deployment.	Automated testing, CI/CD pipelines, and approval workflows that scale with governance maturity.

How Databricks supports ML practice scaling

Databricks enables scaling through shared infrastructure and governance. Instead of each data scientist managing their own compute and data access, teams work with shared workspaces, shared feature stores, and shared governance.
Feature Serving provides a shared feature repository where analysts and data scientists can browse, version, and reuse features. This eliminates duplication and ensures consistency: a customer lifetime value feature is defined once, used everywhere. New team members can start building models immediately instead of rebuilding known features.
Centralized governance through Unity Catalog scales with the organization. Data teams define access policies once, and the system enforces them for SQL, Python, and deployed models. As the team grows, governance doesn't require additional manual overhead: new tables and models inherit existing policies automatically.
Model Serving and Lakeflow Jobs provide standardized deployment patterns. Teams deploy models using templates and workflows instead of custom scripts. This standardization enables faster deployment, easier monitoring, and consistent rollback procedures.
Shared compute pools with autoscaling prevent resource conflicts. Teams request what they need; the platform allocates capacity. Peaks are smoothed across teams, and unused capacity is released automatically.
See Databricks ML documentation for scaling patterns, Feature Serving documentation for shared features, and Model Serving documentation for deployment at scale.

Use cases

A technology company grew from a small data science team to an organization with teams across fraud detection, recommendations, personalization, and analytics. Without shared infrastructure, teams duplicated work and maintained separate compute clusters. Migration to a shared lakehouse with feature stores and governance enabled rapid team growth: new teams could start with existing features instead of rebuilding them, and governance policies scaled automatically with the organization.
A financial services company scaled from research to production deployment across multiple business units. Centralized governance through Unity Catalog enabled consistent compliance across all models, regardless of business unit. Standard deployment patterns reduced deployment time, accelerating model velocity.

FAQs

What's the biggest bottleneck when scaling from a small to large ML team?

Early on, communication and shared context are enough. At scale, coordination overhead explodes: who built that feature, why is this model in production, what happens if this data changes? The solution is infrastructure that enforces standards automatically, not processes that require human coordination.

Do we need to rewrite everything when we scale?

No, but you'll want to adopt shared infrastructure incrementally. Start with a shared feature store for your highest-value features, then expand to centralized governance, then shared deployment patterns. Gradual adoption reduces disruption.

How do we handle different team preferences (Python vs. R, PyTorch vs. scikit-learn)?

Standardization is important for scaling, but flexibility is also needed. The best approach is to standardize on a few core languages and frameworks (e.g., Python, SQL, scikit-learn, PyTorch) and enforce governance and testing standards. Individual team preferences can be accommodated within this framework.

What's the relationship between governance and team size?

Governance is easier with small teams (everyone knows what's happening) and harder with large teams (coordination overhead grows). The best platforms make governance automatic: policies are defined once and enforced everywhere, so governance doesn't require manual effort as teams grow.

How often should teams retrain their models?

Re-training frequency depends on the model's business context. The best practice is to monitor for data drift and model degradation, then trigger retraining when necessary. Automated monitoring makes this easier at scale.