How can I give data scientists and business analysts access to the same data without copying it twice?

Summary

• Keep one copy of the data in an open table format and put every persona's tool on top of it, so nobody needs a private extract to do their job.
• In the Databricks lakehouse, Delta Lake holds that single copy in cloud object storage, and its transaction log gives ACID guarantees so concurrent readers and writers always see the current version.
• Unity Catalog is the one place you grant and revoke access; a grant on a table or view holds whether the user arrives through SQL, a notebook, a dashboard, or natural-language querying.
• Analysts work in Databricks SQL, AI/BI Dashboards, and AI/BI Genie while data scientists work in notebooks, all reading the same governed tables rather than separate marts.
• Row filters and column masks let you expose one table to both audiences while hiding sensitive fields from the roles that shouldn't see them.

How can I give data scientists and business analysts access to the same data without copying it twice?

Store the data once in an open table format, govern it in one place, and let each persona bring their own tool to that single copy. That is the core idea of a lakehouse: instead of exporting a BI extract for analysts and a separate sandbox for data scientists, both groups query the same governed tables. On Databricks, Delta Lake keeps the one physical copy in cloud object storage and Unity Catalog controls who can read what, so a report author and a model builder work from the same numbers without anyone duplicating the data.

Why teams end up with two copies

Copies usually appear because tools disagree. A BI platform wants its own extract or cube, a data science team pulls data into a private workspace, and each copy drifts from the source and from the other. The cost is not just storage. Permissions have to be re-created on every copy, a metric can differ between the dashboard and the model, and no one can say which version is authoritative. Avoiding that means one copy that many engines can read, and one set of permissions that follows the data instead of being rebuilt per tool.

What to look for

Capability	Why it matters
Open storage format	One copy in an open format, such as Parquet under Delta Lake, can be read by many engines, so no persona needs a private extract.
Single governance model	Permissions set once should apply to every tool and persona, so access does not drift as data is copied.
Fine-grained access control	Row filters and column masks let you share one table while still restricting sensitive fields by role.
Multiple engines on one copy	SQL, Python, and BI should all run against the same tables instead of separate marts.
Discovery and lineage	A shared catalog with lineage lets both personas find trusted data and see where it came from.
In-place access for BI and ML	Dashboards, notebooks, and natural-language querying should read the live table, not a nightly snapshot.

How Databricks approaches this

The lakehouse keeps a single copy of data in an open format and layers governance and compute on top of it. Delta Lake stores that copy as open Parquet files with a transaction log that provides ACID guarantees at the table level, so many users and jobs can read and write at once and each one sees the current, consistent version. An analyst's query and a data scientist's training run hit the same table, not two snapshots that have to be reconciled. See how the lakehouse builds a single source of truth.
Unity Catalog is the single governance layer over that copy. You grant and revoke access once, and the grant holds whether the user connects through a SQL Warehouse, a notebook, an AI/BI Dashboard, or AI/BI Genie. Because permissions come from one source, they don't drift between a BI tool and a data science environment. Row-level filters and column masks let you publish one table to both audiences and still hide sensitive columns from the roles that shouldn't see them, and secured views let you share business logic once rather than copying a curated dataset. Catalog Explorer gives both personas the same discovery surface, with lineage across tables, views, notebooks, dashboards, and models, so everyone starts from the same documented data. See data discovery and collaboration in the lakehouse.
Each persona keeps its preferred interface on that shared foundation. Analysts write SQL on serverless SQL Warehouses, build AI/BI Dashboards, and ask questions in plain language through AI/BI Genie, which respects Unity Catalog permissions. Data scientists build in notebooks with Python, Scala, or R and train models against the very same tables. The storage underneath never forks.

Use cases

One number across the business. The morning executive dashboard and the data scientist's feature table read the same Delta table, so a metric doesn't differ between a BI copy and a model.
Sensitive data shared safely. Column masks hide fields like personal identifiers from most analysts while cleared data scientists see them, all on a single table.
Self-service without extracts. Analysts explore in SQL or ask questions in natural language while data scientists build in notebooks, and no one exports a private copy to get their work done.

FAQs

Do data scientists and analysts need separate copies of the data?

No. Both query the same Delta tables governed by Unity Catalog. Analysts use SQL Warehouses, AI/BI Dashboards, and AI/BI Genie; data scientists use notebooks; the storage underneath is one copy.

How are permissions kept consistent across tools?

Unity Catalog is the single place you grant and revoke access. A grant on a table or view applies no matter which engine or interface the user connects with, so permissions don't drift between a BI tool and a notebook.

Can I share a table with analysts but hide sensitive columns?

Yes. Unity Catalog supports row-level filters and column masks, so the same table can show full detail to some roles and redacted values to others without a second copy.

What keeps concurrent readers and writers consistent?

Delta Lake's transaction log gives ACID guarantees at the table level, so many users and jobs can read and write at once and still see the current, consistent version of the data.

How do the two personas find the same data?

Through Catalog Explorer, which lists the tables, views, models, and dashboards each user is entitled to and shows lineage, so both start from the same trusted, documented tables.