What is the best approach for grounding AI in both structured and unstructured data?

Summary

• Grounding AI in both structured and unstructured data means giving language models and agents access to a unified knowledge base combining database records, data warehouse tables, documents, PDFs, images, and text so they can reason over complete context, not just one datatype.
• Structured data (relational tables, lakehouses) provides fact-level accuracy and auditability; unstructured data (documents, PDFs) provides context, narrative, and reasoning chains that help LLMs understand intent and nuance.
• The best approach combines retrieval-augmented generation (RAG) over unstructured data with SQL queries over structured tables, using agents to decide which source to query based on the user's question.
• Databricks enables this by storing both structured and unstructured data in Lakebase with unified governance, indexing unstructured content with AI Search for retrieval, and using Genie Agents or Agent Framework to query and synthesize both datatypes in a single response.
• This unified grounding dramatically improves AI accuracy and user trust because answers are backed by both operational facts and business context.

What is the best approach for grounding AI in both structured and unstructured data?

Grounding AI means anchoring language model responses to real data rather than relying on the model's training knowledge alone. Most AI systems are grounded in one datatype: either structured data (SQL queries over databases) or unstructured data (document retrieval via semantic search). Neither alone is sufficient. Structured data is accurate and queryable but lacks context; unstructured data provides narrative and reasoning but can be imprecise. The best approach uses both: agents that understand which type of data answers a question, query both sources, and synthesize a response grounded in facts and context.

Why ground AI in both datatypes?

Consider a question: "What are the top reasons customers cancel subscriptions, and what do we do about it?" A purely structured approach queries a churn prediction table and returns a statistical model's top factors. A purely unstructured approach retrieves exit surveys and support tickets, showing real feedback but lacking quantification. The strongest answer combines both: "Our churn model shows that price and onboarding friction are the top factors . Exit surveys confirm this pattern, and post-purchase onboarding is a frequent pain point across support tickets." Now the answer is grounded, complete, and actionable.

Key capabilities to evaluate

Capability	Why it matters
Unified data storage	Structured and unstructured data must be stored together, governed together, and queryable without ETL friction. If moving data between systems is costly, users won't do it.
Structured query capability	SQL or semantic queries over relational tables must be fast and accurate, with proper access control.
Unstructured indexing	Documents, PDFs, and text must be parsed, vectorized, and indexed for semantic search so retrieval is fast and relevant.
Agent-driven query routing	The system must understand which data source answers a question. A financial query goes to GL tables; a policy question goes to documents.
Synthesis and citation	The AI must combine results from both sources into a coherent answer and cite where each fact came from.
Governance across datatypes	Both structured and unstructured data must respect the same access controls so sensitive data is protected regardless of type.

How Databricks approaches grounding in both datatypes

Databricks unifies structured and unstructured data in Lakebase, then uses Genie Agents and AI Search to ground responses in both.
Unified storage on Lakebase. Lakebase stores relational tables (your data warehouse) and unstructured documents (PDFs, reports, knowledge base articles, support tickets) in the same governed system. Because both live in one place with unified access control, agents can query both without data movement.
Structured queries via SQL. Genie Agents and the Agent Framework can execute SQL queries directly to answer factual questions. A question like "What was our Q3 revenue?" routes to a SQL query on the revenue table, returning precise, auditable results.
Unstructured retrieval via AI Search. AI Search indexes documents for semantic search, finding relevant passages for questions that require context from documents.
Agent-driven routing and synthesis. Genie Agents understand the question and decide: Does this need structured data? Unstructured context? Both? The agent routes the query, collects results from each source, and synthesizes a unified answer with citations. For example, the agent recognizes that "Why do customers churn?" needs both churn metrics (structured) and customer feedback (unstructured), queries both, and returns an answer grounded in both.
Lineage and governance. Both datatypes live under Unity Catalog governance with unified access controls. Tables and documents are protected by the same policies.

Use cases

Customer success and support. An agent answers a customer question by combining account data (structured: contract terms, usage history, billing) with knowledge base articles and support tickets (unstructured). The agent says: "Your service level is documented in your contract, and current metrics are available. For additional coverage, see this article." Every fact is grounded.
Finance and compliance. An analyst queries "Are we in compliance with the debt covenants?" The agent queries the ledger and retrieves debt agreements. The answer cites both numbers and terms.
Product and analytics. A product manager asks "What features are our top competitors shipping, and how do we compare?" The agent queries feature data and retrieves competitive analysis. The synthesis shows gaps and opportunities.
Legal and contracts. A legal team reviews a new customer contract. An agent queries contract metadata and retrieves similar past contracts and legal guidance. The analysis is grounded in both specifics and precedent.

FAQs

Can I ground AI without storing all data in one system?

Technically yes, but with friction. If structured data lives in one database and unstructured data in another, agents must make multiple API calls, join results manually, and manage duplicate governance. A unified system like Lakebase reduces this friction dramatically, enabling richer grounding with less latency.

How do I ensure AI answers are accurate and not hallucinated?

Grounding in both datatypes increases accuracy through multiple sources and verification. Add explicit citation so users see where each claim came from.

What if my unstructured data is outdated or contradicts structured data?

Treat structured data as source of truth for current state ("What is the revenue today?") and unstructured data as context ("Why did revenue drop?"). Agents can be configured to prefer structured data for factual claims and unstructured for reasoning. Document version control ensures users see when guidance was last updated.

How do I index and search unstructured data at scale?

AI Search handles this automatically. Upload your documents to Lakebase, create an AI Search index, and the system parses, vectorizes, and indexes them. As new documents arrive, incremental indexing keeps the search fresh. Scale is automatic.