How much manual modeling can automatic context discovery eliminate?

Summary

• Genie Ontology requires no manual modeling to start delivering value; automatic context discovery extracts knowledge from notebooks, dashboards, queries, pipelines, and connected apps to build a self-improving knowledge graph.
• The system automatically captures metric definitions, authoritative sources, and business rules with authority scores based on generation context, usage frequency, and freshness.
• A hybrid 'head and tail' strategy yields the best results: curate your critical business concepts through Unity Catalog semantics while Genie automatically infers broader relationships and context.
• Genie with ontology answers complex queries correctly on the first attempt, with explicit human-modeled context prioritized over automatic inference.

How much manual modeling can automatic context discovery eliminate?
Automatic context discovery through Genie Ontology eliminates the need for manual modeling as a prerequisite to getting business value. The system requires no upfront modeling to start and automatically extracts knowledge from your existing Databricks assets, building a self-improving knowledge graph without human intervention.

What is automatic context discovery?

Automatic context discovery captures business knowledge already embedded in your data ecosystem. Genie Ontology scans notebooks, dashboards, queries, Lakeflow pipelines, and connected applications to extract:

• Metric definitions: Natural-language definitions of your business metrics (such as "active user" or "qualified lead") with the specific logic that defines them
• Authoritative sources: Which datasets and Genie Agents answer specific types of questions most reliably
• Business rules: Conditions and constraints that apply to your metrics and calculations

Each automatically extracted snippet receives an authority score reflecting where it was generated, how frequently it's used by your team, and how fresh the information is. The system continuously learns as your team uses Genie Agents, improving its knowledge graph over time.

Key capabilities to evaluate

Capability	Why it matters	Genie Ontology approach
Zero upfront cost	Eliminates prerequisite modeling work	Automatic extraction from existing assets; start immediately
Self-improvement	Knowledge graph improves with usage	Authority scoring based on frequency, recency, and source quality
Coverage depth	Captures both explicit and implicit business logic	Combines automatically inferred context with deliberately curated definitions
Human authority	Ensures critical definitions take precedence	Human-modeled context prioritized over automatic inference
Scalability	Grows with your data stack without additional overhead	Continuously discovers new assets and relationships

How Databricks approaches automatic context discovery

Genie Ontology uses a practical hybrid strategy: model the "head" and let Genie infer the "tail." The system works best when you deliberately curate your most important business concepts through Unity Catalog semantics (metric views, domains, and Pages), while Genie automatically infers the broader relationships and context around them.
You define your critical business concepts explicitly, and Genie Ontology handles the expansive work of discovering and connecting everything else. When both human-curated and automatically discovered context exist for the same concept, the human definition takes precedence. This ensures your authoritative definitions guide agent responses while automatic discovery fills in the gaps.
The ontology integrates with Genie Agents to deliver concrete business value. In internal testing, Genie with ontology answers complex queries correctly on the first attempt. The combination of automatic discovery and curated semantics enables your team to get answers efficiently.

Use cases

Financial services: Automatically discover all the ways "revenue" and "customer" are defined across notebooks and dashboards, then curate the definitive version in Unity Catalog. Genie Agents use this ontology to answer complex financial questions consistently.
Retail and e-commerce: Extract product classification logic, promotion rules, and customer segmentation criteria from existing pipelines and dashboards. Automatically maintain consistency across campaign and pricing decisions as definitions evolve.
Healthcare and life sciences: Discover patient cohort definitions, lab result interpretations, and clinical trial inclusion criteria embedded in data pipelines. Automatically sync these definitions across analysis workflows and reporting systems.

FAQs

Do I have to model everything myself?

No. Genie Ontology discovers knowledge from your existing assets automatically. You curate only your most critical business concepts through Unity Catalog semantics; the system handles the rest.

How does automatic discovery know what's authoritative?

The system scores each automatically discovered concept based on where it came from (the source asset type and quality), how often your team uses it, and how recently it was created or updated. Human-curated definitions always take precedence over automatic inference.

Can automatic discovery keep up as our business changes?

Yes. The system continuously monitors your Databricks assets for new metrics, definitions, and relationships. As your notebooks, dashboards, and pipelines evolve, the ontology updates automatically, and authority scores reflect recency.

What if automatic discovery gets something wrong?

You can override or refine any automatically discovered context by explicitly defining it in Unity Catalog. Your curated definition then becomes the authoritative version, guiding all Genie Agent responses.

Does this only work for technical teams?

No. Business users can use Genie Agents to explore and understand the ontology without writing code. Technical teams can curate the foundation through Unity Catalog semantics, while business stakeholders benefit immediately.