Skip to main content

Spark versus Trino or Presto for interactive SQL?

Summary

  • Trino and Presto are distributed SQL query engines built for analytics: a coordinator parses and plans each query, and workers run its stages and exchange intermediate data. Both projects' docs say they are not general-purpose relational databases and were not designed for OLTP.
  • Apache Spark is a general-purpose engine. Spark SQL can act as a distributed query engine through its Thrift JDBC/ODBC server or command-line interface, and the same cluster also runs batch ETL, Structured Streaming and machine learning.
  • The deepest architectural difference is how each engine treats intermediate data and failures. Trino's docs state that a query fails by default if a node fails, until you enable fault-tolerant execution; Spark writes shuffle files to disk, retries failed tasks and recomputes lost partitions from lineage.
  • Trino or Presto fits teams whose main job is ad hoc SQL across several data sources. Spark SQL fits teams whose interactive queries share tables, code and compute with pipelines and machine learning.
  • The engines can share data: Trino, Presto and Spark all read Delta Lake and Apache Iceberg tables.

Spark versus Trino or Presto for interactive SQL?

Choose by architecture fit, not reputation. Trino and Presto are distributed SQL engines designed to run analytics queries over one or more data sources, with a coordinator that plans each query and workers that pass data between stages. Apache Spark is a general-purpose engine whose SQL layer serves the same queries while sharing a cluster with ETL, streaming and machine learning. If interactive SQL across many sources is the main job, Trino or Presto matches that design. If SQL is one step in a larger pipeline on shared tables, Spark SQL saves you from operating a second engine.

How Trino and Presto run a query

Trino describes itself as "a distributed SQL query engine designed to query large data sets distributed over one or more heterogeneous data sources." The coordinator parses statements, plans queries and manages workers. The plan becomes a hierarchy of stages: each stage runs as tasks on workers, tasks operate on splits, and exchanges move data between stages.
Federation is part of the model. Each catalog is a properties file that binds a connector to a source, and Trino's docs note that a single SQL query can read from two catalogs, for example two data lakes behind separate Hive catalogs.
Trino and Presto share their origins. Trino's 2020 announcement describes it as the new name for PrestoSQL, and Presto continues under the Presto Foundation, part of the Linux Foundation. Presto's classic mode shuffles data between stages over RPC, which keeps producers and consumers running together until the query finishes; optional exchange materialization supports memory-intensive queries. Presto C++ (development name Prestissimo) replaces Java workers with C++ workers based on the Velox library, and its docs mark it as in active development.

How Spark runs interactive SQL

Spark SQL "can also act as a distributed query engine using its JDBC/ODBC or command-line interface." Its Thrift JDBC/ODBC server, started with ./sbin/start-thriftserver.sh, corresponds to HiveServer2, so applications can run SQL queries through it directly. Spark Connect, introduced in Spark 3.4, adds a client-server protocol: the client sends unresolved logical plans over gRPC to a remote Spark server.
Underneath, a driver program coordinates executors on a Standalone, Hadoop YARN or Kubernetes cluster. Shuffles write intermediate files to disk, failed tasks are retried, and RDDs recover from node failures by recomputing lost partitions from lineage. Adaptive Query Execution, on by default since Spark 3.2.0, re-optimizes plans mid-query from runtime statistics, for example by converting a sort-merge join to a broadcast join.

Trino vs Presto vs Spark: how they compare

Aspect Trino Presto Apache Spark (Spark SQL)
Design center (per project docs) Data warehousing and analytics (OLAP) over heterogeneous sources Data warehousing and analytics (OLAP) over heterogeneous sources Data engineering, data science and machine learning on single machines or clusters
Process model One coordinator plus workers Java coordinator; Java or C++ (Velox) workers Driver plus executors on a cluster manager
Data between stages Exchanges; spooled to an exchange manager when fault-tolerant execution is on RPC shuffle in classic mode; optional exchange materialization Shuffle files written to disk
Failures and large queries A node failure fails the query by default; retry-policy set to QUERY or TASK adds retries Exchange materialization, or Presto on Spark, which uses Spark as the execution engine Failed tasks retried; lost partitions recomputed from lineage
Federation One catalog per source; one query can span catalogs One catalog per source, set by connector.name JDBC data source, files and table formats
Beyond SQL SQL query engine SQL query engine DataFrame APIs, Structured Streaming, MLlib

How Databricks approaches interactive SQL

On Databricks, interactive SQL runs on SQL warehouses, and Databricks recommends serverless SQL warehouses where they're available. Photon, the Databricks-native vectorized query engine, is the default engine for all SQL warehouses and is compatible with Apache Spark APIs, so existing Spark SQL and DataFrame code runs without changes. Serverless warehouses use Intelligent Workload Management, which predicts each query's resource needs, starts it when capacity exists and queues it otherwise.
For sources outside the lakehouse, Lakehouse Federation gives governed, read-only access to external databases such as PostgreSQL and MySQL through Unity Catalog foreign catalogs, with query pushdown. Teams that keep Trino don't have to copy data: Unity Catalog's Iceberg REST catalog serves Iceberg clients, and Databricks' docs list Trino among them.

FAQs

Are Trino and Presto the same engine?

They share an origin but are separate projects: Trino is the new name for PrestoSQL, and Presto is governed by the Presto Foundation.

Does Trino retry failed queries?

Not by default. Fault-tolerant execution stays off until you set retry-policy to QUERY or TASK, and the TASK policy requires a configured exchange manager.

Can Trino and Spark query the same tables?

Yes. Both read Delta Lake and Apache Iceberg tables, and Trino's docs include a configuration for reading Databricks Unity Catalog through its Iceberg REST catalog.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.