pinot

apacheApache-2.0活跃维护

面向用户和 AI Agent 的实时分析数据库，支持低延迟 SQL 查询和向量检索，可从 Kafka 实时摄入数据做 OLAP 分析。

Apache Pinot: a real-time analytics database for low-latency SQL and vector search, serving user-facing and agent-facing applications

release-1.6.0发布于 2026-10-23 04:05 UTC6,171star1,532fork同步于 2026-11-10 05:34 UTC

实时分析数据库向量检索 OLAP 低延迟SQL 分布式数据库

对比 GitHub 官网 ai-agents analytics database distributed-database java

怎么上手

由 README 提炼，执行前请确认

原文

docker run --rm --name pinot-quickstart -p 127.0.0.1:9000:9000 apachepinot/pinot:1.6.0 QuickStart -type batch

硬件要求按 README 说明

Docker 快速体验需给 Docker 分配至少 6 GB 内存；README 未说明 GPU 或显存要求。

支持 Apple Silicon有 Docker 镜像

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼

没人工点评？写下你的经验

实时分布式 OLAP 数据库，用低延迟 SQL 和向量检索支撑面向用户与 Agent 的分析应用

面向用户和 AI Agent 的实时分析数据库，支持低延迟 SQL 查询和向量检索，可从 Kafka 实时摄入数据做 OLAP 分析。

核心能力

从 Kafka 流式与 HDFS/S3 等批量源摄入数据
提供标准 SQL 查询接口与内置查询编辑器、REST API
支持向量相似度检索用于语义搜索与 RAG
可插拔索引含倒排、StarTree、Bloom filter、范围索引
列式存储配 Run Length、Fixed Bit Length 等压缩

容易踩的坑

首次运行需下载镜像并加载示例表，要等日志出现 Quick start setup complete 才能查询；从源码构建需 JDK 25+，且要加 -Pbin-dist -Pbuild-shaded-jar 才能产出二进制发行包和 shaded jar；macOS ARM 构建 gRPC Java 插件需先 brew install protobuf protoc-gen-grpc-java。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

Apache Pinot 1.5.1

Apache Pinot 1.5.1 is a security patch release based on 1.5.0. It contains only dependency updates and dependency-exclusion changes to resolve reported CVEs — no functional, API, configuration, or wire-format changes. A vulnerability scan of the 1.5.1 binary distribution reports 0 critical and 0 high findings.

This addresses the CVEs reported in #18593, plus additional Jetty, BouncyCastle, and commons-configuration2 findings surfaced during release verification.

Security fixes

Dependency	1.5.0 → 1.5.1	CVEs addressed
Netty (`netty-bom`)	`4.1.122.Final` → `4.1.134.Final`	CVE-2025-55163, CVE-2025-59419, CVE-2026-33870, CVE-2026-33871, CVE-2026-42579, CVE-2026-42583, CVE-2026-42584, CVE-2026-42587
Apache Log4j Core	`2.25.3` → `2.26.0`	CVE-2026-34478, CVE-2026-34479, CVE-2026-34480, CVE-2026-34481
async-http-client	`3.0.7` → `3.0.10`	CVE-2026-45300
Apache HttpClient 5	`5.6` → `5.6.1`	CVE-2026-40542
Apache Pulsar (bundles BouncyCastle)	`4.0.9` → `4.0.10` (bcprov/bcpkix/bcutil → `1.84`)	CVE-2026-5598, CVE-2026-0636, CVE-2026-5588
commons-configuration2	`2.13.0` → `2.15.0`	CVE-2026-45205

CVE-2026-2332 (Jetty request smuggling)

The only Jetty in Pinot's distribution was Hadoop's embedded HttpServer2, which Pinot never starts. As of 1.5.1 it is excluded from the Hadoop dependencies and stripped from the shaded jars (#18659), so Jetty no longer ships in the distribution at all. (Jetty 9.4.x itself is end-of-life with no available patch; removing the unused artifact is the resolution.)

Upgrade notes

Drop-in security update for 1.5.0 — no behavioral changes.

Full Changelog: https://github.com/apache/pinot/compare/release-1.5.0...release-1.5.1

release-1.5.0Apache Pinot Release 1.5.02026-04-09 · 2026-04-09 16:47 UTC
查看更新说明
What Changed

This release delivers significant improvements across the Multi-stage Query Engine (new UNNEST support, enriched joins, aggregation rewrites), Upsert (offline table support, commit-time compaction, xxhash compression), a new Federation / Multi-Cluster Routing framework, Kafka 4.x support, real-time auto-reset, Time Series Engine enhancements, new indexing capabilities (N-gram, IFST, combined Lucene), Query Resource Isolation, and numerous performance optimizations, security hardening, and bug fixes. It includes over 1100 commits from the community.

Multi-stage Query Engine Enhancements

UNNEST / CROSS JOIN UNNEST Support #17168

The multi-stage engine now natively supports array unnesting via CROSS JOIN UNNEST(...), with optional WITH ORDINALITY. This enables flattening array columns in queries without workarounds, a frequently requested SQL feature.

Enriched Join Operator #16123

The MSE join operator has been enriched with improved execution strategies, enhancing join performance and flexibility for complex multi-table queries.

MSE Lite Mode Improvements
- Disable joins in Lite Mode: Added a configuration to explicitly disallow joins in MSE Lite Mode for workloads that should remain scatter-gather only. #17030
- Fan-out adjusted limit: Lite Mode now supports adjusted limits based on fan-out to reduce data transferred. #16822
- Push collation from Exchange to Sort: Optimizes Lite Mode queries by pushing collation requirements into sort operators. #16551
Physical Optimizer for Logical Tables #17447

Extended the Physical Optimizer to support logical tables, enabling exchange simplification and optimized execution plans for queries against logical table abstractions.

Other MSE Improvements
- Row() value expression support in comparisons. #17317
- Add maxRowsInJoin, maxRowsInWindow, numGroups to query response metadata. #17784
- Broker config to support configuring planner rules disabled by default. #17258
- Query option to exclude virtual columns from schema. #17047
- Fix non-deterministic hash-distributed exchange routing. #17323
- Sort servers for deterministic workerId-to-server mapping across stages. #17342
- Always use singleton worker for final sort stage to ensure hotspot server rotation. #17347
- Reset gRPC connection backoff when server is re-enabled. #17466
- Improved error handling and error messages for leaf stage errors. #17306, #17207, #17161
- New MSE metrics for better observability. #17419
- MSE latency metric for multi-stage queries. #16398
- Support tracing of planner rule productions. #16581
- Added support for UNION (distinct) set operations in addition to the existing UNION ALL, enabling more performant deduplication of results from combined queries. #16570
Federation / Multi-Cluster Routing (New)

Pinot 1.5.0 introduces a new federation framework enabling a single broker to route queries across multiple independent Pinot clusters. This is a foundational capability for organizations ope
release-1.4.0Apache Pinot Release 1.4.02025-09-15 · 2025-09-15 22:14 UTC
查看更新说明
What Changed

This release delivers significant improvements to the Multistage Engine, Pauseless Consumption, Time Series Engine, Logical Table support, Upsert and Deduplication Enhancement, Minion Jobs (including smallSegmentMerger), and Rebalancing capabilities. It also includes numerous smaller features and general bug fixes.

Multistage Engine Lite Mode (Beta) | Runbook

There's an all new query mode added for running Multistage Engine queries against Pinot, heavily inspired from Uber's Presto over Pinot query architecture.

The MSE Lite Mode runs queries following a Scatter-Gather paradigm, same as Pinot's V1 Query Engine. Moreover, it adds a configurable limit on the number of records returned by each instance of the leaf stage. This limit is set to 100k records by default.

With MSE Lite Mode, you can enable MSE access for all users without worrying about them breaking their production workloads. Moreover, MSE Lite Mode can scale to 1000s of QPS with minimal hardware, meaning users can now run complicated multi-stage queries leveraging features such as sub-queries, window functions, etc. at high-qps and low-latencies, with minimal reliability risks.

You can enable this by setting all of the following query options:
```
SET useMultistageEngine=true; 
SET usePhysicalOptimizer=true; 
SET useLiteMode=true;
```
Multistage Engine Physical Optimizer (Beta) | Runbook

We have added a new query optimizer for the Multistage Engine that can automatically eliminate or simplify redundant Exchanges. We aim to make this query optimizer the default in future versions.

Uber adopted this optimizer for one of their workloads that needs Colocated Join support, and it proved to be 5-7x faster with 50% less CPU consumed. issue #15871

To enable this, set the query options:
```
SET useMultistageEngine=true;
SET usePhysicalOptimizer=true;
```
Features
- Capable of simplifying Exchanges for arbitrary complicated queries. No query-hints required.
- Supports group-by, joins, union-all, etc.
- Can solve constant queries within the Broker itself.
- Can simplify Exchange even if the number of partitions of the two join inputs are different. e.g. if the table on the left is partitioned into 8 partitions over 4 servers, and the table on the right is partitioned into 16 partitions over 4 servers, the Physical Optimizer will automatically switch to "Identity Exchanges".
- Can simplify Exchange even if the servers selected for the two sides of a join are different.
Unsupported but Coming Soon
- Support for customizing query parallelism via SET stageParallelism=x.
- Support for dynamic filters for Semi-Join queries.
- Support for SubPlan based execution for eligible queries.
- Support for "Lookup Join" optimization.
- Support for Spools.
Here are some of the key PRs that have been merged as part of this feature
- [multistage] Replace LogicalTableScan with PinotLogicalTableScan #15225
- [multistage] Adding Basic Constructs for Physical Optimization #15371
- [multistage] Add Physical Plan Nodes / Trait Assignment / Logical Agg Rule #15439
- [multistage] Add Leaf Stage Worker Assignment / Boundary / Agg Rules #15481
- [multistage] Add Pushdown and Worker Rules #15658
- [multistage] Support Physical Optimizer E2E #15698
- [multistage] Multistage Engine Lite Mode (prototype) [#15743](https://github.co

用户点评

登录用户的真实使用记录，可按版本和硬件筛选

共 0 条

还没有人点评。第一个说出真实体验的人最有价值。

README 原文

未翻译

在 GitHub 打开

What is Apache Pinot?

What is Apache Pinot?

Apache Pinot is a real-time distributed OLAP datastore, built to deliver scalable real-time analytics with low latency. It can ingest from batch data sources (such as Hadoop HDFS, Amazon S3, Azure ADLS, Google Cloud Storage) as well as stream data sources (such as Apache Kafka).

Pinot also supports vector similarity search for vector database use cases such as semantic search, recommendations, and retrieval-augmented generation (RAG). Its low-latency SQL queries over fresh data make it well suited to agent-facing analytics, including AI assistants and automated decision systems.

Pinot was built by engineers at LinkedIn and Uber and is designed to scale up and out with no upper bound. Performance always remains constant based on the size of your cluster and an expected query per second (QPS) threshold.

For getting started guides, deployment recipes, tutorials, and more, please visit our project documentation at https://docs.pinot.apache.org.

Docker quickstart

With Docker running and at least 6 GB of memory allocated to it, start Pinot with a bundled sample dataset:

docker run --rm --name pinot-quickstart \
  -p 127.0.0.1:9000:9000 \
  apachepinot/pinot:1.6.0 QuickStart -type batch

The first run downloads the image and loads sample tables, which can take a few minutes. Wait for ***** Quick start setup complete ***** in the logs, then open the query console and run this query to find the players with the most recorded home runs:

SELECT playerID, SUM(homeRuns) AS totalHomeRuns
FROM baseballStats
GROUP BY playerID
ORDER BY totalHomeRuns DESC
LIMIT 3;

Expected result for the bundled dataset:

playerID	totalHomeRuns
bondsba01	762
aaronha01	755
ruthba01	714

To stop the demo, run docker stop pinot-quickstart in another terminal. The container and its demo data are removed. For streaming examples and other ways to run Pinot, see the quickstart guide.

Features

Pinot was originally built at LinkedIn to power rich interactive real-time analytic applications such as Who Viewed Profile, Company Analytics, Talent Insights, and many more. UberEats Restaurant Manager is another example of a customer facing Analytics App. At LinkedIn, Pinot powers 50+ user-facing products, ingesting millions of events per second and serving 100k+ queries per second at millisecond latency.

Fast Queries: Filter and aggregate petabyte data sets with P90 latencies in the tens of milliseconds—fast enough to return live results interactively in the UI.
High Concurrency: With user-facing applications querying Pinot directly, it can serve hundreds of thousands of concurrent queries per second.
SQL Query Interface: The highly standard SQL query interface is accessible through a built-in query editor and a REST API.
Versatile Joins: Perform arbitrary fact/dimension and fact/fact joins on petabyte data sets.
Column-oriented: a column-oriented database with various compression schemes such as Run Length, Fixed Bit Length.
Pluggable indexing: pluggable indexing technologies including timestamp, inverted, StarTree, Bloom filter, range, text, JSON, and geospatial options.
Stream and batch ingest: Ingest from Apache Kafka, Apache Pulsar, and AWS Kinesis in real time. Batch ingest from Hadoop, Spark, AWS S3, and more. Combine batch and streaming sources into a single table for querying.
Upsert during real-time ingestion: update the data at-scale with consistency
Built-in Multitenancy: Manage and secure data in isolated logical namespaces for cloud-friendly resource management.
Built for Scale: Pinot is horizontally scalable and fault-tolerant, adaptable to workloads across the storage and throughput spectrum.
Cloud-native on Kubernetes: Helm chart provides a horizontally scalable and fault-tolerant clustered deployment that is easy to manage using Kubernetes.

When should I use Pinot?

Pinot is designed to execute real-time OLAP queries with low latency on massive amounts of data and events. In addition to real-time stream ingestion, Pinot also supports batch use cases with the same low latency guarantees. It is suited in contexts where fast analytics, such as aggregations, are needed on immutable data, possibly, with real-time data ingestion. Pinot works very well for querying time series data with lots of dimensions and metrics.

Example query:

SELECT sum(clicks), sum(impressions) FROM AdAnalyticsTable
  WHERE
       ((daysSinceEpoch >= 17849 AND daysSinceEpoch <= 17856)) AND
       accountId IN (123456789)
  GROUP BY
       daysSinceEpoch TOP 100

Contributing to Pinot

Want to contribute to Apache Pinot? 👋🍷

Want to join the ranks of open source committers to Apache Pinot? Then check out the Contribution Guide for how you can get involved in the code.

If you have a bug or an idea for a new feature, browse the open issues to see what we’re already working on before opening a new one.

We also tagged some beginner issues new contributors can tackle.

Apache Pinot YouTube Channel

Share Your Pinot Videos with the Community!

Have a Pinot use case, tutorial, or conference/meetup recording to share? We’d love to feature it on the Pinot OSS YouTube channel! Drop your video or a link to your session in the #pinot-youtube-channel on Pinot Slack, and we’ll showcase it for the community!

Building Pinot

# Clone a repo
$ git clone https://github.com/apache/pinot.git
$ cd pinot

# Pinot services require JDK 25+ to build and run
# Java/JDBC clients and SPI artifacts continue to target Java 11 bytecode

# Build Pinot
# -Pbin-dist is required to build the binary distribution
# -Pbuild-shaded-jar is required to build the shaded jar, which is necessary for some features like spark connectors
$ ./mvnw clean install -DskipTests -Pbin-dist -Pbuild-shaded-jar

# Run the Quick Demo
$ cd build/
$ bin/quick-start-batch.sh

For UI development setup refer this doc.

Normal Pinot builds are done using the ./mvnw clean install command.

However this command can take a long time to run.

For faster builds it is recommended to use ./mvnw verify -Ppinot-fastdev, which disables some plugins that are not actually needed for development.

More detailed instructions can be found at Quick Demo section in the documentation.

macOS Build Requirements

If you're building Pinot on macOS and encounter issues with the gRPC Java plugin during the build process, you may need to configure the protobuf Maven plugin to use a specific executable path. This is a known issue on macOS ARM (Apple Silicon) systems.

Automatic Profile Activation (macOS ARM64)

Pinot's Maven build now includes dedicated profiles for Apple Silicon (ARM64) Macs to ensure reliable protobuf compilation with Homebrew-installed binaries:

Primary profile: Activates automatically if /opt/homebrew/bin/protoc-gen-grpc-java exists (default for Apple Silicon Macs).
Fallback profile: Activates if /usr/local/bin/protoc-gen-grpc-java exists and the primary path does not (for Intel Macs or custom Homebrew setups).

You do not need to manually edit the pom.xml or set the plugin executable path. The correct profile will be selected based on your system and Homebrew installation.

To install the required tools:

brew install protobuf
brew install protoc-gen-grpc-java

If you installed Homebrew to a non-default location, ensure the protoc-gen-grpc-java binary is available in either /opt/homebrew/bin/ or /usr/local/bin/.

To verify which profile is active, run:

./mvnw help:active-profiles

If you encounter issues, check that the protoc-gen-grpc-java binary is present in one of the expected locations and is executable.

Lucene index recovery during loading

Normal segment preprocessing automatically rebuilds text and HNSW indexes whose Lucene format cannot be read by the running version. Single-column text, multi-column text, and directory/packed HNSW indexes are rebuilt using the segment's original forward-index values and existing table configuration; no recovery flag is needed. Compatible indexes are reused, and subsequent loads do not rebuild the recovered indexes again. Interrupted builds restart from the retained source values before the segment is loaded.

Install this recovery release on the intended rollback target before upgrading Lucene. After a downgrade, loading segments containing newer Lucene indexes rebuilds those indexes with the older writer. Retain the original forward indexes: dictionary/inverted-index reconstruction can lose multi-value ordering and duplicate values. When original values are unavailable, recovery fails before replacing the incompatible index; restore a compatible segment or refresh it from retained source data. Rebuilding adds CPU/I/O and segment-loading time. Existing preprocessing opt-outs remain in effect. Corruption and ordinary I/O errors fail loading instead of triggering format recovery.

Deploying Pinot to Kubernetes

Please refer to Running Pinot on Kubernetes in our project documentation. Pinot also provides Kubernetes integrations with the interactive query engine, Trino, and the data visualization tool, Apache Superset.

Join the Community

Ask questions on Apache Pinot Slack
Please join Apache Pinot mailing lists
[email protected] (subscribe to pinot-dev mailing list)
[email protected] (posting to pinot-dev mailing list)
[email protected] (subscribe to pinot-user mailing list)
[email protected] (posting to pinot-user mailing list)
Apache Pinot Meetup Group: https://www.meetup.com/apache-pinot/

Documentation

Check out Pinot documentation for a complete description of Pinot's features.

License

Apache Pinot is under Apache License, Version 2.0

怎么上手

项目说明

核心能力

容易踩的坑

最近发版

Apache Pinot 1.6.0

Highlights

Fixes and improvements

Upgrade notes and removals

Apache Pinot 1.6.0 RC0

Highlights

Upgrade notes and removals

Apache Pinot 1.5.1

Security fixes

CVE-2026-2332 (Jetty request smuggling)

Upgrade notes

What Changed

Multi-stage Query Engine Enhancements

UNNEST / CROSS JOIN UNNEST Support #17168

Enriched Join Operator #16123

MSE Lite Mode Improvements

Physical Optimizer for Logical Tables #17447

Other MSE Improvements

Federation / Multi-Cluster Routing (New)

What Changed

Multistage Engine Lite Mode (Beta) | Runbook

Multistage Engine Physical Optimizer (Beta) | Runbook

Features

Unsupported but Coming Soon

Here are some of the key PRs that have been merged as part of this feature

用户点评

README 原文

What is Apache Pinot?

Docker quickstart

Features

When should I use Pinot?

Contributing to Pinot

Apache Pinot YouTube Channel

Building Pinot

macOS Build Requirements

Automatic Profile Activation (macOS ARM64)

To install the required tools:

Lucene index recovery during loading

Deploying Pinot to Kubernetes

Join the Community

Documentation

License

相关项目