PolarDB-X is a cloud native distributed SQL Database designed for high concurrency, massive storage, complex querying scenarios. It has a shared-nothing architecture in which computing is decoupled from storage. It supports horizontal scaling, distributed transactions and Hybrid Transactional and Analytical Processing (HTAP) workloads, and is characterized by enterprise-class, cloud native, high availability, highly compatiable with MySQL and its ecosystem.
PolarDB-X was originally created to solve the database's scalability bottleneck of Alibaba Tmall's "Double Eleven" core transaction system, and has grown with AliCloud along the way, and is a mature and stable database system that has been verified by many customers' core business systems.
The core features of PolarDB-X include:
- Horizontal Scalability
PolarDB-X is designed with Shared-nothing architecture, supporting multiple Hash and Range data sharding algorithms and achieving transparent horizontal scaling through implicit primary key sharding and dynamic scheduling of data shard.
- Distributed Transactions
PolarDB-X adopts MVCC + TSO approach and 2PC protocol to implement distributed transactions. Transactions meet ACID characteristics, support RC/RR isolation levels, and achieve high performance through optimizations such as one-stage commit, read-only transaction, and asynchronous commit.
- HTAP
PolarDB-X supports analytical queries through native MPP capability, and achieves strong isolation of OLTP and OLAP traffic through CPU quota constraint, memory pooling, storage resource separation, etc.
- Enterprise-class
PolarDB-X has many capabilities designed for enterprise scenarios, such as SQL Concurrency Control, SQL Advisor, TDE, Triple Authority Seperation, Flashback Query, etc.
- Cloud Native
PolarDB-X has years of cloud native practice on AliCloud, supports managing cluster resources via K8S Operator, supports public cloud, hybrid cloud, private cloud and other forms for deployment.
- High Availability
PolarDB-X achieves strong data consistency through Multi-Paxos protocol, supports cross-data center deployment, and improves system availability through Table Group, Geo-locality, etc.
- Compatiable with MySQL and Its Ecosystem
The goal of PolarDB-X is to be fully compatible with MySQL, which currently includes MySQL protocol, most of MySQL SQL syntax, Collations, transaction isolation level, binary log, etc.
PolarDB-X supports one-click installation by PXD tool, through which you can quickly try the functions of PolarDB-X.
See the PXD Quick Start.
PolarDB-X provides K8S deployment mode, through which you can customize the configuration of PolarDB-X cluster.
See the K8S Quick Start.
Try the following steps under a non-root user to build and run PolarDB-X from the source code.
NOTE: CentOS 7&8 and Ubuntu 18 and higher are currently supported.
- Prerequisites
Make sure the following packages are installed (for most cases, they are installed already):
# For CentOS
yum install -y sudo git make redhat-lsb-core
yum remove -y cmake
yum install -y wget java-11-openjdk-devel cmake3 automake bison openssl-devel ncurses-devel libaio-devel mysql snappy-devel lz4-devel bzip2-devel autoconf libstdc++-static libarchive
yum install -y centos-release-scl
yum install -y devtoolset-10
# For Ubuntu
apt-get update && apt-get install -y sudo git make lsb-releaseCreate and switch to a non-root user with sudo privilege.
useradd -ms /bin/bash polarx
echo "polarx:polarx" | chpasswd
echo "polarx ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
su - polarx
git clone https://github.com/polardb/polardbx.git
cd polardbx- Build
makemake will download all repositories, install build tools and libraries, build and install PolarDB-X.
Repositories are downloaded to ./build dir and binaries are installed to ./build/run.
You can run make clean to remove the installation and try build PolarDB-X again.
You can also run make cleanAll to remove everything under ./build.
This process may take more than half an hour depending on your network bandwidth.
- Run
./build/run/bin/polardb-x.sh startWait about 1 minute and use mysql -h127.0.0.1 -P8527 -upolardbx_root to connect to PolarDB-X.
- Stop
./build/run/bin/polardb-x.sh stopThe core features of PolarDB-X community version will be consistent with the commercial version, and more manuals can be found in the documentations of the commercial version. The documentations of the community version are being compiled and will be released to the public in the near future.
PolarDB-X has a shared-nothing architecture in which compute and storage is decoupled, and the system consists of 4 core components.
- CN (Compute Node)
The Compute Node is the entry point of the system and is stateless, which includes modules such as SQL parser, optimizer, and executor. It is responsible for distributed data routing, 2PC coordination, global secondary index maintenance, etc. It also provides enterprise features such as SQL concurrency control and triple authority separation.
- DN (Data Node)
The Data Node is responsible for data persistence, providing strong data consistency based on the Multi-Paxos protocol, while maintaining distributed transaction visibility through MVCC.
- GMS (Global Meta Service)
The Gloal Meta Service is responsible for maintaining globally consistent Table/Schema, Statistics and other system Meta information, maintaining security information such as accounts and permissions, and providing global timing services (i.e. TSO).
- CDC (Change Data Capture)
The CDC Node provides change data capture capability that is fully compatible with the MySQL binary log format and MySQL DUMP protocol, and master-slave replication capability that is compatible with the MySQL Replication protocol.
PolarDB-X provides tool to manage the above components through K8S Operator, and the RPC between the CN and DN can be done through private protocol component. The corresponding repositories of these components are as follows.
| Component Name | Repository |
|---|---|
| CN (Compute Node) | polardbx-sql |
| GMS (Global Meta Service) | polardbx-engine |
| DN (Data Node) | polardbx-engine |
| CDC (Change Data Capture) | polardbx-cdc |
| RPC | polardbx-glue |
| K8S Operator | polardbx-operator |
polardbx-sql is one component of PolarDB-X, namely CN (Compute Node).
polardbx-sql is under the Apache License 2.0. See the License file for details.
You are welcome to make contributions to PolarDB-X. We appreciate all the contributions. For more information about how to start development and pull requests, see contributing.
You can join these groups and chats to discuss and ask PolarDB-X related questions:
- DingTalk Group: 32432897
- WeChat Group: 阿里云 PolarDB-X 开源交流群 (Contact group manager to get into wechat group. Managers' ID: oldbread3, hustfxj, agapple0002)
polardbx-sql references from many open source projects, such as Calcite, Presto etc. Sincere thanks to these projects and contributors.