Skip to content

Home

Jump to bottom Edit New page
Michael edited this page Mar 19, 2017 · 69 revisions

News

      
Domino v0.1 is released! (06/25/2014)
Domino (SSCC) is integrated into HP’s new generation DBMS Apache Trafodion (01/01/2015)
Domino v0.2 : Some bugs are fixed! (07/25/2015)

Overview


Domino is a transaction engine implementing SUI on the top of HBase. SUI (State-based Update Isolation) provides distributed concurrency control (SSCC:Stareless and Stateful update Concurrency Control) for bigdata transaction processing and has the ability to prevent the anomalies of P0/P1/P2/P3/P4/P4B/A5A. Domino and SSCC has been implemented in Apache Trafodion.

Domino has the following properties:

  1. low lock contention;
  2. one-phase commit;
  3. lightweight rollback and assistant commit.

These features make Domino support high concurrency and low delay. The TPC-C results shows that Domino achieves high score (tpmC/W=12.24) under distributed environment, which is nearly the same with the performance of Oracle with its dedicated disk array. Domino is open source under Apache License and you can also see Apache Trafodion. (It is called SSCC in Apache Trafodion)

System Model

The key contribution of Domino is to design a lock-free concurrency control and one-phase commit (1PC) mechanism for distributed environment. The method is based on our carefully designed data model, which is called 3M data model (Multi-dimension and Multi-version Map). Our primary idea is to save multiple snapshots for a data item. Every snapshot has two kinds of tags to support full-ACID properties: status and version. “Version” indicates the latest available data and “status” is designed to show whether the data is being updated. In our system, the write operations fall into two basic types: stateless-update and stateful-update. Stateless-update can also be called blind-write. It inserts new data regardless of the current data. Stateless-update depends on the current data, such as B’=B+1 where B’ is the new value and B is the current value. The concurrent stateless-update operations are lock-free in our system. The ACID properties are guaranteed through multiple data versions. The data is saved in the backend store engine system. There is no coordinator in the system, and every read/write is directly send to the store engine node where the data is located and returned with data or false. The procedure is similar to that of single-machine database, which only needs 1PC during the transaction.

The above mechanism is called State-based Update Isolation (SUI). Through the correctness verification, we prove that SUI can prevent the following anomalies: P0, P1, P2, P3, P4. In summary, our contributions are enumerated as follows:

  • Multi-dimension and Multi-version Map data model (3M model). 3M model provides the basis for lock-free design.
  • State-based concurrency control (SCC). SCC implements the full-ACID properties preventing the anomalies of P0,P1,P2,P3,P4. SCC also optimises the failure handling through assistant commit and lightweight rollback.
  • One-phase commit (1PC). Unlike the traditional 2PC for distributed environment, the idea of 1PC makes our transaction engine response in real-time.

System Implementation

Domino consists of four components, DETS, TME, DTO and Domino Client. DETS (Domino Endpoint Transaction System) runs on HBase RegionServer with Coprocessor. Therefore, the scalability of DETS is as strong as HBase. The routing protocol is based on the RowKey, which highly saves the maintenance costs. Meanwhile, the routing protocol reduces the most communication between client and server, highly improving the system performance.
Domino also uses Coprocessor Framework to implement DTO (Domino Timestamp Oracle) and TME (Transaction Metadata Endpoint). DTO is responsible for generating increasing transaction id and TME manages the metadata for Domino. Domino client includes HBaseClient and provides APIs for the user.

Interface (APIs)

Initiate Client

  • public Domino(String zookeeperAddress) throws IOException;
  • public Domino(Configuration config) throws IOException;

DDL Handler

The APIs is used to manage data table and start a transaction.
Create a Table

  • public void createTable(HTableDescriptor table) throws IOException;
  • public void createTable(HTableDescriptor table, byte[ ][ ] splitKeys) throws IOException;
  • public void createTable(HTableDescriptor table, byte[ ] startKey, byte[] endKey, int numRegions) throws IOException;
    Delete a Table
  • public void dropTable(byte[ ] name) throws IOException;
Begin a transaction
  • public Transaction startTransaction() throws IOException;
    startTransaction() returns a handler “Transaction” to deal with the transaction.

Transaction operators

. Transaction’s commit, rollback, read and write are all provided by Class Transaction.


  1. Read a record.
    public Result get(Get get, byte[ ] table) throws IOException;
    This api uses HBase’s ‘Get’ as a parameter, and the result is returned in the format of HBase’s ‘Result’.


  2. Write a stateless record.
    public void put(Put put, byte[ ] table) throws IOException;


  3. Write a stateful record.
    public void putStateful(Put put, byte[ ] table) throws IOException;


  4. Delete a record.
    public void delete(byte[ ] row, byte[ ] table) throws IOException;


  5. Scan the given range data.
    public ResultScanner scan(Scan scan, byte[] table) throws IOException;
    This api uses HBase’s Scan as a parameter.


  6. Commit a transaction.
    public void commit() throws IOException;


  7. Rollback a transaction.
    public void rollback() throws IOException;


An example of how to use domino client can be found at https://github.com/domino-succ/domino/blob/master/example.java.

Deployment

It is very simple to deploy Domino in the real-world system. The prerequisite is that there should be an executable HBase cluster and the version of HBase should be higher than 0.94.

Domino consist four library files (JARs) as follows:

  1. domino-client-.jar, the client class library;
  2. domino-common-.jar, Common class library;
  3. domino-core-.jar, the core Endpoint class library；
  4. domino-id-service-.jar, transaction ID service class library.
Server side deployment.

Before HBase’s startup, we should first put the above four library files (JARs) into HBase’s , and then startup HBase. Follow the above steps, the service of Domino is deployed successfully.

Client side.

When using domino at client, we only need to include three jars, domino-client, domino-common and domino-id-service, into . The example how to use the client can be found at https://github.com/domino-succ/domino/blob/master/example.java.

Anomalies Prevention

Domino/SUI has the ability to prevent anomalies of P0/P1/P2/P3/P4/A5A, which is the high level isolation for the modern databases. We prove the correctness in our paper.

Evaluation

The TPC-C results shows that Domino achieves high score under distributed environment, which is nearly the same with the performance of oracle with its dedicated disk array.

For more evaluation results, please refer our papers.

Papers

Download our papers

People

Tieying Zhang
Andy Pavlo
Tianqi Zheng
Zhen Zhao
Weimin Zhang

Welcome you to join Domino.

--------------------------------This page is edited by Michael: [email protected]

Add a custom sidebar
Clone this wiki locally