Giter VIP home page Giter VIP logo

zhaodiankui / zheap Goto Github PK

View Code? Open in Web Editor NEW

This project forked from enterprisedb/zheap

0.0 0.0 0.0 241.9 MB

new heap with in-place update, powered by undo, for eventual integration into PostgreSQL

License: Other

Emacs Lisp 0.01% Makefile 0.71% M4 0.24% Shell 0.16% SQLPL 0.68% C 84.11% PLpgSQL 6.72% Perl 2.04% C++ 2.19% Yacc 1.37% Lex 0.49% Objective-C 0.80% PLSQL 0.06% Ruby 0.17% Python 0.14% Assembly 0.01% Roff 0.07% sed 0.01% DTrace 0.01% XS 0.02%

zheap's Introduction

The purpose of this document is to let users know how they can use zheap (a new storage format for PostgreSQL) and the work that is still pending. This new storage format provides a better control over bloat, reduces the tuple size and reduces the write amplification. The detail design of zheap is present in zheap design document (src/backend/access/zheap/README).

How do I use zheap?

We have provided a storage engine option which you can set when creating a table. For example:

create table t_zheap(c1 int, c2 varchar) USING zheap;

Index creation for zheap tables doesn't need any special syntax.

You can also set the GUC parameter default_table_access_method. The default value is “heap", but you can set it to “zheap”. If you do, all subsequently-created tables will use zheap.

These interfaces will probably change once the storage format API work is integrated into PostgreSQL. We’ll adjust this code to use whatever interfaces are agreed by the PostgreSQL community.

Each zheap page has fixed set of transaction slots each of which contains the transaction information (transaction id and epoch) and the latest undo record pointer for that transaction. By default, we have four transaction slots per page, but this can be changed by setting --with-trans_slots_per_zheap_page=value while configuring zheap.

What doesn’t work yet?

  • Logical decoding
  • Snapshot too old - We might want to implement this after first version is committed as this will work differently for zheap.
  • Alter Table <table_name> Set Tablesapce <tbs_name> - For this feature to work correctly in zheap, while copying pages, we need to ensure that pending aborts gets applied before copying the page.

Tools

  • pg_undo_dump similar to pg_wal_dump: We would like to develop this utility as it can be used to view undo record contents and can help us debug problems related to undo chains.
  • We also want to develop tools like pgstattuple, pgrowlocks that allow us to inspect the contents of database pages at a low level.
  • wal consistency checker: This will be used to check for bugs in the WAL redo routines. Currently, it is quite similar to what we have in current heap, but we want to extend it to check the consistency of undo pages similar to how it checks for data and index pages.

Open Issues

  • Currently, the TPD pages are not added to FSM even if they can be completely reused.
  • Single user mode: This needs some investigation as to what exactly is required. I think we need to ensure that undo gets applied without the need to invoke undo worker.

The other pending code related items are tracked on zheap wiki page: https://wiki.postgresql.org/wiki/Zheap

Test run before pushing code

  • make check-world
  • make installcheck and isolation test (with default_table_access_method = 'zheap')
  • make installcheck with hot standby (with wal_consistency_checking=all)

You can find overall design of zheap in the README: src/backend/access/zheap/README

zheap's People

Contributors

tglsfdc avatar bmomjian avatar petere avatar robertmhaas avatar scrappy avatar hlinnaka avatar alvherre avatar mhagander avatar anarazel avatar adunstan avatar vadim4o avatar simonat2ndquadrant avatar tatsuo-ishii avatar feodor avatar nmisch avatar michaelpq avatar masaofujii avatar sfrost avatar kgrittn avatar jconway avatar itgacky avatar macdice avatar dbdbdb avatar akorotkov avatar grunthos avatar petergeoghegan avatar darcyjcain avatar rhodiumtoad avatar deanrasheed avatar tvondra avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.