Task List

Current high-level objectives

By popular request, Brotli is currently the main focus. However, in order to satisfy that objective, a DEFLATE implementation will be done first.

• Implement flate.Writer
• Implement brotli.Writer

The Brotli format shares many similarities with DEFLATE. At its core, it is built on the ideas of LZ77 and Huffman bit-encoding. Thus, implementing DEFLATE first will allow for a better understanding of how to implement Brotli well. Given that the Readers were able to share much code, it is likely that the Writers will also be able to share much code.

Items to work on by package

• brotli
  • Reader
    • Make use of internal/prefix
    • Add more tests for malicious and strange inputs
    • Eagerly return io.EOF at end of block when possible
  • Writer
    • Implement me!
• bzip2
  • Reader
    • (performance) Add chunking between stages to reduce total memory bandwidth
    • (performance) Investigate other ways to improve performance
  • Writer
    • Optimize assignment of prefix trees for better compression ratio
    • Investigate various BWT construction methods:
      • bwtis, sais, qsufsort, divsufsort
    • (performance) Add chunking between stages to reduce total memory bandwidth
    • (performance) Investigate other ways to improve performance
• flate
  • Reader
    • Make use of internal/dict when available
  • Writer
    • Implement me!
• xz
  • Implement me!
• zstd
  • Implement me!
• internal
  • Rename prefix to bits and abstract the prefix encoding logic as a sub-package bits/prefix
  • Create package dict to implement a generic LZ77 dictionary for use in brotli and flate