The CRaw2 framework

A C and Python framework for Canon Raw v2 hacking

version 0.8, dec 2018

Origin

Because I was curious about how my holidays pictures are written to the memory card by my Canon camera, I started to write the Canon Raw v2 specification, based on others' work (Phil Harvey and Dave Coffin).

But is was not enough to 'open' this proprietary format, the next obvious step is to write code that can be used by others to play with CR2 file content. Python is easy to write and elegant to read, and C is efficient, let's use a mix of them!

Design

The CRaw2 framework is composed of 3 parts: the libcraw2 library written in C, craw2tool a CLI tool and PyCRaw2 a Python extension.

Libcraw2 has two kind of interfaces:

• an C API, mainly used by crawtool, to offer CR2 data handling in a Camera agnostic way
• an Python extension, PyCraw2 a simpler API which goal is to offer all required functions to render RAW data into a 16bits RGB image, either from RAW (RGGB) or sraw/mraw (YCbCr) data.

Features

Using craw2tool or PyCraw2:

• Access to TIFF tags, in a generic way to handle unknown tags
• Jpeg lossless decompression and re-compression* of main image
• Access to low level jpeg properties (bits depth, wide, high, components number, HSF, VSF, slices)
• Unslicing and re-slicing*
• RGGB and YCbCr data statistics
• Automatic computation of RGGB top/left and bottom/right borders (before interpolation)
• YCbCr data can be exported and imported in packed form (4:2:2 or 4:1:1 subsampling) or interpolated (4:4:4)
• 16 bits TIFF export for RGGB data
• 16 bits TIFF export and import for YCbCr data (unofficial TIFF extension, I know)

*not from PyCraw2, only libcraw2 API

In few words: educational, camera agnostic and lossless access to RGGB and YCbCr data

Using PyCraw2, we have additionallly:

• Access to 'As Shot' white balance ratios
• Naive interpolation (you can add better methods)
• YCbCr data to RGB conversion

95% of camera specific values are outside of the code, in dng_info.txt. Only two libcraw2 functions are camera specific.

As a summary, the PyCRaw2 goal is to allow RAW to RGB rendering using only Python and NumPy.

What CRaw2 is not

• a user friendly RAW to Jpeg rendering tool: use the excellent DCraw or Camera Raw. CRaw2 is limited to Canon CR2 files.
• a tool to display or hack TIFF tags: use the standard ExifTool. But CRaw2 can be used to analyze unknown TIFF tags.

Bonus

• cr2_database.txt, camera specific Jpeg properties, autogenerated and as CSV format
• dng_info.txt, camera specific RGB rendering values, autogenerated, from DNG converted pictures
• an experimental Python tool to render RGGB and YCbCr pictures to RGB (aka PyCRaw2 demo).
• Python only code tool to extract Thumbnails pictures of CR2 files (thumbails.py) or get TIFF tags content
• 2 PDF posters: to visually explain the CR2 file format, and Jpeg lossless decompression / unslicing

Availability

• compiled for Windows 10 x64. Tested under Linux x64, Ubuntu 18.04, Gcc 7.3
• for Python 3.6
• as code source

CRaw2 is written by Laurent Clévy (@lorenzo2472) and released under GPLv2 license