Skip to content

Latest commit

 

History

History
58 lines (38 loc) · 3.2 KB

Binary-Data.md

File metadata and controls

58 lines (38 loc) · 3.2 KB
Raw

Binary Data

PostgreSQL stores binary data either as a field with the BYTEA➚ data type or as a large object➚.

Large Objects in taoPQ have their own representation discussed in the Large Object chapter.

The BYTEA➚ Data Type

In PostgreSQL strings are a sequence of bytes that is valid in a given encoding. One or more bytes can represent individual characters or code points. This means that not all sequences of bytes are a valid string and therefore binary data can generally not be represented by PostgreSQL as a string.

Binary data is different from strings, as binary data is a collection of an arbitrary sequence of bytes. Any byte is treated independently of its surrounding bytes and can have any value, including '\0'. It is therefore crucial to represent binary data with the dedicated BYTEA➚ data type.

C++ Binary Data

The individual bytes are represented by std::byte➚. As there is no one-size-fits-all data type to handle binary data in C++, we allow several options.

We mostly represent binary data with tao::pq::binary and tao::pq::binary_view. These are type aliases for std::vector<std::byte>➚ and std::span<std::byte>➚, respectively.

Passing Binary Data

When you pass binary data to taoPQ, we only require a view to be passed. As a view is a non-owning data type, constructing an instance of it is cheap.

If you have other data types, you can create a binary data view by using

auto tao::pq::to_binary_view( const auto* data, const std::size_t size ) noexcept -> tao::pq::binary_view;

auto tao::pq::to_binary_view( const auto& data ) noexcept
{
   return pq::to_binary_view( std::data( data ), std::size( data ) );
}

The former function requires (and checks) that T has a size of 1 byte. If you want to store larger Ts as binary data you need to manually convert the pointer and size appropriately.

The second method requires the data type T to be a suitable candidate for std::data()➚ and std::size()➚, which requires the data to be stored in a contiguous block of memory. We do not offer any convenience methods to create binary data from distributed data structures, i.e. std::list<std::byte> is not supported.

Receiving Binary Data

When receiving binary data, a non-owning view is insufficient, hence we return tao::pq::binary. In some cases other alternatives are offered, i.e. you may provide a buffer that the data is written to.

This document is part of taoPQ.

Copyright (c) 2021-2024 Daniel Frey and Dr. Colin Hirsch
Distributed under the Boost Software License, Version 1.0
See accompanying file LICENSE_1_0.txt or copy at https://www.boost.org/LICENSE_1_0.txt