Class TrxStream

Nested Relationships

Nested Types

Class Documentation

class TrxStream

Streaming-friendly TRX builder that spools positions and finalizes to TrxFile.

TrxStream allows appending streamlines without knowing totals up front. It writes positions to a temporary binary file, tracks lengths, and can add DPS/DPV/group metadata. Call finalize() to produce a standard TRX.

Public Types

enum class MetadataMode

Values:

enumerator InMemory
enumerator OnDisk

Public Functions

explicit TrxStream(std::string positions_dtype = "float32")
~TrxStream()
TrxStream(const TrxStream&) = delete
TrxStream &operator=(const TrxStream&) = delete
TrxStream(TrxStream&&) noexcept = default
TrxStream &operator=(TrxStream&&) noexcept = default
void push_streamline(const float *xyz, size_t point_count)

Append a streamline from a flat xyz buffer.

Parameters:

  • xyz – Pointer to interleaved x,y,z data.

  • point_count – Number of 3D points in the buffer.

void push_streamline(const std::vector<float> &xyz_flat)

Append a streamline from a flat xyz vector.

void push_streamline(const std::vector<std::array<float, 3>> &points)

Append a streamline from a vector of 3D points.

void set_positions_buffer_max_bytes(std::size_t max_bytes)

Set max in-memory position buffer size (bytes).

When set to a non-zero value, positions are buffered in memory and flushed to the temp file once the buffer reaches this size. Useful for reducing small I/O writes on slow disks.

TrxStream &set_metadata_mode(MetadataMode mode)

Control how DPS/DPV/groups are stored during streaming.

InMemory keeps metadata in RAM until finalize (default). OnDisk writes metadata to temp files and copies them at finalize.

TrxStream &set_metadata_buffer_max_bytes(std::size_t max_bytes)

Set max in-memory buffer size for metadata writes (bytes).

Applies when MetadataMode::OnDisk. Larger buffers reduce write calls.

TrxStream &set_voxel_to_rasmm(const Eigen::Matrix4f &affine)

Set the VOXEL_TO_RASMM affine matrix in the header.

TrxStream &set_dimensions(const std::array<uint16_t, 3> &dims)

Set DIMENSIONS in the header.

template<typename T>
void push_dps_from_vector(const std::string &name, const std::string &dtype, const std::vector<T> &values)

Add per-streamline values (DPS) from an in-memory vector.

template<typename T>
void push_dpv_from_vector(const std::string &name, const std::string &dtype, const std::vector<T> &values)

Add per-vertex values (DPV) from an in-memory vector.

void push_group_from_indices(const std::string &name, const std::vector<uint32_t> &indices)

Add a group from a list of streamline indices.

template<typename DT>
void finalize(const std::string &filename, TrxCompression compression = TrxCompression::None)

Finalize and write a TRX file.

void finalize(const std::string &filename, TrxScalarType output_dtype, TrxCompression compression = TrxCompression::None)
void finalize(const std::string &filename, const TrxSaveOptions &options)
void finalize_directory(const std::string &directory)

Finalize and write a TRX directory (no zip).

This method removes any existing directory at the output path before writing. Use this for single-process writes or when you control the entire output location lifecycle.

See also

finalize_directory_persistent for multiprocess-safe variant.

Parameters:

directory – Path where the uncompressed TRX directory will be created.

Throws:

std::runtime_error – if already finalized or if I/O fails.

void finalize_directory_persistent(const std::string &directory)

Finalize and write a TRX directory without removing existing files.

This variant is designed for multiprocess workflows where the output directory is pre-created by a parent process. Unlike finalize_directory(), this method does NOT remove the output directory if it exists, making it safe for coordinated parallel writes where multiple processes may check for the directory’s existence.

See also

finalize_directory for single-process variant that ensures clean slate.

Note

Typical usage pattern: 

// Parent process creates shard directories
fs::create_directories("shards/shard_0");

// Child process writes without removing directory
trx::TrxStream stream("float16");
// ... push streamlines ...
stream.finalize_directory_persistent("shards/shard_0");
std::ofstream("shards/shard_0/SHARD_OK") << "ok\n";

// Parent waits for SHARD_OK before reading results

Parameters:

directory – Path where the uncompressed TRX directory will be created. If the directory exists, its contents will be overwritten but the directory itself will not be removed and recreated.

Throws:

std::runtime_error – if already finalized or if I/O fails.

inline size_t num_streamlines() const
inline size_t num_vertices() const

Public Members

json header