file_io.file_output

Utility functions for writing data from our internal representation of numpy structured arrays to output files.

Functions

`write_csv`(data, filepath[, move_columns])	Write a numpy structured array to a CSV file.
`_store_hdf5`(data, filepath, key, mode)	Write `data` to `filepath` as an appendable HDF5 table.
`write_hdf5`(data, filepath[, key])	Write a numpy structured array to an HDF5 file, overwriting any existing
`append_hdf5`(data, filepath[, key])	Append a numpy structured array to an HDF5 table, creating the file if it

Module Contents

write_csv(data, filepath, move_columns=None)[source]

Write a numpy structured array to a CSV file.

Parameters:

data (numpy structured array) – The data to write to the file.
filepath (str) – The path to the file to write.
move_columns (dict, optional) – Dict of any column names that need moved, paired with their new position.

_store_hdf5(data, filepath, key, mode)[source]

Write data to filepath as an appendable HDF5 table.

mode="w" truncates any existing file first (a fresh write); mode="a" appends to it (creating it if absent).

write_hdf5(data, filepath, key='data')[source]

Write a numpy structured array to an HDF5 file, overwriting any existing file so that repeated calls are idempotent.

To accumulate data across several calls (e.g. writing successive chunks into one file), call this once for the first write and append_hdf5() for the rest; otherwise a second call to a pre-existing file would duplicate rows.

Parameters:

data (numpy structured array) – The data to write to the file.
filepath (str) – The path to the file to write.
key (str, optional) – The key to use in the HDF5 file.

append_hdf5(data, filepath, key='data')[source]

Append a numpy structured array to an HDF5 table, creating the file if it does not yet exist.

Use to accumulate chunked output after an initial write_hdf5(). On its own this grows whatever is already on disk, so it is not idempotent across re-runs – start each run with write_hdf5().

Parameters:

data (numpy structured array) – The data to append to the file.
filepath (str) – The path to the file to write.
key (str, optional) – The key to use in the HDF5 file.