data_processing_utilities

Attributes

`_OBSERVER_POSITION_WARN_KM`
`_MP_CONTEXT`
`logger`

Classes

`FakeSorchaArgs`
`LayupObservatory`	A wrapper around Sorcha's Observatory class to provide additional functionality for Layup.

Functions

`write_fallback_obscodes`()	Decompress the observatory-codes file bundled with layup to a plain JSON
`process_data`(data, n_workers, func, **kwargs)	Process a structured numpy array in parallel for a given function and keyword arguments
`process_data_by_id`(data, n_workers, func, ...)	Process a structured numpy array in parallel for a given function and
`get_cov_columns`()	Get the covariance columns that are expected in the structured numpy array
`has_cov_columns`(data)	Check if the data has the expected covariance columns.
`parse_cov`(orbit_row[, flatten])	Parse the covariance matrix from a structured numpy array representing our
`parse_fit_result`(fit_result_row[, orbit_colm_flag, ...])	Parse the initial guess data from a structured numpy array representing our
`create_chunks`(reader, chunk_size)	For a given reader create a list of lists of object ids such that the total
`layup_furnish_spiceypy`(cache_dir)	A simple wrapper to furnish spiceypy kernels.
`get_format`(data)	Get the orbit parameter format for this data.
`skyplane_cov_to_radec_cov`(cov_xx, cov_xy, cov_yy)	Convert a 2x2 sky-plane covariance into an on-sky error ellipse.

Module Contents

_OBSERVER_POSITION_WARN_KM = 100.0[source]

_MP_CONTEXT = None[source]

logger[source]

write_fallback_obscodes()[source]

Decompress the observatory-codes file bundled with layup to a plain JSON file and return its path.

The MPC observatory-codes file is normally downloaded from minorplanetcenter.net on first use. When that download fails (e.g. the MPC server is unreachable, as has happened on CI runners), we fall back to the copy shipped in layup/data/ObsCodes.json.gz instead of failing the run. Observatory codes change rarely, so a slightly stale fallback is far better than a hard failure. Returns a path suitable for Observatory’s oc_file argument (which reads the decompressed JSON directly).

process_data(data, n_workers, func, **kwargs)[source]

Process a structured numpy array in parallel for a given function and keyword arguments

Parameters:

data (numpy structured array) – The data to process.
n_workers (int) – The number of workers to use for parallel processing.
func (function) – The function to apply to each block of data within parallel.
**kwargs (dictionary) – Extra arguments to pass to the function.

Returns:

res – The processed data concatenated from each function result

Return type:

numpy structured array

process_data_by_id(data, n_workers, func, primary_id_column_name, **kwargs)[source]

Process a structured numpy array in parallel for a given function and keyword arguments. Instead of distributing the data across all available workers it is expected that the data will contain a primary id column. The data will be split by the unique values in the primary id column and each block of data will be processed in parallel.

Parameters:

data (numpy structured array) – The data to process. Expected to contain a primary id column.
n_workers (int) – The number of workers to use for parallel processing.
func (function) – The function to apply to each block of data within parallel.
**kwargs (dictionary) – Extra arguments to pass to the function.

Returns:

res – The processed data concatenated from each function result

Return type:

numpy structured array

get_cov_columns()[source]

Get the covariance columns that are expected in the structured numpy array representing our orbit fit output result.

Columns are a flattened version of the covariance matrix, which is a 6x6 matrix where the first first row and first 6 items are:

[cov_00, cov_01, cov_02, cov_03, cov_04, cov_05]

and the last row and last 6 items of the flattened matrix are: [cov_50, cov_51, cov_52, cov_53, cov_54, cov_55]

Returns:: cov_columns – The covariance columns in the data.
Return type:: list[str]

has_cov_columns(data)[source]

Check if the data has the expected covariance columns.

Parameters:: data (numpy structured array) – The data to check.
Returns:: True if the data has covariance columns, False otherwise.
Return type:: bool

parse_cov(orbit_row, flatten=False)[source]

Parse the covariance matrix from a structured numpy array representing our orbit fit output result.

Parameters:

orbit_row (numpy structured array) – The row of the structured array representing an orbit.
flatten (bool, optional) – If True, return a flattened covariance matrix. If False, return a 6x6 covariance matrix. Default is False.

Returns:

cov – The parsed covariance matrix.

Return type:

numpy array

parse_fit_result(fit_result_row, orbit_colm_flag=True, orbit_para=None)[source]

Parse the initial guess data from a structured numpy array representing our orbit fit output result.

Parameters:: fit_result_row (numpy structured array) – The row of the structured array representing the orbit fit result.
Returns:: res – The parsed fit result.
Return type:: FitResult

create_chunks(reader, chunk_size)[source]

For a given reader create a list of lists of object ids such that the total number of entries in the file for all object ids in a given list, will be less than the chunk size.

Parameters:

reader (ObjectDataReader) – The file reader object for the input file
chunk_size (int) – The maximum number of rows to be included in a single list of ids

Returns:

chunks – A list of lists of object ids that can be passed to the reader’s read_objects method.

Return type:

list[list[ObjIds]]

class FakeSorchaArgs(cache_dir=None)[source]

ar_data_file_path = None[source]

layup_furnish_spiceypy(cache_dir)[source]: A simple wrapper to furnish spiceypy kernels.

class LayupObservatory(cache_dir=None)[source]

Bases: sorcha.ephemeris.simulation_parsing.Observatory

A wrapper around Sorcha’s Observatory class to provide additional functionality for Layup.

cache_dir = None[source]

cached_obs[source]

ObservatoryVel[source]

static _cached_obscodes_present(cache_dir, auxconfigs)[source]

Whether the decompressed observatory-codes file is already cached.

Mirrors the check inside Sorcha’s Observatory – which downloads the codes from the MPC only when this uncompressed file is absent – so we can make the same decision before the download would happen and avoid it (see __init__()). cache_dir defaults to pooch’s per-user cache, exactly as the retriever does.

convert_to_geocentric(obs_location: dict) → tuple[source]

Convert an observatory’s parallax constants to geocentric coordinates.

This overrides Sorcha’s Observatory.convert_to_geocentric, which gates on the truthiness of the parallax constants (obs_location.get("sin", False)). A constant that is legitimately 0.0 – e.g. the geocenter (codes 500/244/248: Longitude=cos=sin=0), Greenwich (code 000: Longitude=0), or an equatorial station such as Quito (code 782: sin=0) – is falsy, so the base class wrongly reports the observatory as having no fixed position. Layup then routes it to the moving-observatory path and raises “invalid coordinates” for plain MPC input that carries no per-observation position (issue #286).

We instead test for the presence of the constants. Codes that have no position keys at all (roving observer 247, space telescopes WISE/TESS/HST, …) still return (None, None, None) and are correctly routed to the ADES per-observation position path. The geocenter resolves to a (0, 0, 0) offset from Earth’s center, which is exactly right.

Parameters:: obs_location (dict) – Dictionary with Longitude and the sin/cos of the observatory latitude.
Returns:: Geocentric position (x, y, z), or (None, None, None) when the observatory has no fixed position.
Return type:: tuple

create_obscode_cache_key(obscode, et)[source]

Create a cache key for the observatory coordinates.

Parameters:

obscode (str) – The observatory code.
et (float) – The ephemeris time.

Returns:

The cache key for the observatory coordinates.

Return type:

str

populate_observatory(obscode, et, data)[source]

Populate the observatory coordinates for a given observatory code and ephemeris time and provide the key that can be used to access the coordinates for the observatory in the cache at the given epoch. This is used to generalize the case where the observatory does not have a fixed position and the coordinates are provided in the data.

Parameters:

obscode (str) – The observatory code.
et (float) – The ephemeris time.
data (numpy structured array) – A row of the structured array of the orbit data to process.

Returns:

obscode_cache_key – The cache key for the observatory coordinates at the given epoch.

Return type:

str

_populate_observatory_velocity(obscode_cache_key, data)[source]

Cache an optional user-supplied geocentric observer velocity (km/s).

ADES allows optional velocity columns (vel1/vel2/vel3) alongside the position of a space-based observer (issue #147). They share the position’s reference frame (sys) and center (ctr), both already validated for the position in populate_observatory(). We convert to km/s and store under the same per-epoch cache key as the position; _barycentric_moving_observatory() then adds Earth’s barycentric velocity to it. Velocity is optional: if the columns are absent or NaN we leave the cache untouched and fall back to Earth’s velocity.

Parameters:

obscode_cache_key (str) – The per-epoch cache key the position was stored under.
data (numpy structured array) – The observation row (already known to be a moving observer).

static _obs_vel_to_km_s(vel, sys)[source]

Convert an ADES OBS_VEL observer velocity to km/s.

Unit convention (matches the SPICE km/s state that _barycentric_moving_observatory() uses for Earth):

ICRF_KM -> km/s (no conversion; SPICE-native)
ICRF_AU -> au/day -> km/s

sys has already been validated to one of these two values in populate_observatory().

static _has_ades_position(data)[source]

Whether the row carries a usable ADES per-observation position.

True only when all of sys/ctr/pos1/pos2/pos3 are present and the three position components are finite. Used to let a user-supplied position override the JPL-Horizons lookup for a space-based observatory (issue #55).

_fetch_space_observatory_states(naif_id, jd_tdb_list)[source]

Geocentric states of a spacecraft at the given JD(TDB) epochs.

Thin wrapper around query_horizons_geocentric() – a seam that tests monkeypatch so they never touch the network. The persistent (naif_id, jd) -> state cache is keyed under this observatory’s cache directory, so it is shared across objects, processes, and runs.

_populate_space_observatory(obscode, et, obscode_cache_key)[source]

Resolve a space-based observatory’s geocentric state via JPL Horizons.

Stores the geocentric position (km) and velocity (km/s) under the per-epoch cache key, exactly as an ADES-supplied moving observer would, so _barycentric_moving_observatory() then adds Earth’s barycentric state to it (issue #55). A no-op if the key is already cached (e.g. warmed by _prefetch_space_observatories()).

_prefetch_space_observatories(data)[source]

Warm the cache for space-based observatories, one request per spacecraft.

Groups every epoch of each space-based obscode (that the user has not overridden with an explicit ADES position) and resolves them in a single batched JPL Horizons request, so a fit with N space-based observations makes one HTTP call per spacecraft rather than N (issue #55).

_barycentric_moving_observatory(et, obscode_cache_key)[source]

Barycentric position and velocity (km, km/s) of a moving observatory.

A fixed ground station’s ObservatoryXYZ entry holds dimensionless parallax constants in the Earth-fixed frame, so barycentricObservatoryRates rotates it to J2000 and scales it by the Earth radius. A moving (e.g. space-based) observatory is different: its position is supplied per-observation as an explicit geocentric vector that populate_observatory has already stored in km and in the J2000/ICRF frame (from the sys/ctr/pos1..3 columns). It must therefore be added to Earth’s barycentric position directly – with no Earth-rotation and no Earth-radius scaling, both of which barycentricObservatoryRates would apply and which would misplace the observatory by a factor of the Earth radius (~6378x).

The observatory’s own geocentric velocity is used when the user supplies it via the ADES vel1/vel2/vel3 columns (cached in ObservatoryVel, km/s, issue #147): the barycentric velocity is then Earth’s velocity plus the geocentric observer velocity, mirroring the position. When no velocity is supplied we fall back to Earth’s barycentric velocity alone; the satellite’s orbital velocity (~km/s) is then a small correction that only enters through aberration.

obscodes_to_barycentric(data)[source]

Takes a structured array of observations and returns the barycentric positions and velocites of the observatories.

This assumes that data must have a column ‘et’ representing the ephemeris time of each observation in TDB.

Parameters:: data (numpy structured array) – The data to process.
Returns:: res – Representing the barycentric positions and velocities of the observatories in the data (x,y,z,vx,vy,vz).
Return type:: numpy structured array

get_format(data)[source]

Get the orbit parameter format for this data.

Parameters:: data (numpy structured array) – The data to check.
Returns:: The format of the data.
Return type:: str

skyplane_cov_to_radec_cov(cov_xx, cov_xy, cov_yy)[source]

Convert a 2x2 sky-plane covariance into an on-sky error ellipse.

The covariance is expressed in the local orthonormal tangent-plane basis used by predict, whose axes are the unit vectors in the directions of increasing RA (a great circle on the sky, i.e. already scaled by cos(dec)) and increasing Dec. The error ellipse is therefore the eigen-decomposition of the 2x2 matrix; no cos(dec) scaling is applied (the input is already an on-sky covariance, not a covariance in RA/Dec coordinates).

Parameters:

cov_xx (numpy array) – (x, x) entry of the sky-plane covariance (radians^2); x is the great-circle RA direction.
cov_xy (numpy array) – (x, y) entry of the sky-plane covariance (radians^2).
cov_yy (numpy array) – (y, y) entry of the sky-plane covariance (radians^2); y is the Dec direction.

Returns:

numpy array – semi-major axis of the error ellipse (arcsec)
numpy array – semi-minor axis of the error ellipse (arcsec)
numpy array – position angle of the major axis (degrees, North through East)