Reading Data
==============

Between "I have a CSV of measurements" and "I have a prophys model" sits a
step usually retyped per project: parse the file, convert the units, decide
what the non-detects mean, turn a wind rose into a distribution.
:mod:`prophys.io` does that step, producing the objects the rest of the
package already consumes.

The CSV, table and wind-rose readers need nothing beyond NumPy. The raster
and NetCDF readers need the ``geo`` extra and say so when it is missing.

Units are declared, never guessed. Every reader takes the unit the file is
written in and the unit the model works in, and converts between them
through :mod:`prophys.units` — so the unit on a model attribute and the unit
in the source file are checked against each other exactly once, at the
boundary.

Laboratory records
-------------------

A results sheet typically contains three kinds of entry: a measurement, a
non-detect below an instrument's limit, and a blank. Substituting a number
for either of the last two changes the likelihood the fit maximizes.
:func:`~prophys.io.read_csv` classifies them instead:

.. code-block:: text

   site,arsenic,depth
   A,0.8,12
   B,<0.5,15
   C,ND,20
   D,1.9,
   E,2.4,31

.. code-block:: python

   from prophys import io

   columns = io.read_csv("lab.csv", unit={"arsenic": "mg/L", "depth": "m"})
   columns["arsenic"].summary()
   # {'n': 5, 'measured': 3, 'censored': 1, 'missing': 1, 'mean': 1.7, ...}

``<0.5`` becomes left-censored at 0.5; ``ND`` with no limit becomes missing,
because a censoring term needs a limit to integrate up to; the blank becomes
missing; the text column is skipped.

:func:`~prophys.io.read_observations` goes straight to a fit-ready record:

.. code-block:: python

   obs = io.read_observations("lab.csv", "arsenic", unit="ug/L", to_unit="mg/L")
   result = prp.finetune(compiled, {"y": obs}, n_steps=3000)

See :doc:`observations` for what each kind contributes to the likelihood.

Wind and wave roses
--------------------

:func:`~prophys.io.read_wind_rose` reads either shape of directional table —
one row per measurement, or one row per sector with a frequency column — and
returns directions in **radians**, the convention every circular
distribution in this package uses:

.. code-block:: python

   rose = io.read_wind_rose("wind.csv", direction_unit="deg", speed_unit="m/s")
   climatology = io.directional_mixture(rose, n_sectors=12)

:func:`~prophys.io.directional_mixture` divides the circle into equal
sectors; each becomes one component of a
:class:`~prophys.distributions.DirectionalMixture`, weighted by its share of
the rose, with a :class:`~prophys.distributions.Weibull` intensity fitted to
the speeds recorded in that sector.

That per-sector fit is by the method of moments — shape from the coefficient
of variation, scale from the mean. It runs per sector with no optimizer, no
starting value and no failure mode, which matters because a rose routinely
has a sector with a handful of records in it. The result is a **starting**
climatology; refine it against the same data:

.. code-block:: python

   result = prp.fit_distribution(climatology, observations, n_steps=2000)

Rasters and gridded fields
---------------------------

.. code-block:: python

   terrain = io.read_raster("terrain.tif", frame=site, unit="m")

The raster's affine transform supplies the field's origin and cell size, so
the resulting :class:`~prophys.structures.Field` lands in ``frame`` at the
coordinates the file declares. No-data cells become ``nan`` unless
``nodata_fill`` is given; any interpolation touching them then yields
``nan``, which is the honest result.

:func:`~prophys.io.read_netcdf` reads one variable, optionally selecting a
single position along named dimensions so a large file need not be
materialized in full:

.. code-block:: python

   wind = io.read_netcdf(
       "reanalysis.nc", "u10", index={"time": 0}, to_unit="m/s"
   )

The variable's own CF-convention ``units`` attribute is used when none is
given, and checked against one that is: a file saying ``m s-1`` while the
caller says ``kt`` is a mistake worth catching at the read.

In-memory tables
-----------------

:func:`~prophys.io.read_table` is the counterpart for rows that came from
somewhere other than a file — a database cursor, an API response, a test
fixture. It takes the same unit arguments and produces the same
:class:`~prophys.io.Column` objects, so downstream code cannot tell the two
apart.

Empirical distributions
------------------------

.. code-block:: python

   empirical = io.empirical_from_column(columns["arsenic"])

A kernel-density distribution over the column's **measured** values.
Censored and missing entries are excluded: an ``Empirical`` describes values
that were observed, and a detection limit is not one. When the non-detects
carry information worth using, fit a censored likelihood instead.

.. automodule:: prophys.io.tabular
   :members: Column, read_csv, read_table, read_observations, read_wind_rose,
             directional_mixture, empirical_from_column, read_raster,
             read_netcdf