atmospheric_table

Generate atmospheric opacity lookup tables for (sub)millimeter calibration using the am atmospheric model by Scott Paine (CfA).

These tables encode the zenith opacity as a linear function of precipitable water vapor (PWV):

tau(nu) = b(nu) * pwv_mm + c(nu)

where b (neper/mm) and c (neper) are stored per frequency grid point, per pressure layer. The output format is compatible with kalibrate.

Prerequisites

• Python >= 3.11
• The am atmospheric model (see installation below)
• GSL (GNU Scientific Library) — required for wavelet compression

Install am from source

# Install build dependencies first:
# Debian/Ubuntu: sudo apt-get install build-essential gfortran libgsl-dev curl
# macOS:         brew install gsl gcc curl

./scripts/install_am.sh              # installs to /usr/local
# or
./scripts/install_am.sh /opt/am      # custom prefix

# Verify:
am --version

Install this package

pip install -e ".[dev]"

Usage

Quick start

# 1. Verify am is working
atm-table check-am

# 2. Generate a small test table (minutes)
atm-table generate configs/ccat_quick.toml -o test-table.dat

# 3. Generate the full NANTEN2+CCAT+SOFIA table (hours, 16M grid points)
atm-table generate configs/nanten2_ccat_sofia.toml -o atm-table_NANTEN2-CCAT-SOFIA.dat

# 4. Benchmark against the legacy table
atm-table benchmark new-table.dat.gz /path/to/legacy/atm-table_NANTEN2-CCAT-SOFIA.dat.gz

# 5. Inspect a table
atm-table info atm-table.dat.gz

Configuration

Tables are configured via TOML files. See configs/ for examples.

[frequency]
f_min_mhz      = 1e5      # grid start (100 GHz)
df_mhz         = 0.5      # resolution (0.5 MHz)
mask_f_min_mhz = 4e5      # useful range start (400 GHz)
mask_f_max_mhz = 5e6      # useful range end (5000 GHz)

[geography]
latitude  = -23.0
longitude = -67.75
localtime = "2024-06-15T00:00:00"

[output]
name              = "atm-table_CCAT"
compression_ratio = 64.0   # 0 for plain (uncompressed) table

[[sites]]
name        = "CCAT-5600m"
pressure_pa = 51013
pwv_low_mm  = 0.05
pwv_high_mm = 0.50

Key parameters:

• pressure_pa: Base pressure at the observing site (Pa)
• pwv_low_mm / pwv_high_mm: Two PWV values used to fit the linear tau-vs-PWV model. Choose values spanning the typical range at each site.
• compression_ratio: Wavelet compression factor (64 = keep 1/64th of coefficients). Set to 0 for a plain text table.

CLI reference

Command	Description
atm-table generate <config.toml>	Generate a table from configuration
atm-table benchmark <new> <ref>	Compare two tables
atm-table info <table>	Show table metadata
atm-table check-am	Verify am installation

Options for generate

Flag	Description
-o, --output PATH	Output file path (default: atm-table.dat)
--plain	Write plain table without wavelet compression
--no-gzip	Don't gzip the output
--am PATH	Path to am binary

How it works

1. For each pressure layer and two PWV values, invoke the am model to compute zenith opacity tau(nu) across the full frequency grid.
2. Fit the linear model tau = b * pwv + c from the two runs.
3. Zero out coefficients outside the configured frequency mask.
4. Optionally compress using GSL B-spline 301 wavelet transform (keeping only the top 1/ratio coefficients by magnitude).
5. Write the table in kalibrate-compatible .dat format, optionally gzipped.

Output format

The .dat file has two sections:

Header (comment lines starting with !):

• UUID, model name, pressure layers, frequency range
• Ends with !EOC

Data (either plain or wavelet-compressed):

• Plain: one line per frequency: freq b1 c1 b2 c2 ... per layer
• Wavelet: sparse coefficient lines prefixed !@, with index/value pairs per layer. Reconstructed via GSL inverse B-spline 301 wavelet transform.

Benchmarking

To verify a new table matches the legacy one:

atm-table benchmark \
    new-table.dat.gz \
    /path/to/kalibrate/shared/atm-table_NANTEN2-CCAT-SOFIA.dat.gz \
    --freq-min 400 --freq-max 5000

This reports per-layer RMS and max relative errors for both b and c coefficients. Differences arise from:

• Different am versions (the model improves over time)
• Different atmospheric profile assumptions
• Wavelet compression artifacts (typically < 0.1%)

Development

pip install -e ".[dev]"
pytest tests/
ruff check src/