This directory contains the SAFARI WireWalker processing pipeline for both Level-2 quality-controlled observations and Level-3 gridded products, along with raw inputs, metadata definitions, and diagnostic plots.
Processing: 6 October 2026 — Benjamin Greenwood.
Processing assistance: OpenAI Codex was used to assist with development and refinement of processing code, quality-control workflows, and diagnostic plots. Final methods and outputs remain subject to scientific review.
Real-time telemetry: A real-time WireWalker data stream was also received through the Certus 9770 modem. The Campbell CR6 and Certus data-return streams are described in the Raw data section; they are complementary because Certus provides near-real-time return while the CR6 files provide the recovered record set.
The Level-2 pipeline decodes Campbell CR6 records containing RBR CTD measurements, applies pressure-aware adaptive envelope and time-domain spike QC, and writes a compressed CF-1.13 NetCDF4 product. The Level-3 pipeline selects QC-approved upward profiles, averages them into one-meter depth bins at two-hour and four-hour resolutions, and provides separate bounded two-dimensional interpolation products that fill only within 10 meters vertically and five time bins on either side.
Install the required Python libraries with python3 -m pip install numpy matplotlib netCDF4 gsw pdfkit. The README PDF also requires the system wkhtmltopdf executable used by pdfkit. From this directory, run python3 L2_process.py to regenerate the L2 NetCDF file, python3 L3_process.py to regenerate the L3 NetCDF file, and python3 L3_plots.py --save to create the L3 diagnostic PNG files. Run python3 L2_plots.py to create L2 diagnostic PNG files, or add --show for interactive display. Run python3 publish_readme.py to rebuild this HTML and PDF README.
SAFARI_Wirewalker/
raw/
wirewalker*.dat
L2_process.py
L2_quality_control.py
L2_netCDF.py
L2_variables.json
L2_globals.json
L2_plots.py
L3_process.py
L3_netCDF.py
L3_variables.json
L3_globals.json
L3_plots.py
publish_readme.py
plots/
SAFARI_wirewalker_L2.nc
SAFARI_wirewalker_L3.nc
README.html
README.pdf
The raw directory contains the WireWalker data files. L2_process.py decodes binary packets and coordinates the L2 pipeline. L2_quality_control.py computes QC flags and up/down profile IDs. L2_netCDF.py assembles the L2 product from JSON metadata. L2_variables.json and L2_globals.json define L2 NetCDF metadata. L3_process.py averages QC-approved upward profiles into 2-hour and 4-hour by 1 m time-depth bins and creates bounded interpolated variants. L3_netCDF.py writes the L3 product from L3_variables.json and L3_globals.json.
The raw directory contains the Campbell CR6 WireWalker files used to create the L2 product.
| File | Size | First CR6 timestamp | Last CR6 timestamp |
|---|---|---|---|
| wirewalker0.dat | 0.18 MB | 2025-10-15 15:40:00Z | 2025-11-09 20:23:00Z |
| wirewalker1.dat | 0.73 MB | 2025-11-10 00:13:00Z | 2025-11-10 18:00:10Z |
| wirewalker2.dat | 32.41 MB | 2025-11-10 18:03:30Z | 2025-12-20 08:59:50Z |
| wirewalker3.dat | 32.88 MB | 2025-12-20 09:13:20Z | 2026-01-28 07:13:00Z |
| wirewalker4.dat | 8.15 MB | 2026-01-28 07:16:20Z | 2026-02-17 22:02:20Z |
| wirewalker5.dat | 8.24 MB | 2026-02-27 14:03:30Z | 2026-03-10 01:21:30Z |
Each Campbell CR6 record contains WireWalker controller fields and a 1000-byte RBR buffer. The buffer can contain up to 125 RBR measurements. RBR timestamps are offset from Campbell record timestamps by one index. The decoder compares consecutive buffers because size_rbr was not logged accurately; changed bytes identify the number of new 8-byte RBR samples.
| ww_time 4 bytes | rbr_time 4 bytes | size_rbr 2 bytes | serial 1 byte | packet 1 byte | voltage 1 byte | current 1 byte |
| time elapsed 1 byte | pressure 3 bytes | conductivity 2 bytes | temperature 2 bytes |
| time elapsed 1 byte | pressure 3 bytes | conductivity 2 bytes | temperature 2 bytes |
| time elapsed 1 byte | pressure 3 bytes | conductivity 2 bytes | temperature 2 bytes |
RBR timestamps are offset from Campbell record timestamps by one index.
Multi-byte WireWalker and RBR integer fields are stored little-endian: the least-significant byte comes first. The 4-byte elapsed-time field and 2-byte conductivity and temperature fields are decoded as little-endian integers; the 3-byte pressure field is assembled least-significant byte first.
Each RBR sample stores elapsed time in the low byte of the raw value and pressure in the upper three bytes. The size_rbr field was not logged reliably. The parser compares the previous and current 1000-byte buffers, finds the changed byte range, and converts it into the number of new 8-byte RBR samples.
A real-time data stream was received through the Certus 9770 modem in addition to the recovered Campbell CR6 record set. Certus data return was assessed by counting unique RBR timestamps, so repeated telemetry packets are not counted more than once. The Certus stream provided near-real-time observations, while the recovered CR6 files provide the complete record set available after recovery.
L2_process.py decodes the raw binary records and coordinates the L2 pipeline. L2_quality_control.py computes pressure, temperature, conductivity, salinity, and up/down profile quality flags. L2 observations retain the native time sampling.
SAFARI_wirewalker_L2.nc is a CF-1.13 NetCDF-4 product assembled by L2_netCDF.py from the JSON metadata definitions. The inventory below is read from the generated file each time this README is published.
record_cr6=22842 time_rbr=2063569 platform=1. The product is NetCDF-4 with enabled for data variables (zlib, level 4, shuffle).
| Name | Type | Dimensions | Units | Long name / flags |
|---|---|---|---|---|
latitude | float32 | platform | degree_north | SAFARI surface mooring latitude |
longitude | float32 | platform | degree_east | SAFARI surface mooring longitude |
record_cr6 | uint32 | record_cr6 | Campbell CR6 datalogger record number | |
record_timestamp_cr6 | uint32 | record_cr6 | seconds since 1970-01-01 00:00:00Z | Campbell CR6 record reception time |
record_timestamp_controller | uint32 | record_cr6 | seconds since 1970-01-01 00:00:00Z | WireWalker controller record timestamp |
controller_current | float32 | record_cr6 | mA | WireWalker controller current draw |
controller_voltage | float32 | record_cr6 | volts | WireWalker controller supply voltage |
rbr_record | uint32 | time_rbr | Campbell CR6 record number associated with each RBR sample | |
time_rbr | uint32 | time_rbr | seconds since 1970-01-01 00:00:00Z | RBR Concerto CTD measurement time |
up_profile | float32 | time_rbr | Upward WireWalker profile identifier | |
down_profile | float32 | time_rbr | Downward WireWalker profile identifier | |
rbr_pressure | float32 | time_rbr | dbar | RBR Concerto CTD absolute pressure |
rbr_pressure_quality_flag | uint8 | time_rbr | RBR Concerto CTD pressure quality-control flag flags [0, 1]; bad good | |
rbr_depth | float32 | time_rbr | m | RBR Concerto CTD depth, positive downward |
rbr_conductivity | float32 | time_rbr | mS cm-1 | RBR Concerto CTD electrical conductivity |
rbr_conductivity_quality_flag | uint8 | time_rbr | RBR Concerto CTD conductivity quality-control flag flags [0, 1]; bad good | |
rbr_temperature | float32 | time_rbr | degree_Celsius | RBR Concerto CTD in-situ temperature |
rbr_temperature_quality_flag | uint8 | time_rbr | RBR Concerto CTD temperature quality-control flag flags [0, 1]; bad good | |
rbr_practical_salinity | float32 | time_rbr | 1 | RBR Concerto CTD practical salinity derived from conductivity, temperature, and pressure |
rbr_practical_salinity_quality_flag | uint8 | time_rbr | RBR Concerto CTD practical salinity quality-control flag flags [0, 1]; bad good |
| title | SAFARI 2025 WireWalker Level-2 CTD observations and quality-control flags |
|---|---|
| summary | The ONR SAFARI Departmental Research Initiative aims to improve our understanding of how air/sea interactions impact the timing, intensity, and impacts of atmospheric river events in the eastern Pacific Ocean. A joint Scripps Oceanographic and Woods Hole Oceanographic partnership has developed a capacity to observe surface fluxes and ocean boundary layer response through previous ONR support. The system combines an air/sea flux buoy (WHOI) with a WireWalker ocean-wave-powered profiling vehicle (SIO). For SAFARI, in consultation with the SAFARI DRI science team, a mooring location in the central mid-latitude Pacific (33°25′N, 158°W) was chosen to provide information for numerical models and to investigate basic questions regarding the influence of air/sea fluxes on atmospheric processes in the critical upstream region. This Level-2 dataset contains decoded WireWalker profile observations from an attached RBR Concerto CTD, including pressure, depth, temperature, conductivity, practical salinity, engineering measurements, and quality-control flags. The WireWalker was deployed alongside the surface mooring on 2025-11-21 at 00:40Z and recovered on 2026-09-08. Communication with the WireWalker was lost on 2026-03-09; consequently, the dataset may not represent the complete deployment period. |
| Conventions | CF-1.13 |
| netcdf_version | 4.9.2 |
| date_created | 2026-10-05T22:51:00Z |
| time_coverage_start | 2025-11-21T00:41:06Z |
| time_coverage_end | 2026-05-10T04:36:44Z |
| time_coverage_duration | P170D |
| time_coverage_resolution | P2S |
| geospatial_vertical_min | 8.810438055890591 |
| geospatial_vertical_max | 303.47246063603524 |
| geospatial_vertical_units | meters |
| geospatial_vertical_positive | down |
| license | CC BY-NC 4.0 |
QC flags use uint8 values with 0=bad and 1=good. Data-variable valid_min and valid_max attributes are calculated from QC-approved values; the vertical geospatial limits are likewise based on pressure-QC-approved depths.
Absolute pressure is screened first using the configured pressure range. Samples failing pressure QC are excluded from temperature, conductivity, and salinity QC.
Temperature, conductivity, and practical salinity use adaptive centered time-window envelopes. Candidate windows are 3, 8, and 16 hours. The shortest candidate window containing the equivalent of at least 1,000 measurements per hour is selected locally; longer windows are used during sparse periods. Broad seed limits exclude gross failures before percentile estimation.
| Variable | Seed range | Lower quantile | Upper quantile | Fixed margin |
|---|---|---|---|---|
| Temperature | 9.5 to 22.0 °C | 0.005 | 0.990 | ±0.07 °C |
| Conductivity | 35.0 to 52.0 mS cm-1 | 0.006 | 0.990 | ±0.2 mS cm-1 |
| Practical salinity | 33.5 to 35.0 | 0.015 | 0.992 | ±0.018 |
The resulting bounds are used for the gross envelope check. Measurements inside the envelope are then evaluated by the time-domain WireWalker spike tests described below.
Samples inside the envelope are tested with the causal WireWalker second-difference method and a bidirectional local-linear prediction test. Temperature, conductivity, and salinity use separate thresholds.
Practical salinity must pass its own envelope and spike tests and also requires both temperature and conductivity to pass QC.
The 24-hour QC and depth plots show blue kept observations and red removed observations. The full L2 time-series and depth plots show only QC-kept observations while retaining estimated envelopes where applicable. The direction plot selects a 12-hour window with more complete up/down profiles; up profiles are black and down profiles are purple. Engineering points use red centered one-hour means for voltage and current.
L3_process.py selects QC-approved upward L2 profiles and averages temperature, conductivity, and practical salinity into one-meter depth bins at two-hour and four-hour resolutions. The 2-hour bins preserve more short-term variability, while the 4-hour bins combine more profiles and generally provide more complete coverage.
The 1 m grid matches the typical sub-meter to approximately 1–2 m L2 vertical spacing, with a median spacing near 0.7 m. It is a grid interval rather than a claim of 1 m effective resolution; per-bin sample counts should be consulted.
The interpolated products use bounded local two-dimensional interpolation within five time bins on either side—up to 10 hours for the 2-hour product and up to 20 hours for the 4-hour product—and within 10 m vertically. Interpolation is restricted to overlapping sampled depth ranges, so large February–May gaps are not bridged and profiles are not extended above or below their sampled limits.
L3_netCDF.py assembles the compressed L3 CF-1.13 NetCDF-4 product from L3_variables.json and L3_globals.json. It includes binned and interpolated temperature, conductivity, practical salinity, sample counts, density, and mixed-layer-depth variables for both temporal resolutions.
time_2hr=2042 time_4hr=1021 depth_l3=290 platform=1. The product is NetCDF-4 with enabled for data variables (zlib, level 4, shuffle).
| Name | Type | Dimensions | Units | Long name / flags |
|---|---|---|---|---|
latitude | float32 | platform | degree_north | SAFARI surface mooring latitude |
longitude | float32 | platform | degree_east | SAFARI surface mooring longitude |
time_2hr | int64 | time_2hr | seconds since 1970-01-01 00:00:00Z | center time of 2-hour L3 averaging interval |
time_4hr | int64 | time_4hr | seconds since 1970-01-01 00:00:00Z | center time of 4-hour L3 averaging interval |
depth_l3 | float32 | depth_l3 | m | center depth of 1 m L3 averaging bin |
temperature_2hr_binned | float32 | time_2hr, depth_l3 | degree_Celsius | 2-hour mean QC-approved upward-profile in-situ temperature |
temperature_2hr_binned_interpolated | float32 | time_2hr, depth_l3 | degree_Celsius | 2-hour mean QC-approved upward-profile in-situ temperature with limited 2D interpolation |
temperature_4hr_binned | float32 | time_4hr, depth_l3 | degree_Celsius | 4-hour mean QC-approved upward-profile in-situ temperature |
temperature_4hr_binned_interpolated | float32 | time_4hr, depth_l3 | degree_Celsius | 4-hour mean QC-approved upward-profile in-situ temperature with limited 2D interpolation |
conductivity_2hr_binned | float32 | time_2hr, depth_l3 | mS cm-1 | 2-hour mean QC-approved upward-profile electrical conductivity |
conductivity_2hr_binned_interpolated | float32 | time_2hr, depth_l3 | mS cm-1 | 2-hour mean QC-approved upward-profile electrical conductivity with limited 2D interpolation |
conductivity_4hr_binned | float32 | time_4hr, depth_l3 | mS cm-1 | 4-hour mean QC-approved upward-profile electrical conductivity |
conductivity_4hr_binned_interpolated | float32 | time_4hr, depth_l3 | mS cm-1 | 4-hour mean QC-approved upward-profile electrical conductivity with limited 2D interpolation |
practical_salinity_2hr_binned | float32 | time_2hr, depth_l3 | 1 | 2-hour mean QC-approved upward-profile practical salinity |
practical_salinity_2hr_binned_interpolated | float32 | time_2hr, depth_l3 | 1 | 2-hour mean QC-approved upward-profile practical salinity with limited 2D interpolation |
practical_salinity_4hr_binned | float32 | time_4hr, depth_l3 | 1 | 4-hour mean QC-approved upward-profile practical salinity |
practical_salinity_4hr_binned_interpolated | float32 | time_4hr, depth_l3 | 1 | 4-hour mean QC-approved upward-profile practical salinity with limited 2D interpolation |
temperature_2hr_count | uint32 | time_2hr, depth_l3 | 1 | number of QC-approved upward-profile temperature samples in each 2-hour bin |
temperature_4hr_count | uint32 | time_4hr, depth_l3 | 1 | number of QC-approved upward-profile temperature samples in each 4-hour bin |
conductivity_2hr_count | uint32 | time_2hr, depth_l3 | 1 | number of QC-approved upward-profile conductivity samples in each 2-hour bin |
conductivity_4hr_count | uint32 | time_4hr, depth_l3 | 1 | number of QC-approved upward-profile conductivity samples in each 4-hour bin |
practical_salinity_2hr_count | uint32 | time_2hr, depth_l3 | 1 | number of QC-approved upward-profile salinity samples in each 2-hour bin |
practical_salinity_4hr_count | uint32 | time_4hr, depth_l3 | 1 | number of QC-approved upward-profile salinity samples in each 4-hour bin |
mixed_layer_depth_2hr_binned | float32 | time_2hr | m | estimated mixed-layer depth from 2-hour binned profiles relative to the shallowest sampled bin |
mixed_layer_depth_2hr_binned_interpolated | float32 | time_2hr | m | estimated mixed-layer depth from 2-hour interpolated profiles relative to the shallowest sampled bin |
mixed_layer_depth_4hr_binned | float32 | time_4hr | m | estimated mixed-layer depth from 4-hour binned profiles relative to the shallowest sampled bin |
mixed_layer_depth_4hr_binned_interpolated | float32 | time_4hr | m | estimated mixed-layer depth from 4-hour interpolated profiles relative to the shallowest sampled bin |
| title | SAFARI 2025 WireWalker Level-3 gridded upward-profile CTD observations |
|---|---|
| summary | This Level-3 dataset contains quality-controlled upward WireWalker profiles from the attached RBR Concerto CTD, averaged into one-meter depth bins at both two-hour and four-hour temporal resolutions. For each resolution, the product includes the original binned observations, a separate two-dimensional interpolation product limited to 10 m vertically and five time bins on either side, and per-bin sample counts. Variables include temperature, electrical conductivity, and practical salinity. |
| Conventions | CF-1.13 |
| netcdf_version | 4.9.2 |
| date_created | 2026-10-06T13:35:50Z |
| time_coverage_start | 2025-11-21T01:00:00Z |
| time_coverage_end | 2026-05-10T03:00:00Z |
| time_coverage_duration | P170D |
| time_coverage_resolution | PT2H |
| geospatial_vertical_min | 8.5 |
| geospatial_vertical_max | 297.5 |
| geospatial_vertical_units | meters |
| geospatial_vertical_positive | down |
| license | CC BY-NC 4.0 |
QC flags use uint8 values with 0=bad and 1=good. Data-variable valid_min and valid_max attributes are calculated from QC-approved values; the vertical geospatial limits are likewise based on pressure-QC-approved depths.
The L3 mixed-layer estimate uses TEOS-10 density from gsw. It identifies the dominant 5 m-smoothed density-gradient peak between 30 and 150 m, applies a minimum gradient threshold, and uses a gap-aware centered 24-hour temporal filter plus short-gap filling.
Binned and interpolated plots show temperature and practical salinity. Interpolated products include temperature and salinity contours. Separate interpolated density plots show temperature, density, and the mixed-layer estimate. The plots end on 10 March 2026, include daily minor time ticks, and show the NetCDF filename and creation date in the upper-right. Count plots show temperature and salinity coverage using fifteen positive-count bins with zero-count cells clear.