Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Enhanced SUNA Data Processing and Quality Control #357

Merged
merged 23 commits into from
Nov 12, 2024

Conversation

JiaxuZ
Copy link
Collaborator

@JiaxuZ JiaxuZ commented Nov 8, 2024

This pull request introduces several key enhancements and updates to the SUNA data processing pipeline within the EcoFOCIpy package. Major improvements include temperature and pressure corrections, advanced quality control based on RMSE, and integration of shipboard nitrate measurements for absolute value correction.

Summary of Source Code Changes:

  • Temperature and Pressure Corrections: Implemented temperature and pressure correction algorithms for SUNA measurements.
  • Quality Control Enhancement: Introduced multi-level QC based on the newly generated RMSE.
  • Calibration with Shipboard Nitrate Measurements: Added functionality to adjust absolute values using shipboard nitrate data, with step-by-step instructions for manual QC integration.

Additional Changes in the Jupyter Notebook Example:

  • Nearest CTD Cast Selection: Optimized the process for finding and using the nearest CTD cast data based on horizontal distance and depth matching.
  • Improved Plotting Functions: Modularized and enhanced plotting functions for better visualization of SUNA data across various correction and QC stages.
  • Documentation Updates: Added and polished documentation, including examples for using user-provided calibration files and interpreting results from multi-level QC.

Notes:

  • This PR includes merging the latest updates from main into feature/suna to ensure compatibility and consistency with recent changes in the primary branch.
  • Additional testing was conducted to confirm the accuracy of the corrections and the robustness of the QC pipeline.

Future Work:

  • Further development for data output following the EcoFOCI data formatting.
  • Integration of additional calibration files.

Please review the changes and provide feedback.

JiaxuZ added 23 commits July 19, 2024 12:13
Update requirements files to keep up with main.
Merge main into feature/suna to keep up with necessary package updates.
@shaunwbell shaunwbell merged commit 7270057 into main Nov 12, 2024
9 of 12 checks passed
@shaunwbell shaunwbell deleted the feature/suna branch November 12, 2024 18:14
@shaunwbell
Copy link
Collaborator

I think this looks great @JiaxuZ and I will accept it as is. Some very simple code-checks are failing (like import order of libraries) but these will be easy to clean on my end as I have some other similar tasks to do to the codebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants