Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: SPEC0 test env, clean up python 3.6 #250

Draft
wants to merge 14 commits into
base: develop
Choose a base branch
from
28 changes: 21 additions & 7 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,25 @@ jobs:
fail-fast: false
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.11", "3.12"]
python-version: ["3.12", "3.13"]
numpy_ver: ["latest"]
test_config: ["latest"]
include:
- python-version: "3.10"
- test_config: "SPEC0"
python-version: "3.11"
numpy_ver: "1.24"
pandas_ver: "2.0"
scipy_ver: "1.10"
xarray_ver: "2022.12.0"
cdasws_ver: "1.7.42"
cdflib_ver: "1.0.4"
netcdf_ver: "1.6"
pysat_ver: "3.2"
os: ubuntu-latest
test_config: "NEP29"
- python-version: "3.9"
- test_config: "Ops"
python-version: "3.9"
numpy_ver: "1.23.5"
os: "ubuntu-20.04"
test_config: "Ops"

name: Python ${{ matrix.python-version }} on ${{ matrix.os }} with numpy ${{ matrix.numpy_ver }}
runs-on: ${{ matrix.os }}
Expand All @@ -43,10 +50,17 @@ jobs:
pip install --no-cache-dir numpy==${{ matrix.numpy_ver }}
pip install --upgrade-strategy only-if-needed .[test]

- name: Install NEP29 dependencies
if: ${{ matrix.test_config == 'NEP29'}}
- name: Install SPEC0 dependencies
if: ${{ matrix.test_config == 'SPEC0'}}
run: |
pip install numpy==${{ matrix.numpy_ver }}
pip install pandas==${{ matrix.pandas_ver }}
pip install scipy==${{ matrix.scipy_ver }}
pip install xarray==${{ matrix.xarray_ver }}
pip install cdasws==${{ matrix.cdasws_ver }}
pip install cdflib==${{ matrix.cdflib_ver }}
pip install netCDF4==${{ matrix.netcdf_ver }}
pip install pysat==${{ matrix.pysat_ver }}
pip install --upgrade-strategy only-if-needed .[test]

- name: Install standard dependencies
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ This project adheres to [Semantic Versioning](https://semver.org/).
## [0.1.0] - 2025-XX-XX
* New Instruments
* Mars Global Surveyor Magnetometer (MGS Mag)
* Maintenance
* Implement SPEC0 tests in main workflow

## [0.0.6] - 2024-10-03
* New Instruments
Expand Down
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
[![PyPI Package latest release](https://img.shields.io/pypi/v/pysatNASA.svg)](https://pypi.python.org/pypi/pysatNASA)
[![Build Status](https://github.com/github/docs/actions/workflows/main.yml/badge.svg)](https://github.com/github/docs/actions/workflows/main.yml/badge.svg)
[![Coverage Status](https://coveralls.io/repos/github/pysat/pysatNASA/badge.svg?branch=main)](https://coveralls.io/github/pysat/pysatNASA?branch=main)
[![SPEC 0 — Minimum Supported Dependencies](https://img.shields.io/badge/SPEC-0-green?labelColor=%23004811&color=%235CA038)](https://scientific-python.org/specs/spec-0000/)


[![Documentation Status](https://readthedocs.org/projects/pysatnasa/badge/?version=latest)](https://pysatnasa.readthedocs.io/en/latest/?badge=latest)
[![DOI](https://zenodo.org/badge/287387638.svg)](https://zenodo.org/badge/latestdoi/287387638)
Expand All @@ -20,17 +22,17 @@ some examples on how to use the routines

pysatNASA uses common Python modules, as well as modules developed by
and for the Space Physics community. This module officially supports
Python 3.6+.
Python 3.9+.

| Common modules | Community modules | Optional Modules |
| ---------------- | ----------------- |------------------|
| beautifulsoup4 | cdflib>=0.4.4 | pysatCDF |
| lxml | pysat>=3.1.0 | |
| netCDF4 | | |
| beautifulsoup4 | cdasws>=1.7.42 | pysatCDF |
| lxml | cdflib>=1.0.4 | |
| netCDF4>=1.6 | pysat>=3.2.0 | |
| numpy | | |
| pandas | | |
| requests | | |
| scipy>=1.4.0 | | |
| scipy>=1.10.0 | | |
| xarray | | |

## PyPi Installation
Expand All @@ -54,7 +56,7 @@ pip install .

Note: pre-1.0.0 version
-----------------------
pysatNASA is currently in an initial development phase and requires pysat 3.1.0.
pysatNASA is currently in an initial development phase and requires pysat 3.2.0.
Feedback and contributions are appreciated.

# Using with pysat
Expand All @@ -68,7 +70,7 @@ from pysatNASA.instruments import icon_ivm
ivm = pysat.Instrument(inst_module=icon_ivm, inst_id='a')
```
Another way to use the instruments in an external repository is to register the
instruments. This only needs to be done the first time you load an instrument.
instruments. This only needs to be done the first time you load an instrument.
Afterward, pysat will identify them using the `platform` and `name` keywords.

```
Expand All @@ -81,10 +83,10 @@ ivm = pysat.Instrument('icon', 'ivm', inst_id='a')
# CDF Integration
For data products stored as CDF files, this package can use either `cdflib` or
`pysatCDF`. Note that `cdflib` is a pure python package and more readily
deployable across systems, whereas `pysatCDF` interfaces with the fortran.
This is a faster approach for loading data, but may not install on all systems.
deployable across systems, whereas `pysatCDF` interfaces with the fortran.
This is a faster approach for loading data, but may not install on all systems.
There are known issues with `numpy`>=1.24. Therefore, `pysatCDF` is optional
rather than required.
rather than required.

You can specify which load routine to use via the optional `use_cdflib` kwarg.
If no kwarg is specified, `pysatNASA` will default to `pysatCDF` if it is
Expand Down
10 changes: 5 additions & 5 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,18 @@ Prerequisites

pysatNASA uses common Python modules, as well as modules developed by
and for the Space Physics community. This module officially supports
Python 3.6+ and pysat 3.1.0+.
Python 3.9+ and pysat 3.2.0+.

================== =================
Common modules Community modules
================== =================
beautifulsoup4 cdflib>=0.4.4
lxml pysat>=3.1.0
netCDF4
beautifulsoup4 cdasws>=1.7.42
lxml cdflib>=1.0.4
netCDF4>=1.6.0 pysat>=3.2.0
numpy
pandas
requests
scipy>=1.4.0
scipy>=1.10.0
xarray
================== =================

Expand Down
14 changes: 7 additions & 7 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ name = "pysatNASA"
version = "0.0.6"
description = "pysat support for NASA Instruments"
readme = "README.md"
requires-python = ">=3.6"
requires-python = ">=3.9"
license = {file = "LICENSE"}
authors = [
{name = "Jeff Klenzing, et al.", email = "[email protected]"},
Expand All @@ -21,10 +21,10 @@ classifiers = [
"License :: OSI Approved :: BSD License",
"Natural Language :: English",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Operating System :: POSIX :: Linux",
"Operating System :: MacOS :: MacOS X",
"Operating System :: Microsoft :: Windows"
Expand All @@ -38,15 +38,15 @@ keywords = [
]
dependencies = [
"beautifulsoup4",
"cdasws",
"cdflib >= 0.4.4",
"cdasws >= 1.7.42",
"cdflib >= 1.0.4",
"lxml",
"netCDF4",
"netCDF4 >= 1.6",
"numpy",
"pandas",
"pysat >= 3.2",
"requests",
"scipy >= 1.4",
"scipy >= 1.10",
"xarray"
]

Expand Down
5 changes: 1 addition & 4 deletions pysatNASA/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,7 @@

"""

try:
from importlib import metadata
except ImportError:
import importlib_metadata as metadata
from importlib import metadata

import os

Expand Down
30 changes: 6 additions & 24 deletions pysatNASA/instruments/methods/_cdf.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,13 +100,8 @@ def __init__(self, filename,
self.meta = {}
self._dependencies = {}

if hasattr(self._cdf_info, 'rVariables'):
self._variable_names = (self._cdf_info.rVariables
+ self._cdf_info.zVariables)
else:
# cdflib < 1.0 stores info as a dict
self._variable_names = (self._cdf_info['rVariables']
+ self._cdf_info['zVariables'])
self._variable_names = (self._cdf_info.rVariables
+ self._cdf_info.zVariables)

self.load_variables()

Expand Down Expand Up @@ -170,13 +165,8 @@ def set_epoch(self, x_axis_var):

"""

if hasattr(self._cdf_file.varinq(x_axis_var), 'Data_Type_Description'):
data_type_description = self._cdf_file.varinq(
x_axis_var).Data_Type_Description
else:
# cdflib < 1.0 stores this as a dict
data_type_description = self._cdf_file.varinq(
x_axis_var)['Data_Type_Description']
data_type_description = self._cdf_file.varinq(
x_axis_var).Data_Type_Description

center_measurement = self._center_measurement
cdf_file = self._cdf_file
Expand Down Expand Up @@ -317,11 +307,7 @@ def load_variables(self):
if not re.match(var_regex, variable_name):
# Skip this variable
continue
try:
var_atts = self._cdf_file.varattsget(variable_name, to_np=True)
except TypeError:
# cdflib 1.0+ drops to_np kwarg, assumes True
var_atts = self._cdf_file.varattsget(variable_name)
var_atts = self._cdf_file.varattsget(variable_name)

for k in var_atts:
var_atts[k] = var_atts[k] # [0]
Expand All @@ -343,11 +329,7 @@ def load_variables(self):
continue

if "FILLVAL" in var_atts:
if hasattr(var_properties, 'Data_Type_Description'):
data_type_desc = var_properties.Data_Type_Description
else:
# cdflib < 1.0 stores this as a dict
data_type_desc = var_properties['Data_Type_Description']
data_type_desc = var_properties.Data_Type_Description

if data_type_desc in ['CDF_FLOAT', 'CDF_REAL4', 'CDF_DOUBLE',
'CDF_REAL8']:
Expand Down
7 changes: 1 addition & 6 deletions pysatNASA/instruments/methods/cdaweb.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,7 @@
from pysat.utils import io
from pysatNASA.instruments.methods import CDF as libCDF

try:
# cdflib 1.0 syntax
from cdflib.xarray import cdf_to_xarray
except ModuleNotFoundError:
# cdflib 0.4 syntax required for backwards compatibility
from cdflib import cdf_to_xarray
from cdflib.xarray import cdf_to_xarray

try:
# Use pysatCDF as default for pandas data sets
Expand Down
4 changes: 2 additions & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
beautifulsoup4
cdasws
cdflib>=0.4.4
cdflib>=1.0.2
lxml
netCDF4
numpy
pandas
pysat>=3.1.0
pysat>=3.2.0
requests
xarray
6 changes: 0 additions & 6 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -1,9 +1,3 @@
# name and version must be maintained here as well for python 3.6 compatibility

[metadata]
name = pysatNASA
version = 0.0.6

[flake8]
max-line-length = 80
ignore =
Expand Down
Loading