Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP noise annotation refactor #69

Draft
wants to merge 57 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
26213e5
Merge pull request #3 from opera-adt/main
seongsujeong Aug 8, 2022
8fddf15
Merge branch 'main' of github.com:opera-adt/s1-reader
Aug 9, 2022
dffe813
Replacing the loaders in Burst* class into class methods, with furthe…
Aug 10, 2022
0ec4fc4
fix on determining beta_naught; addressing PEP8 issues
Aug 10, 2022
b4d3c0e
Bug fix and feature addition to BurstEAP; restructuring LUT exportation
Aug 20, 2022
6d11d3d
Readibility improvement; removing unnecessary imports
Aug 24, 2022
460eecb
Merge branch 'main' of github.com:opera-adt/s1-reader
Aug 24, 2022
142ece4
Format change on `burst_id`; keeping the absolute orbit number inside…
Aug 24, 2022
bae4227
updates on test_bursts.py
Aug 25, 2022
5eb971b
Merge branch 'main' of github.com:opera-adt/s1-reader into burst_id_a…
Aug 25, 2022
45662ce
Merge branch 'burst_id_and_abs_orbit' into correction_and_calibration
Aug 25, 2022
83454cb
keeping the basename of the CADS and NADS for populating RTC metadata
Aug 25, 2022
b29ffe2
Update src/s1reader/s1_annotation.py
seongsujeong Aug 28, 2022
2ae4fa2
Update src/s1reader/s1_annotation.py
seongsujeong Aug 28, 2022
79a4f20
Update src/s1reader/s1_annotation.py
seongsujeong Aug 28, 2022
79032a4
Update src/s1reader/s1_annotation.py
seongsujeong Aug 28, 2022
de49003
Update src/s1reader/s1_burst_slc.py
seongsujeong Aug 28, 2022
9013e4f
Update src/s1reader/s1_burst_slc.py
seongsujeong Aug 28, 2022
64f34e0
Update src/s1reader/s1_burst_slc.py
seongsujeong Aug 28, 2022
e7df7b0
Update src/s1reader/s1_burst_slc.py
seongsujeong Aug 28, 2022
389aa53
Merge pull request #5 from opera-adt/main
seongsujeong Aug 28, 2022
bf029cc
Update src/s1reader/s1_burst_slc.py
seongsujeong Aug 28, 2022
b3f3612
Update src/s1reader/s1_burst_slc.py
seongsujeong Aug 28, 2022
f0b0b82
Update src/s1reader/s1_burst_slc.py
seongsujeong Aug 28, 2022
4109740
Update src/s1reader/s1_burst_slc.py
seongsujeong Aug 28, 2022
6d62feb
addressing comments bt @LiangJYu
Aug 28, 2022
df2b371
Merge branch 'correction_and_calibration' of github.com:seongsujeong/…
Aug 28, 2022
1af92bf
docstring fix; variables renamed for clarity
Aug 28, 2022
d9a76d9
implemented s1_annotation.AucCal.load_from_zip_file()
Aug 28, 2022
3504967
readability improvement
Aug 28, 2022
6b2a07a
s1_annotation.py - code cleanup; excention handling for AUX_CAL; PEP8…
Aug 28, 2022
d5cd5de
docstring for `s1_burst_slc.eap_compensation_lut()`
Aug 28, 2022
db59419
class import scheme changed
Aug 28, 2022
e908892
PEP8 compliance
Aug 28, 2022
a9149e1
Update src/s1reader/s1_annotation.py
seongsujeong Aug 31, 2022
7fb70a4
Update src/s1reader/s1_annotation.py
seongsujeong Aug 31, 2022
53ffce4
Update src/s1reader/s1_annotation.py
seongsujeong Aug 31, 2022
0700e74
Update src/s1reader/s1_reader.py
seongsujeong Aug 31, 2022
fe8252e
Update src/s1reader/s1_reader.py
seongsujeong Aug 31, 2022
90c09bd
command and variable name revised for clarity
Aug 31, 2022
d87c234
Raise error when `burst_noise` is not defined but user attempts to re…
Aug 31, 2022
08acee0
fixing variable names that causing CircieCI to fail
Aug 31, 2022
27f6998
LUT retrieval scheme changed; updates in accordance with the comments…
Sep 6, 2022
637a7d5
Changing the length unit in BurstEAP._anx2roll() to [m]
Sep 6, 2022
9c13a08
change on variable name for clarity
Sep 6, 2022
265a6c2
Addressing PEP8 issue (R0201)
Sep 13, 2022
4aac153
Reverting the unit in `_anx2roll()` into the original documentataion;…
Sep 13, 2022
1b7f805
Update src/s1reader/s1_annotation.py
seongsujeong Sep 14, 2022
14e046c
Update src/s1reader/s1_annotation.py
seongsujeong Sep 14, 2022
69072d9
Update src/s1reader/s1_annotation.py
seongsujeong Sep 14, 2022
de4096e
Update src/s1reader/s1_annotation.py
seongsujeong Sep 14, 2022
663c735
Docstring clarification
Sep 14, 2022
90aa6d7
Addressing PEP8 C0301
Sep 14, 2022
2ce289a
AuxCal.get_aux_cal_instrument_config_id() - check the instrument conf…
Sep 14, 2022
2913b29
get_path_aux_cal() - check if the instrument configuration ID matches
Sep 14, 2022
f019bae
untested refactor to isolate noise loading from annotation
LiangJYu Sep 21, 2022
2a0c138
Merge branch 'main' into noise_refactor
LiangJYu Oct 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
237 changes: 0 additions & 237 deletions src/s1reader/s1_annotation.py
Original file line number Diff line number Diff line change
Expand Up @@ -201,114 +201,6 @@ def from_et(cls, et_in: ET, path_annotation: str):
return cls


@dataclass
class NoiseAnnotation(AnnotationBase):
'''
Reader for Noise Annotation Data Set (NADS) for IW SLC
Based on ESA documentation: "Thermal Denoising of Products Generated by the S-1 IPF"
'''

basename_annotation: str
rg_list_azimuth_time: np.ndarray
rg_list_line: list
rg_list_pixel: list
rg_list_noise_range_lut: list
az_first_azimuth_line: int
az_first_range_sample: int
az_last_azimuth_line: int
az_last_range_sample: int
az_line: np.ndarray
az_noise_azimuth_lut: np.ndarray

@classmethod
def from_et(cls,et_in: ET, ipf_version: version.Version, path_annotation: str):
'''
Extracts list of noise information from etree

Parameter
----------
et_in : xml.etree.ElementTree
Parsed NADS annotation .xml

Return
-------
cls: NoiseAnnotation
Parsed NADS from et_in
'''

cls.xml_et = et_in
cls.basename_annotation = os.path.basename(path_annotation)

if ipf_version < min_ipf_version_az_noise_vector: # legacy SAFE data
cls.rg_list_azimuth_time = \
cls._parse_vectorlist('noiseVectorList',
'azimuthTime',
'datetime')
cls.rg_list_line = \
cls._parse_vectorlist('noiseVectorList',
'line',
'scalar_int')
cls.rg_list_pixel = \
cls._parse_vectorlist('noiseVectorList',
'pixel',
'vector_int')
cls.rg_list_noise_range_lut = \
cls._parse_vectorlist('noiseVectorList',
'noiseLut',
'vector_float')

cls.az_first_azimuth_line = None
cls.az_first_range_sample = None
cls.az_last_azimuth_line = None
cls.az_last_range_sample = None
cls.az_line = None
cls.az_noise_azimuth_lut = None

else:
cls.rg_list_azimuth_time = \
cls._parse_vectorlist('noiseRangeVectorList',
'azimuthTime',
'datetime')
cls.rg_list_line = \
cls._parse_vectorlist('noiseRangeVectorList',
'line',
'scalar_int')
cls.rg_list_pixel = \
cls._parse_vectorlist('noiseRangeVectorList',
'pixel',
'vector_int')
cls.rg_list_noise_range_lut = \
cls._parse_vectorlist('noiseRangeVectorList',
'noiseRangeLut',
'vector_float')
cls.az_first_azimuth_line = \
cls._parse_vectorlist('noiseAzimuthVectorList',
'firstAzimuthLine',
'scalar_int')[0]
cls.az_first_range_sample = \
cls._parse_vectorlist('noiseAzimuthVectorList',
'firstRangeSample',
'scalar_int')[0]
cls.az_last_azimuth_line = \
cls._parse_vectorlist('noiseAzimuthVectorList',
'lastAzimuthLine',
'scalar_int')[0]
cls.az_last_range_sample = \
cls._parse_vectorlist('noiseAzimuthVectorList',
'lastRangeSample',
'scalar_int')[0]
cls.az_line = \
cls._parse_vectorlist('noiseAzimuthVectorList',
'line',
'vector_int')[0]
cls.az_noise_azimuth_lut = \
cls._parse_vectorlist('noiseAzimuthVectorList',
'noiseAzimuthLut',
'vector_float')[0]

return cls


@dataclass
class ProductAnnotation(AnnotationBase):
'''
Expand Down Expand Up @@ -535,135 +427,6 @@ def closest_block_to_azimuth_time(vector_azimuth_time: np.ndarray,
return np.argmin(np.abs(vector_azimuth_time-azimuth_time_burst))


@dataclass
class BurstNoise:
'''Noise correction information for Sentinel-1 burst'''
basename_nads: str
range_azimith_time: datetime.datetime
range_line: float
range_pixel: np.ndarray
range_lut: np.ndarray

azimuth_first_azimuth_line: int
azimuth_first_range_sample: int
azimuth_last_azimuth_line: int
azimuth_last_range_sample: int
azimuth_line: np.ndarray
azimuth_lut: np.ndarray

line_from: int
line_to: int

@classmethod
def from_noise_annotation(cls, noise_annotation: NoiseAnnotation,
azimuth_time: datetime.datetime,
line_from: int,
line_to: int,
ipf_version: version.Version):
'''
Extracts the noise correction information for individual burst from NoiseAnnotation

Parameters
----------
noise_annotation: NoiseAnnotation
Subswath-wide noise annotation information
azimuth_time : datetime.datetime
Azimuth time of the burst
line_from: int
First line of the burst in the subswath
line_to: int
Last line of the burst in the subswath
ipf_version: float
IPF version of the SAFE data

Returns
-------
cls: BurstNoise
Instance of BurstNoise initialized by the input parameters

'''

basename_nads = noise_annotation.basename_annotation
id_closest = closest_block_to_azimuth_time(noise_annotation.rg_list_azimuth_time,
azimuth_time)

range_azimith_time = noise_annotation.rg_list_azimuth_time[id_closest]
range_line = noise_annotation.rg_list_line[id_closest]
range_pixel = noise_annotation.rg_list_pixel[id_closest]
range_lut = noise_annotation.rg_list_noise_range_lut[id_closest]

azimuth_first_azimuth_line = noise_annotation.az_first_azimuth_line
azimuth_first_range_sample = noise_annotation.az_first_range_sample
azimuth_last_azimuth_line = noise_annotation.az_last_azimuth_line
azimuth_last_range_sample = noise_annotation.az_last_range_sample

if ipf_version >= min_ipf_version_az_noise_vector:
# Azimuth noise LUT exists - crop to the extent of the burst
id_top = np.argmin(np.abs(noise_annotation.az_line-line_from))
id_bottom = np.argmin(np.abs(noise_annotation.az_line-line_to))

# put some margin when possible
if id_top > 0:
id_top -= 1
if id_bottom < len(noise_annotation.az_line)-1:
id_bottom += 1
azimuth_line = noise_annotation.az_line[id_top:id_bottom + 1]
azimuth_lut = noise_annotation.az_noise_azimuth_lut[id_top:id_bottom + 1]

else:
azimuth_line = None
azimuth_lut = None

return cls(basename_nads, range_azimith_time, range_line, range_pixel, range_lut,
azimuth_first_azimuth_line, azimuth_first_range_sample,
azimuth_last_azimuth_line, azimuth_last_range_sample,
azimuth_line, azimuth_lut,
line_from, line_to)


def compute_thermal_noise_lut(self, shape_lut):
'''
Calculate thermal noise LUT whose shape is `shape_lut`

Parameter:
----------
shape_lut: tuple or list
Shape of the output LUT

Returns
-------
arr_lut_total: np.ndarray
2d array containing thermal noise correction look up table values
'''

nrows, ncols = shape_lut

# Interpolate the range noise vector
rg_lut_interp_obj = InterpolatedUnivariateSpline(self.range_pixel,
self.range_lut,
k=1)
if self.azimuth_last_range_sample is not None:
vec_rg = np.arange(self.azimuth_last_range_sample + 1)
else:
vec_rg = np.arange(ncols)
rg_lut_interpolated = rg_lut_interp_obj(vec_rg)

# Interpolate the azimuth noise vector
if (self.azimuth_line is None) or (self.azimuth_lut is None):
az_lut_interpolated = np.ones(nrows)
else: # IPF >= 2.90
az_lut_interp_obj = InterpolatedUnivariateSpline(self.azimuth_line,
self.azimuth_lut,
k=1)
vec_az = np.arange(self.line_from, self.line_to + 1)
az_lut_interpolated = az_lut_interp_obj(vec_az)

arr_lut_total = np.matmul(az_lut_interpolated[..., np.newaxis],
rg_lut_interpolated[np.newaxis, ...])

return arr_lut_total


@dataclass
class BurstCalibration:
'''Calibration information for Sentinel-1 IW SLC burst
Expand Down
10 changes: 10 additions & 0 deletions src/s1reader/s1_burst_slc.py
Original file line number Diff line number Diff line change
Expand Up @@ -638,6 +638,16 @@ def thermal_noise_lut(self):

return self.burst_noise.compute_thermal_noise_lut(self.shape)

@property
def thermal_noise_lut(self):
'''
Returns the LUT for thermal noise correction for the burst
'''
if self.burst_noise is None:
raise ValueError('burst_noise is not defined for this burst.')

return self.burst_noise.compute_thermal_noise_lut(self.shape)

@property
def eap_compensation_lut(self):
'''Returns LUT for EAP compensation.
Expand Down
Loading