Merged
Conversation
Member
yunjunz
commented
May 23, 2017
- consistent naming style for all data files
- more generalized load-data
- introduce coherence-based network modification
- referencing data in space and reading modified network while inverting network to timeseries
- introduce auto referencing in time
- introduce auto turbulence detection using timeseries residual standard deviation
- full template list for pysarApp.py - pysarApp_template.txt
- new progress bar
- new time-series viewer python script
- updated scripts for time-series web viewer
- bug fixed for UNAVCO InSAR Archive output
- centralized code for parallel computing check
- bugs fixed
- improved plotting scripts
1. move code of drawing lat/lon label from plot_matrix() to independent function draw_lalo_label() inside class Basemap2 2. minor adjustment to Basemap2.drawscale()
rename —footprint option to —tight option, and setup output filename to file*_tight.ext, instead of a prefix of ‘subset_’
1. bug fixed converting ref_y/x to ref_lat/lon while geocoding 2. bug fixed while convert radar2geo coord with one point, instead of multiply
write full attribute into rsc file before calling geocode.pl from ROIPAC for geocoding, so less warning for missing attributes update Y/XMIN/MAX in attributes for output file
add —coastline option to plot coastline, along with —resolution option to specify the boundary database resolution used in coastline plotting. Add map_group in input argument parsing, to group —coastline, —resolution, —scalebar and —noscalebar options.
add qr code file for pysar github page and um insarmaps
Short URL: q-r.to/pysar Target URL: yunjunz.github.io/PySAR/ Short URL: q-r.to/insarmaps Target URL: insarmaps.miami.edu
1. add auto figure size in view.py and save_kml.py; by default, it calculate a figure size proportional to data shape, and constrain its shorter dimension to 8 inches. 2. update auto_row_col_num() in view.py with better handling of calculation residual for multiple display
…rmaps; add functionality in json_mbtiles2insarmaps to upload mbtiles and json files
… scripts; began work on add_plot_attribute function; remove unnecessary check for table name in add_attribute
__init__.py: add figsize_single_max variable for max size _datetime.py: 1. move progress_bar class from _pysar_utilities.py to _datetime.py; and adjust corresponding scripts 2. return start and end date in auto_adjust_xaxis_date() _network.py: add ymin/max argument in auto_adjust_yaxis() _pysar_utilities.py 1. add function timeseries_std() and timeseries_coherence() to calculate standard deviation and phase coherence in space for each epoch of timeseries 2. add function get_residual_std() to calculate deramped phase residual standard deviation and write it to text file 3. check is inFile exists in update_file() dem_error.py: output phase inversion residual to a new timeseries file pysarApp.py: 1. constrain input subset box in radar coord within the data coverage 2. add new pysar options in the template pysar.residualStd.maskFile - mask file used for residual std calculation pysar.residualStd.ramp - ramp type pysar.residualStd.minStd - threshold for turbulence detection pysar.velocity.excludeDate - customized text file for excluded dates in linear velocity estimation 3. plot all command line even they are not ran 4. use tropospheric delay file if exists instead of call pyaps again 5. calculate linear velocity from tropospheric delay file 6. add new step: calculate timeseries residual standard deviation 7. add new step: automatic referencing in time using min residual std reference_epoch.py 1. use argparse to replace get opt for input argument parsing 2. add —ref-date auto option to find reference date automatically based on residual standard deviation new script - timeseries_std.py to calculate standard deviation of deramped phase residual for each epoch of timeseries timeseries2velocity.py 1. rename -m/M option to —start-date and —end-date option, add —template option to use pysar.velocity.excludeDate option 2. remove obsolete function yyyymmdd2years(), call pysar._datetime module instead 3. add function read_template2inps() to read template options into Namespace object 4. add get_exclude_date() to read all date related options 5. add get_velocity_filename() to get output velocity filename based on input initial name, and with/without exclude date
tropcor_pyaps.py: add support to read bl_list.txt file to get list of date for weather re-analysis data downloading use —exclude by default in select_network.py and view.py
Introduce a default, full template file - pysarApp_template.txt - to pysarApp.py. With parameter setting for all steps run within pysarApp.py. For the first time running, pysarApp.py will generate this txt file with all default settings unless it already exists. If custom template, like KujuAlosAT422F650.template is input, script will read the value from it and update it to the default template file. use pysarApp.py -g to generate/check default template and exit, similar to run -g for Doris. _pysar_utilities.py 1. add function update_template_file() to write template new value into an organized existing template file 2. add function get_spatial_average() to get spatial average list of input file. And change output file extension from .list to .txt _readfile.py: modify read_template() to not read option with empty value load_data.py: adjust script for full template list and the auto value modify_network.py: 1. adjust script for full template list and auto value 2. add —reset option to select all group within input file - drop_ifgram=no 3. support maskAoi.yx/lalo to change area of pixels for average coherence calculation pysarApp.py add full template list with -g option to stop after template reading reference_epoch.py: do not write date to reference_date.txt file and support full template list with auto value seed_data.py: support full template list
_pysar_utilities.py: add multiple input files - list - support for update_file(), and adjust corresponding function usage in _pysar_utilities.py and pysarApp.py diff.py: do not exit if all groups of file1 exists within file2 for timeseries file type multilook.py: 1. fix bug while multi looking mapping transformation file 2. add print_message option for multilook_attribute(), so that no message print while multi looking multi_group files
_network.py 1. add coherence_matrix() to calculate 2D coherence matrix given date12_list and coherence_list 2. add threshold_coherence_based_mst() to return a minimum spanning tree of network of interferograms with inverse of coherence as weight, given date12_list and coherence_list 3. add plot_coherence_matrix() and plot_coherence_history() for plotting 4. modify plot_network() to find color jump based on input date12_list to keep and drop, so that all dropped interferograms shown as red dash line and all kept interferograms shown as solid blue line; change min coherence to display from 0.4 to 0.2 modify_network.py: add MST constrain if using coherence-based network modification, so that network will not become non-fully connected, because of coherence thresholding. plot_network.py: plot two new figures - CoherenceMatrix.pdf and CoherenceHistory.pdf pysarApp.py: add comment about coherence-based network with MST constrain. timeseries_std.py: adjust bar width using common mode of sampling length in time
1. file permission adjustment for _gmt.py and insarmaps_query.py 2. do not plot CoherenceMatrix and CoherenceHistory if no coherence info input 3. tropcor_pyaps.py: get ride of --timeseries option name for timeseries input
…or correction performance 1. rename timeseries_std.py to timeseries_rms.py and calculate timeseries_ECMWF_demErrInvResid_quadratic.h5's RMS instead of STD. Test shows that both of them are the same for deramped timeseries, but RMS are more sentative than STD if timeseries is not deramped, and RMS is more clear in definition. Code is updated in pysarApp.py, _pysar_utilities.py, timeseries_rms.py 2. bug fixed for seed_data.py while using pysar.reference.lalo option with template input 3. use default maskTempCoh.h5 mask in view.py for velocity, timeseries, rmse and interferograms.
add pysarApp.py --subset-data option to create a subset dataset of all required files easily, so that the newly created SUBSET_X*Y* folder could be shared and re-processed. add ROIPAC file support for add_attribute.py, and if new attribute's value is None, drop this item from attribute dictionary reverse back the multilooking for geomap*.trans file, do not divide its value with number of multilooks
add list_ifgram2date12() in _datetime.py to convert ifgram list into date12_list, and adjust corresponding script, mainly the one use this info to shown suffix in progress bar consider difference reference date for timeseries file input of diff.py check corrupted hdf5 file in load_data.py more robust tight croping in pysarApp.py add --exclude option in tsviewer.py to show excluded date as gray dot bug fixed for view.py when input exclude date for timeseries file
fix bug in load_data.py introduced by the full template txt file in pysarApp.py do not tight subset DEM in geo coord if it's not exists in pysarApp.py
1. add pysar.topoError.stepFuncDate option to specify the date of step jump, i.e. date of earthquake or volcanic eruption, to the temporal deformation model; more robust for complicated deformation dataset 2. add pysar.topoError.excludeDate option to exclude date(s) for error estimation, which usually are streamely strong ionosphere 3. add --template option to dem_error.py for input argument, and adjust corresponding code in pysarApp.py
_network.py: 1. fix bug with MST date12 list exchanged master/slave date 2. add coh_thres to specify color jump/cut in network plot with coherence 3. bug fixed with coloring in plot_network() 4. add disp_title option diff.py: support differencing of two timeseries files with different reference date and reference pixel plot_network.py: enable --figsize option save_kml.py: do not read 'X/Y_UNIT', make it more robust
1. add new script prep_gamma.py to prepare a .rsc file for Gamma product. It contains the following functions: a. def attribute_gamma2roipac() to convert exsiting GAMMA attribute to ROIPAC attribute b. def get_perp_baseline() to calculate perpendicular baseline based on master/slave par file and interferogram off file, using Gamma command base_orbit and base_perp c. def get_lalo_ref() to calculate lat/lon of interferogram's four corners based on master par file, using Gamma command SLC_corners d. def extract_attribute() to extract the whole attribute based on interferogram file name 2. add new template optioin pysar.insarProcessor into full template list, for pysarApp.py and load_data.py 3. _readfile.py: add byteorder option into read_real_float32() to support Gamma product 4. load_data.py: adjust to be able to load Gamma unw and cor files into PySAR
hfattahi
pushed a commit
that referenced
this pull request
May 10, 2018
add script for tropospheric delay correction from GACOS at U New Castle
yunjunz
pushed a commit
that referenced
this pull request
Sep 9, 2019
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.