alosStack_tutorial.txt

######################################################################################
# Tutorial for alosStack
# Cunren Liang, October 2020
# updated Eric Fielding, May 2023
######################################################################################

This is the tutorial of alosStack processor.


###########################################
#        0. SET ENVIRONMENT VARIABLE
###########################################

Set environment variable 'ISCE_STACK'
export ISCE_STACK=CODE_DIR/isce2/contrib/stack/

where CODE_DIR is the directory of your isce code. The alosStack is not installed in your ISCE_HOME directory when you install
the software, so CODE_DIR is your ISCE2 source code directory rather than installation directory.


###########################################
#            1. PREPARE DATA
###########################################

1. ALOS-2 data
Currently the processor only supports the processing of a stack of data acquired in the same mode.

To find the acquisition mode code, check the unpacked ALOS-2 product. For example, in the following
file name

IMG-HH-ALOS2183010685-171012-FBDR1.1__A
                             ^^^
FBD (indicated by ^) is the acquisition mode code. Here is the list of acquistion modes:

 Operation Mode   |   Mode (AUIG2)  |    Mode (in file name)
--------------------------------------------------------------
   spotlight      |       SPT       |    SBS
--------------------------------------------------------------
   stripmap       |       SM1       |    UBS, UBD
                  |       SM2       |    HBS, HBD, HBQ
                  |       SM3       |    FBS, FBD, FBQ
--------------------------------------------------------------
   ScanSAR        |       WD1       |    WBS, WBD, WWS, WWD
                  |       WD2       |    VBS, VBD


Create a folder such as 'saf_d169', and in this folder, unpack all frames of each date in an individual folder
named YYMMDD. YYMMDD is the acquistion date, and it must be in this format. Now the data directory should look
like

saf_d169_data-------150225-------IMG-HH-ALOS2041062800-150225-WBDR1.1__D-F1
                 |__150408    |__IMG-HH-ALOS2041062800-150225-WBDR1.1__D-F2
                 |__150520    |__IMG-HH-ALOS2041062800-150225-WBDR1.1__D-F3
                 |__150701    |__IMG-HH-ALOS2041062800-150225-WBDR1.1__D-F4
                 |__...       |__IMG-HH-ALOS2041062800-150225-WBDR1.1__D-F5
                              |__IMG-HH-ALOS2041062850-150225-WBDR1.1__D-F1
                              |__IMG-HH-ALOS2041062850-150225-WBDR1.1__D-F2
                              |__IMG-HH-ALOS2041062850-150225-WBDR1.1__D-F3
                              |__IMG-HH-ALOS2041062850-150225-WBDR1.1__D-F4
                              |__IMG-HH-ALOS2041062850-150225-WBDR1.1__D-F5
                              |__LED-ALOS2041062800-150225-WBDR1.1__D
                              |__LED-ALOS2041062850-150225-WBDR1.1__D

2. DEM and water body

You MUST FIRST have an account to download DEM and water body. See
https://github.com/isce-framework/isce2#notes-on-digital-elevation-models
or
https://github.com/isce-framework/isce2
for more details.

See input xml file alosStack.xml in this folder on how to download DEM and water body.


###########################################
#            2. PROCESS DATA
###########################################

1. Create and enter a folder for processing data, e.g.
mkdir saf_d169_proc
cd saf_d169_proc

2. Input xml file alosStack.xml can be found in code directory. Copy it to current folder and simply set 
the parameters.
cp ${ISCE_STACK}/alosStack/alosStack.xml ./

3. Create command files for processing data. Run
${ISCE_STACK}/alosStack/create_cmds.py -stack_par alosStack.xml

4. Do most of the single date processing. Run
cmd/cmd_1.sh

This script runs a number of steps as can be easily seen from the content of the script. Some steps can
run parallelly. The number of runs can be set in the step script, or in the input file alosStack.xml
before generating these scripts.

Note that cmd_1_7_resample_to_a_common_grid.sh may require a lot of memeory. Each parallel run needs about
7.2 G because the WD1 SLC size may be up to 7.2 G. So total memeory required = 7.2 G * nruns, where nruns
is the number of parallel runs.

5. InSAR processing before ionosphere correction. Run
cmd/cmd_2.sh

6. Ionosphere correction. Run
cmd/cmd_3.sh

If the following parameter of the input xml file is True (default)

    <!--<property name="do ionospheric phase estimation">True</property>-->

Ionospheric phase estimation is performed, otherwise cmd/cmd_3.sh is empty.


After it finishes, actually nothing was done in the last two step scripts as the commands in the two step
scripts are commented out. Before running them:

check the images in folder 'fig_ion' to see if ionosphere estimation is OK for each pair. The anomalies 
include dense fringes or slight phase difference between adjacent swaths/frames in ScanSAR interferograms after 
removing ionosphere. There might also be dense fringes elsewhere. These are all anomalies and the associated 
ionosphere estimation results should not be used in cmd/cmd_3_7_estimate_ionospheric_phase_for_each_date.sh.

If you find some pairs with ionosphere estimation anomalies, specify them by adding argument '-exc_pair'
to the command ion_ls.py in cmd/cmd_3_7_estimate_ionospheric_phase_for_each_date.sh. Make sure all dates are 
still connected after excluding these pairs. You can plot baselines to see if the pairs are fully connected, e.g.
${ISCE_STACK}/alosStack/plot_baseline.py -baseline baseline/baseline_center.txt -pairs_dir pairs_ion -pairs_exc 150520-150701 -output baselines.pdf

Then run cmd/cmd_3_7_estimate_ionospheric_phase_for_each_date.sh.

If the following parameters of the input xml file are True (default)

    <!--<property name="do ionospheric phase estimation">True</property>-->
    <!--<property name="apply ionospheric phase correction">True</property>-->

In cmd/cmd_3_8_correct_ionosphere.sh, uncomment the code marked by '#uncomment to run this command'
and then run it.

7. InSAR processing after ionosphere correction. Run
cmd/cmd_4.sh

If everything is OK, you may consider removing the huge slc files in folder dates_resampled. If you need them in
the future, you can re-run cmd/cmd_1_7_resample_to_a_common_grid.sh.

Furthermore, you may consider removing the huge original data files you unpacked previously.


###########################################
#          3. ADDING MORE DATES
###########################################

Sometimes we want to add new acquistions to the already processed stack. To do this, 

1. Upack the new acquistions in data directory following #1. PREPARE DATA.

2. Repeat the processing in #2. PROCESS DATA.

We recommend saving previous scripts in a folder before new processing. Note that even the previously processed
pairs will be reprocessed again by cmd_4.sh if the following parameters of the input xml file are True (default)

    <!--<property name="do ionospheric phase estimation">True</property>-->
    <!--<property name="apply ionospheric phase correction">True</property>-->

because ionospheric phase of each date will be estimated by cmd/cmd_3_7_estimate_ionospheric_phase_for_each_date.sh
with new pairs included, all steps after this step should be reprocessed.


###########################################
#            4. CHECK RESULTS
###########################################

baseline                    basline files
burst_synchronization.txt   burst synchronization
dates                       original date of each date
dates_ion                   ionospheric phase of each date
dates_resampled             resampled date of each date. Data of all other dates are coregistered to reference date.
                            The parameter xml files including *.track.xml and f*_*/*.frame.xml are in reference date
                            folder. These should be the files you should use in most cases, such as looking for data
                            parameters, preparing for time series analysis etc.
fig_ion                     figures for checking ionosphere estimation results
pairs                       pairs of InSAR processing
pairs_ion                   pairs for ionosphere estimation

If you want to know more details about the files in each folder, read
CODE_DIR/examples/input_files/alos2/alos2_tutorial.txt
File name conventions and directory structures are mostly the same.


###########################################
#            5. KNOWN ISSUES
###########################################

1. Issues with Ionospheric Correction
According to our experience, ionospheric correction works for most of the interferograms. Because it 
relies on coherence and phase unwrapping, it does not work in some cases. These include:

(1) data have low coherence
(2) the majority of the imaged area is low coherence area like lake, ocean...
(3) the imaged area is completely divided into several isolated areas by low coherence areas, such as
    islands.

In addition to the above issues, there are also data-mode-related issues.
(1) ScanSAR-ScanSAR interferometry. While you can process one single subswath, it's better to process 
more than one subswath if the additional subswath has good coherence. This is good for ionospheric 
correction.

(2) Range distortions in JAXA product. This mostly happens in stripmap-stripmap interferometry using 
data not covering Japan. If you see very dense fringes in the corrected inteferogram, probably it is 
caused by this problem. This has been reported to JAXA and JAXA is working on debugging the focusing 
program.

UPDATE: On November 20, 2018 (JST), JAXA updated the software for PALSAR-2 standard products. Therefore, 
if your product is ordered after this time, you don't have this problem.


2. How do I improve ionospheric correction?

First of all, we recommend reading through cmd/cmd_3.sh before manually improving ionosphere estimation results.

Isolated areas lead to relative phase unwrapping errors, and therefore leads to significant errors in ionosphere
estimation result, usually shown as dense fringes in the corrected interferograms. If your scene covers an area 
with two or more isolated areas and you are interested in one of the areas, you can mask out the other areas by 
setting "areas masked out in ionospheric phase estimation" in the input file.

Or if you have processed the data, you can also specify the argument -masked_areas in cmd/cmd_3_4_filter_ionosphere.sh.
Then check the updated results following cmd/cmd_3_6_check_ionosphere_estimation_results.sh.

For ScanSAR, the software uses some accurate values for removing phase difference between adjacent swaths.
This, however, does not work well sometimes as a result of the inconistencies between different JAXA products,
especially products processed by different versions of JAXA software. As a result of this, you may see dense
fringes in the ionospheric correction result. In this case, you can try not to use aforementioned accurate
values by setting -snap in cmd/cmd_3_2_subband_interferograms.sh, and run this command and the remaining commands
to see if ionosphere estimation results have improvement. You should do this only for the problematic pairs.

Note that each time you updated ionosphere estimation results, you need to re-run the steps after
cmd/cmd_3_7_estimate_ionospheric_phase_for_each_date.sh (including this step) in cmd/cmd_3.sh, as well as cmd/cmd_4.sh


3. ScanSAR burst synchronization
For ScanSAR data acquired before February 8, 2015, chances of having enough burst synchronization for 
interferometry are very low. Don't include data acquired before this date in your stack processing.


###########################################
#            6. REFRENCES
###########################################
The methods and algorithms implemented can be found in the following papers.

1. ScanSAR or multi-mode InSAR processing
C. Liang and E. J. Fielding, "Interferometry with ALOS-2 full-aperture ScanSAR data," 
IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 5, pp. 2739-2750, May 2017.

2. Ionospheric correction, burst-by-burst ScanSAR processing, and burst-mode spectral diversity (SD) or 
multi-aperture InSAR (MAI) processing
C. Liang and E. J. Fielding, "Measuring azimuth deformation with L-band ALOS-2 ScanSAR interferometry," 
IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 5, pp. 2725-2738, May 2017.

3. Ionospheric correction
C. Liang, Z. Liu, E. J. Fielding, and R. Bürgmann, "InSAR time series analysis of L-band wide-swath SAR 
data acquired by ALOS-2," 
IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 8, pp. 4492-4506, Aug. 2018.