5.2.5 Getting data from USGS or EMSC into SEISAN, get_web_locations

Written by Terje Utheim, University of Bergen.

The purpose of this program is to download hypocenter information from international data centers. The program has now been implemented for USGS and EMSC.

The program downloads the data to a SEISAN data base in a Nordic format file in Nordic or Nordic2 format. The program downloads a file from the web site called
get_web_locations_EMSC.txt or get_web_locations_USGS.txt
from which it picks up relevant information for the S-file. The download might have data from several other sources which are entered into the S-file. The list of USGS sources can be found at
https://earthquake.usgs.gov/data/comcat/contributor/.
The source 'us' is translated into USG in the S-file but the others are left as they are, e.g. 'ak' is Alaska earthquake Center'.

The input magntude types from the text file are converted to SEISAN magnitude types as follows:

mb	b
ML	L
mB	B
Ms	s
MS	S
MW	W
Mc	C
mw	W
mww	W
ww	W
wr	W
ml	L
md	C

The program has several options:

Set limits for minimum and maximum depths and minimum magnitude

Set time interval for download

Run once for the given parameters from command line

Run at given time intervals to always maintain the local data base. If a new location of the same event is found, the old event is replaced.

The program has no parameter files, all options are given as options on the prompt line.

Typing get_web_locations  -h the following options/info  are shown (platform dependent)

SEISAN_TOP..................................: /home/seismo

Valid program options, int is integer, float is a floating point number and text is an ascii string:

-h		show this help info
-fm	int     SEISAN format: 1-old, 2-new (default: 1)
-ag	int     Agency to query: 0-USGS,1-EMSC (default: 0)
-loop	int 	1-loop forever, 0-one query (0=default)
-tsrt	int	minutes back, start-time. 
                (Number of minutes to start from current time)
-tstp	int	minutes back, stop-time.
                Number of minutes to stop from current time. This should not 
                be too close to real time in order for the data center 
                to calculate the hypocenter. 50 minutes is a reasonable number.
                These two parameters are only used if the program runs 
                continuously. The interval of looking for data is tstp-tsrt.
-mag	float	minimum magnitude
                The minimum magnitude to be included. The magnitude is the 
                first found in case there are several types of magnitudes. 
                If no magnitude is given, all magnitude events are used.
-mind	text	minimum depth (default: 0)
-maxd	text	maximum depth (default: 800)
-db	text	SEISAN database name, max 5 characters
-srt	text	start-time, format: 2018-02-20T03:45:00
-end	text	end-time, format: 2018-02-22T08:23:00
                Start and stop time for a one time extract. Cannot be used 
                at the same time as tsrt and tstp.
 
To run the program in a continuous loop, the following parameters 
MUST be specified:

-loop 1
-tsrt t1 (start time, number of minutes back in time)
-tstp t1 (stop time, number of minutes back in time)

Example 1: Collecting data continuously from command line:

get_web_locations  -loop 1 -tsrt 60 -tstp 55 -mag 3.0 -db TERJE

The program will query data from 60 minutes ago until 50 minutes 
back in time from now.
The program will then sleep for 60 - 50 = 10 minutes and start again. 
Minimum magnitude is 3.0 and the data is written to the data base TERJE.
It is created if not existing.

-srt and -end cannot be specified when running in loop mode.


The next lines are output from the program so that the 
user can see the progress.

start_time: 2021-03-19T15:50:02
end_time  : 2021-03-19T15:55:02
web_agency: 0
web_agency: USGS
sleep 300  5



start_time: 2021-03-19T15:55:13
end_time  : 2021-03-19T16:00:13
web_agency: 0
web_agency: USGS
sleep 300  5

.
.
.
.
.
start_time: 2021-03-19T16:26:23
end_time  : 2021-03-19T16:31:23
web_agency: 0
web_agency: USGS
sleep 300  5



start_time: 2021-03-19T16:31:33
end_time  : 2021-03-19T16:36:33
web_agency: 0
web_agency: USGS
/home/seismo/sei11/REA/TERJE/2021 does not exist, create it
/home/seismo/sei11/REA/TERJE/2021/03 does not exist, create it
sleep 300  5

s-file generated

 2021  319 1633 14.8 D  -7.723 118.701 10.0  USG                        4.7bUSG1
 WEB event-id: us7000dl0y                                                      3
 LOCALITY: 81 km N of Bima, Indonesia                                          3
 ACTION:NEW 21-03-19 17:31 OP:SEIS STATUS:               ID:20210319163314     I
 STAT SP IPHASW D HRMM SECON CODA AMPLIT PERI AZIMU VELO AIN AR TRES W  DIS CAZ7
 


Example 2:" one time" data collections with no options

get_web_locations -db TEST -srt 2018-01-01T00:00:00 -end 2018-01-10T23:59:00



Example 3: Request events from EMSC, from a specific time-window, 
magnitudes higher than 6.0, Nordic2 SEISAN format, database EMSC

get_web_locations -ag 1 -srt 2021-03-20T00:00:00 -end 2021-03-21T15:00:00 -mag 6.0 -fm 2 -db EMSC

SEISAN_TOP..................................: /home/seismo/sei11
web_agency: 1
web_agency: EMSC
/home/seismo/sei11/REA/EMSC_/2021/03 does not exist, create it

Response from EMSC in your current catalog temporary file get_web_locations_EMSC.txt.

#EventID | Time | Latitude | Longitude | Depth/km | Author | Catalog | Contributor | ContributorID | MagType | Magnitude | MagAuthor | EventLocationName
20210320_0000081|2021-03-20T09:09:46.5Z|38.52|141.55|60.0|EMSC|EMSC-RTS|EMSC|961496|mw|7.0|EMSC|NEAR EAST COAST OF HONSHU, JAPAN
20210320_0000043|2021-03-20T05:19:31.3Z|-59.65|150.21|10.0|EMSC|EMSC-RTS|EMSC|961440|mw|6.1|EMSC|WEST OF MACQUARIE ISLAND

Running eev:

eev 202103 EMSC

 2021  3 Reading events from base EMSC_  2 
#    1 20 Mar 2021  5:19 31  D -59.650 150.210 10.0  N      6.1WEMC      ? 
#    2 20 Mar 2021  9: 9 46  D  38.520 141.550 60.0  N      7.0WEMC      ? 1
#    1 20 Mar 2021  5:19 31  D -59.650 150.210 10.0  N      6.1WEMC      ? t

 File name: /home/seismo/sei11/REA/EMSC_/2021/03/20-0519-31D.S202103                        
2021  320  519 31.3 D -59.650 150.210 10.0  EMC                        6.1WEMC1
WEB event-id: 20210320_0000043                                                3
LOCALITY: WEST OF MACQUARIE ISLAND                                            3
ACTION:NEW 21-03-20 15:36 OP:SEIS STATUS:               ID:20210320051931     I
STAT COM NTLO IPHASE   W HHMM SS.SSS   PAR1  PAR2 AGA OPE  AIN  RES W  DIS CAZ7

#    1 20 Mar 2021  5:19 31  D -59.650 150.210 10.0  N      6.1WEMC      ? 2
#    2 20 Mar 2021  9: 9 46  D  38.520 141.550 60.0  N      7.0WEMC      ? t

 File name: /home/seismo/sei11/REA/EMSC_/2021/03/20-0909-46D.S202103                        
2021  320  9 9 46.5 D  38.520 141.550 60.0  EMC                        7.0WEMC1
WEB event-id: 20210320_0000081                                                3
LOCALITY: NEAR EAST COAST OF HONSHU, JAPAN                                    3
ACTION:NEW 21-03-20 15:36 OP:SEIS STATUS:               ID:20210320090946     I
STAT COM NTLO IPHASE   W HHMM SS.SSS   PAR1  PAR2 AGA OPE  AIN  RES W  DIS CAZ7

#    2 20 Mar 2021  9: 9 46  D  38.520 141.550 60.0  N      7.0WEMC      ? q




Example :  Continuous mode as cron job on a Linux system

The setup below will start every hour at 0 minutes in a bash environment
0  * * * * /home/seismo/get_web_locations/cron_restart.bash  2.0 MYDB 0 800 2 1 >/dev/null 2>&1

The setup below will start every hour at 0 minutes in a csh environment
0  * * * * /home/seismo/get_web_locations/cron_restart.csh  2.0 MYDB 0 800 2 1 >/dev/null 2>&1


The script files are in /home/seismo/get_web_locations

cron_restart.bash file 


#!/bin/bash
rm -f /home/seismo/get_web_locations/prog.log
source /home/seismo/sei11/COM/SEISAN.bash
echo MAG: $1 DBASE: $2 MIN_DEPTH: $3 MAX_DEPTH: $4 START_HOUR: $5 STOP_HOUR: $6 AGENCY: $7 FORMAT: $8
start_time=`date "+%Y-%m-%dT%H:%M:00" --date -$5hour`
stopp_time=`date "+%Y-%m-%dT%H:%M:00" --date -$6hour`
echo $start_time
echo $stopp_time
cd /home/seismo/get_web_locations
/home/seismo/PRO/get_web_locations -mag $1 -db $2 -mind $3 -maxd $4 -srt "$start_time" -end "$stopp_time" -ag $7 -fm $8 > prog.log

The script expects the arguments in defined sequence:

arg.1	minimum magnitude
arg.2	SEISAN database, max 5 characters. If not existing it will be created
arg.3	minimum depth in km
arg.4	maximum depth in km
arg.5	start hour, for example 2, 2 hours back in time from now
arg.6	stop hour, for example 1, 1 hour back in time from now
arg.7	agency from where to send the request
arg.8	format, old or new SEISAN format.

The directory /home/seismo/get_web_locations is where data is downloaded and the 
log written, can be any directory.

cron_restart.csh

#!/bin/csh
rm -f /home/seismo/get_web_locations/prog.log
source /home/seismo/COM/.SEISAN
echo MAG: $1 DBASE: $2 MIN_DEPTH: $3 MAX_DEPTH: $4 START_HOUR: $5 STOP_HOUR: $6 AGENCY: $7 FORMAT $8
set start_time=`date --utc "+%Y-%m-%dT%H:%M:00" --date -$5hour`
set stopp_time=`date --utc "+%Y-%m-%dT%H:%M:00" --date -$6hour`
echo $start_time
echo $stopp_time
cd /home/seismo/get_web_locations
/home/seismo/PRO/get_web_locations -mag $1 -db $2 -mind $3 -maxd $4 -srt "$start_time" -end "$stopp_time" -ag $7 -fm $8 > prog.log

Peter Voss : Fri Nov 12 10:33:10 UTC 2021