Skip to content

Noise Toolkit PDF PSD bundle V.1

Jump to bottom
Manoch Bahavar edited this page Nov 12, 2020 · 2 revisions

DESCRIPTION (V.1.0):

This bundle contains Python scripts to:
  • request/read waveforms for given network/station/location/channels using the obspy FDSN client
  • compute and populate an hourly “PSD database”
  • extract PSDs and PDFs from the “PSD database”

QUICK START

Installation:

See also the bundle’s INSTALL.txt file
  • install Python on your computer
  • additional required Python modules are:
    . matplotlib
    . numpy
    . scipy
    . obspy
  • download the package under the installation directory.
Note: if you are installing this bundle on a PC, edit the param/common.py file and set the namingConvention parameter to ‘WINDOWS’

Running the scripts:

  • under the root directory of the Noise Toolkit (IRIS_NTK) execute the following command to plot PSD of NM.SLM.00.BHZ for the first hour of 2015-01-01:

python bin/ntk_computePSD.py param=computePSD net=NM sta=SLM chan=BHZ loc=00 start=2015-01-01T00:00:00 end=2015-01-01T00:30:00 type=period mode=plot

  • review all parameter files to get familiar with the script capabilities
  • use examples given below to run/test the scripts and become familiar with them
  • edit the parameter files under IRIS_NTK/param to change parameters based on your needs

NOTES

  • the “DeprecationWarning: Development and maintenance efforts will focus on the new obspy.fdsn client…..” message
    that you receive from the script is caused by script loading the IRIS module, as described below for the infrasound stations, and should not
    affect the run
  • examples below may turn on the verbose mode. Once you have configured the script, you can turn it off

COMMENTS/QUESTIONS/SUGGESTIONS:

  • We welcome patches and enhancements to this software. When developing patches, please pay particular attention to ease of use and maintenance and also keep dependencies to a minimum.
  • for issues, file a ticket under Issues

HISTORY

  • R 0.8.0 2015-04-07 added check for all parameter values to inform user if they are not defined. Corrected the instrument correction for SAC files that would apply
    sensitivity in addition to instrument correction
  • R 0.8.0 2015-04-06 addressed the variable maximum period issue that was changing based on the smoothing window length
  • R 0.8.0 2015-04-02 based on feedback from Robert Anthony, in addition to nan values other non-numeric values may exist. The write that contains a flot() conversion
    is placed in a try block so we can catch any non-numeric conversion issue and report it as user-defined NAN
  • R 0.7.2 2015-03-30 added a check to number of samples to aviod log of zero (reported by Rob Anthony)
  • R 0.7.1 2015-03-17 added the “waterLevel” parameter to provide user with more control on how the ObsPy module shrinks values under water-level of max spec amplitude
    when removing the instrument response.
  • R 0.7.0 2015-02-24 introduced two new parameters (performInstrumentCorrection, applyScale) to allow user avoid instrument correction also now user can turn od decon. filter
  • R 0.6.0 2014-10-22 supports Windows installation and moved data, param and bin directory from IRIS_NTK/PSD to IRIS_NTK in preparation for Microseism Energy bundle release
  • R 0.5.0 2014-10-17 Beta release

USER’S GUIDE

Use below examples to run/test the scripts and become familiar with them and edit the parameter files under IRIS_NTK/param to change parameters based on your needs.

SCRIPT: ntk_computePSD.py

compute hourly PSDs using command arguments and the associated parameters in the configuration files

USAGE:

                     configuration file name           net     sta    loc      startDateTime        endDateTime              x-axis type        =plot run in plot mode; =time run in timing mode;  =verboserun in verbose mode
                                         |               |      |       |         |                        |                        |           |
          python bin/ntk_computePSD.py param=computePSD net=NM sta=SLM loc=DASH start=2009-01-01T00:00:00 end=2009-01-05T12:00:00 type=period  mode=0
          python bin/ntk_computePSD.py param=computePSD net=TA sta=959A loc=EP  start=2013-10-01T11:00:00 end=2013-10-01T12:00:00 type=period  mode=0

NOTES & EXAMPLES:

IN ALL EXAMPLES IT IS ASSUMED THAT THE COMMAND IS ISSUED UNDER THE IRIS_NTK/ DIRECTORY

Please note that the “DeprecationWarning: Development and maintenance efforts will focus on the new obspy.fdsn client…..” message that you receive from the script is caused by script loading the IRIS module, as described below for the infrasound stations, and should not affect the run Use the run in plot or verbose mode to tune the parameters before a production run (mode=0):

python bin/ntk_computePSD.py param=computePSD net=NM sta=SLM loc=DASH start=2009-01-01T00:00:00 end=2009-01-05T12:00:00 type=period mode=plot

Use the x-axis type to not only define the axis type but also ask the script to create regular sampling in the corresponding log period or log frequency The first sample is placed at xStart second or xStart Hz as defined in the parameter file and log-regular samples are created on both sides. User may set xStart=‘Nyquist’ in the parameter file to set the Nyquist frequency as the reference. The former option will result the same period or frequency samples regardless of the station sampling interval

python bin/ntk_computePSD.py param=computePSD net=NM sta=SLM loc=DASH start=2009-11-01T11:00:00 end=2009-11-01T12:00:00 type=frequency mode=0 At this time the FDSN services is not able to remove instrument response from infrasound data if the response is a polynomial to process infrasound data, in the configuration file change the channel to BDF and the requestClient to IRIS

python bin/ntk_computePSD.py param=computePSD net=TA sta=959A loc=EP chan=BDF start=2013-10-01T11:00:00 end=2013-10-01T12:00:00 type=frequency mode=0

If you already have your waveform data saved on a local storage as valid file format such as SAC, MSEED, CSS, etc. (see the supported-formats at https://docs.obspy.org/packages/autogen/obspy.core.stream.read.html#obspy.core.stream.read) then you can instruct the script to get the waveform data from these files and connect to Web Services for the response information only. To do this you need to update two parameters in your parameter file (such pas param/computePSD.py):

set requestClient = “FILES” to flag the script that the waveform data are coming from file and fileTag = “{IRIS_NTK_PSD}/data/TEST/SAC/W*.SAC” to tell it which files to look at

python bin/ntk_computePSD.py param=computePSD net=TA sta=W5 loc=DASH start=2014-03-17T04:30:00 end=2014-03-17T05:30:00 type=period mode=0

the most time consuming part of the process is the waveform request. Use BH? channel to request multiple channels (3 channels) at the same time. This will cut down on the number of trips to get waveforms you can further cut the number of requests, when possible, by providing only 2 common characters of station name. This will be very helpful in processing data for arrays like TA that uses very uniform station naming, in example below TA.O6 —> TA.O6

python bin/ntk_computePSD.py param=computePSD net=TA sta=O6 loc=DASH start=2013-10-01T11:00:00 end=2013-10-01T12:00:00 type=period mode=0

in example below TA.25 —> TA.25

python bin/ntk_computePSD.py param=computePSD net=TA sta=25 loc=DASH start=2013-10-01T11:00:00 end=2013-10-01T12:00:00 type=period mode=0

keep your requests at reasonable size. Requesting too many stations may cause timeouts. Script will only write the timeout message and has no way of handling it. Even for smaller requests there is a chance that data requests fail when timeout is reported. User should scan the output messages and re-queue the timed out requests again. for production runs run one or more instances of scripts as a background process. In our limited testing each 3-channel 1 hour time window with 50% overlap takes about 15 seconds to process.

nohup python bin/ntk_computePSD.py param=computePSD net=NM sta=SLM loc=DASH start=2010-01-01T00:00:00 end=2011-01-01T00:00:00 type=period mode=0 >& /tmp/psdSLM_2010 &

for more information on FDSN WS specifications see

http://www.fdsn.org/webservices/FDSN-WS-Specifications-1.1.pdf

SCRIPT: ntk_extractPsdHour.py

a Python script to extract PSDs for a given channel and bounding parameters.

USAGE:

                       configuration file name              net    sta     loc       chan       start date-time        end date-time            x-axis type  verbose on/off
                                              |                 |      |       |         |         |                            |                     |              |     
     python bin/ntk_extractPsdHour.py param=extractPsdHour net=NM sta=SLM loc=DASH chan=BHZ start=2009-01-01T10:00:00 end=2009-01-05T12:00:00  type=period     mode=0

INPUT:

The ntk_computePSD.py output files

NOTES & EXAMPLES:

The output is similar to PQLX’s exPSDhour script and will provide the output path/file at the end of the run

python bin/ntk_extractPsdHour.py param=extractPsdHour net=NM sta=SLM loc=DASH chan=BHZ start=2009-01-01T10:00:00 end=2009-01-05T12:00:00 type=period mode=0

SCRIPT: ntk_binPsdDay.py

A Python script to bin PSD’s to daily files for a given channel and bounding parameters.

USAGE:

                          configuration file name        net      sta     loc       chan     start day         end day[inclusive]   x-axis type    verbose on/off
                                         |                 |       |       |         |           |                        |               |                    | 
     python ntk_binPsdDay.py    param=binPsdDay        net=NM sta=SLM loc=DASH chan=BHZ start=2009-01-01         end=2009-01-06    type=period            mode=0

INPUT:

The ntk_computePSD.py output files

NOTES & EXAMPLES:

python bin/ntk_binPsdDay.py param=binPsdDay net=NM sta=SLM loc=DASH chan=BHZ start=2009-01-01 end=2009-01-06 type=period mode=0

Full path to the output data file(s) is provided at the end of the run. You may turn off hourly file output via the configuration file

COPYRIGHT © 2015 Product Team, IRIS Data Management Center

This bundle is provided by the IRIS DMC Data Products Team WITHOUT ANY WARRANTY AND/OR SUPPORT This is a free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This script is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License (GNU-LGPL) for more details. The GNU-LGPL and further information can be found here: www.gnu.org You should have received a copy of the GNU Affero General Public License along with this program. If not, see http://www.gnu.org/licenses.

Noise Toolkit docs:

Previous release (V.1) docs:

Clone this wiki locally