News
Data
User Resources
Provider Resources
Support
  MADIS Radiosonde Quality Control

MADIS Radiosonde Quality Control


This page provides a detailed description of the quality control (QC) processing and data structures for the radiosonde dataset. For a general overview of MADIS quality control, click here.

Automated Quality Control

Level 1 QC checks are considered the least sophisticated, level 2 the most sophisticated checks. The following table lists the radiosonde variables* that are QC'ed, and the checks that are used:


  MADIS Radiosonde Variables with QC
  ----------------------------------------------------------------------------------------------------
  Code       Name                  Max    Level | Level 1: |                  Level 2:                
                                Possible        |          |                                          
                                QC Level        | Validity | Super-Adiabatic  Hydrostatic  Wind-Shear 
                                                |          |   Lapse Rate                             
  ----------------------------------------------------------------------------------------------------

  Integrated Sounding            Sig    Mandatory
  -------------------            ---    ---------
  
  DD        wind direction        1         2        X
  FF        wind speed            1         2        X
  U         u wind component      1         2        X
  V         v wind component      1         2        X
  T         air temperature       2         2        X             X            X + correction
  TV        file temperature   2         2        X             X            X + correction
  TD        dewpoint temperature  1         1        X
  RH        relative humidity     1         1        X
  Q         specific humidity     1         1        X
  DPD       dewpoint depression   1         1        X
  AH        absolute humidity     1         1        X
  WVMR      water vapor mix ratio 1         1        X
  P         pressure              1         1        X
  HT        geopotential height   1         2        X                          X + correction
 
  Mandatory Levels
  ----------------

  DDMAN     wind direction        2                  X                                            X
  FFMAN     wind speed            2                  X                                            X
  UMAN      u wind component      2                  X                                            X
  VMAN      v wind component      2                  X                                            X
  TMAN      air temperature       2                  X             X            X + correction
  TVMAN     file temperature   2                  X             X            X + correction
  TDMAN     dewpoint temperature  1                  X
  RHMAN     relative humidity     1                  X
  QMAN      specific humidity     1                  X
  DPDMAN    dewpoint depression   1                  X
  AHMAN     absolute humidity     1                  X
  WVMRMAN   water vapor mix ratio 1                  X
  PMAN      pressure              1                  X
  HTMAN     geopotential height   2                  X                          X + correction

  Significant Levels
  ------------------

  TSIG      air temperature       2                  X             X
  TVSIG     file temperature   2                  X             X
  TDSIG     dewpoint temperature  1                  X
  RHSIG     relative humidity     1                  X
  QSIG      specific humidity     1                  X
  DPDSIG    dewpoint depression   1                  X
  AHSIG     absolute humidity     1                  X
  WVMRSIG   water vapor mix ratio 1                  X
  PSIG      pressure              1                  X
  HTSIG     geopotential height   1                  X
 
  Wind Levels
  -----------

  DDWIND    wind direction        1                  X
  FFWIND    wind speed            1                  X
  UWIND     u wind component      1                  X
  VWIND     v wind component      1                  X
  PWIND     pressure              1                  X
  HTWIND    geopotential height   1                  X

  Maximum Wind Levels
  -------------------

  DDMXWND   wind direction        1                  X
  FFMXWND   wind speed            1                  X
  UMXWND    u wind component      1                  X
  VMXWND    v wind component      1                  X
  PMXWND    pressure              1                  X
  HTMXWND   geopotential height   1                  X

  Tropopause Levels
  -----------------

  DDTROP    wind direction        1                  X
  FFTROP    wind speed            1                  X
  UTROP     u wind component      1                  X
  VTROP     v wind component      1                  X
  TTROP     air temperature       2                  X             X
  TVTROP    file temperature   2                  X             X
  TDTROP    dewpoint temperature  1                  X
  RHTROP    relative humidity     1                  X
  QTROP     specific humidity     1                  X
  DPDTROP   dewpoint depression   1                  X
  AHTROP    absolute humidity     1                  X
  WVMRTROP  water vapor mix ratio 1                  X
  PTROP     pressure              1                  X
  HTTROP    geopotential height   1                  X

The level 1 validity checks restrict each observation to falling within a TSP-specified set of tolerance limits. Observations not falling within the limits are flagged as failing the QC check. The following table lists the tolerance limits:


  ---------------------------------------------------
  Validity Checks

            Geopotential     
            Height(m)         Temp.(C)     Max Wind
  Level(mb) Low    High        Low  High   Speed(Kts)
  ---------------------------------------------------
  1000      -588    601        -65    60       70
   850       634   1853        -50    45       90
   700      2101   3473        -50    30      120
   500      4505   6121        -57     5      200
   400      5870   7791        -66   -10      250
   300      7726   9952        -72   -20      300
   250      8835  11274        -76   -25      300
   200     10260  12699        -78   -30      300
   150     12094  14533        -85   -30      200
   100     14000  17500        -95   -30      200
    70     16496  19596        -95   -25      200
    50     18402  21602        -95   -15      200
    30     21003  25503        -95    -5      200
    20     23501  28001        -95     5      200
    10     27003  33003        -95    15      200       
   <10      N/A   45000        -95    15      200       

The level 2 hydrostatic, super-adiabatic lapse rate, and wind shear checks ensure hydrostatic consistency between vertical layers, and also reasonable vertical consistency for the temperature and wind data.

The hydrostatic check is based on techniques developed by Schwartz and Govett (1992). In the MADIS implementation, the check is applied to the surface and mandatory level heights and temperatures, which may also be corrected by the check. Moving a layer at a time up from the surface, the height at the top level of the layer is estimated from reported pressure and temperature, then compared against the reported height. The height residual is then tested against a threshold that allows a margin of error based on the depth of the layer, which can span one missing mandatory level. If the residual is greater than the threshold, the next layer up is examined and that information is used in an attempt to correct either the height or the temperature of the level between the two layers. If a correction cannot be made, the estimate is recalculated using any available significant level temperatures in the suspect layer in the hope that this will better represent the layer's temperature and reduce the residual to below the threshold. The check continues until the top of the sounding is reached, or until a gap of more than one mandatory level occurs, or until a large residual is found and the criteria for correction are not met. In that event, both the height and temperature at the suspect level are failed by the QC, and the check is not applied to data higher up in the sounding. The following table lists the absolute value of the allowable residuals for each layer:


  ---------------------------------------------------
  Hydrostatic Check Allowable Residuals (Absolute)

  Layer (mb)              Value (m)
  ---------------------------------------------------
  1099 - 1000              20
  1000 -  925              20
   925 -  850              21
   850 -  700              20
   700 -  500              25
   500 -  400              20
   400 -  300              20
   300 -  250              20
   250 -  200              25
   200 -  150              30
   150 -  100              36
   100 -   70              40
    70 -   50              45
    50 -   30              50
    30 -   20              55
    20 -   10              60
    10 -    7              65
     7 -    5              70
     5 -    3              75
     3 -    2              80
     2 -    1              85

The level 2 super-adiabatic lapse rate check, based on Atkins (1985), fails for the temperature layer between levels n and n+1 if the following detection criteria is met:

    C > [ ( (Pn / Pn+1 ) k  *  Tn+1)  -  Tn ]

where,

   Pn : pressure at level n (mb)
   Pn+1 : pressure at level n+1 (mb)
   Tn : temperature at level n (K)
   Tn+1 : temperature at level n+1 (K)
   k : kappa = 2/7
   C : tolerance
  = 2 K if Pn  <  800 mb
  = 2 K if Pn > 800 mb
Note:    level n is below level n+1, and thus Pn / Pn+1 is always > 1 )

The level 2 wind shear check (DiMego et al., 1985) is applied to mandatory level winds. For each mandatory level, the nearest wind that meets the following criteria is used for the check:

Once the wind to be used has been found (at level "s"), the wind shear check is applied, and if the following criteria are met, the check passes:
    ( FFmean  <  30  kt   and   FFdiff  <   FFthr )    or

    ( FFmean  <  39  kt   and   FFdiff  <   FFthr   and   DDdiff  <  70  deg  )    or

    ( FFmean  <  51  kt   and   FFdiff  <   FFthr   and   DDdiff  <  55  deg  )    or

    ( | FFm1 - FFs |  <   FFthr   and   DDdiff  <  40  deg  )

where,

   m1 : mandatory level being checked
   s : level used to check m1
   FFmean : mean wind speed (kt) between levels m1 and s
   FFdiff : absolute speed difference between the two levels
   FFthr : acceptable speed shear threshold (kt) between levels m1 and s
  = 50 kt if s is a wind level
  = 80 kt if s is a maximum wind or mandatory level
   DDdiff : absolute directional difference between the two levels
If s is a mandatory level, there is an additional restriction that the magnitude of the vector difference between the two levels be less than 80 (kt).

*It should be noted that while the QC checks discussed here are generally applied to the form of the variable stored in the database, the QC results will also be applied to any forms of the variable that are requested by the user and are derived from the primary variable. For example, specific humidity will get the QC results from the checks applied to dewpoint temperature.

Subjective Intervention

Two text files, a "reject" and an "accept" list provide the capability to subjectively override the results of the automated QC checks. The reject list is a list of stations and associated input observations that will be labeled as bad, regardless of the outcome of the QC checks; the accept list is the corresponding list of stations that will be labeled as good, regardless of the outcome of the QC checks. In both cases, observations associated with the stations in the lists can be individually flagged. For example, wind observations at a particular station may be added to the reject list, but not the temperature observations.

Here are the current subjective intervention lists in use:

QC Data Structures

The MADIS QC information available for each variable includes the following QC structures: a single-character "data descriptor", intended to define an overall opinion of the quality of each observation by combining the information from the various QC checks, and for users desiring detailed information, a "QC applied" bitmap indicating which QC checks were applied to each observation, and a "QC results" bitmap indicating the results of the various QC checks.

The following table provides a complete list of the data descriptors and the bits used in the bitmaps:

  ---------------------------------
  MADIS QC Information - Radiosonde
  ---------------------------------

  QC Data Descriptor Values
  -------------------------

  No QC available:

   Z - Preliminary, no QC

  Automated QC checks:

   C - Coarse pass, passed level 1
   S - Screened, passed levels 1 and 2
   V - Verified, passed levels 1, 2, and 3
   X - Rejected/erroneous, failed level 1
   Q - Questioned, passed level 1, failed 2 or 3

       where level 1 = validity
             level 2 = hydrostatic, super adiabatic lapse rate, and wind shear
                       checks
             level 3 = N/A

  Subjective intervention:

   G - Subjective good
   B - Subjective bad

  Interpolated/Corrected observations:

   I - Interpolated
   W - Raw data was "wrong", has been corrected using estimates from automated QC checks
   T - Virtual temperature could not be calculated, air temperature passing all QC 
       checks has been returned

  Bitmask for QC Applied and QC Results
  -------------------------------------

   Bit       QC Check                      Decimal Value
   ---       --------                      -------------
    1        Master Check                        1
    2        Validity Check                      2
    3        Reserved                            4
    4        Reserved                            8
    5        Reserved                           16
    6        Hydrostatic Check                  32
    7        Reserved                           64
    8        Super Adiabatic Lapse Rate Check  128
    9        Wind Shear Check                  256
   10        Reserved                          512

The QC bitmask is used in the QC applied and QC result "words" returned along with the QC data descriptor. By examining the individual bits, the user can determine which checks were actually applied, and the pass/fail status of each check that was applied.

In the QC applied word, a bit value of 1 means the corresponding check was applied, a bit value of 0 indicates the check wasn't applied.

In the QC results word, a bit value of 1 means the corresponding check was applied and failed, a bit value of 0 indicates the check passed (given that the check was applied).

The "Master Check" is used to summarize all of the checks in a single bit. If any check at all was applied, this bit will be set in the QC applied word. If the observation failed any QC check, it will be set in the QC results word.

When read as decimal numbers, the different bits that are set in the bitmask are summed together. For example, a QC applied value of 131 should be interpreted as 1 + 2 + 128, meaning the validity and super-adiabatic lapse rate checks were applied.


References

Atkins, M.J., 1985: Quality Control, Selection and Data Processing of Observations in the Meteorological Office Operational Forecast System. Workshop on the Use and Quality Control of Meteorological Observations, Reading, U.K., European Centre for Medium Range Weather Forecasts, 255-290.

DiMego, G.J, P.A. Phoebus, and J.E. McDonnel, 1985: Data Processing and Quality Control for Optimum Interpolation Analysis at the National Meteorological Center. NMC Office Note 306, NOAA, U.S. Dept. of Commerce, 38 pp.

Schwartz, B. and M. Govett, 1992: A Hydrostatically Consistent North American Radiosonde Data Base at the Forecast Systems Laboratory. NOAA Technical Report ERL FSL-4, 81 pp.

Technique Specification Package 88-21-R2 For AWIPS-90 RFP Appendix G Requirements Numbers: Quality Control Incoming Data, 1994. AWIPS Document Number TSP-032-1992R2, NOAA, National Weather Service, Office of Systems Development.


Last updated 30 October 2014