News
Data
User Resources
Provider Resources
Support
  MADIS Satellite Sounding Quality Control

MADIS Satellite Sounding Quality Control


This page provides a detailed description of the quality control (QC) processing and data structures for the satellite sounding dataset. For a general overview of MADIS quality control, click here.

Automated Quality Control

Level 1 QC checks are considered the least sophisticated, level 2 the most sophisticated checks. The following table lists the satellite sounding variables* that are QC'ed, and the checks that are used. Note that the observed water vapor mixing ratio is converted to dewpoint, then the checks are applied to dewpoint.


  MADIS Satellite Sounding Variables with QC
  --------------------------------------------------------------------------------------------------
  Code       Name                                    Max     Level | Level 1:       Level 2:
                                                   Possible        |          
                                                   QC Level        | Validity   Internal Consistency
                                                                   |          
  --------------------------------------------------------------------------------------------------
  
  Sounding Products                                               
  -----------------

  T         temperature                              1                  X
  TV        virtual temperature                      1                  X                  
  WVMR      water vapor mixing ratio                 2                  X               X
  TD        dewpoint temperature                     2                  X               X  
  RH        relative humidity                        2                  X               X  
  Q         specific humidity                        2                  X               X  
  DPD       dewpoint depression                      2                  X               X  
  AH        absolute humidity                        2                  X               X  
  
  Internal Consistency Checks
  ---------------------------
  
    3   air temperature vs. dewpoint temperature

The level 1 validity checks restrict each observation to falling within a TSP-specified set of tolerance limits. Observations not falling within the limits are flagged as failing the QC check. The following table lists the tolerance limits:


  ---------------------------------------------------
  Validity Checks

            Air and Dewpoint Temperature (C)
  Level(mb)         Low    High        
  ---------------------------------------------------
  1000              -65    60       
   850              -50    45       
   700              -50    30      
   500              -57     5      
   400              -66   -10      
   300              -72   -20      
   250              -76   -25      
   200              -78   -30      
   150              -85   -30      
   100              -95   -30      
    70              -95   -25      
    50              -95   -15      
    30              -95    -5      
    20              -95     5      
    10              -95    15      
   <10              -95    15      

The level 2 internal consistency checks enforce reasonable, meteorological relationships among observations measured at a single location. In this case, the dewpoint temperature must not exceed the temperature observation. If it does, the dewpoint is flagged as failing the internal consistency check.

*It should be noted that while the QC checks discussed here are generally applied to the form of the variable stored in the database, the QC results will also be applied to any forms of the variable that are requested by the user and are derived from the primary variable. For example, specific humidity will get the QC results from the checks applied to dewpoint temperature.

Subjective Intervention

Two text files, a "reject" and an "accept" list provide the capability to subjectively override the results of the automated QC checks. The reject list is a list of satellites and associated input observations that will be labeled as bad, regardless of the outcome of the QC checks; the accept list is the corresponding list of satellites that will be labeled as good, regardless of the outcome of the QC checks. In both cases, observations associated with the satellites in the lists can be individually flagged. For example, moisture observations at a particular satellite may be added to the reject list, but not the temperature observations.

Here are the current subjective intervention lists in use:

QC Data Structures

The MADIS QC information available for each variable includes the following QC structures: a single-character "data descriptor", intended to define an overall opinion of the quality of each observation by combining the information from the various QC checks, and for users desiring detailed information, a "QC applied" bitmap indicating which QC checks were applied to each observation, and a "QC results" bitmap indicating the results of the various QC checks.

The following table provides a complete list of the data descriptors and the bits used in the bitmaps:

  -----------------------------------------
  MADIS QC Information - Satellite Sounding
  -----------------------------------------

  QC Data Descriptor Values
  -------------------------

  No QC available:

   Z - Preliminary, no QC

  Automated QC checks:

   C - Coarse pass, passed level 1
   S - Screened, passed levels 1 and 2
   V - Verified, passed levels 1, 2, and 3
   X - Rejected/erroneous, failed level 1
   Q - Questioned, passed level 1, failed 2 or 3

       where level 1 = validity
             level 2 = internal consistency
             level 3 = N/A

  Subjective intervention:

   G - Subjective good
   B - Subjective bad

  Interpolated/Corrected observations:

   T - Virtual temperature could not be calculated, air temperature passing all QC 
       checks has been returned

  Bitmask for QC Applied and QC Results
  -------------------------------------

   Bit       QC Check                      Decimal Value
   ---       --------                      -------------
    1        Master Check                        1
    2        Validity Check                      2
    3        Reserved                            4
    4        Internal Consistency Check          8
    5        Reserved                           16
    6        Reserved                           32
    7        Reserved                           64
    8        Reserved                          128
    9        Reserved                          256
   10        Reserved                          512

The QC bitmask is used in the QC applied and QC result "words" returned along with the QC data descriptor. By examining the individual bits, the user can determine which checks were actually applied, and the pass/fail status of each check that was applied.

In the QC applied word, a bit value of 1 means the corresponding check was applied, a bit value of 0 indicates the check wasn't applied.

In the QC results word, a bit value of 1 means the corresponding check was applied and failed, a bit value of 0 indicates the check passed (given that the check was applied).

The "Master Check" is used to summarize all of the checks in a single bit. If any check at all was applied, this bit will be set in the QC applied word. If the observation failed any QC check, it will be set in the QC results word.

When read as decimal numbers, the different bits that are set in the bitmask are summed together. For example, a QC applied value of 11 should be interpreted as 1 + 2 + 8, meaning the validity and internal consistency checks were applied.


References

Technique Specification Package 88-21-R2 For AWIPS-90 RFP Appendix G Requirements Numbers: Quality Control Incoming Data, 1994. AWIPS Document Number TSP-032-1992R2, NOAA, National Weather Service, Office of Systems Development.


Last updated 16 June 2016