|
<< Click to Display Table of Contents >> Navigation: Live > Agents > File Agents > File Processor > Examples > Live File Processor Agent Example – AQS RD Transaction |
The Air Quality System (AQS) is an information management system for ambient outdoor airborne pollutants and related meteorological data that is administered by the United States Environmental Protection Agency. The Live File Processor Agent can be used to load AQS Raw Data (RD) Transaction Type data into EQuIS. See the EQuIS AQS Module documentation for more information about AQS data.
Add the following settings to the ST_CONFIG table (CONFIG_SECTION = 'Live.FileProcessor', CONFIG_KEY = 'Agent').
OBJECT_TYPE |
OBJECT_VALUE |
STRING_VALUE |
Description |
|---|---|---|---|
Default |
dateTimeFormats |
en-AU |
Note, this is a default setting (for all processors, unless overridden by a specific processor). |
AQS-RD Transaction |
dateTimeColumnHeaderRegex |
(?i)Date |
DT_LOGGER_DATUM.DATUM_UTC_DT; column header text starts with Date Time. |
AQS-RD Transaction |
dateTimeFormats |
yyyyMMdd |
The format used to process DateTimes. |
AQS-RD Transaction |
datumValueColumnHeaderRegex |
=> NULL || NaN => NULL ||NULL => NULL |
Replace these case-insensitive string datum values (from the file) with these float or NULL values. |
AQS-RD Transaction |
delimiters |
| |
The fields are separated by tabs and/or commas. |
AQS-RD Transaction |
folderWatcherPath |
C:\EarthSoft\Live\AQS-RD-Transaction |
The Folder that will be watched for new data. |
AQS-RD Transaction |
headerLinesDefaultValue |
RD|Action Code |State Code|County Code|Site ID|Parameter|POC|Sample Duration|Unit|Method|Date|Start Time|Sample Value|Null Data Code|Sampling Frequency|Monitor Protocol (MP) ID|Qualifier - 1|Qualifier - 2|Qualifier - 3|Qualifier - 4|Qualifier - 5|Qualifier - 6|Qualifier - 7|Qualifier - 8|Qualifier - 9|Qualifier - 10|Alternate Method Detectable Limit|Uncertainty |
Used if headerLinesRegex is not defined or is empty, or a file does not start with headerLinesRegex.
|
AQS-RD Transaction |
ignoreLinesRegex |
^# |
Line that starts with the given regex will be ignored. |
AQS-RD Transaction |
loggerCodeColumnHeaderRegex |
(?i)State Code || (?i)County Code || (?i)Site ID || (?i)RD || (?i)Transaction Type |
LOGGER_CODE from file column value(s). |
AQS-RD Transaction |
remarkQualifierColumnHeaderRegex |
(?i)Qualifier |
The column header regex to use for the remark qualifier. |
AQS-RD Transaction |
remarkQualifierType |
AQS Qualifier |
The type of qualifier.
Qualifiers are saved in the DT_LOGGER_REMARK table. |
AQS-RD Transaction |
remarkQualifierJoin |
| |
The delimiter to be used when multiple qualifiers are present. |
AQS-RD Transaction |
remarkNullDataCodeColumnHeaderRegex |
(?i)Null Data Code |
The column to be used as the Null Data Code. |
AQS-RD Transaction |
remarkNullDataCodeType |
Null Data Code |
The type of Null Data Code. |
AQS-RD Transaction |
remarkNullDataCodeJoin |
| |
The delimiter to be used when multiple Null Data Codes are present. |
AQS-RD Transaction |
seriesNameColumnHeaderRegex |
(?i)Parameter || (?i)POC || |
SERIES_NAME from file column value(s). |
AQS-RD Transaction |
seriesNameColumnHeaderRegexGroups |
(?'a'.*) |
SERIES_NAME from file column header. |
AQS-RD Transaction |
sysLocCodeColumnHeaderRegex |
(?i)State Code || (?i)County Code || (?i)Site ID |
The column to be used as the Null Data Code. |
AQS-RD Transaction |
timeColumnHeaderRegex |
(?i)Time |
Time (only) column header text pattern. |
Example File
1.Start the agent, and copy the following files to folder C:\EarthSoft\Live\AQS-RD-Transaction.
# RD|Action Code|State Code|County Code|Site ID|Parameter|POC|Sample Duration|Unit|Method|Date|Start Time|Sample Value|Null Data Code|Sampling Frequency|Monitor Protocol (MP) ID|Qualifier - 1|Qualifier - 2|Qualifier - 3|Qualifier - 4|Qualifier - 5|Qualifier - 6|Qualifier - 7|Qualifier - 8|Qualifier - 9|Qualifier - 10|Alternate Method Detectable Limit|Uncertainty RD|I|27|003|1002|11101|1|7|001|091|20120403|0:00||AN|1||||||||||||| RD|I|27|003|1002|11101|1|7|001|091|20120403|1:00||AN|1||DI|DN|EH|MD|MX|NS|QP|SS|SX|TB|| |
2.If headerLinesRegex is not defined, then the default column order defined in headerLinesDefaultValue is used.
Copyright © 2023 EarthSoft, Inc • Modified: 15 Aug 2022