MADIS Plugin Report
by Jim Fluke
Last modified 20 May 08.
- Used the MADIS web page
to get a sample of XML formatted MADIS data. Selected the standard
surface variables. In Firefox I had to select "View/Page Source" to see
the data. Here is a small sample of the data.
- Installed both EDEX and ADE on my development workstation following Joanne's instructions. And insured that I could build and install using ant.
- Downloaded the TO8 version of the EDEX Plugin
tool from Cliff Wong's 3/13/08 email message, and installed it in my
the Plugin Tool from Eclipse by selecting "Window/Show
View/Other.../Other/Plugin Creator". Chose
gov.noaa as the Organization Domain and madis as the Plugin Name. Did
not select IPersistable or "Create Separator". Turned on the "Data URI"
checkbox for all of the data fields. Chose AWIPSEdex/extensions as the
output directory, and let the tool create the plugin-madis
subdirectory. There was no indication that the plugin had been created
other than the "Generate Plugin" button appearing depressed for a short
time. In the "Package" tree, selected the "extensions"
label under "AWIPSEdex" with the 3rd mouse button. This highlighted the
label and brought up the 3rd button menu. Selecting "Refresh" caused "plugin-madis" to appear under "extensions".
- Implemented MadisDecoder.java, the helper class file MadisParser.java and package-info.java under "AWIPSEdex/extensions/plugin-madis/src/gov/noaa/edex/plugin/madis".
eclipse, added the plugin to the Build Path by highlighting the
extensions/plugin-madis/src directory and selecting to
add it to the build path through the 3rd mouse button menu. This causes
the directory to be displayed directly under AWIPSEdex as a
build path selector. Then saw an "Unexpected Package" error even though
the package was correct. Fixed it by shutting down eclipse and starting
it back up again.
- Added plugin-madis to build/deployments/deployment.properties.
- Tried to build, outside of eclipse using ant. Got jibx build errors about
gov.noaa.edex.madis. Corrected them by changing
extensions/plugin-madis/res/binding.xml. Both had gov.noaa.edex.madis
instead of gov.noaa.edex.plugin.madis.
- Retried the build. Got a Calendar
serialize/deserialize error from jibx. Borrowed the
Calendar serializer/deserializer attributes from the pirep plugin binding.xml
for the madis binding.xml.
- Retried the build. It worked.
- Tried to run it after deploying it to my local EDEX install area. Mule crashes. Here is the error log.
edex/opt/esb/conf/madis-ingest.xml from my EDEX working area. Mule runs
just fine, but of course the madis plugin is not included.
- Created a tar file of my plugin from the projects/AWIPSEdex directory. This old file is no longer available. Use the new one from step 26.
- Changed to fully qualified variable type names madis.hbm.xml. Mule now starts up successfully.
- Copied a small madis data file (the sample above)
to <EDEX>/edex/opt/data/sbn/madis. This gives an xml parsing
error about: "...<columnType><UNKNOWN TYPE>...".
- Found the error in madis.db.xml and fixed it. No
longer see the xml parsing error. Now see an error about not being able
to create the MadisDecoder. Can't get its "properties."
- Added Name and Plugin properties elements to
plugin.xml to make it look like the pirep plugin.xml. This fixed the
previous error, but now there is new one complaining about "Could not
- Added logger.info() statements to PluginDecoderProxy.java to try to find the problem. The problem is in my madis java code.
- Added logger.info() statements to
MadisDecoder.java and MadisParser.java. MadisParser get to the "elev"
field and then stops - can't parse it?
- Added a try/catch block around the code in
MadisParser.java that populates the records. Got a
NumberFormatException from Integer.parseInt() for "elev". Ok, so "elev"
does have a floating point format - it always ends in ".00".
- Changed the elev parsing to use
Float.parseFloat() and then convert to an int. Now the exception is:
Error creating data URI for MADIS product:
- Added logger.info() statements to
PluginDataObject.constructDataURI() to find out which madis field is
causing the problem. Its ObTime, but why? constructDataURI() looks like
it deals with the Calendar type. Added more
logging statements. Didn't help. Added a try/catch block around the PropertyUtils.getProperty() call in constructDataURI(). Got:
ERROR 2008-05-16 22:30:38,239 [Awips.Edex.Service.IngestSrv-madis.2]
MadisRecord: java.lang.NoSuchMethodException: Unknown property ObTime'
With no trace back. Strange.
- Looked at the obs plugin per Tom Kretz'
suggestion. It does not use any Calendar type data members for the
dataURI, and it does not look as though any of the other Raytheon
plugins do either. Changed madis.db.xml so that ObTime is not part of
- It works! At least mule says the records are being "Successfully persisted to database".
- Brought up pgadmin3. Had to get Leigh's help to
find the data, but we did find it: Created a server with "awips" as the
password and myself as the user (since this is the edex I installed as
myself). Looked under databases/metadata/Schemas/awips/Tables/madis*.
Used a 3rd button click on eachmadis label to "View Data".
- Created a tar file of my plugin from the projects/AWIPSEdex directory. You can get it here.
Again, you should be able to extract it into your ADE working area with
out any trouble as long as you don't have an extensions/plugin-madis
directory. This tar file does not include
build/deployments/deployment.properties. You will need to add
plugin-madis to the end of this file.