The JMD

The JMD is a tool that facilitates the download of data between sites. It was written in java by NSO staff. It can operate in one of two modes, MIRROR or USER. In MIRROR mode an attempt is made to duplicate data at another site. In USER mode, when a user requests data, the JMD will attempt to download it. The JMD operates in combination with the NetDRMS software, updating the pertinent databases when data is downloaded.

The directory the JMD is installed in varies between installations and for the purposes of this discussion will be referred to a $JMD_HOME (although no such environment variable exists in practice). Under this directory are subdirectories that can be useful for maintenance.

Logging

JMD logs are written to the directory $JMD_HOME/log/ and follow the naming convention YYYY_MM_DD.SITE.stderrout.log, for instance "2014_02_10.NSO.stderrout.log". If the JMD is restarted then the existing log file has the process ID of the JMD appended to it and another log file is started, so the existance of the log files "2014_01_28.NSO.stderrout.log" and "2014_01_28.NSO.stderrout.log.174918515" indicates that the JMD restarted on January 28, 2014 at the NSO site. Log files list downloads that the JMD has attempted. so the line :

Feb 10, 2014 1:12:35 AM org.vso.jmd.Downloader.SCPDownloader call INFO: Th ID:[9261]; SU:[528482873]; RN:[131877486];[aia.lev1];[MIRROR]; Sz:[12490615]; STP:[DONE]; ST:[DONE] [1.0829072MB/s] [SAO]

Indicates a successful download from the SAO site. The line :

Feb 10, 2014 12:07:25 AM org.vso.jmd.Downloader.SCPDownloader call INFO: Th ID:[9243]; SU:[528417871]; RN:[131843058];[aia.lev1];[MIRROR]; Sz:[10443000]; STP:[SCP]; ST:[FAIL] [N/A] [JSOC]

Indicates a failed download attempt from the JSOC site.

If the JMD is running, you should be able to see it with the command "ps aux | grep java | grep JMD" which should show something like this :

jmdUser      25395 13.6  0.4 3018892 400780 ?      Sl   Jan28 2550:19 /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/bin/java -Xmx2048m -Djava.io.tmpdir=/my/JMD/install/JMD_v1.5.1.1/tmp \
-jar /my/JMD/install/JMD_v1.5.1.1/jar/JMD_v1.6.4.0.jar --id NSO --port 8080 --log /my/JMD/install/JMD_v1.5.1.1/log --run /my/JMD/install/JMD_v1.5.1.1/tmp --minThreads 10 --maxThreads 80 --cfg /my/JMD/install/JMD_v1.5.1.1/cfg/JMD.cfg

Different versions of the JMD will have different command line arguments, but they should all show java running a .jar file, and they should all reference a configuration file (in this case /my/JMD/install/JMD_v1.5.1.1/cfg/JMD.cfg). The configuration file defines the parameters the JMD runs with and is generally human readable. It typically resides in the $JMD_HOME/cfg directory.

Control Scripts

Scripts relating to JMD operation reside in the $JMD_HOME/bin directory. The primary script is JMD.sh which can be used to start, stop and restart the JMD. Running this script without any arguments will cause it to print a message explaining its syntax.

The script jmd_admin.pl can be used to initiate a JMD download, reload the config file, set the level of logging verbosity or generate statistics. Again, running the script without arguments prints a message explaining its syntax.

The script jmdij gives access to the underlying derby database (see below).

The JMD database

The JMD incorporates a lightweight third party java-based SQL database. This is used to track what the JMD is currently processing (including errors).

Debugging commands

1. Pinging the JMD The "Are you there" test, and testing data downloading
2. Examining the logs for the JMD - some useful techniques
3. The underlying JMD database
4. What to do if data are offline at the JSOC