Changes between Version 2 and Version 3 of jmdBulkLoad

Timestamp:

02/22/15 05:54:03 (9 years ago)

Author:

joe

Comment:

--

jmdBulkLoad

@@ -28 +28 @@
 
 If we have a list of VSO fileids, we can write queries that will put the appropriate values into the table `public.sunum_queue`, which is used to track new observations that we'd like the JMD to retrieve.
+
+**warning : something is wrong with this method.  See note at end**
+
 
 From IDL, we can get a list of VSO fileids:
@@ -71 +74 @@
 Note that this technique should only be used after the other options.  Due to the way that the JMD pulls records out of the queue table, more recent observations will take precidence.
 
+**note** : somehow, the queue went down by ~5k, and then stopped processing.  I had to manually move the sunums from DRMS to the JMD:
+
+{{{
+alias drms='psql -U postgres -d sdac_drms'
+drms -c "select sunum from sunum_queue where series_name like '%bulk'"' > /tmp/for_mk
+perl -e 'my $i=100,@sunums; while (my $s = <>) { next if $s!~m/^ ?\d+$/; $s=~s/\s//g; push (@sunums, $s); if ( !$s or !--$i) {  my $sunums = join(",", @sunums); print qq{/opt/JMD/bin/jmd_admin.pl --request -- --series="aia.lev1 bulk2" --sunums=$sunums\n}; last if !$s; $i=100; @sunums=() }}' /tmp/for_mk  > /tmp/jmd_load_mk
+sh /tmp/jmd_load_mk
+drms -c '"delete from sunum_queue where series_name like '%bulk%'"
+}}}
+
+