wiki:NetDRMSInPostgres9
Last modified 10 years ago Last modified on 08/13/14 12:03:16

Should I compile NetDRMS against Postgres 9.x?

If you can, you probably should. As of late July 2014, Postgres will not support version 8.4 past 8.4.22. There will be no more patches. For those on federal facilities, "no more patches" means you will be mandated to upgrade the program or you will have lots of explaining to do. That said, the most compelling reason for not updating Postgres is if you are running Slony replication as an active master. If you are a "Slony master", such as the JSOC at Stanford, then you cannot support Slony 1.x against Postgres 9.2. Stanford's replication processes at this time require Slony 1.2, and Postgres 8.4.x is the last version supported for that particular Slony version. However, if you are only a passive Slony recipient, getting slony files and database changes but never issuing any yourself, then you are not confined to any particular database version. Before making any change you should first have a conversation with your systems administrator and your department about changing their database since they might have rules or scripts that require testing.

If you're reading this and have decided to go forward with your 9.x compile, changes required are minimal. The author of this wiki page has only compiled as far as Postgres 9.2 and NetDRMS 7. It is possible that further changes could be necessary for versions > 9.2 and your situation may be different. However, for a NetDRMS 7 compilation against Postgres 9.2, you will need to:

  • Backup and Migration

If you are doing this on the primary data server, first slap your own hand and then go work on a test bed machine. If that is unavailable, then do a full backup of your production database, and be certain you've configured Postgres 9 to run on two different ports (one for the drms database and another for the sums database). Make doubly sure that all your usual keyboard shortcuts and aliases are re-worked (perhaps in a new .profile or .login file) to reference the new database compiling environment. If you are upgrading from an 8.4.x database, recall that you are undergoing a major version change and that involves "data migration". You may find that the migration tool provided by Postgres, pg_upgrade, is unsuccessful. If so, you may need to migrate your data using a plain-text format data dump (with SQL "insert" commands). Check the Postgres website for migration options (http://www.postgresql.org/docs/9.2/static/upgrading.html) and do some testing before assuming the 9.2.x database is fully ready. Better safe than sorry.

  • Change one line in two places in the SUMS code

Once your database is correctly populated and backed up, you can look at the NetDRMS code. Change two files in base/sums/libs/pg named SUMLIB_RmDo.pgc and SUMLIB_RmDoX.pgc and make this modification to line 9

// EXEC SQL TYPE uint64_t IS unsigned long long;
EXEC SQL TYPE uint64_t IS unsigned long int;
  • Change config.local

Ensure config.local has the right paths listed for Postgres-specific variables (config.local file is located in the top netdrms source directory).

  • Local environment variables

You may need to make changes to local environment variables if you have an existing Postgres 8.4.x installed on the machine. Set your $PATH and $LD_LIBRARY_PATH variables properly to reflect the Postgres 9.x paths. Be sure to put paths for the new installation first in the order, ahead of any 8.4.x locales for $PATH and $LD_LIBRARY_PATH so that the 9.x paths are used first. You might want to do this in a .profile file and source it every time you log in to avoid confusing errata later. The full list of environment variables follow - use appropriate values for your location and machines:

export LD_LIBRARY_PATH='/usr/pgsql-9.2/lib/'
export PATH=/usr/pgsql-9.2/bin/:/opt/netdrms7/source/bin/linux_x86_64:$PATH

export JSOC_ROOT=/opt/netdrms7/source
export JSOC_DBHOST=machinename_localhost  --whatever you're calling your database host in your .pgpass file
export JSOC_DBNAME=facilityname_drms
export JSOC_DBUSER=your_user
export JSOC_MACHINE=linux_x86_64  --assuming you're on that linux platform - yours may be different
export DRMS_DBPORT=5433  --author's test port number, default is 5432
export SUMPGPORT=5435    --author's test port number, default is 5434
export DRMS=$JSOC_ROOT
export SUMSSERVER=machinename_localhost

The author had some difficulty with NetDRMS actually using $JSOC_MACHINE's value. Eventually, all lines in $JSOC_ROOT/build/jsoc_machine.csh had to be commented out except for the shebang and "echo linux_x86_64" before compilation of NetDRMS7 was successful. For some reason, the program kept trying to build for linux_avx, and this led to fatal errors. It is possible this was only a local problem, however, it seemed worth noting.

There you go - that's all that is required.


Here's a little program from Igor to test this integer change, and some sample output:

#include <stdio.h>

int main(int argc, char**argv) {
  unsigned long int var1;
  unsigned long long var2;

  printf ("unsigned long long [%d] and unsigned long int [%d]\n",sizeof(var2), sizeof(var1));


  return 0;
}

Save program as: long_int_size.c then do gcc -o long_int_size long_int_size.c and then execute ./long_int_size

Sample results are as follows:

sdo3>  ./long_int_size
sdo3>  unsigned long long [8] and unsigned long int [8]