Ticket #340 (new problem)

Opened 8 days ago

Last modified 4 days ago

vsoui not coping with decimal in wave range - core bug?

Reported by: niles Owned by: niles
Priority: highest Milestone:
Component: (unassigned) Version: 1.4
Severity: blocker Keywords:
Cc:

Description

This vsoui URL with integer wavelengths in Angstroms works :

https://sdac.virtualsolar.org/cgi/vsoui?doAction=startyear&timerange=20110102000000-20110102235959&waverange=6000-7000Angstrom&datatype=IMAGE&physobs=intensity&pptid=10600&fileid=pptid%3d10600%3bdate%3d20110102000000-20110102235959&instrument=NSO.ISOON.ISOON

But if you change it to use a decimal the parsing seems to get messed up.

Using waverange=6000.1-7000Angstrom searches 6000 to 6000 with unit ".1-7000Angstrom"

Using waverange=6000-7000.1 searches 6000 to 7000 unit ".1Angstrom"

Using 6000.3-7000.1Angstrom searches 6000 to 6000 unit ".3-7000.1Angstrom"

I think this is a bug in the core?

Change History

comment:1 Changed 8 days ago by niles

Looking at the file /opt/vso/lib/perl5/Physics/Solar/VSO/UserInterface/VSOUI.pm on vso03.nispdc.nso.edu I see :

$rParams{waverange} .= sprintf("%d-%d%s",(defined($wavemin)&& $wavemin=~/\d+/)?$wavemin:$wavemax,(defined($wavemax)&& $wavemax=~/\d+/)?$wavemax:$wavemin,$params->{wave}->{$_}->{waveunit});

That looks like it will only match integers - I think a better match would be :

$rParams{waverange} .= sprintf("%f-%f%s",(defined($wavemin)&& $wavemin=~/^([+])?(\d+)?\.?(\d+)?$/)?$wavemin:$wavemax,(defined($wavemax)&& $wavemax=~/^([+])?(\d+)?\.?(\d+)?$/)?$wavemax:$wavemin,$params->{wave}->{$_}->{waveunit});

That match should match any positive decimal wavelength.

Ed, what do you think? Do you agree? And can we make this change and push to NSO and SDAC?

comment:2 Changed 8 days ago by niles

Ahh shoot there's another part of the code that needs to change too - looking at the file /opt/vso/lib/perl5/Physics/Solar/VSO/UserInterface.pm I see :

sub parseWaveRange {
        my ($self,$wavestring) = @_;

        # my ( $min, $max, $unit) = ( $wavestring =~ m/^\s+(\d+)(?:-(\d+))?(?:\s*([a-zA-Z]+))?$/ );
        my ( $min, undef, $max, $unit) = ( $wavestring =~ m/^([0-9]+)( *- *([0-9]+))? *(.*)/ );

        if ( !defined($min)  or  $min eq '' ) { return ();         }
        if ( !defined($max)  or  $max eq '' ) { $max  = $min;      }
        elsif ( $max < $min  )                { return ();         }
        if ( !defined($unit) or $unit eq '' ) { $unit = 'Angstrom' }

        return ( { wavemin => $min, wavemax => $max, waveunit => $unit } );
}

I'm not able offhand to put together the regexp needed to match floating point, but what we have is only going to match integers, and that needs to be expanded to floating point.

comment:3 Changed 8 days ago by ed

The main place that will need to be checked and updated is the part of the query for wavemin/wavemax sent to each DataProvider? since the Spectral Range form sends it's query to all Provider/Source? pairs that match the datetime & wave parts of the query. The wave part of the query is formatted in each DPs main package generally (eg. ISOON.pm)

Note that the WSDL is already set to expect wavemin and wavemax to be floats (see Serializer.pm)

I would test by changing the wave part of the query in ISOON.pm only to send to the DB a pair of floats for wavemin/wavemax. I wouldn't change the core just yet to avoid breaking existing functionality for other DPs. Test and confirm ISOON works with the changes as expected before making any changes to the core. Some of those functions mentioned are used for waverange and other matchings.

comment:4 Changed 4 days ago by niles

The immediate problem has been fixed by updating the Oracle database at NSO with integer values for wavelength for ISOON.

This remains an issue, however, because there's no reason it can't happen again. So, not closing the ticket.

Personally I think we should condition whatever the data providers send us to be integers when we put together the URL when we drill down from the multi day view to the single day view. I'm less convinced that we should fix the CGI to cope with floating point wavelengths, but perhaps that should be discussed more. I may not be seeing all the ramifications.

comment:5 Changed 4 days ago by niles

Also the Oracle database fought the update all the way. I had to write some perl to break it down into a day-by-day update rather than just updating the table as a whole, which seemed to cause memory swapping that slowed the DB down to the point of not being usable.

Note: See TracTickets for help on using tickets.