Changes between Initial Version and Version 1 of Meetings/TechReports/2005Oct28


Ignore:
Timestamp:
10/31/05 06:13:23 (19 years ago)
Author:
joe
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Meetings/TechReports/2005Oct28

    v1 v1  
     1= Technical Teleconference, 2005 October 28 = 
     2 
     3== Agenda / Context == 
     4 
     5Igor requested a meeting to discuss how to use VOTable within VSO to make it easier for integration with other Virtual Observatories. 
     6 
     7{{{ 
     8Date: Thu, 27 Oct 2005 19:18:50 -0700 
     9From: Igor Suarez-Sola  
     10To: Joe Hourcle, Karen Tian, Alisdair Davey 
     11Subject: VOTable for VSO 
     12 
     13 
     14Hi all, 
     15 
     16I want to clarify that the adoption of the VOTable format is for our  
     17metadata transactions and not scientific use (e.g. handling of fits  
     18files etc) and that the driver for this change is mainly performance.  
     19Although if we can be compliant with the VO all the better. 
     20 
     21Also , hopefully, the adoption of the VOTable would be used internally  
     22by the CORE, GUI, Catalogs, CART and providers ... 
     23See attached examples of VOTable.xml and VSOTable.xml (the only  
     24difference being in the TABLEDATA part). Both formats can be handle  
     25internally. However the interfaces can switch to one another and that'll  
     26be toggled by passing a flag in the request message.  The VSOTable gains  
     27a little bit of performance over the <TR><TD> one .. mainly in memory  
     28usage for large sets. 
     29 
     30The VOTable structure used in its simplest form E.g. "VOTable.xml" is  
     31quite straight forward. However I see how it can get awkwardly complex  
     32as we get into rough territory like keeping state, linking results set  
     33with Queries etc. For instance for catalog entries in one ResultSet  
     34could yield a new query and to a new result set. Although in principle  
     35all that could be achieved by sticking to the VOTable schema. It can get  
     36very twisted and I don't quite see the advantage of that ... and that's  
     37why I think the VOTable format should be used only for the search  
     38results, while everything else internal to the VSO like states, link to  
     39other queries and links to result set are handle as we do now in our own  
     40format. (mmmh ... it now sounds like the obvious thing to do ...   
     41probably you'll be thinking what the hell is he talking about ... ) 
     42 
     43 
     44Anyway going back to the VOTable.xml example ... things I want you to  
     45pay attention to ... 
     461. We can have multiple RESOURCE elements 
     47           We can take the RESOURCE element as the equivalent of one  
     48provider "search results" 
     49           a) use the DEFINITIONS element to specify the provider and  
     50version maybe other information related to the provider?? 
     51           b) The TABLE element will be the actual result set 
     52                  I- Use the PARAM element to pass information relative  
     53to the search result namely no_of_records_found and no_of_records_returned 
     54                  II - A number of info messages (we can't put much  
     55granularity there but might suffice). If we want something more complex  
     56like  message type , level of  debugging etc we would have to bend the  
     57schema a little bit. 
     58 
     592. Alternatively we could have one RESOURCE and multiple TABLE elements  
     60where each TABLE will be the equivalent to one provider "search results"  
     61see attached "VOTable_2.xml" and the version and provider information  
     62goes as PARAM sub-elements. Note that the xml schema attached is version  
     631.0 and doesn't seem to allow that, however a pdf I have on VOTable 1.1 does 
     64 
     653. The other issue is if you want to pass or keep the 'query input  
     66parameters' in the VOTable ... probably not but I thought of asking ... 
     67 
     68 
     69Unfortunately I can't show you the code at this time. I thought I could  
     70give it to you today but I lost some time thinking in generalizing  the  
     71decoding of the VOTable xml. To be honest is quite straight forward as  
     72long as we keep it simple . I.e the VOTable for search results only.  
     73However the handling on the provider side and the integration with the  
     74DataProvider.pm is mostly done as there weren't much to change there. So  
     75I'm finalizing the xml decoder for the VOTable .. and that will take  
     76tomorrow and then over the weekend I wrap up the test files and  
     77documentation ... yes Joe you heard/read well 'I am DOCUMENTing'  as I  
     78go along ... ;-)  ... so by moday you could have a look at it .. sorry  
     79for the delay! 
     80 
     81 
     82Talk to you tomorrow! 
     83 
     84Igor 
     85}}} 
     86 
     87== Decisions == 
     88 
     89{{{ 
     90Should we extend / modify VOTable? 
     91 
     92        - Don't modify the structure. 
     93        - Maybe add attributes if needed for extra VSO information 
     94        - Try to find some VOTable folks, so we can ask why they laid it 
     95          out how they did, and if they have any special intent for 
     96          'resource', etc. 
     97                - Igor might have local folks to ask 
     98                - Joe might go to the SPASE meeting next week, and has 
     99                  some local folks at GSFC to look up 
     100        - We should try to make our format readable by VOTable tools. 
     101                - Need to find what VOTable tools exist 
     102                  http://viewcvs.cacr.caltech.edu/us-vo/viewcvs.cgi/contrib/VOTable/ 
     103 
     104        - Try to come up with something workable, and see how well it 
     105          performs (speed, memory usage, compatability w/ existing tools), 
     106          rather than try to worry about it beign perfect in the first 
     107          revision. 
     108 
     109Speed / Memory optimizations 
     110 
     111        - Joe will look at the serializer bits, and see if we can avoid 
     112          deserializing the content in the Core, so it's passed through as 
     113          an xml string 
     114                - May have to parse for version compatability issues (eg, 
     115                  deal with outdated providers and/or clients) 
     116        - Get something working first, then find bottlenecks & optimize. 
     117}}}