Changes between Initial Version and Version 1 of Meetings/TechReports/2005Oct28

Timestamp:

10/31/05 06:13:23 (20 years ago)

Author:

joe

Comment:

--

Meetings/TechReports/2005Oct28

@@ +1 @@
+= Technical Teleconference, 2005 October 28 =
+
+== Agenda / Context ==
+
+Igor requested a meeting to discuss how to use VOTable within VSO to make it easier for integration with other Virtual Observatories.
+
+{{{
+Date: Thu, 27 Oct 2005 19:18:50 -0700
+From: Igor Suarez-Sola 
+To: Joe Hourcle, Karen Tian, Alisdair Davey
+Subject: VOTable for VSO
+
+
+Hi all,
+
+I want to clarify that the adoption of the VOTable format is for our 
+metadata transactions and not scientific use (e.g. handling of fits 
+files etc) and that the driver for this change is mainly performance. 
+Although if we can be compliant with the VO all the better.
+
+Also , hopefully, the adoption of the VOTable would be used internally 
+by the CORE, GUI, Catalogs, CART and providers ...
+See attached examples of VOTable.xml and VSOTable.xml (the only 
+difference being in the TABLEDATA part). Both formats can be handle 
+internally. However the interfaces can switch to one another and that'll 
+be toggled by passing a flag in the request message.  The VSOTable gains 
+a little bit of performance over the <TR><TD> one .. mainly in memory 
+usage for large sets.
+
+The VOTable structure used in its simplest form E.g. "VOTable.xml" is 
+quite straight forward. However I see how it can get awkwardly complex 
+as we get into rough territory like keeping state, linking results set 
+with Queries etc. For instance for catalog entries in one ResultSet 
+could yield a new query and to a new result set. Although in principle 
+all that could be achieved by sticking to the VOTable schema. It can get 
+very twisted and I don't quite see the advantage of that ... and that's 
+why I think the VOTable format should be used only for the search 
+results, while everything else internal to the VSO like states, link to 
+other queries and links to result set are handle as we do now in our own 
+format. (mmmh ... it now sounds like the obvious thing to do ...  
+probably you'll be thinking what the hell is he talking about ... )
+
+
+Anyway going back to the VOTable.xml example ... things I want you to 
+pay attention to ...
+1. We can have multiple RESOURCE elements
+           We can take the RESOURCE element as the equivalent of one 
+provider "search results"
+           a) use the DEFINITIONS element to specify the provider and 
+version maybe other information related to the provider??
+           b) The TABLE element will be the actual result set
+                  I- Use the PARAM element to pass information relative 
+to the search result namely no_of_records_found and no_of_records_returned
+                  II - A number of info messages (we can't put much 
+granularity there but might suffice). If we want something more complex 
+like  message type , level of  debugging etc we would have to bend the 
+schema a little bit.
+
+2. Alternatively we could have one RESOURCE and multiple TABLE elements 
+where each TABLE will be the equivalent to one provider "search results" 
+see attached "VOTable_2.xml" and the version and provider information 
+goes as PARAM sub-elements. Note that the xml schema attached is version 
+1.0 and doesn't seem to allow that, however a pdf I have on VOTable 1.1 does
+
+3. The other issue is if you want to pass or keep the 'query input 
+parameters' in the VOTable ... probably not but I thought of asking ...
+
+
+Unfortunately I can't show you the code at this time. I thought I could 
+give it to you today but I lost some time thinking in generalizing  the 
+decoding of the VOTable xml. To be honest is quite straight forward as 
+long as we keep it simple . I.e the VOTable for search results only. 
+However the handling on the provider side and the integration with the 
+DataProvider.pm is mostly done as there weren't much to change there. So 
+I'm finalizing the xml decoder for the VOTable .. and that will take 
+tomorrow and then over the weekend I wrap up the test files and 
+documentation ... yes Joe you heard/read well 'I am DOCUMENTing'  as I 
+go along ... ;-)  ... so by moday you could have a look at it .. sorry 
+for the delay!
+
+
+Talk to you tomorrow!
+
+Igor
+}}}
+
+== Decisions ==
+
+{{{
+Should we extend / modify VOTable?
+
+        - Don't modify the structure.
+        - Maybe add attributes if needed for extra VSO information
+        - Try to find some VOTable folks, so we can ask why they laid it
+          out how they did, and if they have any special intent for
+          'resource', etc.
+                - Igor might have local folks to ask
+                - Joe might go to the SPASE meeting next week, and has
+                  some local folks at GSFC to look up
+        - We should try to make our format readable by VOTable tools.
+                - Need to find what VOTable tools exist
+                  http://viewcvs.cacr.caltech.edu/us-vo/viewcvs.cgi/contrib/VOTable/
+
+        - Try to come up with something workable, and see how well it
+          performs (speed, memory usage, compatability w/ existing tools),
+          rather than try to worry about it beign perfect in the first
+          revision.
+
+Speed / Memory optimizations
+
+        - Joe will look at the serializer bits, and see if we can avoid
+          deserializing the content in the Core, so it's passed through as
+          an xml string
+                - May have to parse for version compatability issues (eg,
+                  deal with outdated providers and/or clients)
+        - Get something working first, then find bottlenecks & optimize.
+}}}