The Virtual Solar Observatory Overview

This document will explain the general structure of the Virtual Solar Observatory's architecture, the goals and objectives that influenced that architecture, and the terms and abbreviations used in this and other supporting documentation.

Goals and Objectives

For a detailed explanation of the design influences for the VSO, please see ​The Virtual Solar Observatory Design Proposal, 2002 November. Relevant excerpts are contained below.

Principal Features

• the ability to search all participating data sources with a common interface,
• the use of existing data query facilities at the current data sources,
• the use of industry-standard protocols such as eXtensible Markup Language (XML) in its internals,
• access through a Web browser interface to a remote VSO server, an Application Programming Interface (API), or via a VSO instance on a local computer, and
• extensibility to multiple additional features, including the registration of searches to enable other solar physicists to, for instance, attempt to reproduce the results of published literature.

Original Prototype

The original VSO prototype, which is occasionally referred to as version 0.5, contained the following features:

• virtualization of data search, data discovery, and query refinement,
• multiple interfaces (browsers, application programming interface),
• leveraging of existing data services (rather than creating e.g. new metadata standards),
• direct user access to data (without the VSO as an intermediary), and
• the ability to expand to several more data sources in the implementation and maintenance phases of the VSO (FY04 and beyond).

The prototype specifically did not include the following:

• a central catalog,
• grid computing
• any features that limit or restrict access to data or software (authentication).

Planned features for later implementation included the following:

• catalog caching (to speed queries),
• multiple instances of the VSO (to make the VSO truly virtual, it could run on local machines instead of on a small number of remote servers), and
• logging of searches (without identifying the party who carried out the original search).

This reference implementation imparted the following ideas into the project:

First, it is not a monolithic server with a database that stores metadata. Instead, it is an agile, lightweight application program that accepts input (user queries) and provides output (data location and access). We store XML descriptions of data providers as a data structure (it can reside in memory) instead of static files or records.

Second, we treat each user input and query not only as a transaction but as a real-time computation. By doing so, we are able to view VSOs inner-parts as instantiated objects and perform operations or manipulate them by applying methods.

Finally, the deployment of VSO may be fully distributed with the ability to have multiple, fully customizable VSOs running on user desktops.

VSO 0.5 Architecture

 UI  --> VSO --> WS --> DP

UI  : User Interface
VSO : VSO 'Core' Application
WS  : Web Service
DP  : Data Provider

VSO 0.6

With VSO 0.6, the individual objects were simplified and attempts were made so that each one could function autonomously. The system was modeled as additional objects, while still using the basic overall concepts from 0.5:

              Registry
                 ^
                 |
 UI --> UQI --> VSO --> DQI --> DP

UQI : User Query Interface
DQI : Data Query Interface

Improvements in 0.6

The changes from 0.5 to 0.6 were refinements of the VSO architecture to assist with later maintenance of the VSO. In addition, the data model significantly changed with this revision. The following additional objectives were realized:

• the addition of new data providers should not require the modification of existing code,
• the modification of the VSO data model should not require significant changes to the VSO Core, and
• the modification of the VSO data model should not require immediate action by VSO Data Providers.

To deal with the implications of these new objectives, there were some minor changes

VSO 0.7

With VSO 0.7, we attempted to move specifics of individual Data Providers out of the User Interface. To meet these goals, we did the the following:

• the ability to request the necessary data provider information to generate query forms, and
• the ability to determine the proper procedure to download data from a particular data provider.

These changes have resulted in the need for an additional component, the Central Server. The Central Server contains the authoritative records for various information that the VSO Core instances and User Interfaces may require to provide a consistent experience with the addition of new data providers. To maintain the virtual aspects of the system, provisions have been made for multiple Central Server mirrors, and for the possibility of no Central Servers being available.

VSO 0.7 was never released to the public, due to flaws in the logic for ordering data, specifically in the cases of a 'partial success' situation.

VSO 0.8

To solve the logic errors present in 0.7, modifications were made to the ordering mechanisms within VSO. The following changes were made:

• Data Providers returned enough information for a User Interface to repeat the query, without having maintained records of what the original query was.

VSO 1.0

The 1.0 release was a series of small, but subtle refinements to VSO, to move it to the point where solar researchers could use it for regular tasks.

The following modifications were made:

• Data Serialization was improved to support the possibility of communicating with components written in strictly-typed languages, and
• Mechanisms were added to permit Data Providers to extend the messages to send additional information.

More obvious to the general public, however, was a revamping of the default web-based User Interface, which included the following:

• Customizable search form generation,
• improved searching by 'nicknames',
• thumbnail previews of data products when available, and
• improved data product ordering capabilities.

VSO 1.2

The 1.2 release incorporated refinements suggested by the solar physics community, and attempted to make the overall system easier to maintain.

The following modifications were made:

• Created a system to browse the VSO Registry,
• Populated additional information in the VSO Registry, including 'data layout',
• Provided a mechanism to allow a provider matching to work for providers with multiple web services,
• Improved handling of unit conversions in spectral ranges searches, and

Significant changes were made to the web-based User Interface, with the result of removing all data provider specific code from the User Interface.

Modifications to the web-based User Interface included the following:

• Introduced the 'catalog search' interface, to allow the time parameter to be derived from solar event catalogs,
• provided online help for search terms,
• added a 'drill down' interface as an alternative to select instruments,
• added a system to allow VSO Cart IDs to be used to reload the results saved from past searches,
• added a mechanism to turn the browse images for VSO data products into a slideshow, and
• provided a mechanism to pass arguments into the search form to pre-select items.

VSO 1.4

The 1.4 release included modifcations to both the underlying core, and the web-based User Interface to enable scientists to more easily identify data sets of interest.

The following modifications were made to to the VSO internals:

• Populated additional information in the VSO Registry, including 'detector' and 'filter',
• and patched a problem with incorrect typing of data returned from the API.

Some modifications were made to the web-based User Interface, to allow users to select based on instruments or observatories, without knowing the specific data repository that maintained the data products. Some cosmetic changes were made to unify the look and feel of the web-based User Interface and the online documentation for VSO.

What VSO isn't

VSO's primary goal is to discover and facilitate the retrieval of data products. Although later VSO User Interfaces may automate the retrieval process, or post processing of data products, this is outside of the scope of VSO's primary objectives, and may not be present through all VSO User Interfaces.

VSO is a lightweight, virtualized system. It provides a framework for finding data products, but does not maintain an itemized list of data products available. The VSO Registry maintains generalized information about the product offerings from each data provider, which may be made more specific in the future to assist in optimization efforts.

Future Plans

The Virtual Solar Observatory project will work with other similar and complementary projects to make data products and services more widely available to the solar physics community. Data providers will be added to provide additional data products. User interfaces will be added to provide for both general and niche purposes. Features will be added and refinements made to assist solar researchers in finding data products that may be of interest.