Ticket #267 (accepted enhancement)

Opened 5 months ago

Last modified 3 weeks ago

Support for Python-based DataProviders as web services

Reported by: ed Owned by: ed
Priority: normal Milestone:
Component: Core Version:
Severity: blocker Keywords: Python, zeep, ladon


To support DataProviders written in Python as either a SOAP service or a RESTful web service, new code in Core.pm is required, which in turn will make use of new meta-data in VSO table "services".

Additionally, validation of the XML Schema of the new web service should be built into the code so that automated notification to both the end User and the DP personnel can be done, thereby ensuring robust error propagation.

Testing support for two of the newer Python modules for SOAP and RESTful services, zeep and ladon, respectively will be done.

Change History

comment:1 Changed 5 months ago by ed

  • Status changed from new to accepted

comment:2 Changed 3 weeks ago by ed

To support DataProviders written in other languages, and/or DataProviders that will provide their data via a RESTful Web service instead of SOAP, one key set of CPAN modules that is needed is Inline and friends. Inline is also available in Python as well.

Inline provides the functionality to call classes and methods in classes written in Python (for example) from Perl.

The Perl code in Core.pm at the heart of the VSO codebase creates a thread for each DataProvider? that matches the User's query. In initializing each thread, a dispatch method is called, which needs to determine (a) is the DataProvider? written in Perl, or some other language, and (b) is the DataProvider? supplying their data via SOAP or a RESTful Web service.

To handle (a) the following code using Inline and JSON was found after testing to allow Python methods to pass their data back to Perl:

In Core.pm


$ENV{PYTHONPATH} = '/path/to/custom/python/classes';


package Physics::Solar::VSO::Core

.... existing use statements...

use Inline::Files; use Inline (Python => 'file', directory => '/tmp/_Inline'); use Inline::Python;

use JSON;

... other existing use statements...

sub _TQueryDispatcher {

... existing code...

my $dp_type = $args->{'dp_type'}; # Type of DataProvider?, stored in meta-data table

if (!$dp_type) {

$soap = SOAP::Lite->new( ...args...)

... existing SOAP code...

} else {

my $pyobj = new VSO_WS # new Python object called from Perl

$pyobj->{'dbquery'} = $block; # pass User query data to Python as attribute

$pyobj->query(); # call Python method to execute the query


... existing code



from VSO import VSO_WS # Python code embedded in Perl

Note: See TracTickets for help on using tickets.