Ticket #305 (new new task)

Opened 4 weeks ago

NCEI data provider for GOES SXI data

Reported by: niles Owned by: niles
Priority: highest Milestone:
Component: Undefined Version: 1.4
Severity: blocker Keywords: NCEI
Cc:

Description

There are GOES SXI data for goes at NCEI, see

https://satdat.ngdc.noaa.gov/sxi/archive/fits/goes15/2018/03/08/

Note that there are daily index files, but these seem to be randomly corrupted. I just wrote a C cfitsio program to read them, and found that out. Joe H had found something similar earlier.

Also the file sizes, which we need, are not in the index files anyway, even if they are not corrupted. The file sizes are not consistent.

So it seems we're going to have to download all the data, parse the relevant parts of the FITS header and the file size into some sort of database, then base a data provider on that.

Also there will need to be a "spider" that does all this going forward for data from GOES satellites that are still updating.

Note: See TracTickets for help on using tickets.