Projects/MidasBeaconReceiver/MidasBeaconReceiver/BeaconDataManagement

Software Radio Beacon Data Management

Overview of existing Data Management at Millstone Hill

An overview of some of the existing data management and user interfaces used at Millstone Hill.

• Madrigal

• OpenMadrigal homepage

• Millstone ISR specific UI - last year's REU project

Project tasks

• Learn required tools listed below

• Understand physical interpretation and scientific uses of beacon data

  • Reference Material

• Analyze how data is presently displayed

• Consider advanced visualization techniques that might be useful and understandable

• Write functional specification as part of overall project plan, including

  • Interface to signal processing chain

  • Loading into Madrigal

  • UI description and detailed design

• Design and code smaller test project - import local GPS scintillation data into Madrigal

• Implement full functional specification

• Test for correctness

• Test for usability with scientific users

Project tools

• Python programming language

• Matplotlib python graphics package

• Madrigal database

• Wiki documentation

• Html/Javascript - other web tools possible

** Exercises **

• Plot Geophyical Data - a python exercise using Matplotlib and the MadrigalWeb libraries.

  • The goal of this exercise is to build a class to plot and print Madrigal geophysical data, and then build a simple command line application to drive the class.

  • All data from file /opt/madrigal/experiments/1950/gpi/01jan50/geo500101g.002 on http://madrigal.haystack.mit.edu/madrigal.

  • The class should take a start datetime and end datetime as initializers (use datetime module). In its init method, it should read in all geophysical data (Kp, F10.7, Ap, Ap3) from Madrigal (use MadrigalWeb module from www.openmadrigal.org).

  • The class should have a plot method which creates a lineplot. Inputs arguments are which parameter to plot, and name of outputfile to write plot to.

  • The class should have a print method which prints the data to a file. Inputs arguments are which parameter to plot, and name of outputfile to write to.

  • A python script should be written to test this class. Its command line arguments are start date, end date, parameter name, plot or print, and output file name.

  • Use all python guidelines listed in main project.

• Create and load fake HDF5 data - create a dummy HDF5 input file with parameters you'd like in it. Write a program to load it into your local test Madrigal

  • Install Madrigal on your local machine

  • Add PyTables to Madrigal's local version of python

  • Decide on a HDF5 format you might like data in

  • Create a dummy HDF5 file in that format

  • Write a program to load it into Madrigal based on my SRI version

  • Iterate with new dummy HDF5 file if information missing