Last Updated: 06/04/2003
The IDFS writer routines are aimed at helping you make an IDFS data set quickly and easily. These routines can not handle every possible scenario that will come up, but it can hopefully cover enough scenarios to get you on your way.
If you have not already read the documentation on creating an IDFS, do so since it will certainly help. Also, you will need to know some special parameters of how you are writing the data which you must decide based on how the data "looks".
You will not need to use all these routines. However, there are some which you must call. You must call the constructor, the Write, the Close, the SetDrTime, and the SetData routines. All others are there for when you need to call them.
The easiest way to use these routines are to look at the examples and try to mold them to your data set.
IDFS (char *virt_name, SDDAS_LONG nss = 1, SDDAS_INT n_sen = 1,
SDDAS_INT max_n_sample = 1, SDDAS_SHORT smp_id = 2,
SDDAS_INT cal_sets = 0,
SDDAS_USHORT max_tdw_len = sizeof (SDDAS_LONG),
SDDAS_LONG max_nss = 0, SDDAS_UCHAR num_status = 0);
The IDFS constructor takes between 1 and 7 parameters. Everything is defaulted
to writing a floating point scalar instrument with one sensor taking one sample at
a time. If you have something different, you need to specify more than one
parameter.
The first parameter is the virtual name of the data you wish to write. Since the filenames are based on the virtual name, this is very important.
This parameter corresponds to the nss field in the data record. These are the number of sensor sets you are writing in the data record. It must be less than max_nss.
The number of sensor defined within this virtual.
The maximum number of samples per sensor. If N_SAMPLE never changes, this value is identical to N_SAMPLE from the header file. If N_SAMPLE may change, set this value to the highest number of samples possible.
This parameter corresponds to the smp_id field in the VIDF.
The number of cal_sets defined in this virtual.
The maximum size of data in bytes that will be put into this virtual. If you have one floating point value and twenty unsigned chars in one virtual, the max_tdw_len would be 4. If that is the case, I recommend breaking out the floating point value and putting it in another virtual. With that, the max_tdw_len would be sizeof (unsigned char) for the virtual with only unsigned chars instead of sizeof (float).
The maximum number of sensor sets possible. This must be greater than or equal to nss. If you send in a zero (the default), it is assumed that this value is equal to nss. This value will be in the VIDF.
The number of mode/status fields. This number will be in the VIDF and the header record. The terms: mode, i_mode, status, and state are sometimes used interchangably.
~IDFS ();
This is the destructor which will free memory allocated by the constructor.
This is called automatically at the end of your program when you delete the
object you created.
int Write ();
When you send a Write, it will write one data record to disk. You MUST call Write somewhere in your program! The header records are kept track of and written to disk automatically.
void Close ();
This will close the data file out by writing an EOF to the header/data files. Call this at the end of your program or when you wish to create new files.
void SetYear (SDDAS_SHORT year);
void SetDay (SDDAS_SHORT day);
void SetTimeUnits (SDDAS_CHAR time_units);
void SetDataAccum (SDDAS_LONG data_accum);
void SetDataLat (SDDAS_LONG data_lat);
void SetSwpReset (SDDAS_LONG swp_reset);
void SetSensorReset (SDDAS_LONG sensor_reset);
void SetScanIndex (short pos, SDDAS_SHORT value);
void SetSensorIndex (short pos, SDDAS_SHORT value);
void SetDataQual (short pos, SDDAS_UCHAR value);
void SetModeIndex (short pos, SDDAS_UCHAR value);
void SetNSample (SDDAS_SHORT value);
All these routines are Set routines and what they take as a parameter
are the field you are setting. In other words, the SetYear takes
the year as a parameter. For what these routines do, see the IDFS Header
File documentation.
The odd routines which take two parameters take a position element as the first parameter. Remember: Scan Index, Sensor Index, and Data Qual are all arrays within the header file so the position is the element you wish to change.
All routines have defaults, but some of them (year and day), you have to set for the data file to make sense. For all values of the header file, the defaults are all set to zero.
If you use SetTimeUnits, I recommend setting the TimeUnits to microseconds (-6) since the header file already has fields doing calculations in microseconds.
Since N_SAMPLE is set as its maximum value in the constructor, you only need to call the SetNSample routine if the number of samples change during processing.
void SetDrTime (SDDAS_SHORT, SDDAS_SHORT, SDDAS_SHORT, SDDAS_ULONG = 0);
void SetDrTime (SDDAS_LONG);
void SetSpin (SDDAS_LONG);
void SetSunSen (SDDAS_LONG);
void SetHdrOff (short pos, SDDAS_LONG);
void SetData (short pos, SDDAS_UCHAR, SDDAS_SHORT sensor = -1);
void SetData (short pos, SDDAS_SHORT, SDDAS_SHORT sensor = -1);
void SetData (short pos, SDDAS_LONG), SDDAS_SHORT sensor = -1);
void SetData (short pos, SDDAS_FLOAT, NumberFormat = IEEE, SDDAS_SHORT sensor = -1);
void SetCalData (short pos, SDDAS_UCHAR);
void SetCalData (short pos, SDDAS_SHORT);
void SetCalData (short pos, SDDAS_LONG);
void SetCalData (short pos, SDDAS_FLOAT, NumberFormat = IEEE);
These are the data record routines. Much like the header routines, these take a parameter which is defined in the IDFS Data File documentation.
Note that the header offsets field takes a parameter for position. Also, keep in mind that you do NOT need to call SetHdrOff yourself as the software will change header offsets when needed.
SetDrTime has two forms. You may call it with either hour, minute, second, millisecond as the parameters or with millisecond of day as a parameter. Either is valid. You MUST call this routine! This information is used to generate the filename and to is written into every data record. This being the case, it must be called for every data record. For the first form, the milliseconds is defaulted to zero.
SetData has four forms. You can write either an SDDAS_UCHAR, SDDAS_SHORT, SDDAS_LONG, or SDDAS_FLOAT variable into the data record. For other data types, you might need to add to IDFS.C. SetData also takes an optional sensor number. If you pass in a fill value, it needs the sensor number to know which quality flag to set. If you know you will not be using any fills, this parameter is optional. Remember that the sensors are ordered starting with zero.
If you defined the number of cal_sets to a number greater than 0, you can define calibration data with the routine SetCalData. It, like SetData, has multiple forms for different kinds of values. Remember, depending on what the max_tdw_len is set to, this may determine which SetCalData to call. The calibration values must be the same size in bits as the largest value of sensor data. If the sensor data has a max_tdw_len of 2, you will need to call the SetCalData with the SDDAS_SHORT as the parameter.
void SetIntFillValue (SDDAS_LONG);
void SetFloatFillValue (SDDAS_FLOAT);
If these routines are called with a value specified for fill data, whenever
you call SetData, SetData will automatically set the DataQual to 0 and write
a new header record. Currently unimplememted.
void SetQuestionableDQual (SDDAS_UCHAR);
If you use fill values, it is recommended to set a quality flag to show that the data is questionable. The data quality value used when this occurs is determined by this procedure. The questionable value is defaulted to 1.
void SetDataDirectory (char *);
If you call this routine, all data for this object will be written into this directory. With this, you can seperate your data into their respective and correct data directories.
void SetDataFileMaxSize (long)
This routine sets the maximum data file size to the value sent in. This is a rough estimate since the data length is dependent on the size of the record. Do not set this expecting to see files of exactly this size. The default is roughly 1000000 bytes, or 1meg. If the file is too big, the files take far too long to promote. If the files are too small, you have a gazillion small files which take up valuable inodes on the file system. Furthermore, each file must have at least one minute of data so there is no name collision. This number must be raised if the data is high density and the data timing is extremely small. If the value is set to zero, the file size is unlimited.
void SetThreshhold (long)
If you have a data gap in your data files, it is generally better to create several new files instead of containing everything in one big file. The reason for this is so the IDFS data catalog will more accurately reflect the times of data availability. This routine allows you to set how big the threshhold of when you close the current file and open a new file in milliseconds. The threshhold is currently set to 300000 milliseconds or five minutes.
For our examples we choose two different data sets. One is the National Oceanic and Atmospheric Administration's TIROS/NOAA-6 through NOAA-11 satellites. The other is the first International Studies of Ionspheric Sciences Satellite or ISIS-1.
Neither was a particularly hard dataset since there are no Spectrogramable quantities in either dataset. Both also process one file at a time and I use a script to process multiple data files.
#include#include #include #include #include #include "isis.h" #include "IDFS.h" extern "C" { #include "intfp.h" // for the floating point to IDFS float routines } // ***************************************************************** // Initialize_ISIS_IDFS takes an IDFS data source and sets some // common parameters which are common to all IDFS' created by this // program. // ***************************************************************** void Initialize_ISIS_IDFS (IDFS *I) { I->SetTimeUnits (-3); // milliseconds I->SetIntFillValue (-32768); I->SetFloatFillValue (-1.0e-14); } // ***************************************************************** // SetTime_ISIS_IDFS takes an IDFS data source and sets the time // ***************************************************************** void SetTime_ISIS_IDFS (IDFS *I, long two_digit_yr, long day, long hour, long min, long sec) { I->SetYear (1900 + two_digit_yr); I->SetDay (day); I->SetDrTime (hour, min, sec); } int main (int argc, char *argv []) { char filename [40]; int ptr_to_isis, empty; // Take a filename on the command line to process, if one is not supplied, // default to one I like to commonly use if (argc != 2) { cerr << "ERROR -> wrong number of arguments, should be" << endl; cerr << " procisis " << endl; cerr << "defaulting to file1.dat" << endl; strcpy (filename, "data/file212.2543"); } else { strcpy (filename, argv[1]); } // With this processing program we are creating a number of IDFS's. // Each IDFS must be given a unique virtual name // The new simply tells the IDFS writer to allocate memory for // our virtual. Nothing is actually written until we set the first time IDFS *OA1 = new IDFS ("ISOA1", 1, 5, 1); // nss = 1, n_sen = 5, n_sample = 1 IDFS *OA2 = new IDFS ("ISOA2", 1, 8, 10); // nss = 1, n_sen = 8, n_sample = 10 IDFS *MAG = new IDFS ("ISMAG", 1, 3, 1); // nss = 1, n_sen = 3, n_sample = 1 // This one we must send all parameters since we are not writing 32 bit // quantities, but 16 bit ones. IDFS *D = new IDFS ("ISD", 1, 7, 225, 2, 0, sizeof (short)); // nss = 1, n_sen = 8, n_sample = 225, smp_id = 2, cal_sets = 0 // max_tdw_len = 16 (sizeof short) IDFS *G = new IDFS ("ISG", 1, 8, 225); // nss = 1, n_sen = 8, n_sample = 225 IDFS *S = new IDFS ("ISS", 1, 3, 225); // nss = 1, n_sen = 3, n_sample = 225 IDFS *PA = new IDFS ("ISPAG6", 1, 1, 225); // nss = 1, n_sen = 1, n_sample = 225 // Initialize all our virtuals and set the data latency Initialize_ISIS_IDFS (OA1); OA1->SetDataLat (60000000); // data latency = one value per minute Initialize_ISIS_IDFS (MAG); MAG->SetDataLat (60000000); Initialize_ISIS_IDFS (OA2); OA2->SetDataLat (6000000); // data latency is 1 minute = 60000000 microsec / 10 Initialize_ISIS_IDFS (G); G->SetDataLat (266667); // 225 values/minute = 60000000 microsec / 225 Initialize_ISIS_IDFS (D); D->SetDataLat (266667); // 225 values/minute = 60000000 microsec / 225 Initialize_ISIS_IDFS (S); S->SetDataLat (266667); // 225 values/minute = 60000000 microsec / 225 Initialize_ISIS_IDFS (PA); PA->SetDataLat (266667); // 225 values/minute = 60000000 microsec / 225 // Ok, we have done all the preliminary IDFS stuff, now it is time to read // the ISIS data. First, open the file... if ((ptr_to_isis = open (filename, O_RDONLY, 0666)) < 0) { cerr << "ERROR opening ISIS data file, abort signaled" << endl; exit (-1); } // This is our main loop. Read the ISIS data and send it to the right place do { empty = read (ptr_to_isis, (char *) &ISIS_test, sizeof (ISIS_test)); if (empty > 0) { ISIS_d *ISIS_data = (ISIS_d *) &ISIS_test; cout << 1900 + (ISIS_data->year) << " " << ISIS_data->day << " " << ISIS_data->hour << ":" << ISIS_data->minute << ":" << ISIS_data->second << endl; // We must always set the time in the loop since we time tag each of our data records. SetTime_ISIS_IDFS (OA1, ISIS_data->year, ISIS_data->day, ISIS_data->hour, ISIS_data->minute, ISIS_data->second); SetTime_ISIS_IDFS (OA2, ISIS_data->year, ISIS_data->day, ISIS_data->hour, ISIS_data->minute, ISIS_data->second); SetTime_ISIS_IDFS (MAG, ISIS_data->year, ISIS_data->day, ISIS_data->hour, ISIS_data->minute, ISIS_data->second); SetTime_ISIS_IDFS (G, ISIS_data->year, ISIS_data->day, ISIS_data->hour, ISIS_data->minute, ISIS_data->second); SetTime_ISIS_IDFS (D, ISIS_data->year, ISIS_data->day, ISIS_data->hour, ISIS_data->minute, ISIS_data->second); SetTime_ISIS_IDFS (PA, ISIS_data->year, ISIS_data->day, ISIS_data->hour, ISIS_data->minute, ISIS_data->second); SetTime_ISIS_IDFS (S, ISIS_data->year, ISIS_data->day, ISIS_data->hour, ISIS_data->minute, ISIS_data->second); // Now we set the data OA1->SetData (0, ISIS_data->pass); OA1->SetData (1, ISIS_data->frame1_num); OA1->SetData (2, ISIS_data->spin, IBM); OA1->SetData (3, ISIS_data->sLST, IBM); OA1->SetData (4, ISIS_data->sLMT, IBM); MAG->SetData (0, ISIS_data->magpts_num); MAG->SetData (1, ISIS_data->smagdec, IBM); MAG->SetData (2, ISIS_data->salpha, IBM); for (int i = 0; i < 8; i++) { for (int j = 0; j < 10; j++) { int pos = i * 10 + j; switch (i) { case 0 : OA2->SetData (pos, ISIS_data->lat [j], IBM); break; case 1 : OA2->SetData (pos, ISIS_data->lon [j], IBM); break; case 2 : OA2->SetData (pos, ISIS_data->alt [j], IBM); break; case 3 : OA2->SetData (pos, ISIS_data->maglat [j], IBM); break; case 4 : OA2->SetData (pos, ISIS_data->maglon [j], IBM); break; case 5 : OA2->SetData (pos, ISIS_data->IL [j], IBM); break; case 6 : OA2->SetData (pos, ISIS_data->L [j], IBM); break; case 7 : OA2->SetData (pos, ISIS_data->B [j], IBM); break; } // switch } } // We set the spin value on all our science units G->SetSpin (fp2intfp (ISIS_data->spin, IBM)); D->SetSpin (fp2intfp (ISIS_data->spin, IBM)); S->SetSpin (fp2intfp (ISIS_data->spin, IBM)); for (int i = 0; i < 8; i++) { for (int j = 0; j < 225; j++) { int pos = i * 225 + j; switch (i) { case 0 : G->SetData (pos, ISIS_data->G1 [j], IBM); break; case 1 : G->SetData (pos, ISIS_data->G2 [j], IBM); break; case 2 : G->SetData (pos, ISIS_data->G3 [j], IBM); break; case 3 : G->SetData (pos, ISIS_data->G6 [j], IBM); break; case 4 : G->SetData (pos, (SDDAS_LONG) ISIS_data->G4L [j]); break; case 5 : G->SetData (pos, (SDDAS_LONG) ISIS_data->G4H [j]); break; case 6 : G->SetData (pos, (SDDAS_LONG) ISIS_data->G5L [j]); break; case 7 : G->SetData (pos, (SDDAS_LONG) ISIS_data->G5H [j]); break; } // switch } // j loop } // i loop for (int i = 0; i < 7; i++) { for (int j = 0; j < 225; j++) { int pos = i * 225 + j; switch (i) { case 0 : D->SetData (pos, ISIS_data->D1 [j]); break; case 1 : D->SetData (pos, ISIS_data->D2 [j]); break; case 2 : D->SetData (pos, ISIS_data->D3 [j]); break; case 3 : D->SetData (pos, ISIS_data->G6 [j]); break; case 4 : D->SetData (pos, ISIS_data->D4L [j]); break; case 5 : D->SetData (pos, ISIS_data->D4H [j]); break; case 6 : D->SetData (pos, ISIS_data->DB2 [j]); break; case 7 : D->SetData (pos, ISIS_data->DB1 [j]); break; } // switch } // j loop } // i loop for (int j = 0; j < 225; j++) PA->SetData (j, ISIS_data->paG6 [j], IBM); for (int i = 0; i < 3; i++) { for (int j = 0; j < 225; j++) { int pos = i * 225 + j; switch (i) { case 0 : S->SetData (pos, ISIS_data->S3 [j], IBM); break; case 1 : S->SetData (pos, (SDDAS_LONG) ISIS_data->S50 [j]); break; case 2 : S->SetData (pos, (SDDAS_LONG) ISIS_data->S80 [j]); break; } // switch } // j loop } // i loop // Write the data to all our data files OA1->Write (); OA2->Write (); MAG->Write (); G->Write (); D->Write (); PA->Write (); S->Write (); } } while (empty > 0); // Close all our data files OA1->Close (); OA2->Close (); MAG->Close (); G->Close (); D->Close (); PA->Close (); S->Close (); // That's it! delete OA1; delete OA2; delete MAG; delete G; delete D; delete PA; delete S; exit (0); }
#include#include #include extern "C" { #include "decode.h" } #include "IDFS.h" const int RECSIZE = 332; // size of NOAA records void tiros_to_time (unsigned long time, SDDAS_SHORT *hour, SDDAS_SHORT *min, SDDAS_SHORT *sec) { unsigned long lsec = time / 1000; // convert to seconds *hour = lsec / 3600; lsec %= 3600; *min = lsec / 60; lsec %= 60; *sec = lsec; } // ***************************************************************** // Initialize_NOAA_IDFS takes an IDFS data source and sets some // common parameters which are common to all IDFS' created by this // program. // ***************************************************************** void Initialize_NOAA_IDFS (IDFS *I) { I->SetTimeUnits (0); // seconds } // ***************************************************************** // SetTime_NOAA_IDFS takes an IDFS data source and sets the time // ***************************************************************** void SetTime_NOAA_IDFS (IDFS *I, long two_digit_yr, long day, SDDAS_ULONG time) // SDDAS_SHORT min, SDDAS_SHORT sec) { I->SetYear ((SDDAS_SHORT) 1900 + two_digit_yr); I->SetDay ((SDDAS_SHORT) day); I->SetDrTime (time); } int main (int argc, char *argv []) { char filename [40]; FILE *ptr_to_noaa; int eof; // Take a filename on the command line to process, if one is not supplied, // default to one I like to commonly use if (argc != 2) { cerr << "ERROR -> wrong number of arguments, should be" << endl; cerr << " procisis " << endl; cerr << "defaulting to file1.dat" << endl; strcpy (filename, "data/n66315"); } else { strcpy (filename, argv[1]); } // With this processing program we are creating a number of IDFS's. // Each IDFS must be given a unique virtual name // The new simply tells the IDFS writer to allocate memory for // our virtual. Nothing is actually written until we set the first time IDFS *OA = new IDFS ("NOAA_OA", 1, 8, 1); // nss = 1, n_sen = 8, n_sample = 1 IDFS *MAG = new IDFS ("N_MAG", 1, 8, 1); // nss = 1, n_sen = 8, n_sample = 1 IDFS *MAG_OA = new IDFS ("N_MAG_OA", 1, 4, 1); // nss = 1, n_sen = 4, n_sample = 1 IDFS *MAG_PA = new IDFS ("N_MAG_PA", 1, 5, 1); // nss = 1, n_sen = 5, n_sample = 1 IDFS *MP_IONS = new IDFS ("NM_IONS", 1, 2, 1); // nss = 1, n_sen = 2, n_sample = 1 IDFS *MP_ELEC = new IDFS ("NM_ELEC", 1, 6, 4); // nss = 1, n_sen = 6, n_sample = 4 IDFS *MP_PROT = new IDFS ("NM_PROT", 1, 10, 4); // nss = 1, n_sen = 10, n_sample = 4 IDFS *MP_OMNI = new IDFS ("NM_OMNI", 1, 3, 4); // nss = 1, n_sen = 3, n_sample = 4 IDFS *HP_PROT = new IDFS ("NH_PROT", 1, 4, 2); // nss = 1, n_sen = 4, n_sample = 2 IDFS *HP_ALPHA = new IDFS ("NH_ALPHA", 1, 2, 2); // nss = 1, n_sen = 2, n_sample = 2 IDFS *HP_S = new IDFS ("NH_S", 1, 5, 2); // nss = 1, n_sen = 5, n_sample = 2 IDFS *NT_S1 = new IDFS ("NT_S1", 1, 12, 4); // nss = 1, n_sen = 12, n_sample = 4 IDFS *NT_S2 = new IDFS ("NT_S2", 1, 16, 1); // nss = 1, n_sen = 16, n_sample = 1 IDFS *NT_M = new IDFS ("NT_M", 1, 8, 1); // nss = 1, n_sen = 8, n_sample = 1 IDFS *NT_BK = new IDFS ("NT_BK", 1, 4, 1); // nss = 1, n_sen = 4, n_sample = 1 IDFS *TEDFX = new IDFS ("TEDFX", 1, 1, 4); // nss = 1, n_sen = 1, n_sample = 4 // Initialize all our virtuals and set the data latency Initialize_NOAA_IDFS (OA); OA->SetDataDirectory ("pdata/NOAA/NOAA/OA/OA"); OA->SetDataLat (80000); // data latency = one value every 8 seconds OA->SetDataAccum (8); // 8 second accum. period Initialize_NOAA_IDFS (MAG); MAG->SetDataDirectory ("pdata/NOAA/NOAA/MAG/MAG"); MAG->SetDataLat (80000); MAG->SetDataAccum (8); // 8 second accum. period Initialize_NOAA_IDFS (MAG_OA); MAG_OA->SetDataDirectory ("pdata/NOAA/NOAA/MAG/MAG"); MAG_OA->SetDataLat (80000); MAG_OA->SetDataAccum (8); // 8 second accum. period Initialize_NOAA_IDFS (MAG_PA); MAG_PA->SetDataDirectory ("pdata/NOAA/NOAA/MAG/MAG"); MAG_PA->SetDataLat (80000); MAG_PA->SetDataAccum (8); // 8 second accum. period Initialize_NOAA_IDFS (MP_IONS); MP_IONS->SetDataDirectory ("pdata/NOAA/NOAA/SEM/MEPED"); MP_IONS->SetDataLat (0); // this is sampled once every 16 seconds MP_IONS->SetDataAccum (16); // 16 second accum. period Initialize_NOAA_IDFS (MP_ELEC); MP_ELEC->SetDataDirectory ("pdata/NOAA/NOAA/SEM/MEPED"); MP_ELEC->SetDataLat (1e6); // this is sampled 4 times every 8 seconds MP_ELEC->SetDataAccum (1); // 1 second accum. period Initialize_NOAA_IDFS (MP_PROT); MP_PROT->SetDataDirectory ("pdata/NOAA/NOAA/SEM/MEPED"); MP_PROT->SetDataLat (1e6); // this is sampled 4 times every 8 seconds MP_PROT->SetDataAccum (1); // 1 second accum. period Initialize_NOAA_IDFS (MP_OMNI); MP_OMNI->SetDataDirectory ("pdata/NOAA/NOAA/SEM/MEPED"); MP_OMNI->SetDataLat (0); // this is sampled 4 times every 8 seconds MP_OMNI->SetDataAccum (2); // 2 second accum. period Initialize_NOAA_IDFS (HP_PROT); HP_PROT->SetDataDirectory ("pdata/NOAA/NOAA/SEM/HEPED"); HP_PROT->SetDataLat (0); // this is sampled 2 times every 8 seconds HP_PROT->SetDataAccum (4); // 4 second accum. period Initialize_NOAA_IDFS (HP_ALPHA); HP_ALPHA->SetDataDirectory ("pdata/NOAA/NOAA/SEM/HEPED"); HP_ALPHA->SetDataLat (0); // this is sampled 2 times every 8 seconds HP_ALPHA->SetDataAccum (4); // 4 second accum. period Initialize_NOAA_IDFS (HP_S); HP_S->SetDataDirectory ("pdata/NOAA/NOAA/SEM/HEPED"); HP_S->SetDataLat (0); // this is sampled 2 times every 8 seconds HP_S->SetDataAccum (4); // 4 second accum. period NT_S1->SetDataDirectory ("pdata/NOAA/NOAA/SEM/TED"); NT_S1->SetTimeUnits (-6); // microseconds NT_S1->SetDataLat (1153846); // 1 + 2/13 of a second NT_S1->SetDataAccum (846154); // 11/13 of a second = 846154 microseconds accum. period NT_S2->SetDataDirectory ("pdata/NOAA/NOAA/SEM/TED"); NT_S2->SetTimeUnits (-6); // microseconds NT_S2->SetDataLat (16000000); // 7 seconds + 2/13 of a second NT_S2->SetDataAccum (0); // 1/13 of a second = 76923 microseconds accum. period NT_M->SetDataDirectory ("pdata/NOAA/NOAA/SEM/TED"); NT_M->SetTimeUnits (-6); // microseconds NT_M->SetDataLat (7153846); // 7 seconds + 2/13 of a second NT_M->SetDataAccum (846154); // 11/13 of a second = 84615 microseconds accum. period NT_BK->SetDataDirectory ("pdata/NOAA/NOAA/SEM/TED"); NT_BK->SetTimeUnits (-6); // microseconds NT_BK->SetDataLat (32153846); // 32000000 (32 s in micro) + 2/13 of a second in microsec. NT_BK->SetDataAccum (76923); // 1/13 of a second = 76923 microseconds accum. period Initialize_NOAA_IDFS (TEDFX); TEDFX->SetDataDirectory ("pdata/NOAA/NOAA/SEM/TED"); TEDFX->SetDataLat (0); // this is sampled 2 times every 8 seconds TEDFX->SetDataAccum (2); // 2 second accum. period // Ok, we have done all the preliminary IDFS stuff, now it is time to read // the NOAA data. First, open the file... if ((ptr_to_noaa = fopen (filename, "rb")) == NULL) { cerr << "ERROR opening NOAA data file, abort signaled" << endl; exit (-1); } fseek (ptr_to_noaa, 0L, SEEK_END); int FileSize = ftell (ptr_to_noaa); /* Position pointer at first record to output */ fseek (ptr_to_noaa, 0, SEEK_SET); struct input_rec inp_rec; struct archive_rec NOAA_data; // This is our main loop. Read the NOAA data and send it to the right place int write_ions = 1; do { // Read a record eof = fread (&inp_rec, 1, RECSIZE, ptr_to_noaa); // If even the first record is short the file must be bad if (eof < RECSIZE) { fclose (ptr_to_noaa); } else { decode (inp_rec, &NOAA_data); SDDAS_SHORT hour, min, sec; tiros_to_time (NOAA_data.IHD [3], &hour, &min, &sec); cout << 1900 + (NOAA_data.IHD [1]) << " " << NOAA_data.IHD [2] << " " << hour << ":" << min << ":" << sec << endl; // We must always set the time in the loop since we time tag each of our data records. SetTime_NOAA_IDFS (OA, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3]); SetTime_NOAA_IDFS (MAG, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3]); SetTime_NOAA_IDFS (MAG_OA, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3]); SetTime_NOAA_IDFS (MAG_PA, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3]); SetTime_NOAA_IDFS (MP_IONS, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - (16 * 1000)); // T0 - 16s SetTime_NOAA_IDFS (MP_ELEC, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - 1000); // T0 - 1s SetTime_NOAA_IDFS (MP_PROT, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3]); SetTime_NOAA_IDFS (MP_OMNI, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - (2 * 1000)); // T0 - 2s SetTime_NOAA_IDFS (HP_PROT, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - (4 * 1000)); // T0 - 4s SetTime_NOAA_IDFS (HP_ALPHA, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - (4 * 1000)); // T0 - 4s SetTime_NOAA_IDFS (HP_S, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - ((int) (1.2 * 1000))); // T0 - 1.2s SetTime_NOAA_IDFS (NT_S1, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - (2 * 1000)); // T0 - 2s SetTime_NOAA_IDFS (NT_S2, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - (2 * 1000)); // T0 - 2s SetTime_NOAA_IDFS (NT_BK, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - (2 * 1000)); // T0 - 2s SetTime_NOAA_IDFS (NT_M, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3] - (2 * 1000)); // T0 - 2s SetTime_NOAA_IDFS (TEDFX, NOAA_data.IHD [1], NOAA_data.IHD [2], NOAA_data.IHD [3]); // T0 // Now we set the data OA->SetData (0, NOAA_data.IHD [5]); OA->SetData (1, NOAA_data.IHD [6]); OA->SetData (2, NOAA_data.IHD [7]); OA->SetData (3, NOAA_data.HEAD [0]); OA->SetData (4, NOAA_data.HEAD [1]); OA->SetData (5, NOAA_data.HEAD [14]); OA->SetData (6, NOAA_data.HEAD [20]); OA->SetData (7, NOAA_data.HEAD [21]); MAG->SetData (0, NOAA_data.HEAD [2]); MAG->SetData (1, NOAA_data.HEAD [3]); MAG->SetData (2, NOAA_data.HEAD [4]); MAG->SetData (3, NOAA_data.HEAD [5]); MAG->SetData (4, NOAA_data.HEAD [8]); MAG->SetData (5, NOAA_data.HEAD [9]); MAG->SetData (6, NOAA_data.HEAD [10]); MAG->SetData (7, NOAA_data.HEAD [11]); MAG_OA->SetData (0, NOAA_data.HEAD [6]); MAG_OA->SetData (1, NOAA_data.HEAD [7]); MAG_OA->SetData (2, NOAA_data.HEAD [12]); MAG_OA->SetData (3, NOAA_data.HEAD [13]); MAG_PA->SetData (0, NOAA_data.HEAD [15]); MAG_PA->SetData (1, NOAA_data.HEAD [16]); MAG_PA->SetData (2, NOAA_data.HEAD [17]); MAG_PA->SetData (3, NOAA_data.HEAD [18]); MAG_PA->SetData (4, NOAA_data.HEAD [19]); MP_IONS->SetData (0, NOAA_data.MEPI [0]); MP_IONS->SetData (1, NOAA_data.MEPI [1]); int pos1, pos2, pos3; pos1 = pos2 = pos3 = 0; for (int j = 0; j < 19; j++) { for (int i = 0; i < 4; i++) { switch (j) { case 0 : case 1 : case 2 : case 3 : case 4 : MP_PROT->SetData (pos1++, NOAA_data.MEP [i][j]); break; case 5 : case 6 : case 7 : MP_ELEC->SetData (pos2++, NOAA_data.MEP [i][j]); break; case 8 : case 9 : case 10 : case 11 : case 12 : MP_PROT->SetData (pos1++, NOAA_data.MEP [i][j]); break; case 13 : case 14 : case 15 : MP_ELEC->SetData (pos2++, NOAA_data.MEP [i][j]); break; case 16 : case 17 : case 18 : MP_OMNI->SetData (pos3++, NOAA_data.MEP [i][j]); break; } // switch } } pos1 = pos2 = pos3 = 0; for (int j = 0; j < 11; j++) { for (int i = 0; i < 2; i++) { switch (j) { case 0 : case 1 : case 2 : case 3 : HP_PROT->SetData (pos1++, NOAA_data.IHEP [i][j]); break; case 4 : case 5 : HP_ALPHA->SetData (pos2++, NOAA_data.IHEP [i][j]); break; case 6 : case 7 : case 8 : case 9 : case 10 : HP_S->SetData (pos3++, NOAA_data.IHEP [i][j]); break; } // switch } } pos1 = pos2 = pos3 = 0; for (int i = 0; i < 4; i++) { for (int j = 0; j < 4; j++) { if (NOAA_data.IHD [8] == 4) NT_BK->SetData (pos1++, NOAA_data.TED [i][j]); else NT_S2->SetData (pos2++, NOAA_data.TED [i][j]); } } for (int i = 0; i < 4; i++) { NT_M->SetData (pos3++, NOAA_data.TED [i][4]); NT_M->SetData (pos3++, NOAA_data.TED [i][5]); } pos1 = pos2 = pos3 = 0; for (int j = 6; j < 18; j++) { for (int i = 0; i < 4; i++) { NT_S1->SetData (pos1++, NOAA_data.TED [i][j]); } } for (int i = 0; i < 4; i++) TEDFX->SetData (i, NOAA_data.TEDFX [i]); // Write the data to all our data files OA->Write (); MAG->Write (); MAG_OA->Write (); MAG_PA->Write (); write_ions = !write_ions; if (write_ions) MP_IONS->Write (); MP_ELEC->Write (); MP_PROT->Write (); MP_OMNI->Write (); HP_PROT->Write (); HP_ALPHA->Write (); HP_S->Write (); NT_S1->Write (); NT_M->Write (); if (NOAA_data.IHD [8] == 4) { NT_BK->Write (); } else NT_S2->Write (); TEDFX->Write (); } } while (eof >= RECSIZE); // Close all our data files OA->Close (); MAG->Close (); MAG_OA->Close (); MAG_PA->Close (); MP_IONS->Close (); MP_ELEC->Close (); MP_PROT->Close (); MP_OMNI->Close (); HP_PROT->Close (); HP_ALPHA->Close (); HP_S->Close (); NT_S1->Close (); NT_S2->Close (); NT_BK->Close (); NT_M->Close (); TEDFX->Close (); // That's it! delete OA; delete MAG; delete MAG_OA; delete MAG_PA; delete MP_IONS; delete MP_ELEC; delete MP_PROT; delete MP_OMNI; delete HP_PROT; delete HP_ALPHA; delete HP_S; delete NT_S1; delete NT_S2; delete NT_BK; delete NT_M; delete TEDFX; exit (0); }