You can also download the most recent (2002 August) version of the ISDS white paper, which describes the ISDS software usage.
In reverse chronological order, the revisions to the ISDS include:
Have begun implementing some new features in the ISDS to simulate instrumental artifacts seen in the first 30 days of flight operations. Will be releasing a new version soon.
Continuing validation of the new ISDS version before release.
Today discovered that the InSb dark frames distributed from the SSC are wrong by a factor of -1.0; i.e., they are BCD-format instead of raw-format. Thus users simulating dark charge in the InSb channels will have difficulties with the dark subtraction process. Users creating BCD format output are unaffected.
Begun writing documentation for ISDSv3.0.
Today while verifying simulations for the deep IRAC IOC observation I discovered another issue with the ISDS dither pattern. Up until now the ISDS was unintentionally adding 1 to the cycling dither position. In other words, when a user was simulating a cycling dither with starting position = N, that user was actually getting starting position N+1. This has been fixed.
I'm testing v3.0 (BETA) of the ISDS now to verify that it can work with David Makovoz's mosaicker. I've got the various elements working but need some time to make sure the photometry is correct. I expect to release the new version within a week.
There is something still odd about the ISDS results when setting iracsim.jansky=yes. I encourage users to avoid this option for the time being, and just create BCD output in DN -- that can be converted 'by hand' to MJy/sr if that's the format you need.
The SSC Mosaicker is up and working here at SAO -- this is a modified version that creates output especially organized for input into what will be version 3.0 of the ISDS.
Preliminary tests with modifications designed to correct the arithmetic error noted on July 2 have been successful. Setting iracsim.jansky=yes will now generate output in units of MJy/sr, as intended. I have advanced the ISDS version number to 2.53, and distributed a copy of the software to the SSC for posting on their web page.
Except for further bug fixes, I plan no more upgrades to the ISDS until I have incorporated David Makovoz mosaicking software into it -- and he delivered a copy of it to SAO this week. My goal is to have a new ISDS version implementing full-up photometric and geometric distortion, as well as position-variable PSFs, on or around the week of July 21.
Erratum: Today during experiments with CR rejection algorithms I realized that setting iracsim.jansky=yes creates BCD output with bad units instead of the intended MJy/sr units. This bug will be corrected in the next ISDS release; it's already corrected in the source code.
BCD-flavored data that are created with iracsim.jansky=no are still in units of ADU; such files should NOT be multiplied by 10^6 as this bug doesn't affect these data.
Work is continuing on integrating the ISDS with the SSC mosaicking software.
I discovered (re-discovered?) a bug in aorexpand that it can fail to read negative declinations properly from the SPOT .aor files when fractional (i.e., <1) degree positions are requested. I will fix this soon but the simple workaround is to do a global search-and-replace on the Declinations specified in the .isds files to make them all negative.
Have successfully installed D. Makovoz' software here at SAO and think integrating it with the ISDS is feasible. However it requires modifying both the ISDS and the SSC software so it will take a while.
Added the ability to recognize Random9 dither patterns -- these had been accidentally left out of prior versions of the ISDS. I verified that the dither pattern is properly handled by aorexpand.
Added the ability to make BCD flavored output that has units of Jy/sr instead of just DN. To accomplish this I had to add a new parameter to the .par file, so it will be necessary to `unlearn iracsim' before using the new version.
The ISDS version has been advanced to 2.5.2. The compiled code is still undergoing testing at SAO and has been distributed to the SSC for testing.
Spoke with Dave Makovoz of the SSC today about incorporating his software into the ISDS -- this would result in simulated images with a near-perfect representation of the geometric and photometric distortion, plus much better PSFs. I don't know how difficult it will be but I'll look into it.
Today I added a 500 s capability to the iracsim program so those who 'hack' the output from aorexpand to create 500 s exposures can run the iracsim program -- this will facilitate the linearity measurements planned for IOC. Zhong Wang is testing the code today. Users of this obscure 'feature' will have to supply their own dark frames in the images/darks/ directory. The 500 s read noise is arbitrarily taken to be the same as in a 200 s frame since they are both Fowler 32 images, but this guess can easily be refined if someone lets me know what read noise to use.
Reviewed IRAC technical memorandum IRAC/TM03-4001. This is the documentation describing the IRAC FOV orientation, i.e., the manner in which IRAC logical coordinates (pixels) map to celestial coordinates. The ISDS conforms to the TM as far as I can tell (see February 24 update).
Distributed a slightly revised version of the ISDS to the SSC for further testing -- Surace discovered that the 36-point Reuleaux triangle pattern was incorrect for large dither scale factors. The small and medium patterns were correct.
Plugged a gap in the library datafiles -- up until now the image library had been missing the 1.2 s darks corresponding to 30 s HDR mode. Gillian Wilson of the SSC supplied model 1.2 s darks derived from ground test data today.
Found that the C&DH time stamps for channels 2-4 had not been updated to match the channel 1 time on February 16. This has been corrected.
Found a bug in the dither pattern implementation that caused dither patterns to be too small -- this was an error in aorexpand.
The ISDS version has been advanced to 2.5. The compiled code is still undergoing testing at SAO and the SSC.
After the IRAC IOC rehearsal conducted during the week of Feb 17, it became apparent that the raw-flavored data output by the ISDS gives astronomical sources that are inverted in the spacecraft Z direction from what they should be. The BCD-flavored data are correct, because the error is applied twice in such a way that it gets canceled out.
The ISDS version has been advanced to 2.43. The compiled code is still undergoing testing at SAO.
Added the capability for the user to set the date on which the observations were acquired. Prior versions of the ISDS defaulted to 30 June 2003. With this release, the user can specify the number of days after (or before) this date by setting iracsim.startday to a positive (or negative) real number.
The ISDS version has been advanced to 2.42. It is undergoing testing here at SAO prior to being released to the SSC.
Changed the output filename suffixes from "10.mipl.fits" to "01.mipl.fits" to conform to the nomenclature used by the current version of the Java SIRTFTelemProc program.
Located and corrected a bug in the formats of the output tracefiles.
Updated the array separation to 6.62 arcminutes for aorexpand. This was done on Dec 20 for iracsim and should have been done for aorexpand at that time as well. Some users may have noticed that their simulated channel 2 and 4 data were displaced from the expected locations by of order 0.4 arcmin, modulo a factor of cos(dec).
The ISDS version has been advanced to 2.41. The compiled code has been delivered to the SSC for testing. I haven't tested it yet however.
I've just learned that the latest version of the ISDS (the not-yet publicly-released v2.4) will NOT compile under RedHat unless IRAFv2.12.1 (or later) is installed. Presumably this will be true for earlier versions as well. Here at SAO we've had trouble compiling the ISDS for RedHat v8.0 until we installed the newest version of IRAF.
Today I found and corrected a bug in aorexpand that prevents it from correctly handling AORs for which FOV1=NO, FOV2=YES. AORs with this configuration were mistakenly treated as having FOV=BOTH. That has now been corrected.
Finished implementing banding effect (first draft) for GLIMPSE.
In addition, to facilitate IRAC IT simulations for the IOC rehearsal, I added parameter TLMGRPS=1 to the FIRST header of the output files to accommodate a planned upgrade to SSC IRAC pipeline. Note that because the keyword GROUPS is a reserved FITS keyword, it is not set by the ISDS even though the attempt is made. If you need the GROUPS=1 keyword for IOC data, you'll have to hand-insert it into the headers somehow.
The ISDS version has been advanced to 2.4. The compiled code has been delivered to the SSC for testing. I haven't tested it yet however.
Began implementing a banding effect simulation for GLIMPSE.
Added tracefile capability to iracsim. Now, if you select tracefile=yes in the parameter file, any time cosmic rays or fluctuating bias is simulated, a tracefile will be generated in the output directory for each simulated DCE to indicate either what exactly the simulated CR positions are, or what offset levels were applied. The trace files are simple ASCII text and should be self-explanatory.
The tracefile for cosmic rays is named exactly as its corresponding FITS image file, but has the suffix "cr.txt" instead of "mipl.fits". Similarly, the tracefile for the fluctuating bias is named with a "bias.txt" suffix.
The ISDS version has been advanced to 2.32. The compiled code has been delivered to the SSC for testing. I've not yet tested it!
Incidentally, the presence of the new tracefile parameter means that you'll have to unlearn iracsim before you can use v2.32.
Learned IRAF help files were not updated in the Dec 16 distribution. I updated them today and forwarded them to the SSC for public release.
In addition, the array separation was updated from 6.22 arcmin (a design value) to the as-built measured value of 6.62 arcmin. In other words, the two IRAC FOVs are slightly further apart than prior versions of the ISDS accounted for.
Distributed the latest ISDS version to the SSC for posting on the web site and use by the Legacy teams. This is the last update planned before geometric and photometric distortion are implemented.
Corrected a long-standing omission for BCD-flavored data. The IRAC pixels are all just slightly larger than 1.2 arcsec in reality, and that's how the ISDS treats them when calculating offsets in its step-and-stare algorithm. However the CDELTx parameters in the output files have always been exactly 1.2 arcsec. This is inaccurate at the 1% level and causes smearing of point sources at the boundaries between IRAC fields when BCD-flavored data are coadded (e.g., with imcombine). Today I fixed this error by changing CDELT1 and CDELT2 by the roughly 1% needed (the change was different for each channel) to make them conform to Table 1 in the writeup. Thanks to the c2d team for helping me figure this out.
After various tests (including some time volunteered by SWIRE) it was determined that the coordinates in the ISDS prior to December 2 were correct and that the changes made on December 2 and 3 were inappropriate. As a result they were removed from the ISDS.
Corrected the implementation of the December 2 release and re-distributed to the SSC for verification. The change implemented was incorrect and mistakenly applied the factor of -1.0 to all four IRAC channels (for BCD type output) instead of just to the Si:As channels.
Corrected a logical bug which caused CDELT1 to be wrong by a factor of -1.0 (sign error) in the BCD-flavored output FITS headers. In essence the WCS was indicating that the east-west direction was the opposite from what was intended. Users can correct existing BCD-flavored output simply by multiplying the CDELT1 parameter by -1.0.
Raw-flavored data were unaffected by this bug, which arises because the ISDS flips the output images to simulate the dichroic in the IRAC. In particular, tests with raw-flavored output processed by the SAO pipeline yield good WCS parameters in the coadded mosaics.
The upgraded version of the ISDS was released today to the SSC for posting on the public web site and directly to the c2d Legacy team.
Corrected a memory management bug that may cause problems when running the ISDS under some versions of Linux. The problem was only present when cosmic rays were simulated. The symptom was that the iracsim program would crash.
Fixed a bug in the cycling dither calculations for aorexpand (not present in earlier releases as far as I know -- only in my development code as of November 1. The ISDS should now understand the dither patterns that are part of SPOTv6.2.
Modified the single-point positioning algorithm in iracsim() substantially to ensure that FOV2 was moved to cover FOV1 in SINGLE_POINT observations. Prior versions of the ISDS had the correct magnitude of FOV offset but the wrong sign. Advanced the code version to 2.3.
Distributed to SSC and SWIRE for testing.
Modified the memory management in the subroutine check_welldepth() of iracutil.x. It was found that under Linux, the prior memory management routines (which work on Sun) were causing a segmentation violation. S. Kim reports ISDS working on her Linux machine now.
Likewise, memory management in the hd_init() and hd_free() subroutines of iracdb.x were updated to include the full set of character arrays used by the main data struct in iracsim(). This (necessary) change was implemented in response to comments made by Zahed Wahhaj and Ed Anderson (Cores to Disks Legacy Team) of North Arizona University.
Updated the iraf/mkpkg file to accommodate Linux architectures, a la Mike Fitzpatrick. Gave to Serena Kim of FEPS to test the new version.
Corrected a bug in the diagnostic runtime output (it did not have any impact on the simulated science data) which caused the reported exposure ID to seem to get out of synch with the data (even though it didn't, really). The fix involved commenting out a line in the irac_doit() routine in the iracreadout.x file which mistakenly incremented AEXPID(hd) by 1.
The PM fields in the SPOT AOR files were causing problems for aorexpand. Some AORs have them, some do not. Today I decided to change the convention and require them, because an increasing number of users specify them (although they're usually zero). However the code doesn't use the PM data in any way.
J. Serena Kim's failed attempt to install iracsim under Linux prompted an evaluation of some Boolean logic operations; it was found by Mike Fitzpatrick of NOAO that whereas the Sun compilers had permitted some Boolean-to-integer variable comparisons, these are forbidden under Linux. irachio.x and iracutil.x both contained expressions comparing Boolean and integer variables. Hence the failure to compile. After he kindly found and corrected the half-dozen mismatches, Mike Fitzpatrick was able to compile iracsim under Redhat 7.3. As of today the changes are being adopted into the source code so that the FEPS Legacy team can re-try their IRAC simulations.
Corrected a bug in aorexpand which caused it to generate negative exposure IDs in the .isds files. This should be benign, i.e., it should have no effect on the simulated data other than confusing users as to simulated data filenames. While I was correcting this bug I also consolidated the high dynamic range commanding slightly so the data are acquired with fewer IRAC_DOIT commands.
Users not taking advantage of high dynamic range mode are unaffected by this change. Those using this mode should be sure to emply the July 13 version (or later) of aorexpand.
It has been brought to my attention that the aorexpand program is having difficulty parsing some of the SPOTv6.0 AOR files. Please do email me right away if this is a problem for you.
Modified the cosmic ray selection algorithm slightly so that the program cycles through all 100 templates available instead of starting with template number 1 for every exposure. This will provide for a little more variety in the cosmic ray simulations, even though the ISDS is currently limited to only 100 different templates in both the InSb and SiAs channels.
Corrected a bug introduced yesterday, in which the revised CRVAL1 and CRVAL2 parameters were reporting Declination and Right Ascension, respectively, instead of vice versa. This simple cut-and-paste error is now fixed.
Corrected a bug in the cycling dither pattern which caused the first exposure at each point in a map to not be dithered; this meant that the first dithers in a map would form perfect MxN grid patterns instead of being distorted by the dithers. 2nd and subsequent dithers were all correctly calculated, however. The bug did not affect any dither patterns except the cycling pattern.
At the request of Mike Regan of STScI and the SSC, the output BCD WCS header information was changed so that pointing errors (generated when iracsim.dopnterr=yes) are NOT included in the CRVAL1 and CRVAL2 parameters. However, two additional parameters called RAPNTERR and DECPNTER were added; they contain the pointing errors in units of arcsec.
The code version has been advanced to v2.2. The ISDS was distributed to the SSC for testing/validation today.
Implemented a bug fix proposed by Mike Regan (STScI) to correct the relative placement of FOVs 1 and 2. In earlier ISDS versions, FOV1 (channels 1 and 3) was correctly placed and oriented on the modely sky, and FOV2 was the correct distance from FOV1, but FOV2 was erroneously placed on the opposite side of FOV1 from where it ought to have been.
The code version has been advanced to v2.1. The ISDS was distributed to the SSC for testing/validation today.
Added 0.6 s and 1.2 s template dark frames to the library used by iracsim, so iracsim can properly handle HDR mode observations. The frames added are median-filtered dark frames taken during ATLO testing in April 2002, with IRAC mounted on the spacecraft.
At the SSC's request, the output FITS file dimensions were changed to 256x256, i.e., to a more intuitively 2D format. Prior versions of the ISDS created 256x256x1 format, which contains the same information but is formally 3D format. This was causing problems for some pipeline algorithms at the SSC and for Legacy teams.
At the SSC's request, WCS (coordinate) information is no longer written to the headers of the 'raw' type of output FITS file. Coordinate data are now written only to the headers of the BCD-flavored output FITS files. This change was made to facilitate automated pipeline processing of the ISDS data by the SSC IRAC BCD pipeline.
At the SSC's request, the BCD-flavored output FITS files have been changed so they contain real-valued pixels. At the same time the format of the raw-flavored output was changed to datatype SHORT. Both changes should make the ISDS output more similar to data users will see during routine flight operations.
An updated version of the code and image library was distributed to the SSC today. The code version was not advanced.
By hand, the cosmic ray templates were edited so that each now contains only 1 event. From now on, users can expect to get exactly:
crrate * frametime
cosmic ray hits on each simulated image. Prior to this date, there was an excess cosmic ray hit rate, particularly in the InSb channels, because some templates contained multiple cosmic ray events.
The code version has been advanced to 2.0, and the code was distributed to the SSC today. This version has several important improvements, especially for read noise; see below.
Corrected an error in the read noise simulation. The nature of the error was to systematically underestimate read noise in exposures with frame times of 12 s or greater.
Corrected an error in the implementation of the cosmic ray simulation -- up until this day the cosmic ray hits were too weak by a large factor. From this point forward, the output FITS files should contain cosmic ray hits exactly as strong as the cosmic ray templates in the images/cosmicrays/ directory. The current implementation is equivalent to assuming that all cosmic rays are incident on the detectors _after_ the pedestal readouts and _before_ the signal readouts.
The correction was tested and verified for channel 4 here at SAO. Since the implementation has no channel dependence it should now be correct for all four channels.
Updated aorexpand to read files from SPOTv6.1, and advanced the version to 2.0 in the notation in the output (.isds) file headers, along with today's date.
Fix 1: Incorporated one of Mike Regan's suggested fixes to the logic of aorexpand, to prevent a segmentation violation on his machine. Only one fix could be implemented, however, since the others (related to string manipulations functions carrying return characters or not) negatively impacted the functioning of aorexpand on my Sun Ultra10 workstation.
Fix 2: Added a dummy line feed to accommodate the extra line for AOT_STATUS.
Fix 3: Added logic to accommodate the lack of a DITHER_SCALE entry when DITHER_PATTERN: TYPE= "none".
Fix 4: Updated the definition of HDR mode to correspond to the specification in the SIRTF Observer's Manual (downloadable from the SSC web site) in Table 6.7 and 6.8, on page 71.
The resulting aorexpand software was compiled successfully and tested on three SWIRE AORs generated with SPOTv6.1 (two maps and one single-pointing observation) as well as a number of IRAC GTO AORs. The test was apparently successful.
Matthew L. N. Ashby