All data obtained at the telescopes of the Observatório do Pico dos Diaz
(OPD) of the Laboratório Nacional de Astrofísica (LNA) are stored on
backup tapes which are kept at the observatory. These tapes constitute
de facto a data bank. In order to become useful, information about
the contents of these observations and their availability must be
disseminated. It is the purpose of
DaBIAS (Data Bank Information
Acquisition System) to keep a complete
record of the observations, to put this information onto the internet,
to provide means for the potential user of the databank to search on-line
for observations of particular objects or at particular locations at the
sky, to request data, and thus to permit a feedback between the scientific
community and the data bank.
The heart of DaBIAS is a UNIX shell script which is also called
DaBIAS [For easier distinction, hereafter DaBIAS
(written in normal font) will stand for the entire software system,
while DaBIAS (written in typewrite font) means the UNIX shell
However, other shell scripts and programs are also part of the system.
In order to minimize the
work load for the data bank manager and to guarantee a stable long-term
performance of the system it is imperative that the process to make
the desired information available to the public is completely automatic.
Thus, the final version of the shell script will be started automatically
each morning after the observing night in order to update the available
information which is then put on the Internet to make it public.
The current version v2.0 has not yet quite achieved this stage: The
script DaBIAS is started manually after the observations of each night
have been transformed from IRAF-format to FITS-format according to the
normal observatory operations.
For the time being DaBIAS will only be applied to the
observations obtained with the various CCD-cameras (and the IR camera)
of the OPD (be it
direct cameras or cameras attached to spectrographs), operated by the
routine OPDPIX under the IRAF environment. This restriction is implemented
in order to deal only with a homogeneous set of data formats. Data
acquisition systems for e.g. the FOTRAP or the FOTEX are quite different
and would constitute serious complications, requiring a structure of
DaBIAS very different from that suitable for the instruments run under
OPDPIX. To which extent data from those instruments can be incorporated
later into the data bank remains to be defined.
The general structure of the DaBIAS system is outlined schematically in
The general structure of DaBIAS
The observed data of each night are stored on disk at the OPD.
Since they were obtained in the IRAF environment they are available as
data files in IRAF-format.
These data are transformed into FITS-files and stored on backup
EXABYTE tapes by the staff of the OPD. This happens during the day following
the observing night (or during the first working day in case of weekends
or holidays). Of course, the observer also takes a copy of the data home.
Due to the limited capacity of the hard disks, data of at most a few nights
are kept on disk. Thereafter, they are deleted.
Before the data are removed from the hard disk - normally immediately
after the observing night - the DaBIAS shell script is
started and performs a series of tasks
which ultimately result in the generation of files which are put on the
Internet in order to make the information about the observed objects
available to the general public. DaBIAS is mainly concerned with
an analysis of the headers of the data files.
Version v2.0 only works
on files in FITS-format. IRAF-formats are not considered. This is due to
the fact that the one of the most important bits of information for the
data bank, the name of the observed object, which is available under the
in FITS-files, is not a keyword in the header of IRAF-files
but is coded in some other (and different for different implementations
of IRAF) way. This -- together this the fact that IRAF cannot be started
in a mode where input commands are read from a command file instead of the
console which makes it impossible for the DaBIAS
script to transform the
data files automatically from IRAF-format to FITS-format - is the principle
why in the current implementation DaBIAS is not started automatically
each morning but has to await the manual transformation of formats before
it is started by the operations crew of the OPD. It is planned to overcome
this difficulty in the next version of DaBIAS.
The DaBIAS script:
which objects have been observed. This is done by an analysis of the
OBJECT keyword in the file headers.
Certain keyword contents are treated in a special manner:
Data files containing object names such as ``test'', ``teste'', ``li'',
``lixo'' etc. are automatically discarded (this does not only hold true
for files with corresponding keyword contents but for filenames as well:
files with such names are not considered).
- prepares a file in HTML format to be placed directly on the
Internet: The nightly summary (NS) file:
This file contains a summary of the observations
taken during the night. Apart from giving information about the instrumental
setup it lists the individual objects observed along
with the number of frames taken of them, and the number of Bias, Dark
and Flat-Field exposures. The WWW page will also contain the general comments
of the observer about the night as well as his comments on the performance
of the instrumentation. The latter information is taken from the night reports.
Obviously, these comments can only be incorporated automatically if
they are available electronically. Currently, this is not yet the case.
The next version of DaBIAS will change this. The NS page contains a link to
a CGI-script which prepares a page with more detailed information about
the observations of the individual objects (individual objects listings; IOL).
From there, another CGI-script can be activated which gives detailed
informations about individual exposures (individual file listings; IFL):
- Individual object listings (IOL): These are HTML pages
which contain listings of
the contents of the most important keywords for each of the data files
of the corresponding object. These pages are meant to provide the potential
user with enough information to enable him to judge if the data might be
of interest to him. (Of course, this question cannot be finally
answered by this kind of information. This is only possible after the user
disposes of the data in their entirety. Thus, the information provided here
can only be a guideline!) The listed information for each data file is:
The IOLs can be reached by anchors included in the NS file.
- universal time of observations
- hour angle
- exposure time
- Individual file listings (IFL): In order to provide the reader with
even more information about the observations the entire contents of the
file header of the individual exposures is also made accessible via
anchors in the IOL pages.
- Data bank summary files (DBSF):
Finally, DaBIAS updates a data bank
summary file where information concerning all observations are cumulatively
stored in a condensed form. Different summary files exist for the data of
each telescope. One file is created for each night with observations.
For operational reasons it is safer and easier
to have a separate data file for each
night than to lump the data of several nights into one file.
These summary files serve a twofold purpose:
- The CGI-scripts used to create the IOL and IFL pages require a source
from where to get the corresponding informations. The DBSF constitute this
- A search machine is provided for the user to look for specific
objects by name or by coordinates. This search machine works on the DBSF
to look for the desired information.
- The web pages containing information about the date for which
data are available must be updated. This is also done by DaBIAS.
- Additional useful information for the user is provided by the
metereological station at OPD which monitors constantly the temperature,
humidity (in both cases within the dome of the 1.6-m telescope and outside),
wind speed, air pressure, and other parameters. These
quantities are stored on disk at a time resolution of 30 minutes.
DaBIAS reads these data and
(Note that this facility is not yet implemented!)
- prepares graphs of the various quantities to be put onto the Internet
as GIF files (to be reached by corresponding anchors from the NS page).
- store them in a cumulative file similar to the DBSF in a condensed
form in order to enable the on-line creation of corresponding graphs in
a later step of realization of the system (to avoid the necessity of having
to store large amounts of GIF images on disk).
The WWW pages
The HTML pages created on a daily basis by DaBIAS are put on the
WWW server of the LNA. The structure of the corresponding Web-site is
shown schematically in Fig. 2.
description of the structure of the data bank
web-site reflects the current status.
Since it is in continuous development, differences may be introduced
at any time! It is an experimental phase and political decisions
concerning the data bank which might have bearings on its structure
are still pending. Moreover,
no particular attention has been payed to the layout of the web pages.
For the time being they are kept very simple. As times goes by
a more appealing layout may be developed.
The data bank web pages (including those which are created on-line by
CGI-scripts) are layered in 6 levels, each of which provides
more specific information about the available data. In detail, contents
of the individual levels are as follows:
Of course, this structure of the data bank web site must not be regarded
as static. It will certainly develop as experience with the performance of
the data bank is being gained.
- The first level consists of the Welcome page of the data bank.
some brief information concerning the data bank and links to the pages on
the next level.
- The second level contains:
- a page informing about the policies of the data bank. (Since these
have not yet been defined, this page is largely empty for the time being!)
- A page with a general description of the databank (basically the
more general parts of this document).
- a search machine. It enables to search the data bank for specific
objects. Since object names are not always unequivocal (different ways of
writing or spelling being imaginable), it is possible to specify various
alias names for a given object. Alternatively, names of several different
objects may be specified, all of which will then be searched for in the
data bank. The problem of different spellings of objects being potentially
quite serious, intelligent routines are sought which enable to find
specific objects even if the search string is not completely identical to
the name found in the OBJECT key word of the data file. As first steps
to solve this problem to strategies are adopted:
- Capital and small letters are not distinguished; spaces within the
object names are ignored. For example: the name >SS Cyg would be
treated as identical to sscYG.
- For each new object in the data bank the SIMBAD data base in Strasbourg
is queried for alias names. (For the time being this is still done manually
by the operator. Ways to automatize this task are sought.) All names and
its aliases are stored in special object name files. Thus, the DaBIAS system
can identify observations of objects stored with different alias names in
the data bank.
- Instead of searching
for specific object names, the user may also search for objects within a
given radius around some celestial coordinates. All data concerning objects
in that specific area of the sky which are found in the data bank are then
listed. The search is performed by a CGI-script.
- a data request form. If the user has found data in the data bank
which are potentially of use to him and which he wants to retrieve, he
can fill in this request form which is then mailed to the data bank
manager. For the time being, this procedure is very simple. A more
sophisticated approach is under development which will include a registration
procedure for users (in order to avoid that any web surfer just for fun
requests data), consistency checks (do the requested data exist at all?),
verification if the requested data are liberated and thus freely available,
- access to the higher levels of the data bank information system
which contain information about the data themselves. To this end, links
to separate pages for the three telescopes of the OPD are provided:
- 1.6-m Perkin Elmer telescope (PE)
- 0.6-m Boller Chivens telescope (BC)
- 0.6-m Zeiss telescope (ZE)
- an option to send an email with suggestions, comments, and request
(other than data retrieval requests) to the data bank manager.
- The third level contains pages listing the dates for which data
are available in the data bank. A kind of calendar is provided with those
days for which data are available constituting links to the corresponding
NS files. In order not to permit these lists to
grow ad infinitum over the years, only the dates of the current
year are listed. Links to corresponding pages for previous years are also
provided. There are separate pages of this kind for each telescope.
- The pages on level four finally get to the data. These are the
NS files mentioned previously. For each day for
which data are available, a summary is given, containing:
The graphs of the metereological conditions during the night available
in future at this level will show the variations of the following quantities
at a time
resolution of 30 minutes from the noon preceding the observing night until
the moment at which the DaBIAS script
is started (or until noon of the day
following the observing night if DaBIAS is only run on a later day):
- the name of the observer
- information about the instrumentation employed during the night
- the observer's comments concerning the general performance of the night
(not yet available)
- the observer's comments concerning the performance of the instrumentation
(not yet available)
- a link to the graphs of the metereological conditions during the night
(not yet available)
- a summary of the exposed CCD frames: Number of Bias, Dark, Flat Field
and Object frames
- a list of the observed objects together with their coordinates and
the number of exposures.
In a later version of DaBIAS it is planned to get this information
together with information about the seeing (for the 1.6-m telescope;
to be taken from the autoguider which is currently under construction) for
each individual observation (to be included into the data file header).
- the temperature measured outside the 1.6-m telescope dome
- the temperature measured inside the 1.6-m telescope dome
- the relative humidity measured outside the 1.6-m telescope dome
- the relative humidity measured inside the 1.6-m telescope dome
- the air pressure
- the wind speed
- On level 5, a page for each of the objects observed during
the night is provided. It lists
the most important items of each exposure of the
- universal Time of exposure
- hour angle at exposure time
- duration of exposure
- Finally, level 6 contains a page for each individual exposure. Here
the contents of all header keywords are listed (with the exception of a few
keywords which are not relevant to judge the usefulness of the data such
as e.g. BSCALE or BZERO).
For the time being, it is not expected that the number of requested data
will be very large. Therefore, it is not yet planned to automatize the
response to a data request by a user.
Several ways to comply with a user request
are possible. Presently, two of these are considered viable
within the current approach for the data bank. They are not exclusive;
and both of them can be followed, depending on the amount of requested
In order to keep the work load of the data bank manager within limits,
he will not bother with too detailed a user request: Instead of copying
individual data files to the FTP Server or the tape, all of a
night's (and telescope's) data are made available to the user.
It is then up to him to down-load (or read from tape) only those files
which he is interested in, discarding the rest.
- The data bank manager
loads down the requested data from the backup tapes onto a FPT server.
The user is then notified by e-mail that the data are available and he
can retrieve them via anonymous FTP. After a given time (say, one week)
the data are automatically removed from the FPT server. Concerning this
alternative, problems with limited Internet capacities might arise when
the amount of requested data is very large (note that since the data are
available only in their raw form, Bias and Flat-Field images will in
general also have to be transferred!).
- The data bank manager copies the requested data to another EXABYE-
or DAT-tape (to be provided by the user!) and sends the data via classical
mail to him. Currently, this appears to be the reasonable way to treat
requests concerning large amounts of data.