Euro3D Visualization Tool
The Euro3D Research Training Network (RTN) was put
forward with the intention to promote integral field spectroscopy
(IFS), or ``3D'' spectroscopy, and to help making it a common user
technique. In order to accomplish this, one of the major tasks was
identified as the need of providing standard software tools for the
visualization and analysis of datacubes. These tools should be general
enough to be entirely independent of the origin of data, i.e.\ 3D
instrument. Previously, a heterogenous collection of
instrument-specific data formats and software tools (e.g.\ XOASIS),
proprietary software packages and a lack of any standard have hampered
a break-through of this powerful observing method, leaving it merely
as an expert technique with comparatively limited scientific impact.
Recognizing the importance of this problem, a work plan was devised to
start creating a package of tools for the analysis and visualization
of IFS data. Entitled 3D Visualization, Task~2.2 of this work
plan foresees the development of a programme, which should be capable
of reading, writing, and visualizing reduced data from 3D
spectrographs of any kind. We have named this tool E3D.
We describe here the progress on the coding of E3D. The last stable version is
distributed in the Euro3D
You can download the last compiled version here:
E3D, 1.3a, source code
E3D, 1.3a, Linux binaries
Expecifications : AIP January mini-meeting
- E3D, The Euro3D visualization tool I: Description of
the program and its capabilities S.F.Sanchez, 2004, AN, 325, 167
- E3D, the
Euro3D Visualization Tool II: mosaics, VIMOS data and large IFUs of
the future S.F.Sanchez, T.Becker, A.Kelz, 2004, AN, 325, 17
31st January, 2005
E3D is distributed freely to the general public.
6th Augost, 2004
New bugs solved. E3D includes new standalone programs, like "interpol"
a program for interpolate non regular grid slices. Version change to 1.3.
E3D distributed within the GEMINI and ESO astronomers. Lots of bugs
detected (as expected!).
30th April, 2004
New tutorial to work with E3D, based in the Euro3D software tutorial
meeting (Potsdam, 28-30 April 2004).
20 of February, 2004
A major bug when reloading new images or plotting new spectra
have been solved. The new version is uploaded.
28 of January, 2004
Some minor cosmetic changes introduced. New version upload.
06 of January, 2004
A lot of changes have been introduced in E3D, during the last month.
A new version (1.2) is available to download.
There are three main lines of modifications:
Finally, I want to remind the E3D users that I am on touch for any
question regarding the program. Ie, if you cannot install it in a not
tested arquitecture (or in a tested one!), please report me.
Our intention is to distribute a static, compiled version of the
program for different arquitectures, but this will take a while.
Memory bug solved. This implies that
now E3D can handle large IFU data without a large cost of memory, and
that it does not collapse when performing consecutive file loads.
We have tested E3D with MUSE like data sets (90000 spectra, 712Mb on
the disk), in a
512Mb Pentium IV. It takes 2m to read the data, and although the
plotting is slow, it actually works. We have also tested with with
consecutive PMAS file loadings, until a limit of 10000 uploads. The
memory use increases from 21Mb to 41Mb, in total.
With the previous version this increase in the memory use was found
in the 5th upload, collapsing the computer at the 30th upload.
- Different bugs solved. They were mainly related to the call of
pointers still not allocated, which causes crashes of the program.
There is still some of them, although they are identified.
I will fix them ASAP.
New features included: (1) Import/Export different data formats,
including datacubes and Row Staked Spectra, and 2D
images or 2D spectra (ie., slit spectra); (2) Different spectra
representation, not only the default pseudo-slit plus average, now the
user can define the best representation for his/her spectra; (3)
Different spaxels selection. To the spaxel-to-spaxel selection, already
implemented, we have added two new selection methods (a) a circular
area selection, with a radius defined by the user, and (b) a
pseudo-slit spaxel selection; (4) Spaxel flux statistics. Basic
statistic on the flux of the selected spaxels has been implemented;
(5) Slice intensity 2D profiles. It is possible to plot the slice
intensity profile of the selected spaxels, and determine the FWHM and
peak intensity, by a simple fitting; (6) Single emission line
fitting. We have implemented a simple and fast single emission line
fitting. The emission line is fitted to a gaussian+continuum. The
initial guess is automatically determined. Although not very precise,
it gives the user a guess of the FWHM, line Flux, EW and continuum
(See the snap shot fitting);
(7) Automatic Kinematic Analysis. Based on the single line fitting, we
have implemented a simple and automatic Kinematic Analysis tool. Once
a spectral region is selected, and running this procedure, the program
derived the Intensity map, velocity map, velocity dispersion map and
continuum flux map of a single line. It gives a first guess of the
kinematic structure of the data. We have compared the results from
this technique with external tools, and the results are surprisingly
precise (considering the simplicity of the method). (8) Changes in the
overall GUI apparency and small details, like the asking of files
- Documentation. Although the documentation (E3D.ps)
included in the doc directory was started in the very
beggining, it has been stacked for a while. I have updated and
modified sustantially this documentation. It is still not complete,
although it contains some valuable information. If you intent to
use E3D, please read the documentation!!!
20 of October, 2003
A bug on the DAR correction has been found. It is related with the
former change from the NCARG interpolation to own coded interpolation
A new version has been uploaded (same name).
It is recommended to use the PEAK DAR correction, when possible, and
the Theoretical does not work.
12-15 of October, 2003
E3D presented to the general public in the XIII ADASS conference, with
a poster and a demo. The demo is a collection of examples of the E3D
scripts. One of these scripts is now included in the distribution of
E3D, under the directory script (plot_map.tcl, an utility for plotting
20 of August, 2003
Some bugs corrected, new options included.
28 of July, 2003
A major change has been introduced on V3D. It will not need anymore
the NCARG libraries. I have included new interpolation routines. Using
the button "MAP" it is possible to select different interpolation
options. The default one is the Cubic Spline, that is fast enough.
This change affects sustantially to V3D, so I have upgrade the version
15 of July, 2003
After the visit of Sebastien Foucaud to the AIP, we have made some
progress on V3D. We have added another system to comunicate with
external routines (like IDL, PYTHON, IRAF...), sending commands to the
visualization tool and getting data from it.
We have tested it under PYTHON (S.Foucaud) and IDL, and it seems to be
stable. Further tests may be required.
A new system to write commands to the visualization has been added,
and the documentation has been substatially updated.
We have also tested V3D with VIMOS mosaic data (7150 spectra).
We have identify some minor problem for large files, that force some
changes in the visualization tool. However, even in actual status the
visualization tool handle these data pretty well. Here you can see two
examples (VIMOS 1,VIMOS
I have upload a new version (1.0c) of V3D in the internal webpage.
3 of June, 2003
Bug with the colormap solved. New version uploaded (16:34)
27 of May, 2003
I discovered a Bug in the SAVE SLICE option. Please download the last
version from the Euro3D internal page!
26 of May, 2003
The Beta version of the visualization tool is available for
downloading. It is distributed under the restrictions of the
Actual Version On the Euro3D internal webpage
It is really very poorly documented.
The major change is the inclusion of a minor version of the NCARG
mathematical libraries within the scripts directory. Instructions of
how to install them are included at the end of the "configure" script.
If you do not have a complete installation of NCARG, this short
version would be enough for compiling V3D.
(Its use should explicitally acknowlodge to Euro3D network:
V3D is a product of the Euro3D Research Training Network, under
contract of the European Comunitty, and
include the reference:
S.F.Sanchez, 2003, proceedings of the Euro3D Science Workshop,
Cambridge, 21th-23th May, 2003, Astronomische Nachrichten)
22 of April, 2003
The last version of the Lyon library has been released, including a
nice documentation of how to produce your own project, and create a
distribution. I have created a 1st version of the documentation
here. It includes an example of how to install
the actual "distribution" of the visualization tool.
Some problems have been solved, and now it is possible to save fits
file from the created map.
6 of March, 2003
Still working on a proper documentation. A basic draf can be found
here. Including new abilities (Map
creating, interpolation, basic arithmetic). VT tested on a Suse Linux,
a RedHat linux and a Solaris Ultra 5. No major problems. New library
included NCARG (proper interpolation/fitting rutines).
For a copy of the V3D (Euro3D visualization Tool), please send me an
17 of February, 2003
New abilities added to the VT: Spectral range selection, Spaxels/Spectra
inspectors, Spaxels selection for Spectra plotting, "Objects"
definition and spectra plot, Postscript out-put.
Tests of the scripting abilities using TCL code plus the new added
commands (100% scriptable system).
Snapshoots & PS output:
10 of February, 2003 Final SHM server created.
Set of C functions to interact with the server,
load a new image, ask for a usual data, E3D information, etc...
Tcl/Tk integration of the server. Creationg of a PGPLOT/TK
interpreter, new commands added to 'mirror' the basic
New Tcl/Tk+SHM server prototype: Load a file, plot the
staked spectra, change of color map, max-min, intensity
- Snap 1: VT main window. Staked Spectra
inspector. TIGER data of NGC1358, the groups info was still not
included in the file (note the circular shape of the spaxels, instead
of the tiger square shape: circles are default)
- Snap 1: VT spaxels inspector.
Cut over an spectral range.
- Snap 2: VT spectra inspector.
Spectra of the central region, selected using the spaxels inspector.
- PS output:Postscript output of Snap
1,generated using the VT.
- PS output:Postscript output of Snap
2,generated using the VT.
1st of February, 2003 Tests of another approach. Instead of
saving all the data on the SHM, we only save the descriptors, and any
peace of data needed by the client will be asked to the server
(a always running program) that 'server' the information to the
client. Problem that solves: The memory limitation on SHM, Problem
that generates: An always running program is mandaroty.
Tests on the candecy has to be performed.
End of January, 2003
Shared Memory tests. SHM standard C library used (System V SHM).
Physical limitation found to the total amount of memory to share
(No use of the SWAP memory). Aditional limitation to the total
amount of memory to share, depending on each computer (standard Suse
instalation 32M, only root can change, and a reboot of the
computer is needed).
How to comunicate? Put all the data on a row on the SHM or
use a semaphores based program that works as a server of the
SHM. Tests: If no further comunication with other external
programs where needed it will prefered the 2nd system.
However, every new development out of the standard SHM serving
will requires a new additional change into the server-program.
1st solution more flexible. Problem: it will require a large
memory computer to handle with VIMOS-like data (not even thought
Prototype remade, splitting the load of the file into the SHM and
the plotting routines. It works faster. All functionality reproduced.
27/28 January, 2003 Presentation of the Prototype.
Proposition of the visualization tool requeriments:
scripting ability, scalability, comunication with different
scripting languages, modularity, comunication with further
PGPLOT still considered the best option for the Graphics handle.
Further studies has to be made of how to comunicate between
different processes: Shared Memory?
PGPLOT TK-driver decided to use for the graphic presentation.
Waiting for the minutes
January, 2003 First Visualization Tool prototype.
Based on the Lyon I/O library and PGPLOT. Monolitic, only
one C-file based.
It reads a Euro3D format file, plot a staked spectra image,
a monochromatic spaxels image (taking in account both
the spaxels structure and the groups), plot an spectrum over
a selected region on the staked image.
Dec 4/5, 2002 Mini-Meeting in Lyon. Discussion
about the situation of the I/O Library. First Beta version
provided. Maybe further changes due to the changes in the
physical way the data are stored on a disk (Euro3D format).
Before Dec, 2002 Studies of the viability of different
Graphics libraries: GTK+, PGPLOT, PLPLOT.
Tests with PGPLOT, considering its driver for TK and its
portability has made it the best candidate.
This page has been visited 836 times since 15.07.03