Monday, March 31. 2008
A new paper has been added to the collection of reproducible papers:
Abstract:
Geophysical inverse problems typically involve a trade off between data misfit and some prior. Pareto curves trace the optimal trade off between these two competing aims. These curves are commonly used in problems with two-norm priors where they are plotted on a log-log scale and are known as L-curves. For other priors, such as the sparsity-promoting one norm, Pareto curves remain relatively unexplored. We show how these curves lead to new insights into one-norm regularization. First, we confirm the theoretical properties of smoothness and convexity of these curves from a stylized and a geophysical example. Second, we exploit these crucial properties to approximate the Pareto curve for a large-scale problem. Third, we show how Pareto curves provide an objective criterion to gauge how different one-norm solvers advance towards the solution.
Wednesday, March 26. 2008
- To place tick labels perpendicular to an axis (rather than parallel to it), use parallel#=n, where # is 1, 2, or 3.
.
- To control the tick selection manually, use n#tic=, o#=num, and d#num=.
- To control the label format, use format#= (the argument is a printf-style string)
The following example is provided in rsf/rsf/sfgraph:
Friday, March 21. 2008
Now you can. See an example, where a figure originally generated with Matlab is replicated with Pylab.
To prepare your figures, follow the rules similar to those for Matlab and Mathematica figures:
- Create a directory called Pylab.
- Put figure-generating python scripts in this directory.
- Each script should have a .py suffix
- Each script should end with a command like savefig('junk_py.eps'); (the name junk_py.eps is important.)
Friday, March 14. 2008
Another old paper has been added to the collection of reproducible papers:
We are already well into 2008 -- and could this year pass without a Madagascar Event?
Of course not!
A coding sprint is coming at full speed, towards us!
Madagascar developers are invited to congregate during May 23-27 in beautiful Golden, Colorado, for the
2008 MADAGASCAR IMPLEMENTATION WORKSHOP:
TOWARDS FULL AUTOMATION AND BETTER ROBUSTNESS
Yes, this is right. Instead of dozing in a dark room listening to others talk, we will actually write code together! We will give Madagascar a real push towards maturity. All necessary info, and more, can be found on the wiki page of the event. See you in Golden!
Monday, February 25. 2008
The first constraint to observe when dealing with wide/full-azimuth data is its sheer volume (tens of Terabytes). Data manipulation becomes the bottleneck procedure that the programmer must pay attention to. In practice, this means that data sorting, FFT-ing, axis reversing and transposing are not trivial operations any longer and their number must be minimized. As a consequence, it will often be preferrable to re-write a particular processing tool to apply to the current form the data is in, instead of re-shaping the data to fit to an existing algorithm. Thankfully, such re-writing would usually only involve re-ordering loops and adding or removing FFTs.
The circumstances above mean that clean, documented, maintainable codes, that can be modified in a pinch without adding bugs are a must when working with wide-azimuth data. The collaboration among geographically separated programmers that do not know each other and do not share a common cultural background necessarily imposes these qualities on open-source software. Considerable discipline is needed by in-house programmers in order to get to the same result. Companies which use open-source software that has the above-described qualities will be able to have a faster wide/full azimuth project turnaround. Conversely, the emergence of wide/full-azimuth data acquisition represents a great opportunity for community-based geophysical open-source software!
Thursday, December 27. 2007
Many of the data processing operations are data-parallel: different traces, shot gathers, frequency slices, etc. can be processed independently. Madagascar provides several mechanisms for handling this type of embarrassingly parallel applications on computers with multiple processors.
- OpenMP
OpenMP is a standard framework for parallel applications on shared-memory systems. It is supported by the latest versions of GCC and by some other compilers.
To run a data-parallel processing task like
sfradon np=100 p0=0 dp=0.01 < inp.rsf > out.rsf
on a shared-memory computer with multiple processors (such as a multi-core PC), try sfomp, as follows:.
sfomp sfradon np=100 p0=0 dp=0.01 < inp.rsf > out.rsf
sfomp splits the input along the slowest axis (presumed to be data-parallel) and runs it through parallel threads. The number of threads is set by the OMP_NUM_THREADS environmental variable or (by default) by the number of available CPUs.
- MPI
MPI (Message-Passing Interface) is a standard framework for parallel processing on different computer architectures including distributed-memory systems. Several MPI implementations (such as MPICH) are available.
To parallelize a task using MPI, try sfmpi, as follows:
mpirun -np 8 sfmpi sfradon np=100 p0=0 dp=0.01 input=inp.rsf output=out.rsf
where the argument after -np specifies the number of processors involved. sfmpi will use this number to split the input along the slowest axis (presumed to be data-parallel) and to run it through parallel threads.
Note: Some MPI implementations do not support system calls implemented in sfmpi and therefore will not support this option.
- MPI + OpenMP
It is possible to combine the advantages of shared-memory and distributed-memory architectures by using OpenMP and MPI together.
mpirun -np 32 sfmpi sfomp sfradon np=100 p0=0 dp=0.01 input=inp.rsf output=out.rsf
will distribute the job on 32 nodes and split it again on each node using shared-memory threads.
- SCons
If you process data using SCons, another option is available. Change
Flow('out','inp','radon np=100 p0=0 dp=0.01')
in your SConstruct file to
Flow('out','inp','radon np=100 p0=0 dp=0.01',split=[0],axis=[3,256])
where the optional split=, axis=, and length= parameters contain a list of the sources that you want to split for parallel processing, the axis that needs to be split, and the size of this axis. Then run something like
scons -j 8 CLUSTER='localhost 4 node1.utexas.edu 2' out.rsf
The -j options instructs SCons to run in parallel creating 8 threads, while the CLUSTER= option supplies it with the list of nodes to use and the number of processes to involve for each node. The output may look like
< inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=0 squeeze=n > inp__0.rsf
< inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=42 squeeze=n > inp__1.rsf
/usr/bin/ssh node1.utexas.edu "cd /home/test ; /bin/env < inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=84 squeeze=n > inp__2.rsf "
< inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=126 squeeze=n > inp__3.rsf
< inp.rsf /RSFROOT/bin/sfwindow n3=42 f3=168 squeeze=n > inp__4.rsf
/usr/bin/ssh node1.utexas.edu "cd /home/test ; /bin/env < inp.rsf /RSFROOT/bin/sfwindow f3=210 squeeze=n > inp__5.rsf "
< inp__0.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__0.rsf
/usr/bin/ssh node1.utexas.edu "cd /home/test ; /bin/env < inp__1.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__1.rsf "
< inp__3.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__3.rsf
/usr/bin/ssh node1.utexas.edu "cd /home/test ; < spike__4.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__4.rsf "
< inp__2.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__2.rsf
< inp__5.rsf /RSFROOT/bin/sfradon p0=0 np=100 dp=0.01 > out__5.rsf
< out__0.rsf /RSFROOT/bin/sfcat axis=3 out__1.rsf out__2.rsf out__3.rsf out__4.rsf out__5.rsf > out.rsf
Splitting the input with sfwindow and putting the output back together with sfcat are immediately apparent. The advantage of the SCons-based approach (in addition to documentation and reproducible experiments) is fault tollerance: If one of the nodes dies during the process, one should be able to restart the computation without recreating parts that are already computed.
All these options will continue to evolve and improve with further testing. Please report your experiences and suggestions.
Monday, December 24. 2007
Another old paper has been added to the collection of reproducible papers:
Saturday, October 27. 2007
The collection of fonts in Vplot is small and goes back to so-called "Hershey fonts" (created originally by Dr. A.V. Hershey at the U. S. National Bureau of Standards).
You can use \F# directives to switch between different fonts.
In general, there are two sorts of escape sequences, those that take an argument and those that do not. Here is a complete list of escape sequences that do not take an argument:
- \> Advance one interletter space
- \< Back up one interletter space
- \^Raise one half of a capital letter height
- \_ Lower one half of a capital letter height
- \g Continue processing text, but don't actually print it ("ghostify it").
This is useful if you want to leave space to go back and add something by hand.
- \G Start printing text again ("deghostify")
- \n Newline
- \h Backspace (control-h also works) back up over the last character
- \- Does nothing; used to prevent a group of characters from being formed into a ligature.
- \\ Print a backslash
The following escape sequences take an integer argument immediately after, with a required space after the integer to delineate the end. This space is not printed.
- \s# Size change. Change to # percent of the size set in the text vplot. \s100 restores the default height.
- \f# Add # to current fatness. Goes out of effect when text printing is finished.
- \F# Switch to font number #. (-1 restores the default font).
- \k# Move by # space widths to the right (in percent; 100 = one space width). Negative numbers are allowed (moves left).
- \r# Move up # character heights (in percent; 100 = the height of a standard capital letter). Negative numbers are allowed (moves down).
- \v# Print ASCII character number # in the current font, stripping it of any special meaning. This and ligatures are the only way that glyphs numbered greater than 255 are available.
- \c# Switch to color number #. \c-1 restores the current drawing color. Vplot's current drawing color is not changed by changing the color inside text.
- \m# Save current position in register number #.
- \M# Restore position saved in register number #.
The following example is from rsf/rsf/sfgraph:
The line to create this title is title="\s100 \F11 Polinom CHebysheva + \F3 T\_\s75 n\^\s100 (cos \F10 q) = \F3 cos n\F10 q" .
You can find a set of tests for different fonts in pens/tests:
 original (designed by Rob Clayton at Stanford)
roman simplex
roman duplex
roman complex
roman triplex
 italic complex
 italic triplex
 script simplex
 script complex
 greek simplex
 greek complex
 Cyrillic complex
\ German style gothic triplex
 Greek style gothic triplex
 Italian style gothic triplex
 mathematics
 miscellaneous
To display a particular font character by character on the screen, try something like
sfplas < RSFSRC/pens/fonts/cyrilc.vplot_font | \
xtpen xcenter=0 ycenter=0 scale=50 pause=1
Thanks to Joe Dellinger for help with this answer! For more information, see his presentation on Vplot and vplotttext.m.
Sunday, October 14. 2007
New stable version is released.
The previous stable versions have been downloaded more than 1,500 times in 16 months. In the same period of time, the development version experienced more than 3,300 read transactions, more than 1,100 write transactions, and more than 7,400 file updates ( statistics from SourceForge).
madagascar has been successfully installed on HP-UX and SGI Irix using native compilers.
Sunday, October 7. 2007
A new paper has been added to the collection of reproducible papers:
A new paper has been added to the collection of reproducible papers:
Saturday, October 6. 2007
New paper has been added to the collection of reproducible papers:
Thursday, September 20. 2007
OpendTect announces plans to implement a Madagascar interface:
http://www2.opendtect.org/madagascar_rsf.html.
OpendTect is a seismic interpretation software system as well as a research and development environment for seismic interpretation.
|