Automatic Program Test (Elk Test)

General

This documentation contains some general information on how to test a new version of the Quantics program. The testing procedure is referred to as the elk-test , named after the famous test that showed up the fatal flaw in the Mercedes A-class. It is not a replacement for good programming practice and testing, but provides an automatic test on a standard set of input files which ensures that features of the program do not get lost by new developments. Note, before one can run the elk_test, one has to generate a comparison data set (if not already existing).

Running the Test

The automatic test program is run by a shell script elk_test. To test a new version of the program set up a directory, type

elk_test prog_name

where prog_name is the name of the new code, and wait. Depending on the machine, and how the code is compiled, the test takes approximately 15 mins to 1 hour. If you are not sure whether the path to your new code is correctly set, better give the complete (i.e. absolute) name of the new code.

The script file is in the $Quantics_DIR/bin directory of the package. If the command elk_test is not found, check that your path contains this directory. If not, add the path.

If there is a problem with finding files or programs, check that the variables in the elk_test script are set correctly:

`testcode`	the name of the executable to be tested, e.g quantics.
`ovlcode`	a version of the overlap program, e.g. overlap.
`compcode`	a version of the compare program, e.g. compare.
`comppath`	the path to the comparison set of test files, i.e. those that have been previously calculated by `elk_test_gen` and are known to be correct.

The script elk_test knows several options. If one types elk_test -h one obtains:


--------------------------------------------------------------------------------
Purpose: Test of the QUANTICS program
Usage: elk_test [-h -c compname -o overlap -t compare -i inppath
       -n number -r -b num -e num -m mpinum -O ompthreads] progname
   -h:  Print this help text.
   -c:  Use "compname" as comparison directory.
        If "compname" ends with a slash, $Quantics_PLATFORM will be appended.
   -o:  Use "overlap" program for overlaps.
   -t:  Use "compare" program for tests.
   -i:  Use test?.inp files from "inppath".
        If inppath=c, i.e. "-i c", comppath is used as inppath.
   -q:  Re-do the compare-checks but do not re-do tests.
   -r:  Re-do tests that failed in a previous elk_test-run.
   -s:  Skip the test with the number that follows -s.
   -n:  Do tests only for the number(s) that follow -n.
        Quote the argument of -n if it contains blanks,
        e.g.:  -n 7   or   -n '3 7 21'
        With option -n elk_test will run the required tests
        regardless whether they already exist or not.
   -b:  Begin the test with the number following -b.
   -e:  End the test with the number following -e.
   -C:  Perform only the comparison step for a single test.
        The number of the test must follow the -C option.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  -m: Specify the number of MPI processes for the elk-test. !
!  -O: Specify the number of OPENMP threads for the elk-test.!
!  If parallel code is tested the appropriate options must be!
!  used. Further a comparison set must be available for the  !
!  chosen number of OpenMP-threads and MPI processes. Even if !
!  the appropriate comparison set is used numerical          !
!  differences can occur.                                    !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 Default overlap    :  $Quantics_DIR/bin/binary/x86_64/overlap
 Default compare    :  $Quantics_DIR/bin/binary/x86_64/compare
 Default comparison :  $Quantics_DIR/elk_comp/x86_64
 Default tests      :  $Quantics_DIR/elk_inputs
--------------------------------------------------------------------------------

One may have to use the -c option if the directory which contains the data for comparison is not the default one. In fact, it is recommended that one adjust the default setting to ones demand before runing elk_test the first time. The defaults are set at the top of the shell script.

If some test did fail, fix the code and re-run elk_test. If the name-directory of a particular test already exists, elk_test will read and analyse the data, but will not run quantics again, if a name-file already exisit. In order to re-do the failed tests use the -r or -n option.

If you want to perform a partial test, you may use the options -b (begin) and -e (end). I.e. elk_test -b 3 -e 15 quantics will run the tests 3,4,..,15. Test50 and higher are testing density operator propagation.

Results Given by the Test

The results are written to two files.

summary:

The title of the test is followed by the test results, which will be one of the following:

testXX : Passed hard limit testXX : Passed *SOFT* limit testXX : FAILED testXX : skipped

Hard limit passed means that the maximum difference found in the numerical comparisons between the source code versions is < 10^-12. The soft limit is < 10^-6.

results:

This file contains more information on the various tests, e.g. CPU-time and memory required by the two versions, and more details on the numerical checks carried out. Thus if the limits are not met, one can look in the results file to see where the problem lies.

Checks Carried Out

The following comparisons are made.

Overlaps between the wavefunctions. The average and maximum difference between the overlaps are calculated.
The numbers in the auto files are compared.
The numbers in the check file are compared.
The numbers in the dicht file are compared.

In addition, some of the analysis programs are also tested.

Generating the Test Set

To generate the comparison set of files, set up a directory, and run the script

elk_test_gen

The test set is generated using the input files in $Quantics_DIR/elk_inputs. The elk_test_gen script also contains variables specifying the code to be used. The script file is in $Quantics_DIR/bin. If the command elk_test_gen is not found, check that your path contains this directory. The output directory is the current directory with $QUANTICS_PLATFORM appended if the pwd-path ends otherwise. elk_test_gen will make the output directory if it does not exist.

To use a version of Quantics with a different name (e.g. a debug version QuanticsD), use elk_test_gen -q prog_name where prog_name is the name of the program to be used for the generation.

One may use other options to change the paths of the input and output files. Running elk_test_gen -h lists the options.

If one or some of the runs failed, fix the error and run elk_test_gen again. The script will not re-do succesfully installed sets.

Adding New Tests

To add new tests, simply set up a new test.inp file in the comparison code test directory, using the next available number, e.g. test31.inp. The .inp file must contain a 3 line title at the beginning. This is written to the output files. The title should start with ### Title of (see other test*.inp files). The psi, auto, and dicht files must be created. If any special checks are to be made, the program compare.F, found in $Quantics_DIR/source/analyse, must be altered. Finally, add the title to the list in this document, $QUANTICS_DIR/doc/quantics/elk.html

By typing elk_test_gen when in the comparison test directory the new test files will be generated. The elk_test can then be re-run, including this new comparison.

To add new tests of Analysis Programs, you will need to

Add the test to be run in the mkanal function of the elk_test_gen script.
Add the test to be run in the testanal function of the elk_test script.
If it is a new code and not just a new test, the program analysis/compare.F90 must be edited to add the tests. Follow the prescription for e.g. adpop .

List of Tests

test1.inp: Test of NOCl1 / CMF integrator
test2.inp: Test of NOCl1 / default VMF integrator
test3.inp: Test of NOCl1 / build option
test4.inp: Test of NOCl1 / FFT basis-section
test5.inp: Test of NOCl1 with full potential
test6.inp: Test of NOCl1 with full potential / CDVR
test7.inp: Test of pyrazine 4-mode multi-set
test8.inp: Test of pyrazine 4-mode single-set
test9.inp: Test of pyrazine 4-mode reading oper file
test10.inp: Test of Propagation of CH3I (combined mode multi-packet version)
test11.inp: Test of Relaxation of NOCl0
test12.inp: Test of Relaxation of CH3I (Single-packet version)
test13.inp: Test of Relaxation of CH3I (Multi-packet version)
test14.inp: Test of Propagation of CH3I (Single-packet version)
test15.inp: Test of Propagation of CH3I (multi-packet version)
test16.inp: Test of NOCl restart adding spfs
test17.inp: Test of NOCL1 continuation
test18.inp: Test of pyrazine 4-mode: natural orbitals, psi = compact
test19.inp: Test of pyrazine 4-mode: psi = natur
test20.inp: Test of H+D2 (J=0) reactive scattering
test21.inp: Test of H2/surface scattering. sphfbr, fft periodic
test22.inp: Test of Propagation of CH3I (multi-set, CMF, h-proj option)
test23.inp: Test of H2/surface scattering. test for autocap
test24.inp: Test of NOCl, multi-packet
test25.inp: Test of pyrazine 4-mode multi-set multi-packet
test26.inp: Test of KLeg. H+D2 reactive scattering (J>0)
test27.inp: Test of expectation / Butatriene linear 5-mode model
test28.inp: Test of N2/surface scattering. PLeg, fft, exp -DVRs. CMF
test29.inp: Test of Relaxation=0 of CH3I (Multi-packet version)
test30.inp: Test of NOCl / Time-dependent operator
test31.inp: Test of CO2 / CMF integrator
test32.inp: Test of NOCl / Time-dep. operator / RK8 integrator and imp-ortho
test33.inp: Test of HONO, N=O excit. / Impr. Relaxation, lock, precon
test34.inp: Test34: pyrazine 4-mode vMCG multi-set
test35.inp: Test35: pyrazine 4-mode vMCG single-set
test36.inp: Test 36: 4D Henon-Heiles vMCG
test37.inp: Test 37: 4D Henon-Heiles vMCG VMF
test38.inp: Test38: pyrazine 4-mode G-Quantics multi-set
test39.inp: DD / Propagation in normal modes
test40.inp: Multi-set DD / Propagation in normal modes
test41.inp: Test41: vMCG relaxation on simple HO. Test of lindep
test42.inp: Test of TRPES: Benzene X, minimal 3-mode 2-state model + ion
test43.inp: Test of LCT: State optimisation of cyclobutadiene
test44.inp: Test: Relaxation 3 bosonic particles, 1D, in a har. trap
test45.inp: Test: Propagation 3 bosonic particles, 1D, in a har. trap
test46.inp: Test of H_2+H_2-Scattering (Propagation) with KLeg
test47.inp: Test of H_2+H_2-Scattering (Propagation) with KLeg
test48.inp: Test48: pyrazine 4-mode G-Quantics multi-set -- Manual Continuation
test49.inp: Test49: pyrazine 4-mode G-Quantics multi-set -- Continuation
test50.inp: Test of: density2 / VMF / closed / Temp / Henon-Heiles
test51.inp: Test of: density2 / VMF / open / Temp / Henon-Heiles
test52.inp: Test of density2 / Pyrazine 4 modes / CMF / s= 3
test53.inp: Test of density2 / Pyrazine 4 modes / VMF / s=3
test54.inp: Test of density2 / Pyrazine 4 modes / CMF / s=2 / Temperature
test55.inp: Test of density2 / Pyrazine 4 modes / CMF / s=2 / linear
test56.inp: Test of density2 / linear / Relax / NOCl S0 surface
test57.inp: Test of density2 / linear / Propagation of NOCl1 to 10 fs.
test58.inp: Test of density1 / VMF / closed / Temp / Henon-Heiles system
test59.inp: Test of density1 / VMF / open / Temp / Henon-Heiles system
test60.inp: Test of density1 / VMF / closed / Temp / Pyrazine 4 moden
test61.inp: Test of density1 / VMF / open / Pyrazine 4 moden
test62.inp: Test of vMCG / LHA = 4 / Salicylaldehyde (vMCG defaults: dynamic allocation, no_cx)
test63.inp: Test63: DD / Propagation in cartesians
test64.inp: Test64: DD / Propagation in normal modes
test65.inp: Test65: DD on QM/MM PES with QoMMMa (In cartesians)
test66.inp: Test66: Some frozen atoms (normal modes)
test67.inp: Test67: Some frozen atoms (cartesians)
test68.inp: Test68: Some classical GBFs (for QM/MM dynamics) (cartesians)
test69.inp: Test69: Multi GBF / (normal modes)
test70.inp: Test70: Multi GBF / (cartesians)
test71.inp: Test71: vMCG / LHA = 4 / Salicylaldehyde (dynamic allocation, time-dependent uncoupled functions)
test72.inp: Test72: vMCG / LHA = 4 / Salicylaldehyde (1ML-vMCG)
test73.inp: Test73: vMCG / LHA = 4 / Salicylaldehyde (1ML-G-Quantics)
test74.inp: Test74: pyrazine 4-mode vMCG multi-set
test75.inp: Test75: pyrazine 4-mode vMCG multi-set (gwpdim .ne. dim)
test76.inp: Test of vMCG / LHA = 4 / Salicylaldehyde (without dynamic allocation)
test77.inp: Test77: pyrazine 4-mode 1ML-vMCG single-set
test78.inp: TUTORIAL: 3D, J=1, H+H2 reactive scattering in Jacobian coordinates.
test79.inp: Test of NOCl1 with cut full potential (point DVR)
test80.inp: Basic test of ML-Quantics: 6D Henon-Heiles model potential
test81.inp: Test81: G-Quantics correlated LHA / Salicylaldehyde
test82.inp: Test82: G-Quantics with correlated DD LHA
test83.inp: Basic test of ML-Quantics with CMF: 6D Henon-Heiles model potential
test84.inp: Test of pyrazine 4-mode ML-Quantics CMF
test85.inp: Test of the read-inwf block in init_wf-section

QUANTICS: Automatic Program Test