QUANTICS: Automatic Program Test

Back to Main Menu

Automatic Program Test (Elk Test)

General

This documentation contains some general information on how to test a new version of the Quantics program. The testing procedure is referred to as the elk-test , named after the famous test that showed up the fatal flaw in the Mercedes A-class. It is not a replacement for good programming practice and testing, but provides an automatic test on a standard set of input files which ensures that features of the program do not get lost by new developments. Note, before one can run the elk_test, one has to generate a comparison data set (if not already existing).

Running the Test

The automatic test program is run by a shell script elk_test. To test a new version of the program set up a directory, type

elk_test prog_name

where prog_name is the name of the new code, and wait. Depending on the machine, and how the code is compiled, the test takes approximately 15 mins to 1 hour. If you are not sure whether the path to your new code is correctly set, better give the complete (i.e. absolute) name of the new code.

The script file is in the $Quantics_DIR/bin directory of the package. If the command elk_test is not found, check that your path contains this directory. If not, add the path.

If there is a problem with finding files or programs, check that the variables in the elk_test script are set correctly:

testcode the name of the executable to be tested, e.g quantics.
ovlcode a version of the overlap program, e.g. overlap.
compcode a version of the compare program, e.g. compare.
comppath the path to the comparison set of test files, i.e. those that have been previously calculated by elk_test_gen and are known to be correct.

The script elk_test knows several options. If one types elk_test -h one obtains:


--------------------------------------------------------------------------------
Purpose: Test of the QUANTICS program
Usage: elk_test [-h -c compname -o overlap -t compare -i inppath
       -n number -r -b num -e num -m mpinum -O ompthreads] progname
   -h:  Print this help text.
   -c:  Use "compname" as comparison directory.
        If "compname" ends with a slash, $Quantics_PLATFORM will be appended.
   -o:  Use "overlap" program for overlaps.
   -t:  Use "compare" program for tests.
   -i:  Use test?.inp files from "inppath".
        If inppath=c, i.e. "-i c", comppath is used as inppath.
   -q:  Re-do the compare-checks but do not re-do tests.
   -r:  Re-do tests that failed in a previous elk_test-run.
   -s:  Skip the test with the number that follows -s.
   -n:  Do tests only for the number(s) that follow -n.
        Quote the argument of -n if it contains blanks,
        e.g.:  -n 7   or   -n '3 7 21'
        With option -n elk_test will run the required tests
        regardless whether they already exist or not.
   -b:  Begin the test with the number following -b.
   -e:  End the test with the number following -e.
   -C:  Perform only the comparison step for a single test.
        The number of the test must follow the -C option.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!  -m: Specify the number of MPI processes for the elk-test. !
!  -O: Specify the number of OPENMP threads for the elk-test.!
!  If parallel code is tested the appropriate options must be!
!  used. Further a comparison set must be available for the  !
!  chosen number of OpenMP-threads and MPI processes. Even if !
!  the appropriate comparison set is used numerical          !
!  differences can occur.                                    !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 Default overlap    :  $Quantics_DIR/bin/binary/x86_64/overlap
 Default compare    :  $Quantics_DIR/bin/binary/x86_64/compare
 Default comparison :  $Quantics_DIR/elk_comp/x86_64
 Default tests      :  $Quantics_DIR/elk_inputs
--------------------------------------------------------------------------------

One may have to use the -c option if the directory which contains the data for comparison is not the default one. In fact, it is recommended that one adjust the default setting to ones demand before runing elk_test the first time. The defaults are set at the top of the shell script.

If some test did fail, fix the code and re-run elk_test. If the name-directory of a particular test already exists, elk_test will read and analyse the data, but will not run quantics again, if a name-file already exisit. In order to re-do the failed tests use the -r or -n option.

If you want to perform a partial test, you may use the options -b (begin) and -e (end). I.e. elk_test -b 3 -e 15 quantics will run the tests 3,4,..,15. Test50 and higher are testing density operator propagation.

Results Given by the Test

The results are written to two files.

summary:

The title of the test is followed by the test results, which will be one of the following:

testXX : Passed hard limit
testXX : Passed *SOFT* limit
testXX : FAILED
testXX : skipped

Hard limit passed means that the maximum difference found in the numerical comparisons between the source code versions is < 10-12. The soft limit is < 10-6.

results:

This file contains more information on the various tests, e.g. CPU-time and memory required by the two versions, and more details on the numerical checks carried out. Thus if the limits are not met, one can look in the results file to see where the problem lies.

Checks Carried Out

The following comparisons are made.

  1. Overlaps between the wavefunctions. The average and maximum difference between the overlaps are calculated.
  2. The numbers in the auto files are compared.
  3. The numbers in the check file are compared.
  4. The numbers in the dicht file are compared.

In addition, some of the analysis programs are also tested.

Generating the Test Set

To generate the comparison set of files, set up a directory, and run the script

elk_test_gen

The test set is generated using the input files in $Quantics_DIR/elk_inputs. The elk_test_gen script also contains variables specifying the code to be used. The script file is in $Quantics_DIR/bin. If the command elk_test_gen is not found, check that your path contains this directory. The output directory is the current directory with $QUANTICS_PLATFORM appended if the pwd-path ends otherwise. elk_test_gen will make the output directory if it does not exist.

To use a version of Quantics with a different name (e.g. a debug version QuanticsD), use elk_test_gen -q prog_name where prog_name is the name of the program to be used for the generation.

One may use other options to change the paths of the input and output files. Running elk_test_gen -h lists the options.

If one or some of the runs failed, fix the error and run elk_test_gen again. The script will not re-do succesfully installed sets.

Adding New Tests

To add new tests, simply set up a new test.inp file in the comparison code test directory, using the next available number, e.g. test31.inp. The .inp file must contain a 3 line title at the beginning. This is written to the output files. The title should start with ### Title of (see other test*.inp files). The psi, auto, and dicht files must be created. If any special checks are to be made, the program compare.F, found in $Quantics_DIR/source/analyse, must be altered. Finally, add the title to the list in this document, $QUANTICS_DIR/doc/quantics/elk.html

By typing elk_test_gen when in the comparison test directory the new test files will be generated. The elk_test can then be re-run, including this new comparison.

To add new tests of Analysis Programs, you will need to

  1. Add the test to be run in the mkanal function of the elk_test_gen script.
  2. Add the test to be run in the testanal function of the elk_test script.
  3. If it is a new code and not just a new test, the program analysis/compare.F90 must be edited to add the tests. Follow the prescription for e.g. adpop .

List of Tests

test1.inp
Test of NOCl1 / CMF integrator
test2.inp
Test of NOCl1 / default VMF integrator
test3.inp
Test of NOCl1 / build option
test4.inp
Test of NOCl1 / FFT basis-section
test5.inp
Test of NOCl1 with full potential
test6.inp
Test of NOCl1 with full potential / CDVR
test7.inp
Test of pyrazine 4-mode multi-set
test8.inp
Test of pyrazine 4-mode single-set
test9.inp
Test of pyrazine 4-mode reading oper file
test10.inp
Test of Propagation of CH3I (combined mode multi-packet version)
test11.inp
Test of Relaxation of NOCl0
test12.inp
Test of Relaxation of CH3I (Single-packet version)
test13.inp
Test of Relaxation of CH3I (Multi-packet version)
test14.inp
Test of Propagation of CH3I (Single-packet version)
test15.inp
Test of Propagation of CH3I (multi-packet version)
test16.inp
Test of NOCl restart adding spfs
test17.inp
Test of NOCL1 continuation
test18.inp
Test of pyrazine 4-mode: natural orbitals, psi = compact
test19.inp
Test of pyrazine 4-mode: psi = natur
test20.inp
Test of H+D2 (J=0) reactive scattering
test21.inp
Test of H2/surface scattering. sphfbr, fft periodic
test22.inp
Test of Propagation of CH3I (multi-set, CMF, h-proj option)
test23.inp
Test of H2/surface scattering. test for autocap
test24.inp
Test of NOCl, multi-packet
test25.inp
Test of pyrazine 4-mode multi-set multi-packet
test26.inp
Test of KLeg. H+D2 reactive scattering (J>0)
test27.inp
Test of expectation / Butatriene linear 5-mode model
test28.inp
Test of N2/surface scattering. PLeg, fft, exp -DVRs. CMF
test29.inp
Test of Relaxation=0 of CH3I (Multi-packet version)
test30.inp
Test of NOCl / Time-dependent operator
test31.inp
Test of CO2 / CMF integrator
test32.inp
Test of NOCl / Time-dep. operator / RK8 integrator and imp-ortho
test33.inp
Test of HONO, N=O excit. / Impr. Relaxation, lock, precon
test34.inp
Test34: pyrazine 4-mode vMCG multi-set
test35.inp
Test35: pyrazine 4-mode vMCG single-set
test36.inp
Test 36: 4D Henon-Heiles vMCG
test37.inp
Test 37: 4D Henon-Heiles vMCG VMF
test38.inp
Test38: pyrazine 4-mode G-Quantics multi-set
test39.inp
DD / Propagation in normal modes
test40.inp
Multi-set DD / Propagation in normal modes
test41.inp
Test41: vMCG relaxation on simple HO. Test of lindep
test42.inp
Test of TRPES: Benzene X, minimal 3-mode 2-state model + ion
test43.inp
Test of LCT: State optimisation of cyclobutadiene
test44.inp
Test: Relaxation 3 bosonic particles, 1D, in a har. trap
test45.inp
Test: Propagation 3 bosonic particles, 1D, in a har. trap
test46.inp
Test of H_2+H_2-Scattering (Propagation) with KLeg
test47.inp
Test of H_2+H_2-Scattering (Propagation) with KLeg
test48.inp
Test48: pyrazine 4-mode G-Quantics multi-set -- Manual Continuation
test49.inp
Test49: pyrazine 4-mode G-Quantics multi-set -- Continuation
test50.inp
Test of: density2 / VMF / closed / Temp / Henon-Heiles
test51.inp
Test of: density2 / VMF / open / Temp / Henon-Heiles
test52.inp
Test of density2 / Pyrazine 4 modes / CMF / s= 3
test53.inp
Test of density2 / Pyrazine 4 modes / VMF / s=3
test54.inp
Test of density2 / Pyrazine 4 modes / CMF / s=2 / Temperature
test55.inp
Test of density2 / Pyrazine 4 modes / CMF / s=2 / linear
test56.inp
Test of density2 / linear / Relax / NOCl S0 surface
test57.inp
Test of density2 / linear / Propagation of NOCl1 to 10 fs.
test58.inp
Test of density1 / VMF / closed / Temp / Henon-Heiles system
test59.inp
Test of density1 / VMF / open / Temp / Henon-Heiles system
test60.inp
Test of density1 / VMF / closed / Temp / Pyrazine 4 moden
test61.inp
Test of density1 / VMF / open / Pyrazine 4 moden
test62.inp
Test of vMCG / LHA = 4 / Salicylaldehyde (vMCG defaults: dynamic allocation, no_cx)
test63.inp
Test63: DD / Propagation in cartesians
test64.inp
Test64: DD / Propagation in normal modes
test65.inp
Test65: DD on QM/MM PES with QoMMMa (In cartesians)
test66.inp
Test66: Some frozen atoms (normal modes)
test67.inp
Test67: Some frozen atoms (cartesians)
test68.inp
Test68: Some classical GBFs (for QM/MM dynamics) (cartesians)
test69.inp
Test69: Multi GBF / (normal modes)
test70.inp
Test70: Multi GBF / (cartesians)
test71.inp
Test71: vMCG / LHA = 4 / Salicylaldehyde (dynamic allocation, time-dependent uncoupled functions)
test72.inp
Test72: vMCG / LHA = 4 / Salicylaldehyde (1ML-vMCG)
test73.inp
Test73: vMCG / LHA = 4 / Salicylaldehyde (1ML-G-Quantics)
test74.inp
Test74: pyrazine 4-mode vMCG multi-set
test75.inp
Test75: pyrazine 4-mode vMCG multi-set (gwpdim .ne. dim)
test76.inp
Test of vMCG / LHA = 4 / Salicylaldehyde (without dynamic allocation)
test77.inp
Test77: pyrazine 4-mode 1ML-vMCG single-set
test78.inp
TUTORIAL: 3D, J=1, H+H2 reactive scattering in Jacobian coordinates.
test79.inp
Test of NOCl1 with cut full potential (point DVR)
test80.inp
Basic test of ML-Quantics: 6D Henon-Heiles model potential
test81.inp
Test81: G-Quantics correlated LHA / Salicylaldehyde
test82.inp
Test82: G-Quantics with correlated DD LHA
test83.inp
Basic test of ML-Quantics with CMF: 6D Henon-Heiles model potential
test84.inp
Test of pyrazine 4-mode ML-Quantics CMF
test85.inp
Test of the read-inwf block in init_wf-section