config_params.md.src

Configuration parameters
========================

    lux [--mode Mode] [ConfigParam]... [File]...

Exit status is 0 if all test cases are successful and 1 otherwise.

Configuration parameters can be given as command line options
`--Var=Val`, as `[config Var=Value]` statements in a script or in a
**architecture specific file**.

An `architecture specific file` is a file with configuration
statements only valid for a certain architecture/platform/system.
The syntax of such a file is the same a normal Lux script, but only
configuration settings (`[config Var=Value]`) are extracted. See
also the configuration parameters `--config_name` and `--config_dir`.
The file extension is `.luxcfg`.

When a test suite (one or more test cases) is to be evaluated, the Lux
engine will determine the software/hardware signature of the system to
construct the name of a architecture specific file. If a file with
that name exists, the architecture specific configuration settings
will be extracted and used as base for the entire test suite. These
settings can however be overridden by command line options and
`[config var=val]` statements in each test case file.

The Lux engine evaluates one or more Lux files. Lux files has normally
`.lux` as extension. See the configuration parameter `--file_pattern`.
If a directory is given as input, all `.lux` files in that directory
and its sub directories are evaluated. The given files (files or
directories) are called test suites and the derived files (actual Lux
scripts) are called test cases. The configuration parameter `--skip`
can be used to conditionally skip test cases.

The lookup of configuration parameters/settings is performed in the
following order:

1. Command line parameters. Eg. `--skip_skip` to ignore all skip settings.

2. Command line parameters from `LUX_FLAGS` environment variable. To
be used interactively.

3. Command line parameters from `LUX_SYSTEM_FLAGS` environment
variable. To be used by makefiles, scripts etc.

4. Test case specific configuration settings.

5. Architecture specific configuration settings from a `.luxcfg` file
on the `--config_dir` directory. See `--config_name` about the naming
convention.

6. Non-architecture specific configuration settings defined in a file
named `luxcfg` on the `--config_dir` directory.

7. Site local default configuration settings defined in a file named
`luxcfg` on the `lux/priv` directory.

8. Environment variables automatically converted to `[config var=val]`
settings..

9. Hardcoded built-in default values.

Test case control
-----------------

**--mode Mode**  
Mode can be one of :

* `execute`  - evaluate the test cases. This is default.
* `validate` - parse all script files and configuration files and
               report syntax errors and warnings.
* `dump`     - parse and dump the internal form for all script files
               and configuration files and report syntax errors and
               warnings.
* `expand`   - parse and expand source files for all script files
               and configuration files.
* `list`     - display a list of all (non-skipped) test cases.
               One file per line.
* `list_dir` - display a list of all directories with non-skipped
               test cases. One directory per line.
* `doc`      - extract all `[doc]` and `[docN]` strings and display
               them on a simple format which is as follows.
               First the script file name is printed on an own line
               ending with a colon, followed by all doc strings, each
               one on a separate line. The doc strings are indented
               with a tab char for each doc level. See [docN].

**--doc Level**  
Implies `--mode=doc`. Restricts how many documentation levels which
should be displayed. `--doc=1` only shows documentation on level 1,
`--doc=2` shows documentation both on level 1 and 2.

The first `[doc]` documentation string in a script is a bit special as
it is regarded as a one line summary of the script. With `--doc=0`
only the oneline summary lines are displayed.

**--rerun Result**  
Rerun old test cases. The test case candidates are found by parsing
old log summary files. If any `File` is explicitly given on command
line these files are interpreted as log directories possibly
containing summary log files. If no `File` is given the log directory
referred to by the `latest_run` link is used.

For each found test case its result must have the same outcome or
higher (worse) than `Result`.`Result` is an enum whose names and
relative values are as follows:

    enable < success < skip < warning < fail < error < disable

For example `--rerun=fail` implies that all old test cases whose
outcome is fail or error will be rerun.

Default is `disabled`, which means that this behavior is disabled.

**-r**  
A shortcut for `--rerun=fail`.

**--file\_pattern**  
Specify file pattern for scripts to be executed when a directory is
given. Defaults to `.*.lux$`.

**--var**  
Overrides environment variable settings. Each entry must be of the
form `var=value`.

**--config\_name ConfigName**  
Normally Lux figures out which system software/hardware it runs on,
but it can explicitly be overridden with the `ConfigName` option. The
`ConfigName` is used to read system architecture specific configuration
parameters from a file named `ConfigName.luxcfg`. By default `ConfigName`
 is obtained from `uname -sm` where `ConfigName` is set to `Kernel-Machine`.
This behavior can be overridden by adding a file named after the name of
the host (`hostname.luxcfg`) on the `ConfigDir` directory.

**--config\_dir ConfigDir**  
A directory where architecture specific configuration files may
reside. The format of the architecture specific files a subset of the
script format. Only `[config var=value]` statements are extracted from
the architecture specific file. The config settings in the
architecture specific file may be overridden by config settings in the
script files. Config settings in script files may be overridden by
command line options. Architecture specific files are by default
located in the subdirectory called `priv` in the `Lux` application.

Non-architecture settings can be put in a file named `luxcfg`. But
those will be overridden by the architecture specific settings.

**--hostname Hostname**  
The `Hostname` overrides the hostname obtained from the operating
system. It may be useful when testing config settings of other
machines or faking the hostname in a test environment with multiple
equivalent slaves.

**--require Var**  
**--require Var=Value**  
Require the given variable to be set. The script will fail if
the variable not is set. This option can be used multiple times,
which means that all given Vars are required to be set.
Typically require is used to test on presence of environment
variables. `--require` is intended to be used as `[config require=Var]`
or `[config require=Var=Value]` statements within scripts. The
construction **Var=Value** is little more restrictive as it
requires the variable to be set to a certain value.

**--skip Var**  
**--skip Var=Value**  
Skip execution of the script if the given variable is set. This
option can be used multiple times, which means that it suffices
that one of the given `Var`s is set in order to skip the test
case. Typically `--skip` is used to test on presence of environment
variables. `--skip` is intended to be used as `[config skip=Var]`
or `[config skip=Var=Value]` statements within scripts. The
construction **Var=Value** is little more restrictive as it requires
that the variable is set to a certain value.

**--skip\_unless Var**  
**--skip\_unless Var=Value**  
Skip execution of the script if the given variable NOT is set. This
option can be used multiple times, which means that it suffices
that one of the given `Var`s NOT is set in order to skip the test
case. Typically `--skip_unless` is used to test on absence of
environment variables. `--skip_unless` is intended to be used as
`[config skip_unless=Var]` or `[config skip_unless=Var=Value]`
statements within scripts. The construction **Var=Val** is little more
restrictive as it requires that the variable is set to a certain
value.

**--unstable Var**  
**--unstable Var=Value**  
Mark a test case as unstable if the given variable is set. This
implies failures to be reported as warnings.The option can be used
multiple times, which means that it suffices that one of the given
`Var`s is set in order to mark the test case as unstable. Typically
`--unstable` is used to test on presence of environment
variables. `--unstable` is intended to be used as `[config
unstable=Var]` or `[config unstable=Var=Value]` statements within
scripts. The construction **Var=Value** is little more restrictive as
it requires that the variable is set to a certain value.

**--unstable\_unless Var**  
**--unstable\_unless Var=Value**  
Mark a test case as unstable if the given variable NOT is set. This
implies failures to be reported as warnings. The option can be used
multiple times, which means that it suffices that one of the given
`Var`s NOT is set in order to mark the test case as unstable.
Typically `--unstable_unless` is used to test on absence of
environment variables. `--unstable_unless` is intended to be used as
`[config unstable_unless=Var]` or `[config unstable_unless=Var=Value]`
statements within scripts. The construction **Var=Val** is little more
restrictive as it requires that the variable is set to a certain
value.

**--skip\_unstable**  
**--skip\_unstable=true**  
Skip unstable test cases. See `--unstable` and `--unstable_unless`.

**--skip\_skip**  
**--skip\_skip=true**  
Forces Lux to not care about `--skip` and `--skip_unless` settings.
Overrides `--skip_unstable`.

**--fail\_when\_warning**  
**--fail\_when\_warning=true**  
Forces Lux to fail if there are any warnings.

Log control
-----------

**--log\_dir LogDir**  
A directory where log files will be written. Default is `./lux_logs`.

**--html Html**  
The `Html` option controls whether the logs should be converted to
HTML or not. It is an enum denoting the outcome of the tests.
If the actual outcome is the same or higher than `Html` then the
logs will be converted. The possible outcome and their relative
values are as follows:

    validate < enable < success < skip < warning < fail < error < disable

Default is `enable`. `validate` behaves as `enable` but will also
perform validation of the generated HTML files. The logs can also be
converted to HTML manually later by using the command line option
`--annotate`.

**--tap LogFile**  
A file where [TAP][TAP] events should be written. The file names
`stdout` and `stderr` are specially handled. They causes the log events
to be written to standard output respective standard error. Multiple
"files" can be given. A log file named lux.tap will always be generated,
regardless of this option.

**-t**  
A shortcut for `--progress=silent --tap=stdout`.

**--junit**  
Generate a JUnit test report for the test run that can be used for example
by Jenkins to show test result using the JUnit plugin. The generated test
report will be named `lux_junit.xml`.

**--case_prefix CasePrefix**  
A prefix string which is prepended to the script names in the user
friendly log files (TAP and HTML). With this the log files can provide
the context for the test case(s), such as subsystem or test suite.

Timeouts
--------

**--multiplier Multiplier**  
In order to be able to run the tests on very slow hardware, there is a
concept of a `Multiplier`. Each time a timer is initiated (except
sleep) its value is multiplied with the `Multiplier` value.
`Multiplier` is an integer and defaults to `1000`, meaning that
`[timeout 10]` actually is 10 seconds. The timeout value 10 is
multiplied with the multiplier value 1000. Perhaps it is easier to
think in milliseconds. 10 means 10*1000=10000 milliseconds. By
changing the multiplier to a higher value, such as 3000, it will cause
Lux to use longer timeouts. E.g. 10*3000=30000 milliseconds.
`--multiplier` is intended to be set in architecture/host specific
files to provide different settings on different systems.

**--timeout Timeout**  
The script expects the shell output to match given
[regular expression][]s. But the output must be received within a
given time limit. The `Timeout` specifies how long it will wait before
the script fails. The `Timeout` defaults to `10000` milliseconds
(`10` seconds). This `Timeout` can be overridden by the statement
`[timeout Timeout]` in the script itself.

**--cleanup\_timeout CleanupTimeout**  
When the script reaches the `[cleanup]` marker, the ordinary
`Timeout` will be set to `CleanupTimeout`. The `CleanupTimeout`
defaults to `100000` milliseconds (`100` seconds).

**--suite\_timeout SuiteTimeout**  
If the duration of the execution exceeds the `SuiteTimeout`, it
is aborted. The `SuiteTimeout` defaults to `infinity`, but can
be any positive integer value in the unit of milliseconds.

**--case\_timeout CaseTimeout**  
If the the duration of a single test case exceeds the
`CaseTimeout`, it is aborted. It can be any positive integer
value in the unit of milliseconds or `infinity`. The default
is `300000` (5 minutes).

**--flush\_timeout FlushTimeout**  
An experimental timeout setting.
All output from a shell is buffered and matched against
[regular expression][]s. It can however explicitly be flushed by
the script. When this is done, the engine first waits a while
before it discards the output. How long it waits is controlled
by `FlushTimeout`. It defaults to `0`. If you want to experiment
with it, `1000` milliseconds (1 second) can be a resonable value.

**--poll\_timeout PollTimeout**  
An experimental timeout setting.
When the Lux engine receives output from a shell it will
wait in `PollTimeout` milliseconds for more output before it
tries to match it against any [regular expression][]s. It defaults
to `0`. If you want to experiment with it, `100` milliseconds
(1/10 second) can be a resonable value.

**--pattern\_mode PatternMode**  
EXPERIMENTAL FEATURE!!! May be changed or removed in a future release.

Changes the default behavior of fail and success patterns. This can
be overridden separately for each shell with the \[pattern\_mode
PatternMode\] command. Valid settings are `all` and `skip`. The
default is `all`.

See the command `\[pattern\_mode PatternMode\]` for details.

**--risky\_threshold RiskyThreshold**  
An experimental timeout setting.
By default Lux warns for risky timers, i.e. timers that are close
to timeout. This may cause intermittent failures in future runs. By
default it is set to `0.85` which means that 85% of the timer was
used.

**--sloppy\_threshold RiskyThreshold**  
An experimental timeout setting.
By default Lux warns for sloppy timers, i.e. timers where only a very
small part of the timer is used. This may cause secondary problems,
e.g. test run abortions by Jenkins or similar systems. Such timers are
also rather inconvenient to debug as they may take hours to time out.
By default it is set to `0.000000001` which means that only 1 ppb
(parts per billion) of the timer was used, or less.

History control
---------------

**--suite Suite**  

The Suite is used for bookkeeping a name which later is used for
printing out the history of test runs. See the
 [command line option](#cmd_line_opts) `--history`.

**--run RunId**  
The `RunId` is used for bookkeeping a name which later is used for
printing out the history of test runs. See the
 [command line option](#cmd_line_opts) `--history`.

**--extend\_run**  
**--extend\_run=true**  
Combines two runs into one. The summary log of an earlier run is
extended with the outcome of the new run. `--log_dir` can be given
explicitly. If not, the symbolic `latest_run` link is used to find
a suitable log directory. Note that the runs cannot be run in parallel
as they will write into the same files. It is much better to use the
`--merge` option to perform the merge of log files when all runs are
done.

**--revision Revision**  
The `Revision` is used for bookkeeping a repository revision
(changeset) which later is used for printing out the history of test
runs. See the [command line option](#cmd_line_opts) `--history`.

**--hostname Hostname**  
The `Hostname`overrides the hostnames extracted from the log files.
It may for example be useful in a test environment where the test
runs are distributed over multiple equivalent slaves. See the
[command line option](#cmd_line_opts) `--history`.

**--html validate**
Performs validation of the generated HTML files.

<a name="debugging"/>

Debugging and tracing
---------------------

**--progress ProgressLevel**  
`ProgressLevel` can be one of `silent`, `summary`, `brief`, `doc`,
`compact` and `verbose`. It defaults to `brief` which means that
single characters are printed to stdout. `doc` is like `brief` but in
this mode doc strings are also printed on stdout. `compact` means that
an event trace is printed on stdout. It is the same event trace that
is written to the `event log`. verbose contains the same info as
compact but is more readable (the newlines are expanded). `summary`
means that no progress is printed. `silent` means that nothing is
printed. The `brief` characters have the following meanings:

       . - a new row in the script is being interpreted
       : - output is being received from a shell
       c - the normal cleanup marker
       C - the cleanup marker during premature termination
       z - is printed out each second while sleeping
       W - is printed out when a dynamic warning is issued
       ( - beginning of a macro, loop or an include file
       ) - end of a macro, loop or an include file
       ? - waiting for shell output. Preceded with lineno.

`[progress String]` can also be used to display progress info.

The `ProgressLevel` can also interactively be changed via the debugger.

**-c**  
A shortcut for `--progress=compact`.

**-v**  
A shortcut for `--progress=verbose`.

**-t**  
A shortcut for `--progress=silent --tap=stdout`.

**--debug**  
The debugger is always available (even without this flag) and waiting
for input on the `stdin` stream. With the `--debug` flag the debugger
is attached to the script before the first line is executed and
waiting for input. The command `attach` (`a` for short) attaches the
debugger to the script and pauses its execution. The command
`progress` (`p` for short) toggles the verbosity level between `brief`
and `verbose`. Use the debugger command `help` to get more info about
the available commands. See also the section [debugging and tracing](#debugging).

**-d**  
A shortcut for `--debug`.

**--debug\_file SavedFile**  
Loads the commands in the `SavedFile` before the first line in the
script is executed. See the debugger command `save` and `load` for
more info. The format of the `SavedFile` is very simple and may be
manually edited. For example `break` and `continue` may be convenient
commands to add to such a file.

Miscellaneous
-------------

**--newshell**  
In `--newshell` mode shells are created with the `[newshell Name]` command
and making another shell active is done with `[shell Name]`. That is the
`[shell Name]` command cannot be used to create new shells in newhell mode.

**--shell\_cmd Cmd**  
**--shell\_args Arg**  
These parameters controls which program that will be started when a
script starts a shell. By default **`/bin/sh -i`** is started as
`--shell_cmd` and `--shell_args` defaults to `/bin/sh` and `-i`
respectively. `--shell_args` is a bit special in how this parameter is
treated by Lux. For example, assume you want to give `-i --rcfile
/another/rcfile` as arguments to the shell. Then you need to give
`--shell_args=-i`, `--shell_args=--rcfile` and
`--shell_args=/another/rcfile` as separate parameters. Each
`--shell_args` is literally treated as one argument. It implies
`--rcfile` to be one argument and `/another/rcfile` another. Further,
the repetition of the `-i` argument is needed as the shell argument
list is reset to scratch once an explicit `--shell_args` is given.

**--shell\_prompt\_cmd PromptCmd**  
**--shell\_prompt\_regexp PromptRegExp**  
When Lux starts a shell the prompt is set to **`SH-PROMPT:`** by
default. In Bourne shell, which is the default shell, the variable
`PS1` is used to set the prompt. This is obtained by using the command
`export PS1=SH-PROMPT:` followed by an explicit match of the prompt
using the regexp `^SH-PROMPT:`. This behavior can be overridden by
using `--shell_prompt_cmd` and `--shell_prompt_regexp` respectively
when using more exotic shells, such as the Bourne Again shell:

     [config shell_cmd=/bin/bash]
     [config shell_prompt_cmd=unset PROMPT_COMMAND; export PS1=SH-PROMPT:]

**--shell\_wrapper**  
**--shell\_wrapper \[Executable\]**  
In order to get the terminal settings to work properly in advanced
interactive cases such as tab completion etc., the shell needs to be
executed in a **pseudo terminal**. This is accomplished by using a
wrapper program setting up the terminal correctly. The arguments to
the wrapper program is the name of the shell program with its
arguments (for example `/bin/sh -i`, see also see `--shell_cmd` and
`--shell_args`). The wrapper is expected to first configure the
terminal and then start the shell.

The built-in executable `lux/priv/runpty` will be used by default as
shell wrapper (if it has been built properly).

It is also possible to use no shell wrapper at all by omitting the
`Executable` value (or simply set it to the empty string "").

     [config shell_wrapper=]

**--shell\_wrapper\_mode WrapperMode**
The WrapperMode is used for debugging the shell wrapper program. By
default it is set to `silent`, but it can also be set to `debug` or
`trace`. Debug will log wrapper internal events, such as signal
handling. Trace will also log the socket payload. When enabled in a
test case, a log file will be created per shell on the extra logs
directory.

     [config shell_wrapper_mode=trace]

**--post\_case**  
**--post\_case \[IncludeFile\]**  
Enable centrally defined cleanup code to be run after each test case
regardless whether they fail or succeed. The purpose of this is to
make it possible to report and possibly undo unwanted side effects
which the cleanup code in the test case has failed to handle.

There may be more than one `post_case` parameter. For each one a shell
will be started when the test case has ended and the Lux code in the
`IncludeFile` will be executed in the context of the test case.

**--line\_term Chars**  
Specify the character sequence added to the end of lines sent to
a shell. It defaults to `\n`.