1. Directory-tree gradle-conform

Similar to the guidelines for arrangement of files in gradle, a directory tree has the following structure:

+-.build      ... build output (maybe redirected to RAM disk)
+-IDE         ... IDEs are separated from sources
|  +-VS15     ... IDE files form Visual Studio Test
|  +-EclCDT   ... some Test projects in Eclipse
|  +-TI_TMS320 ... more IDE projects for other test platforms
|
+-src
  +-main      ... the sources of the module, without test
  |  +-cpp    ... they are C++ and C sources
  |     +-src_emC
  |        +-.git
  |        +-.gitRepository
  |
  +-test    ... all for test
  |  +-cpp    ... they are C++ and C sources
  |  |  +-emC_TestXYZ ... sources for test of special topics
  |  |  +-emC_TestAll
  |  |     +-testmain.cpp  ... test organisation for test of all
  |  |
  |  +-testScripts  ... Some scripts for special tests to start
  |  +-ZmakeGcc ... Test scripts in jzTxtCmd for whole test with gcc
  |     +-ZmakeGcc.jztsh
  |
  +-docs       ... place for documentation
    +-asciidoc ... in Asciidoc

That are all sources, able to commit and compare with git. They are two git repositories present:

  • One for this whole tree, including all test things, but not the sources.

  • A second located in Test_emC/src/main/cpp/src_emC: This is the source repository for the emC, able to include in user applications.

To get this repositories from git hub, use

git clone https://github.com/JzHartmut/Test_emC.git

This gets the Test sources only. But you can start on windows:

build.bat

or on Linux set executable rights and then start:

build.sh

to clone the source tree src_emC, load some more tools, compile and test.

2. Infrastructure on the PC for Test_emC

The Test can be run under Windows or Linux.

Java as version JRE8 should be available. java as command line should be invoke JRE8. If another version of java is installed as standard, the PATH of the system can be changed temporary or all scripts should be invoked with a locally changed PATH environment setting.

3. Test strategies: individual and complete tests, documentation

The test of modules (units) has four aspects:

  • a: The manual step by step test to see what is done in detail, the typical developer test.

  • b: A manual running test while developing for specific situations

  • c: The nightly build test to assure, all is correct. Avoid bugs while improvement.

  • d: Tests document the usage.

The point a: is the most self-evident for the developer, one should use firstly this test aspect by himself.

The point b: is similar a: but without single-step debugging. The results of test are seen as

  • Compiler errors depending on the used compiler and environment settings, an interesting test result. A software may run for example in the IDE’s compiler but fails with an other compiler.

  • Output of test results, see chapter Test check and output in the test files

Both a: and b: works together, can be executed in parallel by selecting the same test environments see chapter Test Selection - arrange test cases

The point c: is the primary for continuous integration.

The point d: is the most important for a user of the sources. One can see how does it works by means of the test (-examples).

3.1. What is tested? C-sources, compiler optimization effects and in target hardware

Firstly the algorithm of the C-sources should be tested. It should be independent of the used compiler and there options. Hence any compiler can be used for test of the sources, for example a Visual Studio compiler, gcc or other.

Secondly, it is possible that an algorithm works proper with the selected compiler, but fails in practice on an embedded hardware. What is faulty? It is possible for example that the target compiler has better optimization, and a property keyword such as volatile is missing in the source. It is a real failure in the source, but it was not detected by the test run with lower optimization. Some other adequate mistakes may be existing in a software.

In conclusion of that, the compiler and its optimization level should be carefully set. The test should be done with more as one compiler and with different optimization levels. For nightly tests the night may be long enough.

The next question is: "Test in the real target hardware". An important answer is: "The test should not only be done in the special hardware environment, the sources should be tested in different environment situations". For example, an algorithm works properly in the special hardware environment because some data are shortened, but the algorithm is really faulty. Ergo, test it in different situations.

But the test in the real target environment, with the target compiler, running inside the real hardware platform may be the last of all tests. It can be done as integration test of course, but the modules can be tested in this manner too.

It means, the test should compile for the target platform, load the result into the target hardware, run there, get error messages for example via a serial output, but run it as module test. Because not all modules may be able to load in one binary (it would be too large), the build and test process should divide the all modules in proper portions and test one after another, or test parallel in more as one hardware board.

3.2. a: Individual Tests in an IDE with step by step debugging

There are some IDE ("Integrated Development Environment") project files:

  • IDE/VS15/AllTest_emC_Base.sln: Visual studio

  • IDE/EclCDT/emC_Test/.cproject: Eclipse CDT, import it in an Eclipse environment, see also ../../../SwEng/html/EclipseWrk.de.html.

  • TODO for embedded platform

Offering special test projects for various topics has not proven successful, because the maintenance of some more projects is a too high effort. Instead, there is exactly one project for any platform (it means at least two, one for Visual Studio and one for Eclipse CDT). To test a special topic there is a main routine which’s calling statements are commented, only the interesting call is used, for single step in debug. This is simple to make.

#ifdef DEF_MAIN_emC_TestAll_testSpecialMain
int main(int nArgs, char const*const* cmdArgs )
{
  STACKTRC_ROOT_ENTRY("main");
  test_Exception();
  test_stdArray();
  //test_Alloc_ObjectJc();
  test_ObjectJc();
  //testString_emC();

This is a snapshot of the current situation. This main routine is used for both IDE.

The include path is IDE- and configuration-specific in the IDE. For both IDEs different path are used for the

#include <applstdef_emC.h>

This file(s) should be changed for several Variants for emC compilation. Of course any commit contains the last used situation, not a developer progress in any case.

The applstdef are located in applstdef_Location_VStudio

D:\vishia\emc\Test_emC\src\test\VS15\All_Test
         1.651 AllTest_emC_Base.sln
<DIR>          applstdef_C1
<DIR>          applstdef_CppObj

It is for Visual Studio. The same set of files, but other files are existing for Eclipse-CDT, see project.

3.3. b: Compile and test for individual situations with gcc or another compiler maybe for the target

This may be seen as preparation for the nightly 'test all' (c) but can also be seen as an intermediate test while development.

For this approach the Test Selection GUI can be used (chapter #testSelection). With the GUI the test cases and test conditions are set. On button [ gen selection ] this gui produces the necessary make..sh files and one file on src/test/cpp/emC_TestAll/fDefSelection.h which contains settings (#define) for compilation. This file is used both by the script controlled test and by the test in the GUI, for example Visual Studio. Both projects may/should include this file to have the same settings. Hence an testing error shown as test result can be explore by testing step by step in the GUI.

3.4. b, c: Generate the scripts for compile and test

In the emC test usual the familiar make approaches are not used. Why?

Standard make files with complex settings are not simple to read, write and understand. Hence a more obvious system named Zmake was established for some years (starting in the 1990th). It uses a [JZtxtcmd]-script to generate shell scripts which invokes the compilation. Such script files are the sources to determine what and how to make.

The output from a Zmake invocation is a shell.sh script which contains the compiler invocation as command line with all obvious options. The script contains the immediately real compiler invocation. It is not a make script which builds the compiler invocation internally using some dependencies, settings etc. The advantage of immediately real compiler invocation is: It is immediately documented what is happen.

To generate this compiler invocation scripts the JZtxtcmd approach is used. This is done by the Test Selection GUI (chapter #testSelection) as also by the specified test files build.sh and inside the src/test/testScripts directory.

All this scripts are short, consisting of two parts:

cd `dirname "$0"`/../../..
pwd
if ! test -e build; then src/buildScripts/-mkLinkBuild.sh; fi

#REM invokes JZtxtcmd as main class of vishiaBase with this file:
java -jar libs/vishiaBase.jar src/test/testScripts/testBasics_Simple.jzTc.sh

##Execute the even yet generated sh scripts, compile and execute:
build/testBasics_Simple.sh
read -n1 -r -p "Press any key to continue..."

exit 0  ##the rest of the file is the JZtxtcmd script

Above is the shell-script part, invoking JZtxtcmd (the main class of the jar) with the script file itself. After them the generated script is executed to compile and confirm the test.

It follows the JZtxtcmd script part:

==JZtxtcmd==

include ../ZmakeGcc/test_Selection.jztsh;

currdir=<:><&scriptdir>/../../..<.>;

##Map ccSet;  ##Settings for compilation

##String ccSet.cc = "clang";



main() {
  call genTestcases(name = "testBasics_Simple", select =
  <:><: >
    1=ObjSiRefl; 2=ReflSi; 3=StrNo; 4=ThSimple; 5=ExcJmp; 6=TestBase
  + 1=ObjCppAdr; 2=ReflFull; 3=StrUse; 4=ThHeapStacktrc; 5=ExcCpp; 6=TestBase
  <.>);  ##Generate all relevant test cases
}

It includes the common generation JZtxtcmd script and invokes in the main() the common genTestCases(…​) routine (from the included script) with given specific arguments. This arguments comes from the Test Selection GUI (chapter #testSelection).

How does the common genTestCases(…​) routine works? It generates texts, selects files from some Fileset etc. See the comments in this script src/test/ZmakeGcc/test_Selection.jztsh.

3.5. a,b: Test Selection - arrange test cases

The problem on testing the core emC sources is the variety of variants (yet 124 combinations) for ObjectJc, Exceptionhandling etc. Writing a lot of scripts, and adjusting the compile switches in applstdef_emC.h is a too high effort. Hence a 'Select Simulation' tool is used, which cames originally from Simulink stimuli selections written by me in the past. It is written in Java and contained in libs/vishiaGui.jar. This tool works with tables.

Select Simulation

The same tables as for the manual operating graphic tool are used to arrange the conditions for the test cases. For example the table for Selecting the kind of reflection generation looks like (File src/test/ZmakeGcc/test_Selection.jztsh):

List tabRefl @name =
[ { name="ObjSiReflNo",       descr="..ObjSiReflNo",       sh="n", def1="DEF_REFLECTION_NO",
, { name="ObjSiReflSi",       descr="..ObjSiReflSi",       sh="i", def1="DEF_REFLECTION_SIMPLE",
, { name="ObjSiReflOffs",     descr="..ObjSiReflOffs",     sh="o", def1="DEF_REFLECTION_OFFS",
, { name="ObjReflNo",         descr="..ObjReflNo",         sh="N", def1="DEF_REFLECTION_NO" }
, { name="ObjReflSi",         descr="..ObjReflSi",         sh="I", def1="DEF_REFLECTION_SIMPLE" }
, { name="ObjReflOffs",       descr="..ObjReflOffs",       sh="O", def1="DEF_REFLECTION_OFFS" }
, { name="ObjReflFull",       descr="..ObjReflFull",       sh="F", def1="DEF_REFLECTION_FULL" }
, { name="ObjCppReflOffs",    descr="..ObjCppReflOffs",    sh="P", def1="DEF_REFLECTION_OFFS",
, { name="ObjCppReflFull",    descr="..ObjCppReflFull",    sh="Q", def1="DEF_REFLECTION_FULL",
, { name="ObjCppAdrReflOffs", descr="..ObjCppAdrReflOffs", sh="R", def1="DEF_REFLECTION_OFFS",
, { name="ObjCppAdrReflFull", descr="..ObjCppAdrReflFull", sh="S", def1="DEF_REFLECTION_FULL",
];

It is a data list in ../../../JZtxtcmd/html/JZtxtcmd.html. You see the magic character in the list and in the 'Select Simulation'. The table contains immediately the necessary compiler switches for each of the four test variants.

With the adequate information about the selected lines the sub routine

sub genSelection(Map line1, Map line2, ..., String testoutpath) { ...

is invoked. It gets the selected line in each table. line1 is from the table above. With the information in the line the compiler switches in the test script can be arragenged in a simple way. The texts are contained in the line.

For execution of some test cases the same information is used.

sub genTestcases(String select, String name) { ...

contains in the select String the information, which lines are used, for example in the snippet of src/test/testScripts/testBasics_Simple.jzTc.sh:

main() {
  ##Generate all relevant test cases
  call genTestcases(name = "testBasics_Simple", select =
  <:><: >
    1=ObjSiReflNo; 2=StrNo; 3=CppAll; 4=ThSimple; 5=ExcJmp; 6=TestBase
  + 1=ObjCppAdrReflFull; 2=StrUse; 3=CppAll; 4=ThHeapStacktrc; 5=ExcCpp; 6=TestBase
  <.>);  ##Generate all relevant test cases
}

The syntax of the select String is described in ../../../smlk/html/SmlkTimeSignals/SmlkTimeSignals.html#truegenerating-manual-planned-test-cases which uses the same tool for another approach. The combination is done with the called Java routine (part of vishiaBase.jar). See ../../../Java/docuSrcJava_vishiaBase/org/vishia/testutil/TestConditionCombi.html.

3.6. Arranging the necessary compile units

The 6. table in the 'Select Simulation' contains, which is to test. (The other tables contains, 'under which condition is to test'). It looks like (shortend):

List tabTestSrc =
[ { name="TestBase",  srcSet="srcTestBasics", def1="DEF_TESTBasics_emC"}
, { name="TestEvMsg", srcSet="srcTestEvMsg",  def1="DEF_TESTALL_emC" }
];

The srcSet is the name of a file set, defined in the script src/test/ZmakeGcc/filesets.jzTc. It determines which files should be used, whereby a reference to further filesets are contained too:

##
## main file for Basic tests.
##
Fileset srcTestBasics =
( src/test/cpp:emC_TestAll/testBasics.cpp
, src/test/cpp:emC_TestAll/test_exitError.c
, &srcTest_ObjectJc
, &srcTest_Exception
, &src_Base_emC_NumericSimple
);

A Fileset is a core capability from ../../../JZtxtcmd/html/JZtxtcmd.html. It names some files and sub Filesets.

In a Fileset some files are named for some application goals. This information can be used to select which emC files are need as part of a maybe simple application:

Fileset src_Base_emC_NumericSimple =
( src/main/cpp/src_emC:emC_srcApplSpec/SimpleNumCNoExc/fw_ThreadContextSimpleIntr.c
, src/main/cpp/src_emC:emC_srcApplSpec/SimpleNumCNoExc/ThreadContextSingle_emC.c
, src/main/cpp/src_emC:emC_srcApplSpec/applConv/LogException_emC.c
);

The fileset for the core files:

##
##The real core sources for simple applications only used ObjectJc.
##See sub build_dbgC1(), only the OSAL should be still added.
##
Fileset c_src_emC_core =
( src/main/cpp/src_emC:emC/Base/Assert_emC.c
, src/main/cpp/src_emC:emC/Base/MemC_emC.c
, src/main/cpp/src_emC:emC/Base/StringBase_emC.c
, src/main/cpp/src_emC:emC/Base/ObjectSimple_emC.c
, src/main/cpp/src_emC:emC/Base/ObjectRefl_emC.c
, src/main/cpp/src_emC:emC/Base/ObjectJcpp_emC.cpp
, src/main/cpp/src_emC:emC/Base/Exception_emC.c
, src/main/cpp/src_emC:emC/Base/ExceptionCpp_emC.cpp
, src/main/cpp/src_emC:emC/Base/ExcThreadCxt_emC.c
, src/main/cpp/src_emC:emC/Base/ReflectionBaseTypes_emC.c
, src/main/cpp/src_emC:emC_srcApplSpec/applConv/ExceptionPrintStacktrace_emC.c
##Note: Only for test evaluation
, src/main/cpp/src_emC:emC/Test/testAssert_C.c
, src/main/cpp/src_emC:emC/Test/testAssert.cpp
, src/test/cpp:emC_TestAll/outTestConditions.c
, &src_OSALgcc
, src/main/cpp/src_emC:emC_srcApplSpec/applConv/ObjectJc_allocStartup_emC.c
);

are also the core sources for test. Maybe not all, but from this selection may be necessary to use as core sources for an application, which uses emC. It documents the necessities and indirectly also the dependencies.

3.7. Distinction of several variants of compilation

The distinction between C and C++ compilation can be done using either gcc for *.c-Files or g++ which always compiles as C++. This is the content of the special build_…​ routine. Some more build_…​ routines are existing for different used files and for decision between C and C++ compilation.

The distinction between conditional compilation (variants, see ../Base/Variants_emC.html are done with the different content of the cc_def variable. It contains '-D …​' arguments for the compilation. The other variant may be selecting different <applstdef_emC.h> files which is recommended for user applications. Then the include path should be varied. It needs some applstdef_emC.h files. This can be done too, the part of the include path to <applstdef_emC.h> is contained in the cc_def variable.

3.8. Check dependency and rebuild capability

A file should be compiled:

  • If the object file does not exist

  • If the source file is newer than the object file (or more exactly: The content of the source file was changed in comparison to the content of the last compilation).

  • If any of the included source files (e.g. header) is newer than the object file (respectively changed after last using).

The first two conditions are checked only with the 'is newer' aspect from an ordinary make file. For the third condition (indirect newly) the dependencies between the files should be known. For a classic make files this dependencies can be given - if they are known. In practice the dependencies depends on the include situation, it is not simple. Hence the real dependencies can only detect for a concretely version of the file, and the make script should be corrected any time. IDEs use their native dependency check usual proper.

Because this cannot be done easily, often there is a 'build all' mentality.

For repeated compilation the 'build all' mentality needs to much time.

For this approach a Java package org.vishia.checkDeps_C is used. See

This tool uses a comprehensive file deps.txt which contains the dependency situation of each file and the timestamp and content situation (via CRC checksum). The tool checks the time stamp and the content of all depending files from the list. If one file is changed, it is parsed by content, find out include statements and build newly the dependencies from this level. Ones of course the object should be recompiled, because another content may be changed. Secondly the dependencies for the test later are corrected..

Because the dependency file contains the time stamp of any source file, it is detected whether an older file is given. The comparison of time stamps is not the comparison between source and object, it is the comparison between the last used source and the current source time stamp. The newly compilation is done also if the file is older, not only newer than the object file. This is an expectable situation, if a file is changed by checkout from a file repositiory with its originally time stamp (the older one). Because git and some other Unix/linux tools stores an older file with the current timestamp this problem is not present on linux, but Windows restores or preserves the time stamp of a copied file, which may be the better and here supported approach.

If the dependency file is not existing, it means, the dependencies should be detected, build all is necessary and the dependency file is built. This is the situation on first invocation after clean.

The dependency file is stored inside the object directory:

...\build\objZmake\test_ObjRefl_ReflFull_ThSi_ExcNo_StrNo_TestEvMsg
<DIR>          emC
<DIR>          emC_Exmpl_Ctrl
<DIR>          emC_srcApplSpec
<DIR>          emC_srcOSALspec
<DIR>          emC_TestAll
<DIR>          emC_Test_Container
<DIR>          emC_Test_Ctrl
<DIR>          emC_Test_C_Cpp
<DIR>          emC_Test_ObjectJc
<DIR>          emC_Test_Stacktrc_Exc
       362.272 deps.txt                 <<=======
         8.330 checkDeps.out
       295.817 emCBase_.test.exe
           296 fDefSelection.h
             0 ld_out.txt

It is a snapshot from the root of the object dir tree. The deps.txt has about 260 kByte, it is not too long. The Java algorithm to check the dependencies of all files reading this file needs only milliseconds, because like known, Java is very fast. It runs of course also in Linux.

You can view this file to explore the individual dependencies of each file, which may be informative.

The dependency check is part of each make..sh shell script (generated):

...\build\objZmake
         2.965 deps_test_ObjRefl_ReflFull_ThSi_ExcNo_StrNo_TestEvMsg.args
        72.677 make_test_ObjRefl_ReflFull_ThSi_ExcNo_StrNo_TestEvMsg.sh
<DIR>          test_ObjRefl_ReflFull_ThSi_ExcNo_StrNo_TestEvMsg
....
echo run checkDeps, see output in build/...testCase/checkDeps.out
java -cp libs/vishiaBase.jar org.vishia.checkDeps_C.CheckDeps ...
  ... --@build/objZmake/deps_test_ObjRefl_ReflFull_ThSi_ExcNo_StrNo_TestEvMsg.args ...
  ... > build/objZmake/test_ObjRefl_ReflFull_ThSi_ExcNo_StrNo_TestEvMsg/checkDeps.out

( The java invocation is a long line).

The check of the unchanged situation does only need reading the time stamps of all depending files, it is very fast because the file system is usual cached. If dependencies should be evaluate newly all source files are parsed. Of course already parsed included files are not proceeded twice. The parsing, and checking for # include statement, does only need a short time because Java is fast. The gcc compiler itself supports a dependency check too, but that is very slower (not because C++ is slow, but because it may be more complex). The checkDeps dependency check is more simple, for example it does not regard conditional compilation (a conditional include). It means, it detects a dependency to a included file which is not active in the compiling situation. But that is not a disadvantage, because the dependency can be exist, and the unnecessary compilation because of one conditional include does not need more time than the elaborately dependency check.

If the object file should be recompiled, the checkDeps algorithm deletes it and forces a recompilation because existence check of the object file before compilation. It is a simple liaison between this independent tools.

4. Testing of sources on a PC platform with evaluable text output

The test has gotten some ideas from the Google test framework (see https://chromium.googlesource.com/external/github.com/pwnall/googletest/+/refs/tags/release-1.8.0/googletest/docs/Primer.md) but is more easier.

The primary goal is, control a proper test output to a file while the program is running. This needs organization.

All (very less) sources are contained in emC/Test/*, especially emC/Test/testAssert.h.

For test any operation (C-function) can be used. It is not necessary to mark it as "TEST" operation. Usual you have a main() and in the main() you invoke some operations one after another, which executes some tests.

4.1. Frame for test

Inside an operation the begin and the end of the test is marked with:

int anyOperation(int mayhaveArgs) {
  STACKTRC_ENTRY("anyOperation");
  TEST_TRY("Description of the test case") {
    //
    //... any stuff to test
    //
  }_TEST_TRY_END
  STACKTRC_RETURN;
}

The macros STACKTRC_…​ are necessary for exception messaging with stacktrace-report, see ../Base/ThCxtExc_emC.html.

The TEST_TRY(…​) macro declares a bTESTok variable and notice the start. For PC usage this macro is defined as

//in source: src/main/cpp/src_emC/emC/Test/testAssert.h
#define TEST_TRY(MSG) bool bTESTok = true; \
  TRY { msgStartFileLine_testAssert_emC(MSG, __FILE__, __LINE__);

The compiler intrinsic FILE and LINE writes the file and line information in the test output, important for tracking of test results in the sources.

The _TEST_TRY_END checks the bTESTok variable and writes an proper message or stores any data information for the test result. It contains also the exception handling for unexpected exceptions while test. For PC usage this macro is defined as

//in source: src/main/cpp/src_emC/emC/Test/testAssert.h
#define _TEST_TRY_END  } _TRY  CATCH(Exception, exc) { \
    bTESTok = false; \
    exceptionFileLine_testAssert_emC(exc, __FILE__, __LINE__); \
  } END_TRY  msgEndFileLine_testAssert_emC(bTESTok); 

The same can done also manually, with more writing overhead, but maybe better obviously what’s happen:

int anyOperation(int mayhaveArgs) {
  STACKTRC_ENTRY("anyOperation");
  TEST_START("test_ctor_ObjectJc");
  TRY {
    //
    //... any stuff to test
    //
  }_TRY;
  CATCH(Exception, exc) {
    TEST_EXC(exc);
  }
  END_TRY;
  TEST_END;
  STACKTRC_RETURN;
}

The adequate macros are defined as:

//in source: src/main/cpp/src_emC/emC/Test/testAssert.h
#define TEST_START(MSG) bool bTESTok = true; \
  msgStartFileLine_testAssert_emC(MSG, __FILE__, __LINE__)

#define TEST_EXC(EXC) bTESTok = false; \
  exceptionFileLine_testAssert_emC(EXC, __FILE__, __LINE__)

#define TEST_END   msgEndFileLine_testAssert_emC(bTESTok); 

That is all for the test frame.

Of course selecting some test cases with a specific tool (gradle tests or such) is not possible. But there is a normal programming for test cases. Selection of specific test cases can be done with ordinary conditional compiling. It doesn’t need special knowledge.

4.2. Exceptions on test

For exceptions see ../Base/ThCxtExc_emC.html.

The test should be done under Exception control. On PC platform this should not be an problem. Either the C++ try-catch can be used, or the other possibilities. But it is the same as elsewhere setted in the <applstdef_emC.h>. Hence: If complicated algorithm are tested, the exception handling should be activated. Only on test of the exception handling itself, it would be set to DEF_NO_Exception_emC or DEF_NO_THCXT_STACKTRC_EXC_emC which disables the exceptions.

This TRY-CATCH frame does only except if an exception is not catched inside the tested algorithm. For a catched exception it is part of the algorithm itself.

If an unexpected exception occurs during the test, then the msgEndFileLine_testAssert_emC(…​) routine is not reached, instead the exceptionFileLine_testAssert_emC(…​) routine is invoked. It writes the exception with file and line and if possible (depending on setting of DEF_ThreadContext_STACKTRC) it writes the stack trace to evaluate where and why the exception was occurring.

4.3. Execute tests

The tests itself checks and logs results. For example the test in emc_Test_ObjectJc/test_ObjectJc.c:

bool bOk = CHECKstrict_ObjectJc(&data->base.obj
           , sizeof(myData1const), refl_MyType_Test_ObjectJc, 0);
CHECK_TRUE(bOk, "checkStrict_ObjectJc with refl and without instance id.")

Any boolean test result condition is built. Then the CHECK_TRUE with the test result is invoked, with a text which describes the positive expected result. The CHECK_TRUE produces only an output if the check is false:

ERROR: checkStrict_ObjectJc with refl and without instance id.
                          (@38: emC_Test_ObjectJc\test_ObjectJc.c)

in one line. Instead using TEST_TRUE outputs also the positive test:

ok: checkStrict_ObjectJc with refl and without instance id.

This documents which tests are done. The output is valid for PC test or for test on an embedded platform with text output capability.

The test can also output values, in a printf manner. For example in emC_Test_Math/Test_Math_emC.c the results of mathematics are checked:

TEST_TRUE(rs == (int32)values[ix].rs
         , "muls16_emC %4.4X * %4.4X => %8.8X"
         , (int)(as) & 0xffff, (int)bs & 0xffff, rs);

It outputs for example:

ok: muls16_emC 4000 * FFFF => FFFFC000
....

and informs about numeric test result, here for a fix point multiplication. The test is important because the fix point multiplication can fail on an embedded platform if register widths etc. are other as expected. Maybe using CHECK_TRUE may be more proper, information only on faulty results. The calculated faulty value can be used to detect the reason of the fault.

Furthermore the text can be prepared and reused for more test outputs. See the example testing a smoothing control block (T1) in test1_T1_Ctrl_emC() (src/test/cpp/emC_Test_Ctrl/test_T1i_Ctrl.c) the output of the control block for a step response is compared with the known mathematic result. The maximum of the abbreviation (error) is built in a variable dysh1_eMax. A small abbreviation is expectable because the T1 FBlock has only integer accuracy with a defined step width. It means in comparison to the exact mathematic behaviour not in the step time a small difference is expected and admissible. Hence the test is written as

CHECK_TRUE(dysh1_eMax < thiz->abbrTsh1, checktext[0]);

The abbreviation is compared with a given value. Because some other tests are done too, the text is stored in an array which is used for more tests. Example:

char checktext[8][100];
snprintf(checktext[0], 100, "%1.3f < abbreviation of Tsh1 4 Bit fTs", thiz->abbrTsh1);
snprintf(checktext[1], 100, "%1.3f < abbreviation of Tsh2 8 Bit fTs", thiz->abbrTsh2);
....

You see, the functionality what and which to test is complex due to the technical approach. But the building of the test result is very simple. The CHECK_TRUE(COND, TEXT,…​) macro expects only a false or true as first argument, TEXT as second arguemnt which describes the expected behavior, and maybe some more variable arguments.

This test output macros are defined as:

//in source: src/main/cpp/src_emC/emC/Test/testAssert.h
/**Test, output ok MSG if ok, only on error with file and line. */
#define TEST_TRUE(COND, MSG, ...) \
  if(!expectMsgFileLine_testAssert_emC(COND, MSG, __FILE__, __LINE__, ##__VA_ARGS__)) bTESTok = false;
/**Checks only, output only if error*/
#define CHECK_TRUE(COND, MSG, ...) \
  if(!(COND)) { expectMsgFileLine_testAssert_emC(false, MSG, __FILE__, __LINE__, ##__VA_ARGS__); bTESTok = false; }

The difference between TEST…​ and CHECK…​ is only the conditional call of the inner operation expectMsgFileLine_testAssert_emC on CHECK…​, hence nothing is outputted on true.

The known C macro `VA_ARGS`is used to forward the variable arguments to the macro expansion. The operation for test output itself is defined as:

//in source: src/main/cpp/src_emC/emC/Test/testAssert_C.c
bool expectMsgFileLine_testAssert_emC ( bool cond, char const* msg, char const* file, int line, ...) {
  char text[200];
  va_list args;
  va_start(args, line);
  vsnprintf(text, sizeof(text), msg, args);  //prepare the positive text maybe with variable args
  va_end(args);
  if(cond) { printf("  ok: %s\n", text ); }
  else {
    char const* filename = dirFile(file);
    printf("  ERROR: %s (@%d: %s)\n", text,  line, filename);
  }
  return cond;
}

As you see, not too complicated, using the known variable arguments in C programming together with the vsnprintf(…​) as save variant of the traditional known sprintf(…​).

4.4. The test output

For test on PC or test on a text supporting embedded platform a text output is created.

With distinguish TEST_TRUE and CHECK_TRUE one can add more information about executed tests. A test output with some executed tests looks like:

Test: test_ctor_ObjectJc: (emC_Test_ObjectJc/test_ObjectJc.c @ 89) ...
  ok: refl type is ok
  ok: INIZ_VAL_MyType_Test_ObjectJc
  ok: checkStrict_ObjectJc
ok

You see the test case starts with Test: …​ left aligned, and ok after the test is also written left aligned as the finish line. Between them some messages ` ok: …​` which documents which tests are executed, or ` ERROR: …​` if a test fails:

Test: test_ctor_ObjectJc: (emC_Test_ObjectJc\test_ObjectJc.c @ 89) ...
  ok: refl type is ok
  ERROR: INIZ_VAL_MyType_Test_ObjectJc (@99: emC_Test_ObjectJc\test_ObjectJc.c)
  ok: checkStrict_ObjectJc
ERROR

The simple form looks like:

Test: test_cos16: (emC_Test_Math\Test_Math_emC.c @ 228) ...
ok

If during processing the test algorithm an exception is thrown, then (using the TEST_TRY(…​) macros) this test is aborted with an error message:

Test: test_ObjectJcpp_Base: (emC_Test_ObjectJc\test_ObjectJcpp.cpp @ 109) ...
EXCEPTION 1 (24, 0) @21: ..\..\src\test\cpp\emC_Test_ObjectJc\test_ObjectJcpp.cpp

RuntimeException: : 24=0x00000018
  at THROW (..\..\src\test\cpp\emC_Test_ObjectJc\test_ObjectJcpp.cpp:21)
  at test_ObjectJcpp_Base (..\..\src\test\cpp\emC_Test_ObjectJc\test_ObjectJcpp.cpp:109)
  at test_ObjectJcpp (..\..\src\test\cpp\emC_Test_ObjectJc\test_ObjectJcpp.cpp:214)
  at main (..\..\src\test\cpp\emC_TestAll\testBasics.cpp:21)
ERROR

4.5. Unexpected exceptions while test

This exception in the chapter above had occurred because the macro for INIZsuper_ClassJc(…​) was faulty, not all elements are initialized. The type is tested inside a C++ constructor outside of the test itself, and that causes the exception. For that case the test is finished with the EXCEPTION …​ line, the other tests of this block are not executed. That is fatal.

In addition, the stack trace is output. With that the source of the exception was able to found without elaborately debug tests: In line 21 an ASSERT_emC(…​) was found, which checks the base type. Setting a breakpoint there (Visual Studio) shows, the information about the ClassJc…​superClass was missing, which was caused by the faulty macro for the initialization.

It is also possible to write `TRY { …​. }_TRY CATCH { …​. } ` statements inside the test, to catch an exception in the test algorithm. Then the CATCH block should contain:

  CATCH(Exception, exc) {
    TEST_EXC(exc);
  }

This logs the not expected exception for the test output. But if the exception behaviour is tested itself as test case, it can be written (see emC_Test_Stacktrc_Exc/TestException.cpp):

  TRY{
    //raise(SIGSEGV);
    CALLINE; float val = testThrow(thiz, 10, 2.0f, _thCxt);
    printf("val=%f\n", val);
  }_TRY
  CATCH(Exception, exc) {
    CHECK_TRUE(exc->line == 46, "faulty line for THROW");
    bHasCatched = true;
  }
  FINALLY {
    bHasFinally = true;
  } END_TRY;
  TEST_TRUE(bHasCatched && bHasFinally, "simple THROW is catched. ");

In the CATCH clause a test can also confirm, but to check whether the CATCH and also here FINALLY has entered, a boolean variable is set and test after the TRY block.

The evaluation can be done with a not sophisticated text analyse, see ../TestOrg/testStrategie_en.html#trueview-of-test-results

5. Evaluation of textual test outputs

Each test for emC has two stages of test execution:

  • Compilation and linking, successfully or not, errors and warnings

  • if the first one is succeeded, the textual result of the tests.

It means in this case three files where outputted:

  • testcase.cc_err with the compiler and linker error messages and warnings

  • testcase.out with the test results while running.

The testcase.out is produces as empty file (1 space) if the test does not run. Additonally

  • testcase.err with commonly error messages and the #define …​ of the test cases

is produces.

5.1. Manual view of test results

Now you can visit in this folder to all this files (sorted by extension). If the file length is 0, then the compilation is ok and without warnings.

Checking the test results is more complicated. You should search the word ERROR on start of line. Inside the text a message can contain ERROR, may be. That is costly.

5.2. Evaluation of all files

If it is expected that most of tests are correct, you can find out the less number of faulty tests and look what’s happen. But often a systematically error is given, and a raw of tests failes because of the same error. Not an overview which tests are failed, and thinking about equal conditions is necessary.

Thats why a test evaluation, which only counts the number of bad tests, is not sufficient.

The script file

src/test/testScripts/evalTests.jztsh

which calls an operation of the Java class: org.vishia.testutil.EvalTestoutfiles

checks all cc_err file and the out file in a given directory (build/result folder).

In the script it is iterated through all tables of the test cases (SimSelector), the relevant file names from the test case name are gathered. The file names are build from the tables, the name entry. The Java operation then checks the content of the files and build only one character per test case.

With this character and an overview description which cases, an output file is written as simple text file in a matrix, one character per file. With them, for example 72 * 36 = 2592 results are able to present in a few (39 with header) lines. Using a 'well presented' html output file with red, yellow and green lights is possible for that tool also, of course. But the simple text file is more compact for a fast overview.

  • An non existing file, not planned test, is a space, nothing visible.

  • E: If no out file was found but the cc_err file, there are compiler errors.

  • X: If the output file contains more as 2 errors, it means the test isproblematic

  • x: If the output file contains 1 or 2 errors, somewhat is faulty.

  • v: If there are test cases without "Ok" but also without errors, the test organization faults.

  • W: as X but also Warnings are in the cc_err file

  • w: as x but also Warnings are in the cc_err file

  • u: as v but also Warnings are in the cc_err file

  • F: internal Failure in evaluation

  • a..j: Test is ok as 0 .. 9 but there are warnings

  • 0..9: Test ok. The digit is (number of test cases + 9)/10, it means 1 for 1..10 test cases per test, 3 for 21..30 test cases per test, 9 for more as 81 test cases per test. With this number an estimation can be done how comprehensive is the test.

It means the successfully test should be marked with 0..9. Spaces documents not tested variants.

It looks like (shorten):

------------------------------------SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS  -:StrNo  S:StrUse
iiiiiiuuuuuussssssIIIIIIUUUUUUSSSSSSiiiiiiuuuuuussssssIIIIIIUUUUUUSSSSSS  i:ThSimple  u:ThStackUs
nnjjppnnjjppnnjjppnnjjppnnjjppnnjjppnnjjppnnjjppnnjjppnnjjppnnjjppnnjjpp  n:ExcNo  j:ExcJmp  p:Ex
CpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCpCp  C:C_Cpp  p:CppAll
 1 1 1 1 1 1 1 1 1             1 1 1 1 1 1 1 1 1 1 1 1             1 1 1 - ObjSiReflNo -  - TestB
 1 1 1 1 1 1 1 1 1             1 1 1 1 1 1 1 1 1 1 1 1             1 1 1 - ObjSiReflSi -  - TestB
 1 1 1 1 1 1 1 1 1             1 1 1 1 1 1 1 1 1 1 1 1             1 1 1 - ObjSiReflOffs -  - Tes
 1 1 1 1 1 1 1 1 1             1 1 1 1 1 1 1 1 1 1 1 1             1 1 1 - ObjReflNo -  - TestBas
 1 1 1 1 1 1 1 1 1             1 1 1 1 1 1 1 1 1 1 1 1             1 1 1 - ObjReflSi -  - TestBas
 1 1 1 1 1 1 1 1 1             1 1 1 1 1 1 1 1 1 1 1 1             1 1 1 - ObjReflOffs -  - TestB
                                                                         - ObjReflFull -  - TestB
 1 1 1 1 1 1 1 1 1             1 1 1 1 1 1 1 1 1 1 1 1             1 1 1 - ObjCppReflOffs -  - Te
                                     1 1 1 1 1 1 1 1 1             1 1 1 - ObjCppReflFull -  - Te
 1 1 1 1 1 1 1 1 1             1 1 1 1 1 1 1 1 1 1 1 1             1 1 1 - ObjCppAdrReflOffs -  -
                                     1 1 1 1 1 1 1 1 1             1 1 1 - ObjCppAdrReflFull -  -
 3 3 3 3 3 3 3 3 3             3 3 3 3 3 3 3 3 3 3 3 3             3 3 3 - ObjSiReflNo -  - TestE
 3 3 3 3 3 3 3 3 3             3 3 3 3 3 3 3 3 3 3 3 3             3 3 3 - ObjSiReflSi -  - TestE
 3 3 3 3 3 3 3 3 3             3 3 3 3 3 3 3 3 3 3 3 3             3 3 3 - ObjSiReflOffs -  - Tes
 3 3 3 3 3 3 3 3 3             3 3 3 3 3 3 3 3 3 3 3 3             3 3 3 - ObjReflNo -  - TestEvM
 3 3 3 3 3 3 3 3 3             3 3 3 3 3 3 3 3 3 3 3 3             3 3 3 - ObjReflSi -  - TestEvM
 3 3 3 3 3 3 3 3 3             3 3 3 3 3 3 3 3 3 3 3 3             3 3 3 - ObjReflOffs -  - TestE
                                                                         - ObjReflFull -  - TestE
 3 3 3 3 3 3 3 3 3             3 3 3 3 3 3 3 3 3 3 3 3             3 3 3 - ObjCppReflOffs -  - Te
                                     3 3 3 3 3 3 3 3 3             3 3 3 - ObjCppReflFull -  - Te
 3 3 3 3 3 3 3 3 3             3 3 3 3 3 3 3 3 3 3 3 3             3 3 3 - ObjCppAdrReflOffs -  -
                                     3 3 3 3 3 3 3 3 3             3 3 3 - ObjCppAdrReflFull -  -
                                                                         - ObjSiReflNo -  - TestS
                                                                         - ObjSiReflSi -  - TestS

In this example the C compilation is not done, all files are compiled with C++. Hence the column for c is empty. The tests with DEF_ThreadContext_HEAP_emC but not with DEF_ThreadContext_STACKTRC are also missed, may be unnecessary because it was tested without DEF_ThreadContext_STACKTRC. All tests with DEF_REFLECTION_FULL without String capability are not meaningfull. That are the spaces here.

5.3. Selection in the evaluation file, open and reconstruct faults

Now another tool can be used, also controlled by the JZtxtcmd file:

src/test/testScripts/evalSelector.jztsh

EvalSelector

As you see you can mark lines to see which test case on right side. But the test case which is after the cursor (current position) is shown in the line below. Additionally you can press one of the buttons. Then the proper file is opened with a text editor. Because which text editor should be able to select, here a batch file ed++.bat with the proper file is opened:

sub openOut(Obj widget) {
  Map testcase;
  testcase = call showTestcase();
  <+out><&testcase.testcase><.+n>
  cmd cmd.exe /C ed++.bat <:>build\result\test_<&testcase.testcase>.out<.> ;
  <+out><&testcase.testcase><.+n>
  onerror {
    <+out><&error><.+n>
  }
}

This is the so named event handler of the button. But it is called not in the graphical thread, it is called in a JZtxtcmd special thread to prevent blocking the GUI on possible errors. The cmd.exe call can be adapted, it is here for MS-Windows and the mentioned text editor call.

The advantage of the JZtxtcmd scripting is, it can be changed simple in the text file. No further tooling is necessary for adaption. The fast work is done in Java, which can be immediately called here. But the Java code is independent of specific settings. Of course it can be also adapted.

6. How does it works, genTestCases(…​)

The test file for the simple basic test is (src/test/testScripts/testBasics_Simple.jzTc.sh, 2021-04-25):

#REM: should be invoked anytime from the root of the Working tree, change to it:
cd `dirname "$0"`/../../..
pwd
if ! test -e build; then src/buildScripts/-mkLinkBuild.sh; fi

#REM invokes JZtxtcmd as main class of vishiaBase with this file:
java -jar tools/vishiaBase.jar src/test/testScripts/testBasics_Simple.jzTc.sh

##Execute the even yet generated sh scripts, compile and execute:
build/testBasics_Simple.sh
read -n1 -r -p "Press any key to continue..."

exit 0  ##the rest of the file is the JZtxtcmd script

==JZtxtcmd==

include ../ZmakeGcc/test_Selection.jztsh;
currdir=<:><&scriptdir>/../../..<.>;

main() {
  ##Generate all relevant test cases
  call genTestcases(name = "testBasics_Simple", select =
  <:><: >
    1=ObjSiReflNo; 2=StrNo; 3=CppAll; 4=ThSimple; 5=ExcJmp; 6=TestBase
  + 1=ObjCppAdrReflFull; 2=StrUse; 3=CppAll; 4=ThHeapStacktrc; 5=ExcCpp; 6=TestBase
  <.>);  ##Generate all relevant test cases
}

The genTestCases in the src/test/ZmakeGcc/test_Selection.jztsh looks like (only shorten because of pagewidth)

##
##
##This routine will be called from inside the Java programm org.vishia.simSelector.SimSelector
##  on the button gen testcases. It generates all selected test cases.
##
sub genTestcases ( String select, String name = "testSelectionGUI"
    , Obj ccSet=ccSetDeflt      ##Variables which determines the compilation, setting

    ){
  <+out><&scriptdir>/<&scriptfile>: genTestcases( name = <&name>, select=
  <&select><.+n>
  Num ixcase = 1;
  Openfile fAllsh = <:>build/<&name>.sh<.>;                         ##build/testCase.sh for all t
  ##Openfile fcsv = <:><&dirSimulink>/<&fileTestCases_m>.csv<.>;
  <+fAllsh><:>
===========echo off
===========echo all output > build/result/all.out
===========if test -e ../build; then cd ..; fi
===========if test -f build/result/<&name>.out; then rm build/result/<&name>.out; fi
===========if test -f build/result/gcc_nocc.txt; then rm build/result/gcc_nocc.txt; fi
===========echo "==== new test select=<&select> ====" > build/result/<&name>.out
===========##date >> build/result/<&name>.out ##hint: the date disturbs comparability!
===========echo "==================================" >> build/result/<&name>.out
===========#All test cases
===========<.><.+>

  Openfile fAllBat = <:>build/<&name>.bat<.>;                         ##build/testCase.bat for al
  <+fAllBat><:>
============echo all output > build/result/all.out
============::if exist build cd build
============if exist build\result\<&name>.out del /S/Q build\result\<&name>.out
============echo "==== new test select=<&select> ====" >build\result\<&name>.out
============::date >> <&name>.out
============echo "==================================" >>build\result\<&name>.out
============::All test cases
============<.><.+>

  Obj testcases = java org.vishia.testutil.TestConditionCombi.prepareTestCases(select, 6);
  for(testcase: testcases) { //The order in the testcase is always the order in the tabs.
    Obj lineRefl =    tabRefl.get(   testcase[0].sel);
    Obj lineStr =     tabStr.get(    testcase[1].sel);
    Obj lineCpp =     tabCpp.get(    testcase[2].sel);
    Obj lineThCxt =   tabThCxt.get(  testcase[3].sel);
    Obj lineExc =     tabExc.get(    testcase[4].sel);
    Obj lineTestSrc = tabTestSrc.get(testcase[5].sel);

    <+out>Select: <&lineRefl.name> <&lineStr.name> <&lineCpp.name> <&lineThCxt.name> <&lineExc.na
    call genSelection(line1=lineRefl, line2=lineStr, line3=lineCpp, line4=lineThCxt, line5=lineEx
                      , fAllsh = fAllsh, fAllBat = fAllBat, ccSet=ccSet, testoutpath = <:>build/r
    ixcase = ixcase + 1;

  } ##for testcases
  ##<+fcsv>"Name", "Description", "todo",<.+n>
  ##<+fAllsh>read -n1 -r -p "Press any key to continue..."<.+n>
  fAllsh.close();
  <+fAllBat>pause<.+n>
  fAllBat.close();
  Obj fileAllsh = new java.io.File(<:>build/<&name>.sh<.>);
  fileAllsh.setExecutable(true);   ##for linux, chmod to executable
  ##fcsv.close();
}

In the text parts visible with =========== from line start there are some commands written in the determined files. After that part the input select string is evaluated in the class ../../../Java/docuSrcJava_vishiaBase/org/vishia/testutil/TestConditionCombi.html. It returns a list of all relevant testcases (expanded).

With this list (for(testcase: testcases)) the line information from the tabs are gotten. This lines are arguments for ,,genSelection(…​),, which is (shorten, see original file)

##
##This operation is kind of common but adapted to the test cases.
##It is called here from execSelection button and from genTestcases
##
sub genSelection ( Map line1, Map line2, Map line3, Map line4, Map line5, Map line6
    , Obj ccSet      ##Variables which determines the compilation, setting
    , Obj fAllsh
    , Obj fAllBat
    , String testoutpath
    ){

  List defines;
  defines += line1.def1;
  if(line1.def2) {
    defines += line1.def2;
  }
  ....
  defines += line6.def1;

  ##testCase is the name of the script file to compile.
  String testCase = <:>test_<&line1.name>_<&line2.name>_<&line3.name>_<&line4.name>_<&li ....
  <+out>
  <:>
  Selection creates make_<&testCase>.sh
  <.><.+>
  ##
  ##                                ## writes to fAllsh, it is the shell script to invoke all tes
  <+fAllsh><: >
    <:><: >
    echo TEST invokes build/objZmake/make_<&testCase>.sh
====build/objZmake/make_<&testCase>.sh
====<.><.+>
  ##
  ##                                ## Writes a header for visual Studio test
  <+out>Generates: src/test/cpp/emC_TestAll/fDefSelection.h<.+n>
  Openfile fDefH = "src/test/cpp/emC_TestAll/fDefSelection.h";
  <+fDefH><: >
    <:><: >
    //This file is produced by running the sim selection tool.
====#define DEFINED_fDefSelection
====
====//The next defines contains the selection:
====<:for:define:defines>#define <&define>
====<.for>
====<.><.+>
  fDefH.close();
  ##
  ##                                ## The following subroutine generates the script with compili
  ##                                ## which invokes the test also.
  call build_dbgC1(testCase=testCase, defines = defines, srcSet = &(line6.srcSet), ccSet=ccSet );
  ##
  ##                                ## to repeat the specific test only the last one routine need
}

The last call is the subroutine which builds the make file for compilation. It is contained in the included file ,,src/test/ZmakeGcc/genScripts_emC.jztc,,:

##
##Generates the files for compile and test (shell script)
##This file can be repeated executed after this generation.
##written to build/objZmake/make_<&testCase>.sh
##
sub build_dbgC1 ( String testCase
    , List defines
    , Obj ccSet      ##Variables which determines the compilation, setting
    , Obj srcSet) {

  <+out>Generates: build/make_test_emC.sh for compilation and start test ...
  <&srcSet>
  <.+n>
  String cc_defh = <:><:for:define:defines> -D <&define><.for> -Isrc/test/ZmakeGcc/applstdef_Use

  mkdir <:>build/objZmake/<&testCase><.>;

  Openfile depArgs = <:>build/objZmake/deps_<&testCase>.args<.>;
  <+depArgs>-currdir:<&currdir><:n><: >
    -obj:build/objZmake/<&testCase>/*.o<:n><: >
    -cfg:src/test/ZmakeGcc/cfgCheckDeps.cfg<:n><: >
    -depAll:build/objZmake/<&testCase>/deps.txt<:n><: >
  <.+>

  ##<+makeAll>build/objZmake/make_<&testCase>.sh<.+n>
##  Openfile filedefineDef = <:>build/objZmake/<&testCase>/fDefSelection.h<.>;    ##fDefSelectio
##  <+filedefineDef><:>//This file is produces by running a test case
##====#define DEFINED_fDefSelection
##====//The next defines contains the selection:
##====<:for:define:defines>#define <&define>
##====<.for>
##====<.><.+close>
  String sMake = <:>build/objZmake/make_<&testCase>.sh<.>;
  <+out>Generates: <&sMake><.+n>
  Openfile makesh = sMake;
  <+makesh># call of compile, link and execute for Test emC_Base with gcc
    <:>
====if test -d ../../src/main; then cd ../..; fi  #is in build directory, should call from SBOX
====if test -d ../src/main; then cd ..; fi
====echo ----------------------------------------------
====echo -
====echo working dir to compile should be the SBOX root
====pwd                                                                          ##first invoke
====if ! test -d build/result; then mkdir build/result; fi
====if ! test -d build/objZmake/<&testCase>; then mkdir build/objZmake/<&testCase>; fi
====if test -f build/objZmake/<&testCase>/emCBase_.test.exe; then rm build/objZmake/<&testCase>/
====echo run checkDeps, see output in build/...testCase/checkDeps.out
====java -cp libs/vishiaBase.jar org.vishia.checkDeps_C.CheckDeps --@build/objZmake/deps_<&testC
====rm -f build/objZmake/<&testCase>/gcc*.txt ##clean output files
====rm -f build/result/<&testCase>.cc_err
====
====#rm -r Debug  #for test
====##echo <&testCase> 1> build/objZmake/<&testCase>/compile_Defs.txt
====echo <&testCase> > build/result/<&testCase>.out
====<:for:define:defines>echo "#define <&define>" >> build/result/<&testCase>.out
====<.for>
====echo Starting compilation >> build/result/<&testCase>.out
====##date >> build/result/<&testCase>.out  ##hint: the date disturbs comparability!
====<.><.+>
  ##                                   ## zmake for compilation, then for link
  zmake <:>build/objZmake/<&testCase>/*.o<.> := ccCompile( &c_src_emC_core
  , &srcSet
  , cc_def = cc_defh, makesh = makesh, depArgs = depArgs, testCase=testCase, ccSet=ccSet
  );
  ##                                   ## link, both zmake write to makesh, see called opration
  zmake <:>build/objZmake/<&testCase>/emCBase_.test.exe<.> := ccLink(&c_src_emC_core
  , &srcSet
  , makesh = makesh, testCase=testCase);

  <+makesh><: >##                          ## statements for execute in the makesh-script
    <:>
====echo End compilation >> build/result/<&testCase>.out
====##date >> build/result/<&testCase>.out  ##hint: the date disturbs comparability!
====if ! test -f build/objZmake/<&testCase>/emCBase_.test.exe; then
====  echo ERROR build/objZmake/<&testCase>/emCBase_.test.exe not built. See linker output.
====  echo MISSING: <&testCase>/..exe >> build/result/_all_result.txt
====  cat build/result/<&testCase>.cc_err >> build/result/_all.cc_err
====  echo ==========================
====else
====  echo ==== execute the test ====
====  echo TEST   : <&testCase>/..exe >> build/result/_all_result.txt
====  build/objZmake/<&testCase>/emCBase_.test.exe 1>> build/result/<&testCase>.out 2> build/res
====  echo ==== Test cases ==========
====  cat build/result/<&testCase>.out
====  echo
##====  echo ==== Test failures =======
##====  cat build/result/<&testCase>.err
##====  echo
====  echo ==========================
====fi
====<.><.+>

  depArgs.close();
  makesh.close();
  Obj fMake = new java.io.File(sMake);
  fMake.setExecutable(true);           ## for linux, chmod to executable
  <+out>success<.+n>
}

,,ccCompile(…​),, and ,,ccLink(…​),, are also sub routines in the JZtxtcmd script. The ,,Zmake,, approach is not complex. Only a specific ',,ZmakeTarget,,' instance is built and used.

##
##Creates a snippet in the generated make shell file for compiling all sources with gcc:
##
sub ccCompile(Obj target:org.vishia.cmd.ZmakeTarget
    , String cc_def
    , Obj makesh     ##Openfile for the make.sh file
    , Obj depArgs    ##Openfile for arguments of the checkDeps_C tool
    , Obj ccSet      ##Variables which determines the compilation, setting
    , String testCase) {
  for(c_src1: target.allInputFilesExpanded()) {
    String src1Base = c_src1.basepath();
    if(src1Base.length() >0) {
      <+depArgs>-src:<&c_src1.basepath()>:<&c_src1.localfile()><.+n>  ##writes the file for check
    } else {
      <+depArgs>-src:<&c_src1.file()><.+n>  ##writes the file for checkDeps
    }
    ###<+out><&infoDeps><.+n> ##show state, info and file name on console.
    <+makesh><: >
    <:>
    #echo ==== gcc <&c_src1.localfile()> 1>> <&target.output.localdir()>/gcc_err.txt
    if ! test -e <&target.output.localdir()>/<&c_src1.localname()>.o; then
      mkdir -p <&target.output.localdir()>/<&c_src1.localdir()>
      <&ccSet.cc> <&cc_options> -Wa,-adhln <&cc_def> <&inclPath> -o <&target.output.localdir()>/<

      if test ! -e <&target.output.localdir()>/<&c_src1.localname()>.o; then
        echo gcc ERROR: <&c_src1.localfile()>
        echo ERROR: <&c_src1.localfile()> >> build/result/gcc_nocc.txt;
      else
        echo gcc ok: <&c_src1.localfile()>
      fi
    else
      echo exist: <&c_src1.localfile()>
    fi
    <.><.+>
  }
}

In the of course here truncated shown long line for the compiler invocation starting with ,,<&ccSet.cc> …​,, both immediately compiler options and some more options from JZtxtcmd variables are written in the ,,make…​sh,, file. It is possible to change here for specific approaches, and look the result in the generated ,,make…​sh,, file. The compiler invocation itself is done only from the shell script on beginning of this chapter, via

build/testBasics_Simple.sh

It may be possible to start the script from inside JZtxtcmd too, but then it runs as Process inside Java. Possible but maybe more complicated to follow which is happen. Hence the more simple way is gone here.

7. Test environment in embedded control without text output

The macros can be written to store only values instead output the texts via printf.

Either the text result are visit via debugger access after all tests, or via any data transfer.

TODO describe an example.

The complete test for embedded control compiles for the target, load the target, start, and reads out the result via the communication line which may be JTAG or usual a serial connection.