boppi/doc/report-tests.tex

% Test plan and results. Discussion of the correctness test, using the criteria described in <20>D.5. You
% should provide a set of test programs demonstrating the correct functioning of your compiler. The test
% set should contain, next to programs testing the various language featurs, also programs containing
% syntactic, semantic or run-time errors.
% All tests should be provided as part of the zip-file. One test programshould be included as an appendix
% in the report (see below).
Testing the Boppi language is done with fully automated ANTLR4 tests. The test suite is designed to quickly add new syntactic, semantic and runtime tests and check them for errors. For this purpose, the \emph{pp.s1184725.boppi.test.BoppiTests} class contains various helper methods that capture errors and pass input and output.\\
Each feature of the Boppi language is tested for correctness with a set of automated tests. Each set of tests checks for both correct and incorrect syntax, semantics and runtime evaluation. Moreover, the dynamic allocator \cref{memlib} is tested for correctness in a separate test suite.\\

\section{Basic expressions}
The basic expressions are tested for:
\begin{itemize}
    \item correctly parsing a valid program
    \item reporting errors for programs with invalid lexemes and invalid expressions
    \item correctly checking a valid program
    \item reporting errors for programs with type errors
    \item correctly executing a valid program and printing the results of expressions
    \item reporting a warning for expressions that require more than the recommended number of registers
    \item crashing the virtual machine when a division by zero is executed
\end{itemize}

\section{Variables}
Simple variables are tested for:
\begin{itemize}
    \item correctly parsing variable names and linking them to a single object
    \item rejecting invalid variable names
    \item correctly matching a variable use to a declared variable
    \item reporting an error when a variable is used without being declared before
    \item correctly redeclaring variables in a deeper scope
    \item reporting an error if a variable is out of scope
    \item reporting an error if a block ends with a declaration
    \item reporting an error if a constant variable is assigned more than once (statically checked)
    \item reporting an error if a variable is used before it is assigned (statically checked)
\end{itemize}

\section{Conditional code}
Conditionals are tested for:
\begin{itemize}
    \item correctly parsing if expressions, while statements and if statements with and without an alternative
    \item correctly returning the inner type of an if expression
    \item correctly returning void of an if or while statement
    \item reporting an error if the condition in a conditional does not return a boolean type
    \item correctly opening a scope for the variables in the condition and in the bodies of a conditional
    \item correctly taking the right branch in an if statement or expression
    \item correctly iterating a while loop as long as the condition is true
\end{itemize}

\section{Functions}
Functions are tested for:
\begin{itemize}
    \item correctly parsing functions with and without a return type
    \item correctly parsing functions with zero, one or multiple typed parameters
    \item correctly parsing function calls
    \item reporting an error when a block ends with a function declaration
    \item correctly matching a name to a declared function
    \item correctly matching the return type of a function to the type of the function body, if applicable
    \item correctly matching the actual parameter types of a function call to the formal parameter types
    \item correctly setting the return type of a function call to the function definition's return type
    \item reporting an error if the number of actual parameters in a function call does not match the number of formal parameters
    \item correctly opening a scope for parameters
    \item correctly checking and running recursive functions
    \item correctly declaring functions inside functions
    \item correctly assigning and reassigning function variables
\end{itemize}

\section{Closures}
Having functions as first-class citizens (i.e. function references can be passed around) calls for the possibility of closures: a context for a function with all the non-local variables in case the function requires them when run.\\
Closures are tested for:
\begin{itemize}
    \item correctly parsing function type declarations
    \item correctly passing function variables as parameters and return values
    \item correctly using non-local variables when a function is called outside its declared context
    \item correctly using the same non-local variables across multiple function calls with the same context
\end{itemize}
The tests for garbage collection are skipped, due to the way activation records (\emph{AR}) are deallocated. See \cref{future-work} for more information.


\section{Arrays}
Arrays are tested for:
\begin{itemize}
    \item correctly parsing array types and nested arrays
    \item correctly parsing array accessors
    \item correctly parsing variable properties (\emph{array.length})
    \item correctly checking array literals
    \item correctly checking array constructors and assigning them to arrays
    \item correctly returning the element type of an array access
    \item correctly performing a bounds check on an array access during run-time
    \item correctly performing equality checks between arrays
\end{itemize}

\section{Allocator}
The \verb|memlib| library is tested for:
\begin{itemize}
    \item correctly allocating objects
    \item correctly deallocating an object when its reference count goes to zero
    \item correctly incrementing and decrementing reference counts
    \item correctly merging freed space
    \item correctly allocating objects in fitting free slots
    \item halting the machine when a program tries to free empty space
    \item halting the machine when a null pointer has its reference count incremented or decremented
\end{itemize}