Skip to content

Commit

Permalink
Address some changes in writeup
Browse files Browse the repository at this point in the history
  • Loading branch information
mjdecker committed Mar 26, 2024
1 parent 29902c9 commit 65eea92
Showing 1 changed file with 14 additions and 15 deletions.
29 changes: 14 additions & 15 deletions _projects/software-testing/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ layout: markdown

## Overview ##
* You will be writing a tool to [instrument](https://en.wikipedia.org/wiki/Instrumentation_(computer_programming)) a given project to collect information on what statements and conditions are executed when running a given test suite or executable.
* Use `simple_main.cpp` and `foo.cpp` for testing
* The following are provided for testing:
* `simple_main.cpp` and `foo.cpp`
* `test_output.cpp` and `bigint.cpp`

## Details ##
* The program must build and run using the provided *CMakeLists.txt* on voyager and be committed/pushed to your class GitHub repository.
Expand All @@ -33,15 +35,22 @@ layout: markdown
* \<expr_stmt> - an expression statement (e.g., *a = 5 + x; std::cin >> x; ++i; foo(30);*)
* \<condition> - a condition of an if/for/while, etc.
* \<return> - a return statement (e.g., return result;)
* Note: function will be used to mean function and method.
* Note: function will be used to any function like element: functions, methods, constructors, destructors, and operator functions/methods. Provided for you is `is_function_tag` which returns true if an element is a function-like element.
* Specification
* The following is a complete working usage of the coverage tool:


# Instrumenting simple_main.cpp & foo.cpp
./coverage simple_main.cpp.xml foo.cpp.xml
# Run cmake prior to next (one-time only)
make
./c-simple_main

# Instrumenting test_output & bigint.cpp
./coverage test_output.cpp.xml bigint.cpp.xml
# Run cmake prior to next (one-time only)
make
./c-test_output

* There are three parts to this assignment:
* The main which parses the command-line arguments and runs the code to profile the project. For simplicity, the first argument to the command line tool will be the file with the main function. The remaining arguments will be additional files (with no main) to be profiled. The main program is provide for you as *main.cpp* (you **MUST** use this).
* An object to collect the coverage information during runtime. This is provided for you as *coverage_map.hpp* (you **MUST** use this).
Expand All @@ -57,7 +66,7 @@ layout: markdown
* `syntax_tree::main_header` inserts an include of *coverage_map.hpp* at the start of the file. In addition, `syntax_tree::main_header` will also insert global variable declarations (one for each file) with type *coverage_map_t* after the include. The method parameter contains the names of these variable for you.
* `syntax_tree:file_header` also inserts an include of *coverage_map.hpp* at the start of the file. However, it inserts one `extern` declaration referencing one of the variables declared in the main file. The name of the variable is passed to the function for you.
* `syntax_tree::main_report` is run on the main file. It will have to search for the function named `main` and before the return statement (for simplicity, there is guaranteed to be only one return statement, and it will be directly in the function body and not nested in a statement), it will call print on each of the global variables inserted by `syntax_tree::main_header`. The names of the variables are provided for you as a parameter.
* `syntax_tree::instrument_functions` inserts all the coverage instrumentations using `syntax_tree::count_elements` and `syntax_tree::instrument_statements` as helper methods. For each function found (you will have to search for each), first, `syntax_tree::instrument_functions` finds the function body and after the `{` it inserts a call to append on the variable name provided as a parameter. The arguments to this call are the name of the function (use `__FUNCTION__` for simplicity) and the number of statements (and conditions if you are doing the extra credit). Use `syntax_tree::count_elements` to get this number. Lastly, it calls `syntax_tree::instrument_statements` to insert statement coverage (and condition coverage if you are doing the extra credit).
* `syntax_tree::instrument_functions` inserts all the coverage instrumentations using `syntax_tree::count_elements` and `syntax_tree::instrument_statements` as helper methods. For each function found (you will have to search for each), first, `syntax_tree::instrument_functions` finds the function body and then the body's content (i.e., \<body_content>) where it inserts a call to append on the variable name provided as a parameter. The arguments to this call are the name of the function (use `__FUNCTION__` for simplicity) and the number of statements (and conditions if you are doing the extra credit). Use `syntax_tree::count_elements` to get this number. Lastly, it calls `syntax_tree::instrument_statements` to insert statement coverage (and condition coverage if you are doing the extra credit).
* `syntax_tree::count_elements` counts the number of expression statements (`expr_stmt`), declaration statements (`decl_stmt`), and, if you are doing the extra credit, conditions (`condition`). This is a recursive function. `is_coverage_tag` may be of use. Make sure to call this on the function element.
* `syntax_tree::instrument_statements` inserts calls to `executed` immediately after each expression statements (`expr_stmt`) and declaration statements (`decl_stmt`). If doing the extra credit, it also inserts a call to `executed` at the start of each condition (`condition`). Some conditions have a `(` and others do not that you will have to place the call after. The variable the call is placed on is provided as a parameter to `syntax_tree::instrument_statements`. Additionally, the second parameter to `executed` is a unique (by function) identifier for that statement/condition. The parameter `current_statement_number` to `syntax_tree::instrument_statements` is used to provide this unique identifier. Make sure to increment after each call to executed is inserted. This is a recursive function.

Expand Down Expand Up @@ -119,21 +128,11 @@ layout: markdown
return i;
}
</pre>
* `coverage.txt`
* Here is how to run and execute `test_output` with coverage.

./coverage test_output.cpp.xml bigint.cpp.xml
# Run cmake prior to next (one-time only)
make
./c-test_output

* Look at the output and the source (`bigint.cpp` and `bigint.hpp`) as well as the instrumented source `c-bigint.cpp`. Can the missed code ever be run? Pay special attention to the magic number in `operator<<`.

* Simplifications
* *ONLY* invoked functions are reported.
* You may *ASSUME* there are no usages of the ternary operator.
* You may *ASSUME* all loops and conditionals have a block.
* You may *ASSUME* the *main* function only has one return-statement and is a child of `block`.
* You may *ASSUME* the *main* function only has one return-statement and is a child of `block_content`.
* You may *ASSUME* all function names are unique.
* You may *ASSUME* that there are not functions within functions.
* You may *ASSUME* that there are no uses of ranged-based for-statement.
Expand Down

0 comments on commit 65eea92

Please sign in to comment.