Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Programming exercises: Improve test build logs for Haskell exercises #9792

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

b-fein
Copy link
Contributor

@b-fein b-fein commented Nov 15, 2024

Checklist

General

Changes affecting Programming Exercises

  • High priority: I tested all changes and their related features with all corresponding user types on a test server configured with the integrated lifecycle setup (LocalVC and LocalCI).
  • I tested all changes and their related features with all corresponding user types on a test server configured with Gitlab and Jenkins.

Motivation and Context

As noted in #9159, the test runner disabled the console output to produce the XML instead. Therefore, the test results no longer show up in the build log which makes it harder to debug what is happening when setting up exercises.

Description

Updates the test template to produce both the console logs that then show up in the build log, and the XML needed to report the result back to Artemis.

Steps for Testing

Prerequisites:

  • 1 Instructor
  1. Create a new Haskell programming exercise.
  2. The solution should pass all tests.
  3. Look at the build logs in Jenkins, or download the build logs for the solution build in the build history tab of the course. The test results should show up in the log as well (example below).
Tests
  Properties
    Checked by SmallCheck
      Testing filtering in A:                  OK
        475 tests completed
      Testing mapping in A:                    OK
        19 tests completed
      Testing filtering in B:                  OK
        475 tests completed
      Testing mapping in B:                    OK
        19 tests completed
      Testing filtering in C:                  OK
        475 tests completed
      Testing mapping in C:                    OK
        19 tests completed
    Checked by QuickCheck
      Testing A against sample solution:       OK
        +++ OK, passed 100 tests.
      Testing B against sample solution:       OK
        +++ OK, passed 100 tests.
      Testing C against sample solution:       OK
        +++ OK, passed 100 tests.
  Unit Tests
    Testing selectAndReflectA (0,0) []:        OK
    Testing selectAndReflectB (0,1) [(0,0)]:   OK
    Testing selectAndReflectC (0,1) [(-1,-1)]: OK

All 12 tests passed (0.01s)

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.







Review Progress

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Test Coverage

unchanged


@coderabbitai ignore

@b-fein b-fein added enhancement component:Programming small programming Pull requests that affect the corresponding module labels Nov 15, 2024
@b-fein b-fein requested a review from a team as a code owner November 15, 2024 21:38
@github-actions github-actions bot added template and removed programming Pull requests that affect the corresponding module labels Nov 15, 2024
@b-fein b-fein added the programming Pull requests that affect the corresponding module label Nov 15, 2024
@HawKhiem HawKhiem temporarily deployed to artemis-test3.artemis.cit.tum.de November 16, 2024 13:02 — with GitHub Actions Inactive
Copy link

@HawKhiem HawKhiem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on TS3. The solution build passed and the mentioned lines were indeed to be found in the build log

Screenshot 2024-11-16 151709
Screenshot 2024-11-16 151821

@github-actions github-actions bot added lock:artemis-test1 and removed programming Pull requests that affect the corresponding module labels Nov 18, 2024
@chrisknedl chrisknedl added programming Pull requests that affect the corresponding module deploy:artemis-test1 and removed lock:artemis-test1 labels Nov 18, 2024
@chrisknedl chrisknedl temporarily deployed to artemis-test1.artemis.cit.tum.de November 18, 2024 16:23 — with GitHub Actions Inactive
Copy link
Contributor

@chrisknedl chrisknedl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected on TS1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Ready For Review
Development

Successfully merging this pull request may close these issues.

3 participants