Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IT file changes- TestFramework.scala, Runner.scala, RunnerFactory.java #41

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

IT file changes- TestFramework.scala, Runner.scala, RunnerFactory.java #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a91cdc3
IT file changes- TestFramework.scala, Runner.scala, RunnerFactory.java
Aug 29, 2024
a7c1459
New version for IT tests
Oct 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

<groupId>edu.illinois.cs</groupId>
<artifactId>testrunner</artifactId>
<version>1.3-SNAPSHOT</version>
<version>1.4-SNAPSHOT</version>

<modules>
<module>testrunner-core-plugin</module>
Expand Down
2 changes: 1 addition & 1 deletion testrunner-core-plugin/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<groupId>edu.illinois.cs</groupId>
<artifactId>testrunner</artifactId>
<version>1.3-SNAPSHOT</version>
<version>1.4-SNAPSHOT</version>
</parent>

<artifactId>testrunner-core-plugin</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion testrunner-gradle-plugin/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<groupId>edu.illinois.cs</groupId>
<artifactId>testrunner</artifactId>
<version>1.3-SNAPSHOT</version>
<version>1.4-SNAPSHOT</version>
</parent>

<artifactId>testrunner-gradle-plugin</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion testrunner-maven-plugin/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<groupId>edu.illinois.cs</groupId>
<artifactId>testrunner</artifactId>
<version>1.3-SNAPSHOT</version>
<version>1.4-SNAPSHOT</version>
</parent>

<artifactId>testrunner-maven-plugin</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion testrunner-running/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<groupId>edu.illinois.cs</groupId>
<artifactId>testrunner</artifactId>
<version>1.3-SNAPSHOT</version>
<version>1.4-SNAPSHOT</version>
</parent>

<artifactId>testrunner-running</artifactId>
Expand Down
144 changes: 96 additions & 48 deletions ...nner-running/src/main/scala/edu/illinois/cs/testrunner/data/framework/TestFramework.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import java.util.List;
import scala.collection.JavaConverters._
import scala.collection.mutable.ListBuffer
import scala.util.Try
import java.util.logging.{Level, Logger}

object TestFramework {
def testFramework(project: ProjectWrapper): Option[TestFramework] = {
Expand Down Expand Up @@ -106,98 +107,145 @@ trait TestFramework {
}

object JUnit extends TestFramework {
val methodAnnotationStr: String = "org.junit.Test"
val methodAnnotationStr: String = "org.junit.Test"

override def tryGenerateTestClass(loader: ClassLoader, clzName: String): Option[GeneralTestClass] = {

override def tryGenerateTestClass(loader: ClassLoader, clzName: String)
: Option[GeneralTestClass] = {
val annotation: Class[_ <: Annotation] =
val annotation: Class[_ <: Annotation] =
loader.loadClass(methodAnnotationStr).asInstanceOf[Class[_ <: Annotation]]

try {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove most empty lines please

val clz = loader.loadClass(clzName)

Logger.getGlobal().log(Level.INFO, s"Loaded class: $clzName")

if (!Modifier.isAbstract(clz.getModifiers)) {

val methods = clz.getMethods.toStream

Try(if (methods.exists(_.getAnnotation(annotation) != null)) {
// Check if it is an integration test by name or directory

val isIntegrationTestByName = clzName.endsWith("IT")
val classFilePath = clz.getProtectionDomain.getCodeSource.getLocation.getPath
val isIntegrationTestByDirectory = classFilePath.contains("/target/test-classes/")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there other places in testrunner that hard codes paths? Looking specifically in /target/test-classes/ may be best to be hard coded with other paths in testrunner


Logger.getGlobal().log(Level.INFO, s"Class $clzName integration test by name: $isIntegrationTestByName")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding debug messages to INFO logging, please change the level to DEBUG or something equivalent.

Logger.getGlobal().log(Level.INFO, s"Class $clzName integration test by directory: $isIntegrationTestByDirectory")

Try(if (methods.exists(_.getAnnotation(annotation) != null) ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should Try be try?

(isIntegrationTestByName && isIntegrationTestByDirectory)) {
Logger.getGlobal().log(Level.INFO, s"Class $clzName recognized as a test class.")
Option(new JUnitTestClass(loader, clz))
} else if (loader.loadClass("junit.framework.TestCase").isAssignableFrom(clz)) {
Logger.getGlobal().log(Level.INFO, s"Class $clzName recognized as a TestCase class.")
Option(new JUnitTestCaseClass(loader, clz))
} else {
Logger.getGlobal().log(Level.INFO, s"Class $clzName is not a test class, skipping.")
Option.empty
}).toOption.flatten

} else {
Logger.getGlobal().log(Level.INFO, s"Class $clzName is abstract, skipping.")
Option.empty

}

} catch {
case e: NoClassDefFoundError => Option.empty

case e: NoClassDefFoundError =>
Logger.getGlobal().log(Level.WARNING, s"Class $clzName could not be loaded due to NoClassDefFoundError.")
Option.empty

}

}

override def toString(): String = "JUnit"

// the splitor to split the class name and method name
// for the full qualified name of JUnit 4 test
// the delimiter to split the class name and method name
// for the fully qualified name of JUnit 4 test
// like com.package.JUnit4TestClass.TestA
override def getDelimiter(): String = "."

}

object JUnit5 extends TestFramework {

val methodAnnotationStr: String = "org.junit.jupiter.api.Test"
val nestedAnnotationStr: String = "org.junit.jupiter.api.Nested"
val disabledAnnotationStr: String = "org.junit.jupiter.api.Disabled"

override def tryGenerateTestClass(loader: ClassLoader, clzName: String)
: Option[GeneralTestClass] = {
val testAnnotation: Class[_ <: Annotation] =
loader.loadClass(methodAnnotationStr).asInstanceOf[Class[_ <: Annotation]]

val disabledAnnotation: Class[_ <: Annotation] =
loader.loadClass(disabledAnnotationStr).asInstanceOf[Class[_ <: Annotation]]
override def tryGenerateTestClass(loader: ClassLoader, clzName: String): Option[GeneralTestClass] = {

try {
val clz = loader.loadClass(clzName)
val testAnnotation: Class[_ <: Annotation] =
loader.loadClass(methodAnnotationStr).asInstanceOf[Class[_ <: Annotation]]

val disabledAnnotation: Class[_ <: Annotation] =
loader.loadClass(disabledAnnotationStr).asInstanceOf[Class[_ <: Annotation]]


try {

val clz = loader.loadClass(clzName)
Logger.getGlobal().log(Level.INFO, s"Loaded class: $clzName")

if (clz.getAnnotation(disabledAnnotation) != null) {
// Skip disabled class
Logger.getGlobal().log(Level.INFO, s"Class $clzName is disabled, skipping.")
return Option.empty

}

if (clz.getAnnotation(disabledAnnotation) != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be removed?

// skip disabled class
if (clz.isMemberClass) {
val nestedAnnotation: Class[_ <: Annotation] =
loader.loadClass(nestedAnnotationStr).asInstanceOf[Class[_ <: Annotation]]

if (Modifier.isStatic(clz.getModifiers) ||
clz.getAnnotation(nestedAnnotation) == null) {
// Skip unqualified nested test class
Logger.getGlobal().log(Level.INFO, s"Class $clzName is an unqualified nested test class, skipping.")
return Option.empty
}

if (clz.isMemberClass) {
val nestedAnnotation: Class[_ <: Annotation] =
loader.loadClass(nestedAnnotationStr)
.asInstanceOf[Class[_ <: Annotation]]
if (Modifier.isStatic(clz.getModifiers) ||
clz.getAnnotation(nestedAnnotation) == null) {
// a nested test class should
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should keep these comments

// (1) be non-static
// (2) have @Nested annotation
//
// Skip unqualified test class
return Option.empty
}
}
}

if (!Modifier.isAbstract(clz.getModifiers)) {
val methods = Utility.getAllMethods(clz)
// Check if it is an integration test by name or directory

val isIntegrationTestByName = clzName.endsWith("IT")
val classFilePath = clz.getProtectionDomain.getCodeSource.getLocation.getPath
val isIntegrationTestByDirectory = classFilePath.contains("/target/test-classes/")

Logger.getGlobal().log(Level.INFO, s"Class $clzName integration test by name: $isIntegrationTestByName")
Logger.getGlobal().log(Level.INFO, s"Class $clzName integration test by directory: $isIntegrationTestByDirectory")


if (!Modifier.isAbstract(clz.getModifiers) &&

(clz.getDeclaredMethods.exists(_.getAnnotation(testAnnotation) != null) || (isIntegrationTestByName && isIntegrationTestByDirectory))) {
Logger.getGlobal().log(Level.INFO, s"Class $clzName recognized as a test class.")
Option(new JUnit5TestClass(loader, clz))

} else {

Logger.getGlobal().log(Level.INFO, s"Class $clzName is not a test class, skipping.")
Option.empty

Try(if (methods.exists(_.getAnnotation(testAnnotation) != null)) {
Option(new JUnit5TestClass(loader, clz))
} else {
Option.empty
}).toOption.flatten
} else {
Option.empty
}
} catch {
case e: NoClassDefFoundError => Option.empty
}
}

override def toString(): String = "JUnit5"
} catch {

case e: NoClassDefFoundError =>
Logger.getGlobal().log(Level.WARNING, s"Class $clzName could not be loaded due to NoClassDefFoundError.")
Option.empty

}

}
override def toString(): String = "JUnit5"

// the splitor to split the class name and method name
// for the full qualified name of JUnit 5 test
// like com.package.JUnit5TestClass#TestA
override def getDelimiter(): String = "#"
}
}
76 changes: 49 additions & 27 deletions testrunner-running/src/main/scala/edu/illinois/cs/testrunner/runner/Runner.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ import scala.collection.mutable.ListBuffer
import scala.io.Source
import scala.util.{Failure, Try}

import java.util.logging.{Level, Logger}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong line. put inbetween line 4-5


trait Runner {
def outputPath(): Path
def classpath(): String
Expand Down Expand Up @@ -56,36 +58,56 @@ trait Runner {

def generateTestRunId(): String = System.currentTimeMillis() + "-" + UUID.randomUUID.toString

def runWithCp(cp: String, testOrder: Stream[String]): Try[TestRunResult] =
TempFiles.withSeq(testOrder)(path =>
TempFiles.withTempFile(outputPath =>
TempFiles.withProperties(Configuration.config().properties())(propertiesPath => {
val builder = makeBuilder(cp + File.pathSeparator + Configuration.config().getProperty("testplugin.classpath"))

val info = execution(testOrder, builder)

val testRunId = generateTestRunId()

val exitCode = info.run(
testRunId,
framework().toString,
path.toAbsolutePath.toString,
propertiesPath.toAbsolutePath.toString,
outputPath.toAbsolutePath.toString).exitValue()

if (exitCode == 0) {
autoClose(Source.fromFile(outputPath.toAbsolutePath.toString).bufferedReader())(reader =>
Try(new Gson().fromJson(reader, classOf[TestRunResult])))
} else {
// Try to copy the output log so that it can be inspected
val failureLog = Paths.get("failing-test-output-" + testRunId)
Files.copy(info.outputPath, failureLog, StandardCopyOption.REPLACE_EXISTING)
Failure(new Exception("Non-zero exit code (output in " + failureLog.toAbsolutePath + "): " ++ exitCode.toString))
}
}))).flatten.flatten.flatten.flatten
val logger = Logger.getLogger(getClass.getName)

def runWithCp(cp: String, testOrder: Stream[String]): Try[TestRunResult] = {
logger.log(Level.INFO, "runWithCp method is being executed")


TempFiles.withSeq(testOrder)(path => {
println(s"Temporary path created: $path") // Print after creating the sequence of test paths

TempFiles.withTempFile(outputPath => {
println(s"Temporary output path created: $outputPath") // Print after creating the temporary output file

TempFiles.withProperties(Configuration.config().properties())(propertiesPath => {
println(s"Properties path created: $propertiesPath") // Print after creating the properties path

val builder = makeBuilder(cp + File.pathSeparator + Configuration.config().getProperty("testplugin.classpath"))

val info = execution(testOrder, builder)

val testRunId = generateTestRunId()
println(s"Generated testRunId: $testRunId") // Print the generated testRunId

val exitCode = info.run(
testRunId,
framework().toString,
path.toAbsolutePath.toString,
propertiesPath.toAbsolutePath.toString,
outputPath.toAbsolutePath.toString).exitValue()

println(s"Execution finished with exit code: $exitCode") // Print the exit code

if (exitCode == 0) {
autoClose(Source.fromFile(outputPath.toAbsolutePath.toString).bufferedReader())(reader =>
Try(new Gson().fromJson(reader, classOf[TestRunResult])))
} else {
// Try to copy the output log so that it can be inspected
val failureLog = Paths.get("failing-test-output-" + testRunId)
Files.copy(info.outputPath, failureLog, StandardCopyOption.REPLACE_EXISTING)
println(s"Non-zero exit code. Output copied to: $failureLog") // Print the failure log location
Failure(new Exception("Non-zero exit code (output in " + failureLog.toAbsolutePath + "): " ++ exitCode.toString))
}
})
})
}).flatten.flatten.flatten.flatten
}
}


trait RunnerProvider[A <: Runner] {
def withFramework(framework: TestFramework, classpath: String,
environment: java.util.Map[String, String], outputPath: Path): A
}

32 changes: 27 additions & 5 deletions testrunner-running/src/main/scala/edu/illinois/cs/testrunner/runner/RunnerFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@
import java.util.stream.Stream;
import scala.Option;

import java.util.logging.Logger;
import java.util.logging.Level;

public class RunnerFactory {
public static Option<Runner> from(final ProjectWrapper project) {
return TestFramework.testFramework(project)
Expand Down Expand Up @@ -44,11 +47,29 @@ public static Option<Runner> from(final MavenProject project) {
}

public static List<Runner> allFrom(final MavenProject project) {
return TestFramework.getListOfFrameworks(project).stream()
.map(framework ->
create(framework, new ProjectClassLoader(project).classpath(),
surefireEnvironment(project), project.getBasedir().toPath()))
.collect(Collectors.toList());
System.out.println("RunnerFactory.allFrom is being called");

List<Runner> runners = new ArrayList<>();

List<TestFramework> frameworks = TestFramework.getListOfFrameworks(project);
System.out.println("Number of frameworks detected: " + frameworks.size());

TestFramework.getListOfFrameworks(project).forEach(framework -> {
System.out.println("Detected framework: " + framework.toString());

Runner runner = create(framework, new ProjectClassLoader(project).classpath(),
surefireEnvironment(project), project.getBasedir().toPath());

if (runner != null) {
runners.add(runner);
System.out.println("Created runner for framework: " + framework.toString());
} else {
System.out.println("Failed to create runner for framework: " + framework.toString());
}
});

return runners;

}

private static <T> Stream<T> emptyIfNull(final T t) {
Expand All @@ -72,3 +93,4 @@ public static Map<String, String> surefireEnvironment(final MavenProject project
.collect(Collectors.toMap(Xpp3Dom::getName, v -> v.getValue() == null ? "" : v.getValue()));
}
}

2 changes: 1 addition & 1 deletion testrunner-testing/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<parent>
<groupId>edu.illinois.cs</groupId>
<artifactId>testrunner</artifactId>
<version>1.3-SNAPSHOT</version>
<version>1.4-SNAPSHOT</version>
</parent>

<artifactId>testrunner-testing</artifactId>
Expand Down