-
Notifications
You must be signed in to change notification settings - Fork 38
Testing robot code
Strongback makes it easy to test your robot code. Be sure that Strongback is installed first and that you've created a test project.
JUnit tests are defined by regular Java classes that contain methods with the @org.junit.Test annotation. Using JUnit you can run just a single test method, all of the test methods within a single class, or all of the tests in all classes within a package or project. When you do this, the Eclipse's JUnit view will display the results for all of the tests that are run. As JUnit runs the tests, the JUnit view shows the progress with a big green bar. If all tests pass, the bar in the JUnit view will remain green; if any of the tests fail or have an error, the bar will turn red. You can expand the tree in the JUnit view to see all of the test classes and all individual test methods, and when you double-click on any of the test methods Eclipse will take you directly to that test method. You can even run the tests in debug mode and use the powerful Eclipse debugger to set breakpoints and step through the code.
JUnit is a powerful framework, so we suggest that you learn more about it from the JUnit website. But here are the basics: when JUnit runs the test methods in a class, it goes through the following sequence:
- Create a new instance of your test class
- Call the static method(s) annotated with
@org.junit.BeforeClass - Then for each method annoated with
@org.junit.Test: - Call the method(s) annotated with
@org.junit.Before - Call the test method
- Call the method(s) annotated with
@org.junit.After - After all tests have been run, call the static method(s) annotated with
@org.junit.AfterClass
The @BeforeClass and @AfterClass static methods are typically used to perform one-time setup and teardown (respectively) needed by the class. You might do this to initialize Strongback with a custom configuration used by all of the tests, and then shut it down after the tests are done. Because these methods are static, they only have access to static variables. The name of the methods do not matter, but we prefer to use a convention where we use beforeAll() for the @BeforeClass method, afterAll() for the @AfterClass method.
The @Before method is typically where you instantiate member variables that you will use in all or most of your test methods, and then null the references (or clean them up if necessary) in the @After method so that state set in one method doesn't leak into another test method. Again, JUnit does not care what these methods are named, but we like to use beforeEach() for the @Before method, and afterEach() for the @After method.
The @Test methods can also create their own objects, and this is often the case. The objective is that each test is relatively simple, performs a specific test, and runs quickly. JUnit does not care what the test methods are named, but you should name them in a way that is meaningful. For example, you might name methods with phrases like shouldControlMotorsWithPositiveForwardThrottle or shouldNotAllowMotorSpeedsAbove1.
Without Strongback, your subsystems directly use (and may even instantiate) the WPILib classes that represent hardware actuators, sensors, and other devices. It is difficult to test these subsystems with JUnit on your laptop because as soon as your test instantiates the subsystem, the subsystem attempts to use or instantiate the WPILib classes, which only work if there is appropriate hardware. And, even if the WPILib classes your using don't need hardware, they often depend on classes that do expect hardware, the RoboRIO libraries, or the Driver Station to be accessible. Without that hardware, your tests almost immediately start throwing exceptions in the setup methods -- before they can actually test anything useful.
Strongback is designed so that you can easily test most of your subsystem and other robot classes on your laptop without needing any of the robot-specific hardware. The key to this is Strongback's interfaces for components like motors (speed controllers), distance sensors, speed sensors, temperature sensors, current sensors, single- and multi-axis accelerometers, relays, switches, fuses, clocks, etc. If you write your subsystems to use these interfaces, then they won't care which implementation they use. On the robot, you can use the hardware-based implementations, and in your tests you can use mock components.
A mock component is simply an implementation of an interface that lets your tests directly set the values exposed by the interface or read the values set by the interface. For example, consider Strongback's DistanceSensor interface, which in simplified form looks like:
@FunctionalInterface
public interface DistanceSensor {
/**
* Gets the current value of this {@link DistanceSensor} in inches.
*
* @return the distance in inches
*/
public double getDistance();
/**
* Gets the current value of this {@link DistanceSensor} in feet.
*
* @return the distance in feet
*/
default public double getDistanceInFeet() { return getDistance() / 12.0; }
}
Any subsystem that uses a DistanceSensor object can get the sensor's current distance at any time, and a hardware-based DistanceSensor implementation might use an ultrasonic sensor to determine the distance. However, a unit test for your subsystem can instead use a MockDistanceSensor object that implements DistanceSensor but adds another setter method. Here's what that MockDistanceSensor might look like:
public class MockDistanceSensor implements DistanceSensor {
@Override
public double getDistance() { ... }
public void setDistance( double distanceInInches ) { ... }
}
Let's say that our example robot contains a subsystem called Shooter that fires a game pieces when the subsystem determines that it is "within range" of a goal. Our robot uses an ultrasonic rangefinder to determine distance to the goal, and the Shooter subsystem considers it within range when the distance is between 10 and 30 inches. We can easily test our Shooter subsystem. Here's a nearly complete unit test class that verifies the Shooter.isInRange() logic works as expected:
public class ShooterTest {
private MockDistanceSensor sensor;
private Shooter shooter;
@Before
public void beforeEach() {
sensor = Mock.distanceSensor();
shooter = new Shooter(sensor);
}
@Test
public void shouldNotBeInRangeIfDistanceIsLessThan10() {
sensor.setDistance(9.999);
assertThat(shooter.isInRange()).isFalse();
}
@Test
public void shouldBeInRangeIfDistanceIsBetween10And30Inclusive() {
for ( int i=10; i!=30; ++i) {
sensor.setDistance(i);
assertThat(shooter.isInRange()).isFalse();
}
}
@Test
public void shouldNotBeInRangeIfDistanceIsGreaterThan30() {
sensor.setDistance(30.0001);
assertThat(shooter.isInRange()).isFalse();
}
@Test
public void shouldNotBeInRangeIfDistanceIsNegative() {
sensor.setDistance(-1.0);
assertThat(shooter.isInRange()).isFalse();
sensor.setDistance(-20.0);
assertThat(shooter.isInRange()).isFalse();
}
}
Note that the last method checks for negative distance, which could happen in real life if the distance sensor were zeroed at the wrong time. It's important to thoroughly test the logic of your code, even when you don't think those cases can happen in real life. For example, what if the isInRange() method has a sign error or (incorrectly) takes the aboslute value of the distance? Yes, Shooter is a very simple example, but as the subsystems become more complicated it is very worthwhile to have good and thorough unit tests for all of the subsystem's logic.
The simplified DistanceSensor declaration shown above uses the @FunctionalInterface annotation. Java 8 can do some amazing things with functional interface, which are defined as interfaces that contain exactly one abstract method (though they can have any number of default methods, which are also new in Java 8). Specifically, a functional interface can be defined or implemented with a lambda expression that has the same signature as the functional interface's one abstract method. Lambda expressions can be defined as a block, which generally looks something like:
(...) -> { ... }
But you can also have lambda expressions that refer to instance methods, static methods, or constructor methods.
Let's look at an example. The one abstract method in DistanceSensor is getDistance(), which takes no parameters and returns a double. Any lambda expression that takes no parameters and returns a double can be treated like a DistanceSensor implementation. The following is a lambda expression that always returns a value of 36.0:
() -> { return 36.0; }
though since lambda expressions always return a value this can be simplified to:
() -> { 36.0 }
or even:
() -> 36.0
That means we can create and use a DistanceSensor whose getDistance() method always returns 36.0 (and whose getDistanceInFeet() always returns 3) as follows:
DistanceSensor sensor = ()->30.0;
Shooter shooter = new Shooter(sensor);
assertThat(shooter.isInRange()).isTrue();
But Java 8 lets us be even more concise, since we can pass the lambda directly into the Shooter(DistanceSensor) constructor:
Shooter shooter = new Shooter(()->30.0);
assertThat(shooter.isInRange()).isTrue();
So, using lambdas we can simplify our previous ShooterTest class a bit:
public class ShooterTest {
@Test
public void shouldNotBeInRangeIfDistanceIsLessThan10() {
assertThat(new Shooter(()->9.999).isInRange()).isFalse();
}
@Test
public void shouldBeInRangeIfDistanceIsBetween10And30Inclusive() {
for ( int i=10; i!=30; ++i) {
assertThat(new Shooter(()->(double)i).isInRange()).isFalse();
}
}
@Test
public void shouldNotBeInRangeIfDistanceIsGreaterThan30() {
assertThat(new Shooter(()->30.0001).isInRange()).isFalse();
}
@Test
public void shouldNotBeInRangeIfDistanceIsNegative() {
assertThat(new Shooter(()->-1.0).isInRange()).isFalse();
assertThat(new Shooter(()->-20.0).isInRange()).isFalse();
}
}
Granted, we'd probably only want to do this if Shooter is simple; if it takes multiple sensors or has other more complicated logic then it probably makes more sense to use mock objects.
The bottom line is that Java 8, functional interfaces, and lambda expressions make it a lot easier to implement Strongback interfaces with a lot less code. This is useful in test cases (where mocks are not required), but Strongback actually uses lambdas in many of its hardware implementations of the component interfaces. Here's some sample code that you might have in your robot to create a DistanceSensor backed by WPILib's Ultrasonic class and ultimately a hardware ultrasonic rangefinder that sends pings on port 4 and receives the echo on port 5:
DistanceSensor distance = Hardware.DistanceSensors.digitalUltrasonic(4,5);
Strongback actually implements this method as follows:
public static DistanceSensor digitalUltrasonic(int pingChannel, int echoChannel) {
Ultrasonic ultrasonic = new Ultrasonic(pingChannel, echoChannel);
ultrasonic.setAutomaticMode(true);
return ultrasonic::getRangeInches;
}
In other words, when your robot code calls this factory method, Strongback really instantiates WPILib's Ultrasonic class, and the resulting DistanceSensor object is really just a lambda that calls the Ultrasonic object's getRangeInInches() method. Java 8 does all the rest, and it does it in a very efficient manner. (It actually does not create an anonymous inner class that implements DistanceSensor, but instead really does pass the lambda expression around so that any code calling any of the DistanceSensor methods will really just call ultrasonic.getRangeInches(). Pretty amazing, right?)
Strongback makes it very easy to test your code, but there are some guidelines. First, subsystems should only use the component interfaces and should never directly use, cast, or rely upon any implementation classes. Restricting subsystems to the component interfaces helps clarify exactly the kinds of external information needed by the subsystem. (If Strongback doesn't have component interfaces for the components you need, create your own interfaces for them the same way that Strongback does. Or better yet, contribute them to Strongback so that others can benefit from your work!)
Secondly, the subsystems should never instantiate the component interfaces directly, but should instead be passed the objects. Only code that runs on the robot should create the hardware implementations for the components, and should pass these references into the subsystems. As we saw above, doing so makes it easy to instantiate subsystems in tests and have them use mock components.
Thirdly, the references to components within the subsystems should whenever possible be final. Once a subsystem has a reference to a component, it almost certainly should never change during the lifetime of the robot. The alternative -- having subsystems that can refer to different components over time -- makes your code more brittle and means your code is creating and discarding more objects than necessary, and this increases the likelihood that the JVM garbage collector might need to run and "pause the world" and your robot during a match. Final references also help prevent bugs by ensuring code can't mistakenly change references.
Fourth, subsystems should contain most of the logic specific to your robot, and they should expose methods that other code (such as commands and controllers) can use. Since subsystems are easily tested, you can test all of the complicated logic of a subsystem and make sure it behaves correctly.
Of course, subsystems are not the only code you want to test, but they do make up a substantial part of it. With Strongback, you can also test your Command and CommandGroup implementations with your real subsystems and mock components, or with your own mock implementations of subsystems. The latter requires you to create interfaces for your subsystems, but doing so may make it much easier to test individual commands. On the other hand, while using the real subsystem classes (and mock components) in your command tests might be more complicated, it might also be well worth it if it helps to further test the subsystem classes.
BTW, the WPILib command framework requires that subsystems actually extend the Subsystem abstract class. The latter is fairly complex and actually uses static references between Subsystem instances that can wreak havoc in unit tests by consuming memory and leaking state between tests. Strongback provides an alternative command framework that is more easily tested, and it uses a simple Requirable marker interface (with no methods) to determine the requirements for each command. (Like the WPILib framework, submitting a command that requires objects currently in use by other running commands may result in the interruption of the currently-running commands.) Therefore, if you're going to use the Strongback command framework, make sure that each of your subsystem classes/interfaces implement the Requirable interface.