Add testing documentation.

Signed-off-by: Eric Paniagua <epaniagua@google.com>
pull/898/head
Eric Paniagua 4 years ago
parent 517856ad66
commit 87b2d5091f

@ -7,6 +7,10 @@ It is, as yet, incredibly incomplete.
Entries should be included in dictionary order. When describing an identifier of any kind from the codebase, it should be Entries should be included in dictionary order. When describing an identifier of any kind from the codebase, it should be
written using identical capitalization to its use in the code and surrounded by backticks: `identifierName` written using identical capitalization to its use in the code and surrounded by backticks: `identifierName`
## Firmware Terminology
These terms commonly arise when discussing the firmware.
### Cycle ### Cycle
The `loop` method in one's sketch file is the heart of the firmware. It runs - The `loop` method in one's sketch file is the heart of the firmware. It runs -
@ -40,3 +44,54 @@ A point where the core firmware calls [event handlers](#event-handler), allowing
An Arduino library prepared to work with Kaleidoscope. They implement methods An Arduino library prepared to work with Kaleidoscope. They implement methods
from the `kaleidoscope::Plugin` (usually a subset of them). See [event from the `kaleidoscope::Plugin` (usually a subset of them). See [event
handlers](#event-handler) above for a list of methods. handlers](#event-handler) above for a list of methods.
## Testing
These terms arise when discussing the testing framworks and related tests.
### Sim Harness
An abstraction used in testing to inject events into or invoke actions on the
simulated firmware. This abstraction comprises half of the interface to the
[test simulator](#test-simulator).
### Sim State
An abstraction used in testing to encapsulate, snapshot, and examine firmware
state. This abstraction comprises half of the interface to the [test
simulator](#test-simulator).
### Test
An indivial assertion or expectation that must hold for a [test
case](#test-case) to pass.
### Test Case
An indivual `TEST*` macro invocation. Its body consists of one or [tests](#test)
and optionally other code, e.g. to invoke the [test harness](#test-harness).
Note that gtest uses the non-standard term _Test_ for what we call a [Test
Case](#test-case).
### Test File
An individual file containing one or more [test suites](#test-suite).
### Test Fixture
A class comprising setup, teardown, and other code and common state to make
writing [test cases](#test-case) easieer. A fresh object of the fixture class
associated with a [test suite](#test-suite) is constructed for each run of each
[teset case](#test-case) in the [test suite](#test-suite).
### Test Simulator
An abstraction wrapping a virtual firmware build that allows performing
actions against the virtual firmware and reading state out of the virtual
firmware. The interface to the test simular is comprised of the [sim
harness](#sim-harness) and the [sim state](#sim-state).
### Test Suite
A collection of related [test cases](#test-case), optionally with an associated
[test fixture](#test-fixture).

@ -0,0 +1,56 @@
# Automated Testing
## Testing with gtest/gmock
Before feature requests or bug fixes can be merged into master, the folowing
steps should be taken:
- Create a new test suite named after the issue, e.g. `Issue840`.
- Add a test case named `Reproduces` that reproduces the bug. It should fail if
the bug is present and pass if the bug is fixed.
- Merge the proposed fix into a temporary testing branch.
- The reproduction test should fail.
- Add a test called "HasNotRegresed" that detects a potential regression.
It should pass with the fix and fail before the fix.
- Comment out and keep the `Reproduces` test case as documentation.
For an example, see keyboardio:Kaleidoscope/tests/issue_840.
### Adding a New Test Case
For general information on writing test with gtest/gmock see [gtest
docs](https://github.com/google/googletest/tree/master/googletest/docs) and
[gmock docs](https://github.com/google/googletest/tree/master/googlemock/docs).
1. Create a new test file, if appropriate.
1. Choose a new test suite name and create a new test fixture, if appropriate.
1. Write your test case.
The final include in any test file should be `#include
"testing/common/setup-googletest"` which should be followed by the macro
invocation `SETUP_GOOGLETEST()`. This will take care of including headers
commonly used in tests in addtion to gtest and gmock headers.
Any test fixtures should inherit from `VirtualDeviceTest` which wraps the test
sim APIs and provides common setup and teardown functionality. The appropriate
header is already imported by `setup-googletest.h`
### Test Infrastructure
If you need to modify or extend test infrastructure to support your use case,
it can currently be found under `keyboardio:Kaleidoscope/testing`.
### Style
TODO(obra): Fill out this section to your liking.
You can see samples of the desired test style in the [example tests](#examples).
### Examples
All existing tests are examples and may be found under
`keyboardio:Kaleidoscope/tests`.
## Testing with Aglais/Papilio
TODO(obra): Write (or delegate the writing of) this section.

@ -2,6 +2,8 @@
Before a new release of Kaleidoscope, the following test process should be run through on all supported operating systems. Before a new release of Kaleidoscope, the following test process should be run through on all supported operating systems.
Always run all of the [automated tests](automated-testing.md) to verify there are no regressions.
(As of August 2017, this whole thing really applies to Model01-Firmware, but we hope to generalize it to Kaleidoscope) (As of August 2017, this whole thing really applies to Model01-Firmware, but we hope to generalize it to Kaleidoscope)
# Tested operating systems # Tested operating systems
Loading…
Cancel
Save