“In God, we trust. In all else, we test.” (Unknown)
Testing is a vital aspect of software development. It is also a very complex discipline within software engineering. Definitions may vary and even overlap, but there are two primary categories for testing:
- Functional – does the software work correctly
- Performance – does the software work within desired metrics
Functional tests yield a deterministic pass/fail result. They can be written at many layers:
- Unit – very short tests for individual “units” (functions, methods, or classes) of code
- Integration – tests that cover where two components come together
- End-to-End – lengthier tests that cover an execution path through a system
These layers form the Testing Pyramid. Unit tests should be white-box tests written in the same language as the product code and run automatically as part of product builds with near-100% code coverage. Integration and end-to-end tests should be black-box tests run against live, deployed product builds. They are more complex and require more investment.
Performance tests yield measurement under given system conditions to be evaluated based on ranges of acceptability (example: < 500ms response time). Performance tests often need special tools or instrumentation like JMeter.
There are two ways to run tests:
- Manual – a person performs test steps themselves
- Automated – special software is developed to automatically run tests
Test automation is my professional specialty. It makes it possible to reliably run tests automatically and repeatedly, which is essential for continuous integration and continuous delivery. However, not all tests can or should be automated due to varying returns-on-investment. Furthermore, manual exploratory testing is a better way to discover unexpected problems or edge cases. The two approaches complement each other: automated testing is best for defensive test procedures, and manual testing is best for offensive exploration.
- 10 Gotchas for Automation Code Reviews
- 10 Things You Lose Without Automation
- Please Hang Up and Dial Again: Handling Test Interruptions in CI/CD
- Quality Metrics 101: The Good, The Bad, and The Ugly
- Test Automation Myth-Busting
- The Best Programming Language for Test Automation
- To Infinity and Beyond: A Guide to Parallel Testing
- Unpredictable Test Data
- Why is Automation Full of Duplicate Code?
Testing is a big part of behavior-driven development. Check out the BDD page for more information.
Lessons from Video Games
These fun posts join my love for gaming with my passion for software testing: