What is software testing about? Finding bugs in a computer program (or more generally)Making sure a computer program is fit for some purpose (or more generally)Exploring the behavior of a computer program, in order to better understand what it should be doing and of course what it actually is doing Usually, we add that "testing"Continue reading "A Very Brief Overview of All of Software Testing"
Welcome
This blog will serve two purposes. First, it will be a testing ground for content for an eventual book on software testing (and analysis in general) whose structure and approach will be in large part based on George Polya's How To Solve It. This way I can write up the rough ideas in a wayContinue reading "Welcome"
A Brief Primer on Using TSTL and DeepState In Docker
You will need to install Docker! At a command line (I will not talk about how to use the Docker GUI!): docker pull agroce/deepstate_examples_aflpp docker run -it agroce/deepstate_examples_aflpp This second command will start a container running. Inside this "machine" you can work with a completely installed DeepState. It also works well to run TSTL andContinue reading "A Brief Primer on Using TSTL and DeepState In Docker"
Letting A Thousand Flowers Bloom: The Uses of Diversity in Software Testing
"Diversity's the spice of life,That gives it all its flavour."- William Cowper, "The Task" Discovering all the bugs in a software system is an extremely difficult task. It is so difficult, in fact, that in the real world it is seldom really attempted, for most kinds of software. However, when software bugs can produce catastrophicContinue reading "Letting A Thousand Flowers Bloom: The Uses of Diversity in Software Testing"
Oracles and Mutation Testing: Adding Bugs to Code for Fun and Profit
Once you understand the concept of code coverage and why it might be (probably is!) helpful, one of the objections to code coverage starts to become more prominent than the others. This objection is also naturally brought to mind by the habitual use of fuzzers like afl. Namely, you can run code all day long,Continue reading "Oracles and Mutation Testing: Adding Bugs to Code for Fun and Profit"
A Brief Overview of Code Coverage
What is code coverage? Code coverage is a way to talk, at a high level, about what a run of a program "did." In particular, the most common kind of coverage is statement coverage: Which "parts" of my code (in terms, usually, of "lines of code") ran? The other kind of coverage anyone but softwareContinue reading "A Brief Overview of Code Coverage"
A Whirlwind Introduction to Static Analysis Concepts and Implementation
Rather than spend the weeks (or entire semester) we could spend on static analysis, I want to give you a very quick overview of how (roughly) static analysis tools work. First, the basic idea of static analysis is that it analyzes source code without running the code. This has advantages, but also serious disadvantages, whenContinue reading "A Whirlwind Introduction to Static Analysis Concepts and Implementation"
Basic Unit Tests, and Two Ways to Generalize Them
Most of you are probably familiar with the basic idea of a unit test. Let's take a very simple unit test, without worrying about language or unit testing framework details: TEST(RoundTrip, BasicRoundTrip) { String s1 = "{\"name\": \"Harry Lime\", \"job\": \"cuckoo clock repairman\"}"; JSONObject j1 = JSON.toJSON(s1); String s2 = JSON.fromJSON(j1); JSONObject j2 = JSON.toJSON(s2);Continue reading "Basic Unit Tests, and Two Ways to Generalize Them"
An Overview of Python “Assurance” Tools
There are two basic types of automated analysis to help us assure/reassure ourselves that our code is correct (or at least correct enough) that we are going to care about in this class: static and dynamic. Static analysis involves looking for bugs by "scanning" the source code. The underlying algorithm used to look for bugsContinue reading "An Overview of Python “Assurance” Tools"
Testing and Philosophy
https://cdn.pixabay.com/photo/2020/10/12/21/20/plato-5650237_1280.png Our topic is testing, broadly conceived. By "broadly conceived" I mean that there is a narrow definition of testing that's important to know: executing a program with given inputs, to observe a possible failure. That kind of testing can be formally thought of in its simplest form as defining some function test: () ->Continue reading "Testing and Philosophy"
Testing Embedded Systems
What's so special about testing embedded systems? In one sense: nothing. If you are well-versed in the general ideas of aggressive, diligent, smart software testing, and know how to apply those ideas to "normal" software, you can probably apply those ideas to embedded systems. The only likely obstacle would be a lack of experience withContinue reading "Testing Embedded Systems"
How to Test It 
