There are two recommended ways of setting up testing on code and resources in a library project:

• You can set up a test project that instruments an application project that depends on the library project. You can then add tests to the project for library-specific features.
• You can set up a set up a standard application project that depends on the library and put the instrumentation in that project. This lets you create a self-contained project that contains both the tests/instrumentations and the code to test.

How to achieve the first of these is pretty obvious, but the second (to me, at least) rather less so. I wasn’t able to find an example, so I thought that I’d post how I managed to get it working.

Although it does work, I’m not 100% happy with it, and there may well be a nicer way to achieve this (see the end of this post for a discussion about my concerns). I would be very grateful for any suggestions for improvements.

Note, we don’t build with Eclipse (partly because we’ve found its reliability leaves a lot to be desired, but mainly because we need a command line build in order to be able to integrate with the rest of our build system and continuous integration server). So all of the following uses the Android command line tools.

The steps

1. Create a new library project as follows:

   android create lib-project -n ExampleLib -t android-8 \
   -p examplelib -k com.example.lib

2. In that project, create src/com/example/lib/Widget.java containing:

   package com.example.lib;
    
   public class Widget {
       public String getColour() {
           return "blue";
       }
    
       public String getDisposition() {
           return "awesome";
       }
   }

3. In the examplelib directory, create a test project with:

   android create test-project -m .. -p test -n ExampleTest

4. Edit the AndroidManifest.xml in the test project and change the line:

   android:targetPackage="com.example.lib"

   to:

   android:targetPackage="com.example.lib.tests"

5. Delete the following line from the build.properties in the test project:

   tested.project.dir=..

   and add the following:

   android.library.reference.1=..

6. Delete the src/com/example/lib/ACTIVITY_ENTRY_NAMETest.java file.
7. Create src/com/example/lib/WidgetTest.java containing:

   package com.example.lib;
    
   import junit.framework.TestCase;
    
   public class WidgetTest extends TestCase {
    
       private Widget widget;
    
       @Override
       protected void setUp() {
           widget = new Widget();
       }
    
       public void testColour() {
           assertEquals("blue", widget.getColour());
       }
    
       public void testDisposition() {
           assertEquals("awesome", widget.getDisposition());
       }
   }

8. Start an emulator and install with ant install.
9. Run the tests with:

   adb shell am instrument \
   -w com.example.lib.tests/android.test.InstrumentationTestRunner

Reservations

It’s clear from the above that the Android tools (the command line tools, at least—perhaps the Eclipse plugin is better?) don’t really fully support this way of working. Not least the fact that they assume that we’re always testing an activity in another application (hence the strangely named ACTIVITY_ENTRY_NAMETest.java file).

1. In step 4, we changed the targetPackage. This is necessary because if we leave it as com.example.lib, Android will try to launch an application with that package name, which doesn’t exist. However, it’s something of a lie to suggest that the targetPackage is com.example.lib.tests
2. In a normal Android test project, we can run the tests by typing:

   ant run-tests

   We have to run the tests explicitly with adb, however, because in step 5, we deleted the tested.project.dir line. If we don’t, the ant build system will try to rebuild the library project (and library projects can’t be build independently).

   An alternative to the solution presented above is to set tested.project.dir to the current directory (.). This enables ant run-tests, but at the expense of building the project twice.

So, it all works, but it’s a bit messy. Suggestions for improvements very welcome!

Update

Updated to simplify WidgetTest to use JUnit directly instead of InstrumentationTestCase.

This book is highly recommended as it contains a whole lot of wisdom and experience from the field of software engineering. It’s only 190 pages, so you should be able to get through very quickly. So, you can’t lose on this one.

— ElegantCode

For more reviews, see Debug It! Review Roundup.

I think that this book should be read by all new software developers as well as all junior or mid-level developers. There’s plenty to be learned by the senior developer types too. Even if you’ve been doing these things for a long time, this is a good read to make sure you’re keeping your feet planted firmly on the ground.

— lostechies.com

For more reviews, see Debug It! Review Roundup.

So I thought that I’d create this page and hopefully save anyone else looking for the same information some time.

$ scalac -version
Scala compiler version 2.8.0.RC1 -- Copyright 2002-2010, LAMP/EPFL

Advanced Options

$ scalac -X
Usage: scalac <options> <source files>
Possible advanced options include:
  -Xassem-extdirs <dirs>         List of directories containing assemblies, defaults to `lib'
  -Xassem-name <file>            Name of the output assembly (only relevant with -target:msil)
  -Xassem-path <path>            List of assemblies referenced by the program (only relevant with -target:msil)
  -Xcheck-null                   Emit warning on selection of nullable reference
  -Xcheckinit                    Add runtime checks on field accessors. Uninitialized accesses result in an exception being thrown.
  -Xdisable-assertions           Generate no assertions and assumptions
  -Xelide-below                  Generate calls to @elidable-marked methods only if method priority is greater than argument.
  -Xexperimental                 Enable experimental extensions
  -Xfuture                       Turn on future language features
  -Xgenerate-phase-graph <file>  Generate the phase graphs (outputs .dot files) to fileX.dot
  -Xlog-implicits                Show more info on why some implicits are not applicable
  -Xmigration                    Warn about constructs whose behavior may have changed between 2.7 and 2.8
  -Xno-forwarders                Do not generate static forwarders in mirror classes
  -Xno-uescape                   Disables handling of \u unicode escapes
  -Xnojline                      Do not use JLine for editing
  -Xplugin-disable:<plugin>      Disable a plugin
  -Xplugin-list                  Print a synopsis of loaded plugins
  -Xplugin-require:<plugin>      Abort unless a plugin is available
  -Xplugin:<file>                Load a plugin from a file
  -Xpluginsdir <path>            Path to search compiler plugins
  -Xprint-icode                  Log internal icode to *.icode files
  -Xprint-pos                    Print tree positions (as offsets)
  -Xprint-types                  Print tree types (debugging option)
  -Xprint:<phase>                Print out program after <phase> or "all"
  -Xprompt                       Display a prompt after each error (debugging option)
  -Xresident                     Compiler stays resident, files to compile are read from standard input
  -Xscript <object>              Compile as a script, wrapping the code into object.main()
  -Xshow-class <class>           Show class info
  -Xshow-object <object>         Show object info
  -Xshow-phases                  Print a synopsis of compiler phases
  -Xsource-reader <classname>    Specify a custom method for reading source files
  -Xsourcedir <directory>        When -target:msil, the source folder structure is mirrored in output directory.
  -Xstrict-warnings              Emit warnings about lots of things.
  -Xwarninit                     Warn about possible changes in initialization semantics
  -Y                             Print a synopsis of private options

Private Options

$ scalac -Y
Usage: scalac <options> <source files>
Possible private options include:
  -Ybrowse:<phase>               Browse the abstract syntax tree after <phase> or "all"
  -Ybuild-manager-debug          Generate debug information for the Refined Build Manager compiler.
  -Ybuilder-debug:<method>       Compile using the specified build manager (none,refined,simple)
  -Ycheck:<phase>                Check the tree at the end of <phase> or "all"
  -Yclosure-elim                 Perform closure elimination
  -Ycompact-trees                Use compact tree printer when displaying trees
  -Ydead-code                    Perform dead code elimination
  -Ydebug                        Output debugging messages
  -Ydetach                       Perform detaching of remote closures
  -Yfatal-warnings               Fail the compilation if there are any warnings.
  -Yide-debug                    Generate, validate and output trees using the interactive compiler.
  -Yinline                       Perform inlining when possible
  -Yjenkins-hashCodes            Use jenkins hash algorithm for case class generated hashCodes.
  -Ylinearizer:<which>           Linearizer to use (normal,dfs,rpo,dump)
  -Ylog-classpath                Output information about what classpath is being applied.
  -Ylog:<phase>                  Log operations in <phase> or "all"
  -Yno-completion                Disable tab-completion in the REPL
  -Yno-generic-signatures        Suppress generation of generic signatures for Java
  -Yno-imports                   Compile without any implicit imports
  -Yno-predefs                   Compile without any implicit predefined values
  -Ypmat-debug                   Trace all pattern matcher activity.
  -Ypmat-naive                   Desugar matches as naively as possible..
  -Yrangepos                     Use range positions for syntax trees.
  -Yrecursion                    Recursion depth used when locking symbols
  -Yrepl-debug                   Trace all repl activity.
  -Yself-in-annots               Include a "self" identifier inside of annotations
  -Yshow-trees                   Show detailed trees when used in connection with -print:phase
  -Yskip:<phase>                 Skip <phase> or "all"
  -Ysqueeze:<enabled>            if on, creates compact code in matching (on,off)
  -Ystatistics                   Print compiler statistics
  -Ystop:<phase>                 Stop after phase <phase> or "all"
  -Ystruct-dispatch:<method>     Selects dispatch method for structural refinement method calls (no-cache,mono-cache,poly-cache,invoke-dynamic)
  -Ysuppress-vt-typer-warnings   Suppress warnings from the typer when testing the virtual class encoding, NOT FOR FINAL!
  -Ytailrecommend                Alert methods which would be tail-recursive if private or final.
  -Ytyper-debug                  Trace all type assignements
  -Ywarn-catches                 Emit warnings about catch blocks which catch everything.
  -Ywarn-dead-code               Emit warnings for dead code
  -Ywarn-shadowing               Emit warnings about possible variable shadowing.

1. Install sbt
2. Create a new project as follows:

   $ mkdir aproject
   $ cd aproject/
   $ sbt
   Project does not exist, create new project? (y/N/s) y
   Name: aproject
   Organization: com.example
   Version [1.0]: 
   Scala version [2.7.7]: 2.8.0.RC1
   sbt version [0.7.3]:

3. Create a build configuration file in project/build/AProject.scala containing:

   import sbt._ 
    
   class AProject(info: ProjectInfo) extends DefaultProject(info) { 
     val scalaToolsSnapshots = ScalaToolsSnapshots
     val scalatest = "org.scalatest" % "scalatest" %
       "1.0.1-for-scala-2.8.0.RC1-SNAPSHOT"
   }

   This tells sbt that you will be using the pre-release snapshot version of ScalaTest for Scala 2.8.0.RC1.

4. Download the ScalaTest library with sbt update. After running this, you should see that you have a lib_managed directory containing the ScalaTest jar.
5. Create src/main/scala/Widget.scala containing:

   package com.example
    
   class Widget {
     def colour = "Blue"
     def disposition = "Awesome"
   }

6. Create src/test/scala/WidgetSpec.scala containing:

   package com.example.test
    
   import org.scalatest.Spec
   import com.example.Widget
    
   class WidgetSpec extends Spec {
    
     describe("A Widget") {
    
       it("should be blue") {
         expect("Blue") { new Widget().colour }
       }
    
       it("should be awesome") {
         expect("Awesome") { new Widget().disposition }
       }
     }
   }

7. Run your tests with sbt test. You should see:

   [info] == com.example.test.WidgetSpec ==
   [info] A Widget
   [info] Test Starting: A Widget should be blue
   [info] Test Passed: A Widget should be blue
   [info] Test Starting: A Widget should be awesome
   [info] Test Passed: A Widget should be awesome
   [info] == com.example.test.WidgetSpec ==

8. Create src/main/scala/Main.scala containing:

   package com.example
    
   object Main {
     def main(args: Array[String]) {
       val w = new Widget()
       println("My new widget is "+ w.colour)
     }
   }

9. Run your program with sbt run. You should see:

   [info] Running com.example.Main 
   My new widget is Blue

Update: Use the pre-defined ScalaToolsSnapshots repo (thanks to Erick Fleming).

Update: Fixed a typo in step 8 (thanks to Jason).

A fuzz test interacts with the System Under Test in a random, yet structured manner. The object of the exercise is to construct sequences of API calls (or network traffic, or whatever) that flush out the kind of bugs that are less likely to be discovered by unit tests tests and the like. Fuzz tests are particularly good at finding things like security problems, buffer overruns, memory leaks etc.

Clearly randomness is a large part of a fuzz test, so I’m creating some utilities that make it easier to randomly choose between several different actions. I thought that I’d publish them just in case anyone else finds them useful, and also to solicit feedback to help me improve my use of Scala (bear in mind that I’m still finding my feet, so be gentle!).

So, here’s my first attempt:

val random = new Random
 
def oneAtRandom(actions: () => Unit*)() =
  () => actions(random.nextInt(actions.length))()

oneAtRandom takes a set of functions and returns another function which, each time it’s called, executes one of them at random. For example:

val randomAction = oneAtRandom(
  () => print('a'), () => print('b'), () => print('c'))
 
for(_ <- 1 to 10)
  randomAction()

Which gives the following output:

aabaacbbcc

So far, so good. But what if we want different functions to be executed with different frequencies? We might have some actions that we want to only happen occasionally? Here’s another version of oneAtRandom that takes a weighted set of functions and calls them in proportion to their relative weights:

def oneAtRandom(actions: (Int, () => Unit)*)() = {
  val totalWeight = actions.foldLeft(0) {_ + _._1}
  () => {
    val r = random.nextInt(totalWeight)
    var w = 0
    val (_, action) = actions.find {
        case(weight, _) => w += weight; w > r
      }.get
    action()
  }
}

Here’s how it’s used:

val randomAction = oneAtRandom(
    (1, () => println("rare")),
    (10, () => println("occasional")),
    (50, () => println("frequent"))
  )
 
for(_ <- 1 to 100)
  randomAction()

And to prove that it’s doing what it claims:

$ scala random2.scala | sort | uniq -c
  88 frequent
  11 occasional
   1 rare
 
$ scala random2.scala | sort | uniq -c
  82 frequent
  12 occasional
   6 rare

So I’ve got it working, which I’m happy about. But I’m sure that it could be improved. I’m sure, for example, that there must be a “nice” functional way to implement the loop inside the weighted version (instead of using var w as an accumulator), but each of my attempts to do so have ended up being considerably more complex than the current version.

I’d also love to use the -> notation to pass the arguments, for example:

val randomAction = oneAtRandom(
    1 -> () => println("rare"),
    10 -> () => println("occasional"),
    50 -> () => println("frequent")
  )

But doing so results in a parse error—the only way I’ve found to avoid it is to add additional parentheses:

val randomAction = oneAtRandom(
    1 -> (() => println("rare")),
    10 -> (() => println("occasional")),
    50 -> (() => println("frequent"))
  )

which is starting to feel a bit too much like Lisp for my taste

Finally, I’m really missing some of the handy little utilities provided by Ruby. In Ruby, for example, I could write the loops that call randomAction like this:

10.times { randomAction }

I know that it’s trivial to create this kind of utility, but one of the nice things about Ruby is that they all come as standard out of the box.

In any case, I’d be very grateful for any and all feedback on the above. Thanks in advance!

It’s clear that the author’s years of debugging experience were superbly distilled into this collection of high impact advice.

— www.drdobbs.com

A must have book for anyone writing programs.

— www.felgall.com

Perhaps it’s the fact that it’s Paul Butcher’s first book that gives Debug It! its remarkable charm and conviction. The fact that very few other books cover the theme on such an intellectual and psychological level undoubtedly helps set it apart from the few books currently out there too. Either way you look at it, Paul’s take on debugging is certainly essential reading for those looking to clean up their act and, more importantly, their code.

— www.linuxuser.co.uk

Overall, this book is going to be a huge win, and I think it’s a worthy successor to the Release It! reputation. Development managers and team leads should get a copy for the junior developers on their team as a Christmas gift, but only after the senior developers have read through it as well.

— blogs.tedneward.com

Excellent discussion of the strategy of debugging. Buy, it read it and kill bugs.

— i-programmer.info

Frankly, I wish this book would be read by a lot of developers

— www.jasonbock.net

This book is highly recommended as it contains a whole lot of wisdom and experience from the field of software engineering. It’s only 190 pages, so you should be able to get through very quickly. So, you can’t lose on this one.

— ElegantCode

I think that this book should be read by all new software developers as well as all junior or mid-level developers. There’s plenty to be learned by the senior developer types too. Even if you’ve been doing these things for a long time, this is a good read to make sure you’re keeping your feet planted firmly on the ground.

— lostechies.com

With my many years of experience in supporting and debugging large existing enterprise systems, I have to say that Paul Butcher summarize and structure all the knowledge (and more) that I have, sometimes painfully, accumulated during this activity. This is therefore an excellent book that I will recommend to everybody that is involved in software development in general and maintenance activities specifically.

— blog.martinig.ch

I know that this book influenced the way I work now, and there aren’t many books I could say something like this about.

— devlicio.us

All in all, there is just a lot of tremendously valuable information in this book. And it’s only about 190 pages so it definitely won’t take you a long time to read it. I’ve frequently been amazed at the inability of developers to efficiently debug issues when they occur. And i’m not just talking about bad developers. I’ve seen plenty of good or even great developers having trouble with debugging efficiently. This book would definitely get them on the right track, with just a little bit effort.

— davybrion.com/blog

Understanding is everything: that is at the heart of Paul Butcher’s comprehensive study of the science and psychology of debugging.

— softtalkblog.wordpress.com

The book does a great (and truly pragmatic) job of covering all these aspects and addressing a wide range of topics related to debugging software.

— hamagudi.com

If you are too busy to read this delightful book in its entirety, then at least read the final chapter whilst mandating that your entire development team read “Debug It!” from cover to cover.

— javablog.co.uk

As usual, the publisher has an excellent book on a practical subject that answers oractical qustions on debugging.

— matt.eifelle.com

I would highly recommend Debug It! to any junior-level programmer who’s interested in developing a more disciplined approach to debugging. If you’re not a junior-level programmer but still feel like you waste a lot of time debugging, you will probably find this book helpful as well. It’s like having a mentor sitting there with you, teaching you how to take your debugging game to the next level.

— lylejohnson.name

It’s really good seeing these ideas in words because it’s quite easy to forget about the best way to approach problems in the heat of the moment and the approaches suggested by Paul certainly aren’t done everywhere in my experience.

— www.markhneedham.com

While I was familiar with many of the practices discussed in the book … I learned quite a few new things.

— starglider.blogspot.com

It does a great job of setting the scene for debugging and getting you into the right mind set, while also talking about the complications that can arise once the bug is found and squashed. It’s almost worth looking at for the anecdotes alone, to understand the lengths that you sometimes have to go to when trying to understand some truly bizarre defects.

— blog.jonmdickinson.com

This is a recommended read for anyone who works closely in the software industry, be it a developer, a tester or even a product manager.

— vidhujoshua.blogspot.com

Debug It! is making its way around the world, as this review by Luiz Marques in Brazil demonstrates.

Luiz is kind enough to say:

While I was familiar with many of the practices discussed in the book … I learned quite a few new things.

He goes on to say:

Well worth the time, specially as it is not very long.

Many thanks, Luiz.

What if I were to tell you that in the first unit, which on the face of things makes 10 times fewer mistakes, nurses don’t report errors because they’re concerned that “heads will roll.” Would that change how you feel? How many errors are being swept under the carpet? Do you think that they’re likely to be learning from their mistakes or repeating them over and over again? (This example comes from Hard Facts, Dangerous Half-Truths & Total Nonsense by Jeffrey Pfeffer and Robert I. Sutton, Harvard Business School Press, ISBN: 1-59139-862-2.)

In software, we have our own name for mistakes—we call them bugs. And every bug is an opportunity to learn.

To read the rest of this article, take a look at Issue 6 of PragPub magazine