junit testing vs system.out printing

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

slightlydead
Posts: 115
Joined: Wed Sep 23, 2009 10:52 pm UTC

junit testing vs system.out printing

Postby slightlydead » Wed May 11, 2011 3:48 pm UTC

It's probably that my methods are sloppy and I'm not used to junit testing yet, but so far I've found myself seeing that junit testing is kind of limiting for specific kinds of methods. And you have to kind of know what will break your calculations before you enter test cases? I'm finding myself using print lines and making my own test main methods instead of using junit. I'm 99 percent I'm not doing it right, as i've only read about junits basic uses in the tutorials I've found. How do you guys use junit testing fully?

Ankit1010
Posts: 135
Joined: Fri Feb 11, 2011 11:32 am UTC

Re: junit testing vs system.out printing

Postby Ankit1010 » Mon May 30, 2011 1:25 am UTC

The way I see it is that adding println's and stuff will let you figure out why your code is failing, whereas JUnit's just really tell you that its failing.
One of the things you can do with JUnit is run loops with a million inputs, and feed them into your program, and then use reference code and make sure they have the same output for every single input.

Parsifal
Posts: 113
Joined: Thu Feb 28, 2008 1:35 am UTC

Re: junit testing vs system.out printing

Postby Parsifal » Mon May 30, 2011 9:34 am UTC

You won't see the benefit of JUnit with just a few tests or even a few iterations of a single application. I maintain a code base of perhaps 300,000 lines right now and let me tell you, I'd much rather have JUnit run 400 tests every time I rebuild than do it myself. The main benefits of JUnit are

1) reproducibility (think continuous integration)
2) integration with other tools, e.g. HttpUnit
3) reporting (from jenkins, cruisecontrol etc.)

and so on.

It also encourages test-driven development, which is in my opinion the foremost benefit. You would be amazed at the number of programmers - I hesitate to call them developers - I have worked with over the years who not only don't test their code, they can't even tell me precisely what the code is supposed to do! Sure, I've gone overboard a few times trying to get 95% test coverage, but I can tell you from hard experience that it takes far fewer iterations to get an application right when you test as you go.

You don't need to know what will 'break' your application before you test. Success cases are every bit as useful because they help prevent regression errors from creeping into the code. I've also developed a habit of starting almost every bug fix with a new, failing unit test. That way, I know that once a bug is fixed it STAYS fixed.

User avatar
evilbeanfiend
Posts: 2650
Joined: Tue Mar 13, 2007 7:05 am UTC
Location: the old world

Re: junit testing vs system.out printing

Postby evilbeanfiend » Wed Jun 01, 2011 9:48 pm UTC

printing out stuff is more of a poor man's debugger than a poor man's unit test (though its actually logging)

in industry we would use unit test to find out what is failing and debuggers to find out why.

lastly there still is a place for 'printing stuff out' but we call it logging, it is primarily useful for deployed software especially if the end user is unable or unwilling to provide the data to reproduce a bug, logs may be the only thing you have to indicate what went wrong
in ur beanz makin u eveel

User avatar
Berengal
Superabacus Mystic of the First Rank
Posts: 2707
Joined: Thu May 24, 2007 5:51 am UTC
Location: Bergen, Norway
Contact:

Re: junit testing vs system.out printing

Postby Berengal » Thu Jun 02, 2011 5:15 pm UTC

Eh, printing stuff out is still my preferred way of debugging. Debuggers are too cumbersome to use.

(Also, debuggers aren't. The bugs are still there even if you run a debugger on it.)
It is practically impossible to teach good programming to students who are motivated by money: As potential programmers they are mentally mutilated beyond hope of regeneration.

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: junit testing vs system.out printing

Postby You, sir, name? » Fri Jun 03, 2011 12:05 am UTC

I resort to debuggers when print-debugging is fruitless.

Printing is easier to use as it produces a consistent diagnostic of the code. So if you make perturbations to locate the source of the problem, you can immediately compare the results between runs. The results of a debugger are much harder to compare between code changes.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

EvanED
Posts: 4331
Joined: Mon Aug 07, 2006 6:28 am UTC
Location: Madison, WI
Contact:

Re: junit testing vs system.out printing

Postby EvanED » Fri Jun 03, 2011 4:10 am UTC

Both techniques definitely have their uses. The problem with printf debugging is you have to know beforehand what you want to track, at least with almost all languages. With debuggers, you can decide more on-the-fly; when you're at a breakpoint, you can say "oh, this would be interesting to look at" and go look at it instead of recompiling your program and getting back to that point. And once time-traveling debuggers become more standard fare (I think they will in a few years), I suspect they'll blow printf debugging out of the water for most tasks.

Parsifal
Posts: 113
Joined: Thu Feb 28, 2008 1:35 am UTC

Re: junit testing vs system.out printing

Postby Parsifal » Fri Jun 03, 2011 10:45 am UTC

I think that coding a modular app and using test-driven development is the best route, although it does take a lot more practice and discipline. Interactive debugging is a close second, and in my experience most people who think troubleshooting that way is too hard simply don't understand how to make full use of the debugger. Changing the application to log to the console, then remembering to change it back before finishing the program, should be a method of last resort.

For what it is worth, I'm the only one in my software shop who does unit testing. Everyone else says it is too hard, or takes too long, or they just admit they don't 'get it'. Want to guess who they come to for any kind of non-trivial debugging, sometimes several times a day? This is not a coincidence - writing good unit tests teaches you to write better code. Being a better coder teaches you to write better tests.

In a nutshell : write to stdout when that is what the program is supposed to do, not to diagnose bugs. If you must log to a file, use a real logger (log4j, commons logging, java.util in the Java word) that can be enabled and disabled without rebuilding.

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: junit testing vs system.out printing

Postby Xeio » Sat Jun 04, 2011 1:03 am UTC

There is absolutely no comparison between printf and a good debugger. Period.

Setting breakpoints on the fly, conditional breakpoints, breakpoints that don't actually break and just print (literally obsoleting printing). Also, edit-and-continue is pretty neat (if you happen to be making changes that are compatible).

As for unit testing. We don't really use it where I work for much. There is a Unit Test solution, but I've never had to open it, and it's primarily used for architecture stuff. We do use logging pretty extensively in some of the logic in the architecture layers as well. I believe that QA has a lot of test scripts they run, but developers for the most part don't have a hand in them.

Rysto
Posts: 1460
Joined: Wed Mar 21, 2007 4:07 am UTC

Re: junit testing vs system.out printing

Postby Rysto » Sun Jun 05, 2011 2:58 pm UTC

Berengal wrote:(Also, debuggers aren't. The bugs are still there even if you run a debugger on it.)

I've chased down way too many bugs where this is not true. :|

stevecrox
Posts: 5
Joined: Tue Oct 14, 2008 9:18 pm UTC

Re: junit testing vs system.out printing

Postby stevecrox » Sun Jun 05, 2011 4:20 pm UTC

Whenever your write a class you should have worked out its possible input's and the expected outputs. JUnit is useful because you can write a series of tests which make sure when the class is given input A it will output B. Which may sound obvious but for example I've just written a 60K SLOC Eclipse RCP application which translates data from one format to another and our team is now disbanding. In the future a team could pick up my component and make a minor fix/enhancement which could break some minor aspect of that data translation. Because I've written JUnit tests for about 75% of the translation code (around 1500 so far) if a team were to do that it should get flagged in the next build kick off by Jenkins (a continuous integration tool)

I also use it to do regression testing, every bug I get given I work out which class is misbehaving then I write a JUnit test to reproduce that bug. I personally find it useful because it forces you to think about that section of code and I have occasionally spotted larger underlying problems that way. Plus regression tests stop people from re-introducing old bugs.

My next use of JUnit is to test a class can take bad input, so things like null, numbers outside the possible range, objects set in almost impossible configurations. The aim is to make sure the class doesn't die horribly, you should also note that you don't have to supply every possible bad input or every possible good input. If your method accepts int's between 0-100 you only really need to test 0, 50 & 100 I would also check -1 and 101 but anything else would be a waste of time.

My last use of JUnit is reproduction of system tests, basically I've often written a projects system tests as a series of unit tests. While this won't catch everything that running a system test would, these unit tests can help you be reasonably confident that a system test isn't going to fail. Not all system tests can translate to a series of unit tests and sometimes its better to just develop a system level test tool.

As for system.out.println I prefer Apache's Log4J, it does the same thing but you can set it to store the output to a file (with different packages reporting to different files), you can assign levels to the statements e.g. info, debug, error, etc.. and when Log4J output's it gives you class, line and system time. Log4J is helpful on larger projects or times when a problem may occur on the 2nd or 3rd time around but not the first. I also use it for debugging reasons when checking for a valid value if I get a invalid value I will output a log4J comment.

You should also learn to use the debugger as well, system.out may tell you where the problem is but often it can be faster to chuck a breakpoint near there and step through the code.

Just saw Evilbean's post, I agree with him you wouldn't believe how hard it can be to get information out of a user.

User avatar
evilbeanfiend
Posts: 2650
Joined: Tue Mar 13, 2007 7:05 am UTC
Location: the old world

Re: junit testing vs system.out printing

Postby evilbeanfiend » Wed Jun 08, 2011 8:39 pm UTC

Berengal wrote:Eh, printing stuff out is still my preferred way of debugging. Debuggers are too cumbersome to use.

(Also, debuggers aren't. The bugs are still there even if you run a debugger on it.)


*Good* debuggers are easy to use. If you are adding new code to diagnose something you either need to remember to remove it again (a risk i would rather not take) or make sure it is ok to leave it in production code i.e. log it properly, which may start to pull in other issues e.g. documentation may need updating, log message may need to be localized etc. course that will depend on what you are writing and to what standard
in ur beanz makin u eveel

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: junit testing vs system.out printing

Postby You, sir, name? » Thu Jun 09, 2011 6:45 am UTC

Code: Select all

#define DEBUG


Problem of finding and removing debug code solved.

You can now slap DEBUG behind any debug line or debug block. When you don't want it, you can define it to "if(0)", or easily identify the debugging code with grep.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
evilbeanfiend
Posts: 2650
Joined: Tue Mar 13, 2007 7:05 am UTC
Location: the old world

Re: junit testing vs system.out printing

Postby evilbeanfiend » Fri Jun 10, 2011 8:04 am UTC

You, sir, name? wrote:

Code: Select all

#define DEBUG


Problem of finding and removing debug code solved.

You can now slap DEBUG behind any debug line or debug block. When you don't want it, you can define it to "if(0)", or easily identify the debugging code with grep.


well you are 80% of the way to just using proper logging now anyway, and there are certainly plenty of reasons to do that. plus there are still downsides, (assuming you leave the code in) you are making the actual code less readable by filling it with debug or logging stuff so it aint just a matter of shove debugging info/logging everywhere and forgetting.

I'm not sure i can really conceive of a debugger which is harder to use than writing debug out wrapped in preprocessor statements, possibly including header/module to support the output (also in preprocessor statements) and then grepping the source code to remove it all afterwards. sure its possible, but is it really better than just adding a breakpoint?
in ur beanz makin u eveel

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: junit testing vs system.out printing

Postby You, sir, name? » Fri Jun 10, 2011 11:00 am UTC

evilbeanfiend wrote:I'm not sure i can really conceive of a debugger which is harder to use than writing debug out wrapped in preprocessor statements, possibly including header/module to support the output (also in preprocessor statements) and then grepping the source code to remove it all afterwards. sure its possible, but is it really better than just adding a breakpoint?


Ever heard of gdb?

Or hard is the wrong word. Inconvenient is better.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.

User avatar
Xeio
Friends, Faidites, Countrymen
Posts: 5101
Joined: Wed Jul 25, 2007 11:12 am UTC
Location: C:\Users\Xeio\
Contact:

Re: junit testing vs system.out printing

Postby Xeio » Fri Jun 10, 2011 11:39 am UTC

You, sir, name? wrote:Ever heard of gdb?
<- So spoiled by Visual Studio's debugger.

User avatar
evilbeanfiend
Posts: 2650
Joined: Tue Mar 13, 2007 7:05 am UTC
Location: the old world

Re: junit testing vs system.out printing

Postby evilbeanfiend » Fri Jun 10, 2011 9:16 pm UTC

You, sir, name? wrote:
evilbeanfiend wrote:I'm not sure i can really conceive of a debugger which is harder to use than writing debug out wrapped in preprocessor statements, possibly including header/module to support the output (also in preprocessor statements) and then grepping the source code to remove it all afterwards. sure its possible, but is it really better than just adding a breakpoint?


Ever heard of gdb?

Or hard is the wrong word. Inconvenient is better.


have no problem using gdb via emacs gdb mode.
in ur beanz makin u eveel

Derek
Posts: 2181
Joined: Wed Aug 18, 2010 4:15 am UTC

Re: junit testing vs system.out printing

Postby Derek » Sat Jun 11, 2011 10:33 am UTC

I used to do lots of printf debugging. It gets the job done for simple problems, but its really not a great habit. Since then I've mostly used gdb, but recently I've been using Visual Studios and it is so nice. There is really no comparison to what I've used before.

User avatar
You, sir, name?
Posts: 6983
Joined: Sun Apr 22, 2007 10:07 am UTC
Location: Chako Paul City
Contact:

Re: junit testing vs system.out printing

Postby You, sir, name? » Sat Jun 11, 2011 5:52 pm UTC

evilbeanfiend wrote:
You, sir, name? wrote:
evilbeanfiend wrote:I'm not sure i can really conceive of a debugger which is harder to use than writing debug out wrapped in preprocessor statements, possibly including header/module to support the output (also in preprocessor statements) and then grepping the source code to remove it all afterwards. sure its possible, but is it really better than just adding a breakpoint?


Ever heard of gdb?

Or hard is the wrong word. Inconvenient is better.


have no problem using gdb via emacs gdb mode.


But then you have to use emacs. Which I have a problem with.
I edit my posts a lot and sometimes the words wrong order words appear in sentences get messed up.


Return to “Coding”

Who is online

Users browsing this forum: No registered users and 11 guests