Java Testing Principles Java Tips

General Java Testing Principles

In this chapter, we discuss the things that can be done to specifically augment the testing process.  In particular, we show how each part of the project team interacts with other members in various ways to provide helpful input for the both the testing process and test execution.

As we describe each of the functional areas? needs, we attempt to define the necessary interactions, tasks, and communications needed between the various groups during the debug and test phase of the project.  We also note important aspects of the planning and design phase that relate back to testing.

      Although not a book about java design, there is always the over-riding principle of ?keep it simple!?  That, indeed, is a theme you will see throughout the rest of the book.  Any java development project, no matter how complex, can (and should) be broken down to the smallest object, class and/or method possible.  This provides several benefits:

?       Ease of documentation

?       Smaller, more easily tested units

?       Each unit can be tracked as a task

?       Easier javadoc commenting[1]

?       Potential for modularity and reuse

This applies not just to the code, but also to almost everything else discussed in this chapter.  The simpler an item is, be it a use case template, a status report, or any documentation, the more likely it will be used effectively which will lead to a more easily managed and successful project.

 

Helping the testers

      In this section, we talk about things that can help the test and QA team do their job more effectively.  We dwell on documentation and code comments and how they relate to javadoc and discuss use cases in some detail.  Since this section covers generalities for the process, it is highly recommended that hands-on testers also read the next chapter on debugging as well to aid in an understanding of how use cases should be developed.

Comments

Commenting code, in general, is a practice that a lot of developers steer clear of.  Some think it's not worth the time (particularly with ?obvious? code), others believe obfuscation leads to continued employment and still others are just sluggish.  Other than lack of them, there are also other issues with code comments; too much verbosity and/or poorly written English (both of which can lead to confusion), duplicating the code, and ?bookmark? comments.  See the following for examples of the various types of comments.  (Note that the examples are in standard Java format, javadoc comments are described further on).

 

 

 

 

 

As you can see, commenting has its idiosyncrasies but good comments in Java are essential for two reasons; general comments let people know what's going on in the code and, javadoc demands properly formed comments to generate usable documentation.

 

Javadoc

The most vital tool in helping everyone ?down the line? in a java development project is, of course, javadoc.  After all, the API documentation for the Java standard libraries is created with it!

It should be noted that Sun describes two general uses for javadoc comments; API specifications and programming guide documentation.  Nevertheless, through the use of custom tags, the javadoc comments can be extended to provide all manner of documentation, from bug reports, to finished design documentation, and even use cases. The drawback is that each custom taglet must be coded which leaves a project team with several choices:

?       Create custom taglets

?       Use existing taglets from within the company (if they exist)

?       Expand on those we will create in this book

?       Utilize 'standard? documentation

The choice for an individual project will depend on the available resources; obviously small, quick, single-use, in-house projects will probably not want to expend the resources developing a taglet library whereas a large, commercial project might want to consider it.

The following illustrations show a well-formed javadoc comment and the generated output from Sun's document ?How to Write Doc Comments for the Javadoc Tool?[2].

 

 

 

As we develop the project throughout the remainder of the book, we will show examples of how and where to use custom taglets to produce some of the required documentation.

It is important to note that javadoc, by default, produces HTML based records, however, there is a doclet implemented (the MMF Doclet) that can create PDF documentation using Adobe FrameMaker's interchange format should there be a requirement for more 'secure? documentation.

Test Plan and Strategy

The testing group, with input from development, writes the overall test plan. The scope of the testing plan is broad and includes the application as a whole and any other applications or processes that interact with the system to be tested.

System testing should be extremely structured and methodical. It is a focused effort to test every aspect of the program in an attempt to anticipate as many user errors as possible, trying to bend, break, or crash the software. The overall goal of the system test phase is to verify the system has been coded according to the design documentation and to test all modules and components of the application as a fully integrated system. System testing should include, and resources must be allocated for, functional testing, boundary testing, stress/performance testing, recovery, security testing, and regression testing (if appropriate).

      The test strategy should allow for the creation of use cases as described below such that, to keep it simple, testers are able to build from smaller functional units to larger integrated pieces up to the fully completed application.

Use Cases

      The proper creation of use cases is critical given that these are what the test team relies on to ensure the application meets the requirements.   These documents should be as short as possible (consistent with required results), specific, and thorough.   Each use case should concentrate on a single functional unit, be it a particular screen from the user interface or a load test, as far as practical.  Groups of use cases can comprise larger, integrated portions for larger projects that, in turn distill the entire system test document.

      A general layout for a simple use case template is shown below and is detailed in the next chapter.

 

      As mentioned, use cases are absolutely necessary to ensure that testers know what to expect.  Developers can offer a lot of help in defining parameters for bounds checking, correct data types[3], expected behaviors (especially if these differ from the design), and dependencies on other processes or applications.

Java Load Testing

      Although Java is scalable, it is best not to take it solely on faith.  As discussed later in the book, resources and user perception can taint an application's reputation.  Therefore, if it is at all possible, some method for providing load scripts to the test team is very useful.  It is also incumbent on the project office to provide sufficient resources if load testing is warranted.  This can include, but is not limited to, load testing software, additional hardware, databases sized and populated as close to the ultimate target as possible, load scripts, etc.

We?ve covered most of the obvious material that can help the testers; let's look at what can be done for the developers.

Helping the developers

Developers are always at odds with having the time to do what they do and delivering it on schedule.  Coding is the major part of their job, but there are other aspects that aren't always provided for or not given enough priority.

Resources

The primary resource a developer needs is time.  There is never enough of it to ensure the code will work exactly as required without error.  There's always something that's compromised; comments, unit testing, even style.  Once a development team understands the requirements and have selected an appropriate architecture, they should be able to produce fairly reasonable estimates of how long it will take to do the actual coding including a fudge factor for ?unknowns?.  Given known models of software development, the project office should be able to make a reasonable assumption regarding additional coding time for bug fixes and rewrites based on testing.

Other factors to include in terms of time resources for development will include any learning curve necessary to bring the team up to speed on any new coding or architecture paradigms as well as time to bring late additions to the team up to the mark.

Additionally, tools may be required or requested (and there are several out there), but be wary of providing every shiny new thing that's available; they can be expensive, may introduce their own set of problems, and add to the learning curve.

Requirements

      Without doubt, requirements, descriptions, user interface mockups, and so forth are essential in aiding the developers, not just from a ?what is needed? perspective, but, if well enough developed, will provide good information to help in the code estimation process.   Moreover, a good descriptive document, well organized, also provides requirements that can be tracked from construction through the testing phase.  Each of these requirements may be expanded on during the detailed design and implementation and, in turn, each of these smaller requirements can be developed to provide use case scenarios and/or objectives.

      Any subsequent detailed design documents, if created, should be considered ?living? documents in that they may change over the design and coding phase, thus, when used to distill test documentation, it is imperative that the latest versions be used.

Java Naming Standards

      This ties back to the what we?ve discussed regarding comments, documentation, and use cases; naming conventions can be used to trace from base requirements through to testing such that anyone familiar with the design should know exactly where a bug or test failure may lie.  As always, it will be up to the team to determine which naming conventions make sense, be they industry standard or already established in-house.

Error Messages

      Here is another relatively easy area that gets overlooked.  Error messages are key in helping the end user understand what, if anything, has gone wrong.  Error messages that simply say ?A General Error Has Occurred, Contact Your System Administrator? are less than helpful.  Java, fortunately, can provide detailed stack traces and other useful facilities for both displaying error details and, more importantly, for detecting and correcting errors when they occur.  It would be best for the team to develop a coordinated approach in terms of error handling and error messages such that there is consistency between modules.

Change Tracking and Source Control

There are a number of tools and techniques that can be utilized for tracking changes.  If the team is using some form of source code control, (such as RCS or CVS) it is important to ensure some standards are established for version numbering and check in notes.  Additionally, most source code control systems utilize macros embedded in comments that can contain such information as current version and release status, among other items.  Once again, it is incumbent on the team to determine the most sensible use of these.

Code Reviews

A lot of developers shudder at the mere thought of code reviews.  Nonetheless, it is well known that, if code reviews are conducted properly, they can be an excellent venue for junior programmers to learn their craft.  Even developers with many years of experience can be surprised at what turns up in a peer code review.  Another use for code reviews is to ensure that interfaces between various modules are consistent and meets each developer's expectations.  Other than the general guidelines for meetings described below, there are several unique rules for code reviews:

?       Deliver the latest code available to the review team with sufficient time for at least one read through

?       The meeting should be on no more than one or two modules at a time

?       The meeting can last as long as one and a half to two hours, but no longer

?       Be professional at all times, no sarcasm is allowed; helpful advice is what's needed

?       Be critical of the code, not the developer; senior developers have made the same mistakes so explain what the issue is and how best to correct it

Bear in mind, these rules apply to peer code reviews (i.e. those held within the development team) if you are using external code, you may want to consider a review of correct usage and interfaces with that development team.  Also, certain types of projects may require code reviews with your customers.  However, it is best to keep to the rules described if at all possible to ensure a successful assessment of the code.

Finally, a brief word or two about style.  There are a number of coding styles and standards out there.  It is in the best interests of the development team to settle on an agreed style.  This is important for a number of reasons:

?       Any standardization of coding style should include all that has been mentioned regarding comment usage, error messages, and so on such that coordinated documentation can be produced

?       Anyone reading different module or source listings will be better able to comprehend the source

?       Programmers should be able to develop good coding habits, particularly when combined with peer code reviews

?       If it is a large, long term project, standardization will allow for easier upgrades by subsequent generations of developers

 

Now we have some guidance for the developers, we really should try and help the project office, after all, they have to keep things moving along.

 

Helping Java Project Management

      The project office always needs help and support.  Since they are generally responsible for the overall project, they are the first in line when a project goes bad.  They need help in determining task definition, what resources are needed, and what team is doing what, when.  The reciprocal of all this is that the project staff can help ameliorate any issues that may arise during the project lifetime.  Additionally, some of what is discussed here will apply to small projects to help individual developers be better organized to discuss their project with management.

Design

      The bane of any project, design will generally make or break a venture, no matter what size.  From initial concept to final documentation, if the design isn't any good, it won't get any better without a lot of rework and wasted effort.  Therefore, it is essential that any design be as a lucid and clear-cut as possible. 

Planning and Estimation

      This is the project manager's bread and butter; if done well, everyone is happy, everything clicks together when needed and minor missteps aren't all that painful.  Do it badly and chaos reigns.  Here are some ways to support the project office.

      A project manager cannot (or should not) attempt all planning in a vacuum.  It's up to each key area to provide him or her with as much detailed information as possible.  How long will it take to code a given module?  How long to test it, given a reasonable set of criteria?  How long will it take to document a given item?  Needless to say, the longer you have been doing your job, the better able you are to provide these estimates; if in doubt, consult a senior member of your team, perhaps they can provide you with a rationale for your approximation or help expand on it.

      Estimates, whether for time or materials, are a key component where a project manager will need good input.  Failure to provide accurate data can result in a poorly planned project that will become a ?death march? to complete, at best, or doomed to complete failure.

Java Documentation

      The overriding principle of ?keep it simple? applies as much (if not more so) to project office required documentation as anything else in the project.  If your company requires specific documentation and formats, it is essential to determine (and restructure, if necessary) how simple it can be made.

Documentation is obviously important because without some sort of written description, no matter how terse or vague, nobody really knows what the application is supposed to do or how things are progressing.  Sounds obvious, but think back to a project that was written on the back of a cocktail napkin (and never expanded on) and recall how badly it probably turned out, or how much re-write there was.  Any level of documentation must be tempered with the need for maintaining simplicity.  However, at a minimum, the following documents (with suggested content) should be prepared:

 

?       Requirements/concept document

?       This is the key document because it describes precisely what the team is trying to accomplish.  Stating the obvious, this document should describe, in as much detail as possible, what the application is supposed to do, who the target audience is, what platform it is expected to run on, and what data is needed to accomplish the tasks.  If it is well organized, it can be useful in generating design documents with requirements that can be tracked throughout the lifecycle.

?       Design Document

?       Once a description of the application has been developed, this document should distill the functional parts and requirements into something that will provide the development team an architecture and requirements set to work to. Subsequently, the requirements can be used to provide test objectives.

?       Project Schedule

?       This can be as simple as a list of tasks to be accomplished to a full-blown project schedule ala Microsoft Project?/span>  (or whichever tool might be used).  Whatever system is used, it will need an easy methodology to maintain.

?       Test Plan/Strategy

?       Now that it's known what is being created and how it should function, this document provides the general approach and strategy as to how it should be tested, from unit testing through to the complete application.

?       Use Cases

?       Considered a sub-part of the test plan, these are the key elements for a tester to complete their assigned tasks.  A general example is shown in this chapter and details are filled in along the way.

?       User Manual

?       If the application is to be released to end-users, this document is necessarily required.  It can range from a simple installation document (with help available in the application itself) to a multi-page, bound manual depending on the complexity of the delivered application.

?       Troubleshooting Guide

?       If the application is to be supported via a help desk or maintained within a corporate server environment, this document is invaluable if one doesn't wish to be called every time there is a minor glitch, day or night.  Again, this can be very simple, containing information on how to start and stop the application, what system(s) it may be running, any database information, and so on.

Once again in order to maintain simplicity, not all of these documents are required nor do they need to be overly verbose, but they will certainly help over the lifetime of the project. In fact, the last two will actually be useful beyond the project's end for the reasons described.

A final word about documentation and reviews.  Despite all efforts to maintain simplicity, there are certain circumstances whereby documentation standards (along with their unique convolutions and complexities) are cast in stone.  This is most obvious with Government or Military contracts.  For all concerned, this can be a massive undertaking and, as such, requires an additional set of resources with their attendant costs just to deal with the paperwork.  The best a project team can do under these circumstances is to ensure sufficient additional time and resources are granted to each phase of the project to allow for the additional documentation burden.  Furthermore, there are additional reviews generally required at each phase of the project and resources will need to be allocated for attendance by technical staff as well as project management.

Status

      The only way the project office knows what's going on (and how close they are to schedule) is if everyone keeps them updated with some sort of status report.  Once again this should be of the simplest format possible while maintaining some method to track what is being reported versus what task assignments or code modules are required.  Timeliness is important such that the project staff are kept current.  The reporting period, depending on the length of the project, should be one to two weeks.

Meetings

      The groans can already be heard.  Very few technical people relish the idea of spending time in a room, sitting around just talking.  Unfortunately, meetings are useful to ensure that everyone is on the same page and aware of any glitches or kinks that have become apparent, thus it is best to establish some general rules about meetings:

?       Meetings should last no longer than one hour

?       If meetings must extend beyond an hour, allow for breaks of 10 or 15 minutes about every 90 minutes to two hours

?       If there are to be routine meetings, have them scheduled and held as set

?       Pagers, cell phones and the like should be off or in silent mode; unless it's an emergency, don't respond

?       Prepare and distribute an agenda well in advance of any meeting

?       Do not have meetings on Mondays, Fridays, or close to shift starting or ending times

?       Have a moderator/facilitator, to keep things on topic, and a note-taker to ensure that everyone knows what was discussed and who has what action items

?       Distribute the minute of the meeting as quickly as possible such that rebuttals can be incorporated as needed

      In this section, we?ve attempted to provide information that will be useful to the project management staff to ensure some level of tracking progress, allocating resources and generally herding all those cats.

 

In this chapter we have discussed and emphasized the need to keep it simple.  The biggest caveat to that principle is that the project team is allowed to define the process.  If, however, they are constrained by a mandated set of specifications, it is incumbent on management to reduce, as far as practical, the burden on the technical staff (developers, programmers, testers and so on).

 

The following concepts to improve the planning and execution of the test cycle for a java development project were covered in detail.

To help the testers, we detailed the following concepts:

?       Comments and Javadoc

?       Use clear, concise and meaningful comments. Use javadoc to the greatest extent possible to aid in overall documentation.

?       Test Plan and Strategy

?       Ensure that the test plan and strategy is as complete and through as possible and that use cases are properly created

?       Use Cases

?       These are the detail documents that testers will need to ensure that the application works as described and needed.

?       Load Testing

?       If warranted, ensure that the testing folks have everything necessary to complete load tests including additional hardware, load testing software, scripts, etc.

 

To help the developers, we discussed the following ideas:

?       Resources

?       Ensure that the developers have what they need; time being the most valuable commodity.

?       Requirements

?       Clear and distinct information about what is desired in the design.

?       Naming Standards and Error Messages

?       Decide on standards as a team and use them consistently.

?       Change Tracking and Source Control

?       Try to use available tools with agreed upon standards and consistency.

?       Code Reviews

?       Use these as learning/teaching opportunities and, when needed, to verify interfaces between units.

?       Style

?       Choose a given style and stick with it to ensure uniformity over the lifetime of the application.

 

Finally, for the project office, we covered:

?       Design

?       This needs to be as complete and accurate as possible, beware of scope creep if things aren't defined early on. 

?       Planning and Estimation

?       The more accurate the design is to begin with, the easier and more accurate resource estimates will be.

?       Documentation

?       The need for clear, concise documentation that is as simple as possible was discussed.

?       Status and Meetings

?       Strategies for conducting meetings and simplifying the status reporting process were discussed