Design Validation and Cost Estimation

Donald K. Burleson

Requirements validation

The requirements validation is performed to ensure that the system meets the infrastructure requirements of the end-user.  A requirements evaluation is a simple assessment and mapping of the components of all inputs, data storage and outputs to ensure that al data items are present.  The requirements analysis does not attempt to validate the functionality of the system, and is commonly performed after the non-functional prototype is completed.  For most database systems development, the non-working prototype screens, prototype Oracle table ad index design and the output design (mockup screens and reports) can be done very quickly. 

Working from the use-case documents and functional analysis document (data flow diagram, process logic specifications), the designer can create dummy screens in a matter of minutes. For an online system with 100 tables and 200 screens, a non-working prototype can be created in a few weeks by a single designer.

This non-working prototype is shown to the end-user representative to ensure that the infrastructure requirements have been met.  Once approved by the end-user community, the development team begins the process of "fleshing-out" the prototype and turning it into a working system. This is the bulk of the development process and commonly takes more effort than all of the other development areas combined.

In sum, the requirements evaluation has the following tasks: 

• EUI requirements validation – Do the online screens and output reports contain the stated data items?
   
• Input requirements validation – Do each of the input screens properly validate the data?  In the IT industry this is called Chimpanzee testing because the validation involves randomly pressing keys and filling-in inputs in an attempt to confuse the input screen.
   
• Database requirements validation – Do the data structures meet the requirements specified in the EUI?
   
• Requirements data mapping – Each of the data inputs are mapped to the database and to the output screens and reports.  Again, this is a very simple mapping, and it says nothing about the functionality (or lack thereof) of the existing system.

Again the requirements evaluation is nothing more than the identification of input, storage and output data items and the mapping of the data items to and from the database, and it says NOTHING about the whether the system actually does anything.

It is also sad that many academicians (and others without real-world experience) fail to understand that the requirements validation is so simple that it is almost trivial.  In a dynamic analysis project, screens, reports and Oracle tables can be created in minutes using wizards, HTML generators, Oracle OEM, etc.  Even a complex system with 500 screens can be prototyped by a single analyst in just a few weeks.   

Once we have confirmed that the system components (input screens, database objects, output screens, and reports) contain the proper data elements and mappings, we are ready for the true validation of the system, the functional validation.

How to identify a poor requirements evaluation

The experienced Oracle professional can quickly identify an inept requirements validation.  Sadly, it is not uncommon for companies to hire inept academicians.  These PhD fools are so common that IT professionals have a special name for them, and call them many name, the most common being the Educated Idiot, or EI, for short.  

The EI usually has an impressive resume, with PhD degrees and dozens of citations to research projects with no practical application and publications in obscure academic journals that few people ever read.  They may site books that they have written, but fail to tell you that their books were so obtuse and unpopular that they only sold a thousand copies. 

The EI is very dangerous because non-technical non IT managers may give the EI more credit than they deserve.

The EI commonly does not possess any real-word development experience and lack the necessary technical skill to perform a complete requirements analysis.  They may be able to view the input and output screens and look at the reports, but they do not have enough skill to example the critical requirements component of all, e structure of the underlying database.

Functional validation

The most important part of any systems design is evaluating the functionality of the existing system.  A functional analysis of the existing system gives the designer insights into the three major areas of functional evaluation, inputs processes and outputs.  All systems, regardless of size, complexity or implementation, have the following functional characteristics: 

• Input functional validation – End user interfaces via online screens or batch interfaces (FTP, document submission) are validated to ensure that they accept the correct datatypes and conform to all required check constraints (as defined in the Oracle dba_constraints view.
   
• Processing functional validation – This step examines how the system takes the inputs are transformed and stored in the database.  This is the most critical area of functional evaluation because the data transformations are the core of the system.  Without meaningful processes, the system is nothing more than an empty shell of prototype screens.
   
• Output functional validation – All system provide output.  The output may be in the form of EUI display (both online and batch) or via stored in the database.

In order to properly conduct a functional evaluation, we must carefully look under the covers.  The analyst must hand-execute all of the code within the system to ensure that the internal; data transformations match the requirements.

Remember, only the functional evaluation can tell you anything about the functional quality of the existing system.  Relying solely on the requirements validation is insanity, because a system has no use without process logic to transform the data.

A legitimate functional validation must include a complete description of all process logic and validation data.  In Plain English, the existing system must be exercised to ensure that expected inputs produce the proper outputs.

How to spot a poor functional analysis

Because the work of specifying and validating the functional requirements is commonly the task on a non-technical r semi-technical person, there are many opportunities for serious misunderstandings and errors.

The most common hallmark of a poor functional analysis is a misunderstanding of the purpose of functional analysis.   The Educated Idiot (EI) is noted for this type of error.  IN their quest to appear erudite, the EI will commonly attempt to cloud the issue with wordy, verbose and redundant jargon, replete with non-sequiters and inaccurate conclusions.

I once say an alleged expert claim that it was not necessary to test the system or inspect the database to perform a valid functional analysis! (Swear to God, I'm not making this up)

Next, let's examine how to estimate the costs for the existing system.

Evaluating the worth of an existing system

As an Oracle professional you may be asked to evaluate an existing system and estimate the amount of money that was (or should have been) expended on the development effort for the system.  Estimating the costs of an existing system is always problematic because of the following issues: 

• Technology factors – The type of technology has a huge influence on the costs.  A system developed in an older technology (i.e. DC Cobol system with CICS BMS screens) may cost  ten times m ore than a state-of-the art Java system with reusable components and HTML screens that are generated quickly with an HTML generator (FrontPage, Cold Fusion, Dream Weaver, etc).
   
• Productivity factors – It has long been recognized that IT develops have a non-linear relationship between productivity and costs (Figure x).  On the low-end, we see that a $30/hour beginner has a cost/productivity factor far less than the $200/hr Guru.  This is because IT developers with more than a decade of full-time development experienced know exactly how to solve a problem.  Further, experienced IT developers almost always have a personal library of re-usable code that performs common business functions.  These libraries allow the experienced developer to be far "cheaper" than the less-expensive than the inept beginner.
   
• Cost Factors – The widespread difference in costs between USA and overseas IT consulting can differ by an order of magnitude.  In today's virtual world, many companies abandon the USA resources and outsource their IT development efforts where experienced programmers can be acquired for less than $500 per month.  As if 2003, Bangalore India, Moscow Russia and Eastern Europe are all courting USA customers.

Before we see the problems that these variables introduce into any attempt to lace an estimated cost on a system, let's examine the different methods that are employed to estimate the costs for an existing system.  The following are approaches have been tried in attempts to estimate the "worth" of an existing system: 

• Compare your system with a known, similar system – In this approach, common metrics are gathered for the existing system and then compared to similar system. The common metrics might include the programming language, a subjective complexity rating for each function, and estimated productivity rates for the developers.  Of course, this approach is fraught with problems, the foremost being the subjective nature of the estimates and the problem of finding a truly similar system.  The subjective nature of the inputs allows you to rig the result to say anything that you desire.
   
• Use Mathematical Models – There are numerous numerical tools that have been designed to attempt to determine how much money that a system may have cost.  These tools include COCOMO, Cost Xpert any many others, none of which are recognized as statistically valid.  All of these tools have a fatal flaw that makes then totally useless as a post hoc cost estimation tool.  The major flaw is the subjective nature of the estimation parameters.  For example, the COCOMO tool allows the analyst to specify the following non-quantifiable and subjective factors:

 ·        Module Size – This is flawed because of the widespread use of object-oriented languages such as Java.  Te hallmark of an object-oriented language is the ability to re-use components to greatly reduce development time.  It is invalid for any tool (like COCCOMO) to assume that a Java method of a given size required any specific amount of effort to produce.

 ·        Labor Rates – As we have noted, this is an invalid parameter because it does not consider the non-linear relationship between experience and productivity.

 ·        Effort Multipliers – If the analyst does not like their result numbers, they need only adjust one of the subjective multipliers to receive any cost they desire.

• Accept External Bids – Using this approach, the IT analyst presents a requirements document (or maybe a functional prototype with process logic specification) to a series of vendors for an estimate.  In order to avoid the immoral act of asking for an estimate under false pretenses (because you have no intention of actually accepting the estimate), the analyst engages the third party consultancy with the promise to pay the fair market price for all estimation services if they fail to engage the consultancy to develop the system.  To ensure that the external bid is accurate, the bidder is only provided with the functional analysis document and they have no access whatsoever to the existing system

As we can see, the comparison approach and math model approach also have a problem with current-value dollars.  The costs of developing a system three years ago might be quite different than the costs today.  IN sum, the only valid way to determine the real costs for an existing system is to commission a consultancy to bid on the system, seeing only the functional specifications. By offering up-front to pay for the estimate removes the moral issue of asking for a bid when you do not intend to accept the bid.  Also, not telling the consultancy that you have no intention of using the bid ensures that you receive a competitive bid.

The above is an excerpt from:

Physical Database Design Using Oracle
by Donald K. Burleson, Auerbach Publications, ISBN: 0849318173