Design Validation and Cost Estimation
Donald K. Burleson
The requirements validation is performed to
ensure that the system meets the infrastructure requirements of the
end-user. A requirements evaluation is a simple assessment and
mapping of the components of all inputs, data storage and outputs to
ensure that al data items are present. The requirements analysis does
not attempt to validate the functionality of the system, and is
commonly performed after the non-functional prototype is completed.
For most database systems development, the non-working prototype
screens, prototype Oracle table ad index design and the output design
(mockup screens and reports) can be done very quickly.
Working from the use-case documents and
functional analysis document (data flow diagram, process logic
specifications), the designer can create dummy screens in a matter of
minutes. For an online system with 100 tables and 200 screens, a
non-working prototype can be created in a few weeks by a single
This non-working prototype is shown to the
end-user representative to ensure that the infrastructure requirements
have been met. Once approved by the end-user community, the
development team begins the process of "fleshing-out" the prototype
and turning it into a working system. This is the bulk of the
development process and commonly takes more effort than all of the
other development areas combined.
In sum, the requirements evaluation has the
- EUI requirements validation – Do the
online screens and output reports contain the stated data items?
- Input requirements validation – Do each
of the input screens properly validate the data? In the IT industry
this is called Chimpanzee testing because the validation involves
randomly pressing keys and filling-in inputs in an attempt to
confuse the input screen.
- Database requirements validation – Do
the data structures meet the requirements specified in the EUI?
- Requirements data mapping – Each of the
data inputs are mapped to the database and to the output screens and
reports. Again, this is a very simple mapping, and it says nothing
about the functionality (or lack thereof) of the existing system.
Again the requirements evaluation is nothing more
than the identification of input, storage and output data items and
the mapping of the data items to and from the database, and it says
NOTHING about the whether the system actually does anything.
It is also sad that many academicians (and others
without real-world experience) fail to understand that the
requirements validation is so simple that it is almost trivial. In a
dynamic analysis project, screens, reports and Oracle tables can be
created in minutes using wizards, HTML generators, Oracle OEM, etc.
Even a complex system with 500 screens can be prototyped by a single
analyst in just a few weeks.
Once we have confirmed that the system components
(input screens, database objects, output screens, and reports) contain
the proper data elements and mappings, we are ready for the true
validation of the system, the functional validation.
How to identify a poor requirements evaluation
The experienced Oracle professional can quickly
identify an inept requirements validation. Sadly, it is not uncommon
for companies to hire inept academicians. These PhD fools are so
common that IT professionals have a special name for them, and call
them many name, the most common being the Educated Idiot, or EI, for
The EI usually has an impressive resume, with PhD
degrees and dozens of citations to research projects with no practical
application and publications in obscure academic journals that few
people ever read. They may site books that they have written, but
fail to tell you that their books were so obtuse and unpopular that
they only sold a thousand copies.
The EI is very dangerous because non-technical
non IT managers may give the EI more credit than they deserve.
The EI commonly does not possess any real-word
development experience and lack the necessary technical skill to
perform a complete requirements analysis. They may be able to view
the input and output screens and look at the reports, but they do not
have enough skill to example the critical requirements component of
all, e structure of the underlying database.
The most important part of any systems design is
evaluating the functionality of the existing system. A functional
analysis of the existing system gives the designer insights into the
three major areas of functional evaluation, inputs processes and
outputs. All systems, regardless of size, complexity or
implementation, have the following functional characteristics:
- Input functional validation – End user
interfaces via online screens or batch interfaces (FTP, document
submission) are validated to ensure that they accept the correct
datatypes and conform to all required check constraints (as defined
in the Oracle dba_constraints view.
- Processing functional validation – This
step examines how the system takes the inputs are transformed and
stored in the database. This is the most critical area of
functional evaluation because the data transformations are the core
of the system. Without meaningful processes, the system is nothing
more than an empty shell of prototype screens.
- Output functional validation – All
system provide output. The output may be in the form of EUI display
(both online and batch) or via stored in the database.
In order to properly conduct a functional
evaluation, we must carefully look under the covers. The analyst must
hand-execute all of the code within the system to ensure that the
internal; data transformations match the requirements.
Remember, only the functional evaluation can tell
you anything about the functional quality of the existing system.
Relying solely on the requirements validation is insanity, because a
system has no use without process logic to transform the data.
A legitimate functional validation must include a
complete description of all process logic and validation data. In
Plain English, the existing system must be exercised to ensure that
expected inputs produce the proper outputs.
How to spot a poor functional analysis
Because the work of specifying and validating the
functional requirements is commonly the task on a non-technical r
semi-technical person, there are many opportunities for serious
misunderstandings and errors.
The most common hallmark of a poor functional
analysis is a misunderstanding of the purpose of functional
analysis. The Educated Idiot (EI) is noted for this type of error.
IN their quest to appear erudite, the EI will commonly attempt to
cloud the issue with wordy, verbose and redundant jargon, replete with
non-sequiters and inaccurate conclusions.
I once say an alleged expert claim that it was
not necessary to test the system or inspect the database to perform a
valid functional analysis! (Swear to God, I'm not making this up)
Next, let's examine how to estimate the costs for
the existing system.
Evaluating the worth of an existing system
As an Oracle professional you may be asked to
evaluate an existing system and estimate the amount of money that was
(or should have been) expended on the development effort for the
system. Estimating the costs of an existing system is always
problematic because of the following issues:
- Technology factors – The type of
technology has a huge influence on the costs. A system developed in
an older technology (i.e. DC Cobol system with CICS BMS screens) may
cost ten times m ore than a state-of-the art Java system with
reusable components and HTML screens that are generated quickly with
an HTML generator (FrontPage, Cold Fusion, Dream Weaver, etc).
- Productivity factors – It has long been
recognized that IT develops have a non-linear relationship between
productivity and costs (Figure x). On the low-end, we see that a
$30/hour beginner has a cost/productivity factor far less than the
$200/hr Guru. This is because IT developers with more than a decade
of full-time development experienced know exactly how to solve a
problem. Further, experienced IT developers almost always have a
personal library of re-usable code that performs common business
functions. These libraries allow the experienced developer to be
far "cheaper" than the less-expensive than the inept beginner.
- Cost Factors – The widespread
difference in costs between USA and overseas IT consulting can
differ by an order of magnitude. In today's virtual world, many
companies abandon the USA resources and outsource their IT
development efforts where experienced programmers can be acquired
for less than $500 per month. As if 2003, Bangalore India, Moscow
Russia and Eastern Europe are all courting USA customers.
Before we see the problems that these variables
introduce into any attempt to lace an estimated cost on a system,
let's examine the different methods that are employed to estimate the
costs for an existing system. The following are approaches have been
tried in attempts to estimate the "worth" of an existing system:
- Compare your system with a known, similar
system – In this approach, common metrics are gathered for the
existing system and then compared to similar system. The common
metrics might include the programming language, a subjective
complexity rating for each function, and estimated productivity
rates for the developers. Of course, this approach is fraught with
problems, the foremost being the subjective nature of the estimates
and the problem of finding a truly similar system. The subjective
nature of the inputs allows you to rig the result to say anything
that you desire.
- Use Mathematical Models – There are
numerous numerical tools that have been designed to attempt to
determine how much money that a system may have cost. These tools
include COCOMO, Cost Xpert any many others, none of which are
recognized as statistically valid. All of these tools have a fatal
flaw that makes then totally useless as a post hoc cost estimation
tool. The major flaw is the subjective nature of the estimation
parameters. For example, the COCOMO tool allows the analyst to
specify the following non-quantifiable and subjective factors:
Module Size – This is flawed because of the
widespread use of object-oriented languages such as Java. Te
hallmark of an object-oriented language is the ability to re-use
components to greatly reduce development time. It is invalid for
any tool (like COCCOMO) to assume that a Java method of a given
size required any specific amount of effort to produce.
Labor Rates – As we have noted, this is an
invalid parameter because it does not consider the non-linear
relationship between experience and productivity.
Effort Multipliers – If the analyst does not
like their result numbers, they need only adjust one of the
subjective multipliers to receive any cost they desire.
- Accept External Bids – Using this
approach, the IT analyst presents a requirements document (or maybe
a functional prototype with process logic specification) to a series
of vendors for an estimate. In order to avoid the immoral act of
asking for an estimate under false pretenses (because you have no
intention of actually accepting the estimate), the analyst engages
the third party consultancy with the promise to pay the fair market
price for all estimation services if they fail to engage the
consultancy to develop the system. To ensure that the external bid
is accurate, the bidder is only provided with the functional
analysis document and they have no access whatsoever to the existing
As we can see, the comparison approach and math
model approach also have a problem with current-value dollars. The
costs of developing a system three years ago might be quite different
than the costs today. IN sum, the only valid way to determine the
real costs for an existing system is to commission a consultancy to
bid on the system, seeing only the functional specifications. By
offering up-front to pay for the estimate removes the moral issue of
asking for a bid when you do not intend to accept the bid. Also, not
telling the consultancy that you have no intention of using the bid
ensures that you receive a competitive bid.
The above is an excerpt from:
Physical Database Design Using Oracle
by Donald K.
Burleson, Auerbach Publications,