NYCPHP Meetup

[Tim Gales]

Elliotte Harold wrote:
> Tim Gales wrote:
> 
>> Valid XML documents must adhere to their DTD/Schema and to that
>> degree they have fields -- called 'elements'.
>> like
> 
> 
> Which is why we don't necessarily use valid XML documents. For many 
> applications, well-formed is good enough. In practice, validation is 
> usually one of the first things to be turned off in a production app 
> because it just costs too much. However there are also good theoretical 
> reasons not to insist on enforcing a schema.
> 
> At design time, you usually don't know all the characteristics of the 
> data you're modeling. It is common to uncover new attributes months and 
> years after you've deployed, especially in rapidly changing fields like 
> medicine. The less structure you impose up front, the more freedom you 
> have to adapt and evolve your database and application to changing 
> circumstances. 
True, at design time you may not have all you need to know about
the data. But that's not all you may be missing during the early
stages of building a system. Sometimes stake holders are too busy
with day-to-day affairs to give you a full run-down of all the
business rules. It can even happen that because of deadline
pressure you have to start building before all the security
policies have been reviewed by whatever department reviews security.

But by implementation time you have what you need -- at least
in most cases (if you if youThere are always some corner cases where this is
not so --  but 99 percent of the time you have what you need)

It is not only in "rapidly changing fields like medicine"
where flexibility is a must. (By rapidly changing fields, I understand
you to be speaking about cases where the underlying information
shifts and evolves.)

Financial institutions can be hit with new rules by federal regulators.
For instance, banks can receive a new edict from the FBI which declares
financial institutions must make their information systems
compliant with the latest anti-money-laundering policy.
This can cause banks and even those stodgy old insurance companies
to have to 'make over' (sometimes large) subsystems.

It turns out financial computer systems have to be pretty flexible
to account for a whole host of things that can happen.
That is banks had better be pretty 'agile' when it comes
to developing systems -- or they won't be around for long.

> 
> As Scott Ambler has noticed, the data community has not yet graduated 
> from the waterfall, big-design up-front school of application design. 
> First they gather their requirements. Then they build their schemas. 
> Then they build their application on top of that.  Once an app is 
> deployed, even a simple addition of a field can be a major operation. 
> Lord help them if they need to remove a field or restructure a table. 
> Relational databases do not lend themselves to agile development.
> 
To say "the data community has not yet graduated from the waterfall..."
is a blanket statement. The data community is not some monolithic
homogeneous group which moves together in lock-step.

There are dozens, if not hundreds, of system building styles in
the data community. The system development styles I have seen have
all been hybrids of methodologies.

This is, I think, because you not only have to migrate data
when you build a new system, but you also have to migrate
the developers' thinking and habits when you 'modernize'.

Usually throwing out the old staff which has a lot
of subject matter expertise garnered from years of experience
and replacing them with fresh developers steeped in some
methodology is not really a viable option -- if you want
to stay out of court (or jail). That is to say, you
want to avoid a too hastily adopted 'sashimi' model
which can result in the building of a 'so-sue-me' system.

> By contrast, if you don't lock in any schema at all (as is possible with 
> an XML DB) then you can adapt your data to meet changing and newly 
> discovered requirements as they become apparent. You can also design and 
> deploy your application in short iterations that progressively add 
> functionality. You don't need to lock down your requirements before 
> writing any code.
> 
A relational schema is not somehow congenitally stiff and unchangeable
-- it is as flexible as you make it.
(plenty of XP-RAD-AGILE developers use them all the time)

> This also enables and requires much greater integration between the 
> database admins and the programming teams. Too many organizations today 
> treat these as separate fiefdoms. The DBAs spend all their time 
> optimizing the database and defending its purity from the demands of the 
> programmers while the programmers spend their time trying to work around 
> the strictures the DBAs have imposed. (I've usually been on the 
> programmer side of this particular battle so my perspective here is a 
> little biased.)
Okay, businesses need to prevent kingdom-building, in order to prevent
departments from working at cross purposes.

Enabling (and requiring) greater integration is 'good' thing as is
locking down requirements in an (often unsuccessful) attempt to
'feature creep'. (But it always seems that some boss somewhere
can't live without some snap-shot report, which he never mentioned
until the system is in acceptance testing -- and after you build
him one for his desktop laser printer, he wants all the headings
re-done in to print in curlicue-times-roman lettering)

(I am not suggesting that you mean using XML is good because it
alleviates the difficulties of learning how to work with others as a
team  -- but reading what you wrote could be misconstrued along those
lines.)
> 
> A more flexible, less schema focused database will not require 
> programmers to wait for weeks, months, or years for the DBAs to make 
> changes applications require.
> 
If an application requires a change, and a DBA impairs the business
by refusing to make that change in a timely manner, he will most
assuredly be removed (and in a timely manner) -- this may not
apply in some civil service situations where the DBA has tenure --
but, as I said before, there are always some corner cases.

-- 

T. Gales & Associates
'Helping People Connect with Technology'

http://www.tgaconnect.com