CYA security

I read this article when it came out a few years back;

http://www.schneier.com/blog/archives/2007/02/cya_security_1.html

Really good coverage on why the existing security measures where ineffective against the threats encountered until *after* the threat had already been encountered, which is too late.

reading this article now:

http://www.rollingstone.com/politics/story/26793903/the_big_takeover/print

reminded me of it, especially this bit:

"In the final three months of last year, (AIG) lost more than $27 million every hour. That's $465,000 a minute, a yearly income for a median American household every six seconds, roughly $7,750 a second. And all this happened at the end of eight straight years that America devoted to frantically chasing the shadow of a terrorist threat to no avail, eight years spent stopping every citizen at every airport to search every purse, bag, crotch and briefcase for juice boxes and explosive tubes of toothpaste. Yet in the end, our government had no mechanism for searching the balance sheets of companies that held life-or-death power over our society and was unable to spot holes in the national economy the size of Libya (whose entire GDP last year was smaller than AIG's 2008 losses)."

So ok, terrorists haven't managed to fly anymore planes into buildings since 9/11, thats a good thing. In the meantime personal liberties in western countries have been eroded by an enormous degree, the US has invaded a country (that still shocks me!) on false pretences, and become a party to illegal detention and the use of torture. What price your soul?

At the same time, the greatest threat to US (perhaps the world?) National Security since the end of the Cold War has been brewing, in the form of the current freeze-up of the financial markets triggered by the sub-prime mortgage collapse. So we successfully defended against future repeats of the last threat (arguably some might say), while completely missing the next great threat. If you tally up financial cost and increased mortality due to financial-related suicide & murders during this period, I dare say the cost in dollars and human lives will outstrip 9/11.

Yes we need to guard against attacks similar to what's been encountered in the past, but we also need to have open eyes in seeing dangers from new and unexpected directions. Prior to 9/11 there was intelligence about a potential attack using planes that was ignored, leading up the current crisis there were warning signs that were ignored. We need to stop ignoring good intelligence just because we haven't seen it before.

Sheeple or the wisdom of crowds?

This is disturbing:

http://brainz.org/ten-most-revealing-psych-experiments/

But on the other hand,what of the wisdom of crowds?

http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds

Whats going on here? It seems that on the one hand individuals can be easily manipulated using peer pressure from groups, and on the other crowds seem to make better decisions than individuals? Perhaps these are complementary points of view, not contradictory? Perhaps the conscious action of the majority of people can be easily swayed, but the unconscious action of groups (displaying emergent behavior?) tends to cluster to optimal points of maxima far faster and more accurately than the conscious efforts of experts?

Could it be that while human psychology is deeply flawed, the behavior that emerges from interactions between individuals and environments can be insightful?

How do we engineer society to avoid, or at least minimise the harmful effects of our own nature? Sometimes we need to follow the crowd, sometimes we need to ignore the crowd and travel against the flow, how do we decide when to do which?

A lot of questions, no real answers.

(edit)
Coming back to this post later, it occurs to me that perhaps we should observe the crowd and be aware of what is causing it to cluster or disperse, but not necessarily follow the crowd. Know thyself...

The joy of refactoring

Imagine you have an infrastructure that allows you to run batch jobs asynchronously. However, you can't specify exactly when you want it to run (à la cron) but rather all you have is a polling interface, so your "service" will start up every x mins, do something, then sleep again. Finer grained control is up to you.

This is our situation, and we usually specify a start time, eg - "23:00", an end time "24:00", and have a bit of logic somewhere to abort the run if the current time is outside that window. I finally noticed the bit of logic that does this, and realised that it was duplicated in every service has a *copy* of the same code. behold:



Yuck.

Needless to say, I didn't like the duplication, or the code itself. I didn't have scope to retrofit the other services that used similar code, but I could fix this one and build a foundation for later.

First step was to refactor that mess out to a common util class, leaving us with this:


...
and this:



The first cut was just a raw refactor extract, the next step was to not use ugly Calendar code and handle the midnight cutover correctly (which is actually quite hard!). After a few false starts with Dates and dealing with GMT and daylight savings offsets, we ended up with this (convenience methods and javadoc not shown), which is unit-tested and guaranteed to work (except maybe during the daylight savings cutover itself..)



Much nicer, well we think so anyway.

If you can see a bug in there please let me know!!

The joy of Spring!

Had the opportunity to Springify a co-workers project while he was on leave the other day. It already sort-of used Spring, but he had inverted the inversion-of-control (yeah... think about it, passing the context everywhere and getting single beans out, scary) and because of the architecture we were variously running into "got minus one from read call" errors (Database-side connections exhausted) or "No ManagedConnections available within configured blocking timeout" (JBoss Connection pool exhausted) depending on whether you used his oracle config (multiple connection pools with max-size of 100) or mine (max-size 10 per pool).

So! Time to throw out cruft, delete code and do proper dependency-injection! always great fun. I once did a project where most of the risk was in the business algorithm, but the data requirements started fairly simple. So I rolled my own database layer and got going. Later in the project the data requirements started to grow, transaction demarcation and correct handling became an issue so I took the time to Spring-ify it, and the joy of going from raw JDBC sql to JdbcTemplates and Hibernate HQL was huge! dozens of lines of verbose JDBC try-catch-finally evaporated to one-liners. just great!

I do love taking well-written OO code and re-wiring it with Spring, you can throw out huge amounts of code and eliminate bugs that you might have never known about! Connection pooling issues, correct rollbacks, host of junk like that. A lot of the ugly casting and boilerplate code just evaporates, and you're left with get-setters (which is another issue entirely, we should have properties!)

At the moment I tend to use a Builder pattern to boot-strap Spring from the init code (not a full spring stack yet...) and get a handle to a Spring bean configured with all the resources (it knows how to orchestrate the DAOs and usually contains most of the bus. logic) which then takes over processing. I could probably do better, but it works pretty well for the moment and is easy enough to unit test, so I'm happy.