"In technology, the conceptual age is defined by cognitive or creative assets, including design, storytelling, artistry, empathy, play, and emotion. Good engineering or good computer science is no longer enough; design must be just as good." 

-- Michael Zyda, Computer Science in the Conceptual Age, Communications of the ACM, Vol. 52 No. 12, Pages 66-72

• Benjamin S. Blanchard, Systems Engineering Management, 2008
• Dennis Buede, The engineering and design of systems: Models and Methods (2nd ed)
• Andrew P. Sage and William B. Rouse, Handbook of Systems Engineering and Management, 2009
• John Boardman and Brian Sauser, Systems Thinking: Coping with 21st Century Problems, 2008

Innovation and creativity

• Ken Robinson, Out of Our Minds: Learning to be Creative
• Genrich Altshuller, Innovation Algorithm: TRIZ, systematic innovation and technical creativity, 1999

Math

• Visualizing Order in Chaos, What Does Randomness Look Like?, Daniel Becker
• Using visualization to find prime numbers

Biology and Medicine

• Martin Krzywinski's Circos: Visualizing the Genome
• Maize genome, NSF

Other Applications of Martin Krzywinski's Circos

• Visualizing tabular data,  Martin Krzywinski
• Visualizing political debates; roll-over adds context-sensitive text
• Other applications of Circos

Visualization Applications

• A Panoply of Applications, NSF Visualization focus area: Visualization: A Way to See the Unseen

More links here.

12/14/09 Architect Competencies

Competencies of IT Architects, 2nd edition, 2009 is published as a Google ebook, and looks like it will be quite useful (thanks to Don Hirst for the pointer). But... it does make some claims that I consider ill-advised:

Perhaps, a little more of that "opposable mind"??? A little less either-or and a little more and? This typology may meet with approval in the "boys don't cry" school, but humanity is complex, and people have different shades of propensity--what is more, what characteristics a person punches up (in the sense of emphasis or drawing out more strongly in themselves), is often situational. Context factors. We need to try to get a handle on complexity, but shoving it into either-or buckets doesn't serve architects very well!

Ok, I know plenty of architects who'd be hugely relieved by this typology. Many of them would be perfectly happy to accept that they are masculine-analytics. But then they'd have to partner with a feminine-visionary who can handle the fuzzy front end, get people on board, and hand over a tightly defined problem for them to carefully solve. And given that women are in short supply in our field, make that a male feminine-visionary. Did we just lose you? Couldn't we instead accept that enough people can flex across this space that we should not neutralize their talents and contributions with a polar taxonomy?

Tom Peters refers to some work that suggests that men are more dominance-hierarchical and women more network-collaborative. There, too, I objected. Plenty of men work in collaborative-networked ways, and many women work in dominance-hierarchical ways. And some move between these styles. So don't stereotype and don't, please don't, assign supposed gender qualities to an architect typology!

By the way, I thought this piece of research was interesting. We're in a business climate where savage brutish dominance behaviors are having their day in the sun. But it is not the ultimate survival strategy.     

"The non-cooperator tries to get a bigger piece of the pie, but the cooperator helps the pie be bigger," Pepper said.

"The naive view of Darwinian evolution is that it always favors the most savage, brutal and selfish behaviors. It doesn't – and this is one example of that. In nature, groups of cooperative individuals are more successful than groups of selfish individuals."

-- Why Nice Guys Usually Get the Girls, UA News, November 6, 2009

12/15/09 What are Models Good For?

No, not that kind of model! The kind the guys putting together the Agile Manifesto swept under the table! ;-)

[I'm still gathering thoughts and notes... this is just scratching the surface...]

A model is a representation or abstraction of something that we are trying to understand.

Here we are focusing in particular on visual models in system design and evolution. Models help

• reason about and solve problems
• evaluate various outcomes, to make decisions
• sharpen our understanding
• compare alternatives
• surface and make assumptions explicit
• guide risk assessment
• discover new questions
• allow thought experiments (for example, exploring "how does/will this work" or "what if")
• enable simulations to be constructed
• predict future trends
• communicate clearly and compellingly
• provide a basis for explanation and training

If we think of the model as the visualization (as in the case of a model expressed in UML, for example) we need to remember that "pictures are worth 1,000 words"--but only 1,000. The rest of the words still need to be written down, if we want to accomplish goals like communication and preservation of design and intent.

When to model?

To support reasoning:

• limitations of the human mind/complexity of the subject of inquiry
• difficult to keep track of all dimensions/variables at the same time
• make relationships among factors/variables explicit
• reveal the structure
• explore and illuminate dynamics
• draw on analogies and models or theories from other domains

To support collaboration:

• facilitate understanding
• lead more structured discussions
• discuss specifics in the context of the "big picture"

To support experimentation:

• modeling effort is small, in comparison with the magnitude of the outcome
• perform sensitivity analysis
• test assumptions/beliefs

To support persuasion and influence (visual rhetoric):

• improves confidence (team: "we can do it"; stakeholders: "they demonstrate they know what they're up against/need to do")
• interesting
• the power has influenced our idiom: "I see what you mean," "it draws people in,"

(When) Is a diagram a model?

In software design contexts, I've seen some people distinguish "diagrams," or informal sketches of the architecture, system topology, use context, etc. from "models," and the distinction seems to be between more formal use of notation like UML and informal, ad hoc pictures. When these informal sketches are abstractions or representations of the system, these kinds of diagrams (from my perspective) are still models; they are just informally sketched. So perhaps this means that a model might have a lifecycle? Grin.

Improving visualizations

Epstein's essay titled simply "Why Model?" is targeted at (social) scientists, but makes a number of pertinent points--and he references one of my favorite Feynman essays. This paragraph echoes points Grady Booch makes (though Grady makes them in the software context):

Starting in Meta-Architecture, we recommend that architects look for analogies and metaphors. And yes, our doing so predates XP, and yes, we still do so. If a suitable analogy is found, it can help think about how to design the system. The point I've been making (these--groan*--past fifteen years) is that analogies can help us find the structures, relationships and qualities that unify and help explain our system. Epstein makes a different point--modeling can help us find analogies. I think these are complementary points. It is not a dictate, as per early XP. Only a suggestion; an aide to discovery.

We try to find the natural fit to business, use context and purpose, and the natural interstices within a system designed for such a fit. But some of what we do is pure invention, and working without a priori knowledge of the invention, we are bringing knowledge, insight, intuition from other systems to bear--that is, we use analogy. Moreover, notions like fit and interstices leverage analogical thinking. Being analogically impaired must, I suppose, adversely impact creativity, problem creation (asking new questions and inventing) and novel problem solution.  

* the groan is at the speed with which time runs its course through us; not the slowness of the field to get my points. ;-)

12/16/09 Visualizing Software Visualization

I'm trying to feel my way towards a structure for the software visualization space. Naturally I'm in the habit of seeing things through a VAP lens, so that influences what I see and that is good (VAP is, afterall, righteous profound; wink) and bad (I'm fallible; less so than many, but more so than you; wink). 

So right now what I see is smoke and mirrors. Like this: envisioning and intentional design (smoke); and reflection of the design as built in the system (mirrors).   [An archman cartoon sketch ghosts across my mind at this thought; I need to catch hold of it and pen pin it down.]

12/17/09 Learning How to See

• Betty Edwards, The New Drawing on the Right Side of the Brain, 1999  

12/17/09 Change and Adaptation

• Constance E. Helfat, Sidney G. Winter, Sydney Finkelstein, David Teece, and Harbir Singh, Dynamic Capabilities: Understanding Strategic Change in Organizations, 2006

12/17/09 Technology Leading Change

"Canton argues that the pace of change is quickening and leading to new challenges—and opportunities—for business. And, according to Canton, it's the CIO who should be at the forefront of these changes, fusing technology innovation with business strategy and leading the rest of the company into tomorrow."

The Change Agent CIO positions where CIO's need to be, which in turn is highly suggestive of the role of architects in IT. Here are some snippets:

"The next phase of this is CIOs being the ones who propose new products, new services and new strategies. ...

I have many big clients who have a culture that says, "We can't predict anything, so, therefore, we're just going to be reactive." So the first step is a mindset shift to where you believe that it's possible to make accurate predictions. ...

There is a new generation of CIOs emerging. The more they understand where the business is going and can help shape it, the better they're going to be in their careers, the better they're going to be able to serve their organizations and the better they're going to be able to deliver shareholder value."

-- James Canton, interviewed by John McCormick, "The Change-Agent CIO," CIO Insights, 12/19/2007

And for a different, and characteristically provocative, perspective:

"In the long run, the IT department is unlikely to survive, at least in its familiar form. It will have little left to do once the bulk of business computing shifts out of private data centers and into 'the cloud.' Business units and even individual employees will be able to control the processing of information directly, without the need for legions of technical specialists."

"But the information management and information strategy elements will become, if anything, more important. The ways companies take advantage of digitized information will become more important, not less.

The big question in the long run is, do those types of skills—information management and thinking—remain in a separate IT department or do they naturally flow into business units and other traditional parts of the business? My guess is that over time, they'll begin to flow into the business itself and that will be accelerated as individual workers and business units get more control over the way they are able to organize and manipulate their own information. I would be surprised if maybe 20 years from now there are still IT departments in corporations."

-- Nicholas Carr interviewed by Edward Cone, "Nicholas Carr: Why IT Will Change,"  CIO Insights, 1/9/2008

12/17/09 Lincoln Speaks to UN Climate Change Conference!

"The dogmas of the quiet past, are inadequate to the stormy present. The occasion is piled high with difficulty, and we must rise -- with the occasion. As our case is new, so we must think anew, and act anew. We must disenthrall ourselves, and then we shall save our country." [and world]

— Abraham Lincoln, Message to Congress, December 1, 1862

12/17/09 Immersive Visualization

One of my first reactions to Second Life was that a "Past Lives" virtual world would be an awesome way to recreate and experience history. Naturally it should be much easier to get birthed into a decent avatar look-alike to oneself or historical personage, and go to the costume hall to be fitted out in period costume, and go to the places and events in history you wanted to experience through your avatar's adventures and encounters with other historical figures. Grin.  (If you peeked in on my avatar's birthing travails, you'll understand the wry grin.) Schools could put on Roman tournaments, wage battles, kids could give famous speeches in their historical context, put on a Shakespeare play with the dude himself directing,... whatever... I suppose an entry fee or donation could make the thing viable, much like wikipedia...

Anyway, today I was excited to see a different take on an immersive  experience of history has been integrated into Second Life. Naturally it's much better, in key ways, than what I imagined! :-)  A little more scouting, reveals some neat stuff happening in this area right in Second Life: a collection of historical maps, Paris 1900... Of course, Second Life has some experience problems... like finding this stuff (I found out about Paris 1900 outside SL)... and economic-stratification (designer avatars, buildings and decor are costly in SL--just like plastic surgery and a personal trainer can advantage the wealthy in RL)... I'm not against economic stratification per se, but it would be nice if an immersive history world didn't qualify the educational opportunities and experience, based on RL discretionary spending allowances...

12/18/09 Etsy Rocks!

Ok, this is twitterish long form, so skip if you don't want to celebrate the cottage revolution etsy style with me:

Christmas around the corner, and a niece in London... no problem with etsy's "shop local" feature. (Well, I could find UK artists using the "shop local" feature, but it wouldn't let me enter a UK address in the "ship to" area once I got to payment... Fortunately I could override the address with an email to the artist--who has already replied to me saying she's packaged up the gift and will post it tomorrow!) I have mentioned it before, but if you weren't following along then (around Valentine's Day) let me say this--I love having access to a world of artists from my base station. Grin. It makes beautiful handmade gifts easy and I love to honor the work of talented people by gifting some of their fine creations.

Larry Spears is our local favorite potter, and we're excited that he has a presence on etsy too. Larry is always innovating and creating. His lovely wife, Jan, teaches ballet and puts on productions in Brown County. His sons are both artists (Kyle's photography is also featured in the Spears Gallery), and they treat Larry with such tender solicitous respect that one knows that the goodness of the man runs deep.  

12/18/09 Aren't those sketches great... (or... not)

Looking for my post back in February where I first raved about etsy, I saw my "giving voice to the earth's pain" sketch.  I scroll over that month's sketches and photos and have to say I'm sure glad I have a Ruth around to say nice things to me! ;-) 

Well, ok, often it takes someone else pointing out the good in a piece of work, before others see it. So one might just see a child-like sketch of Archman running with a bunch of tightly coupled trash accumulating on his shoelaces and not realize that it is a remarkable visualization of the profound paradox that agile is vulnerable to--speed can slow the project down; moving fast and loose and not paying attention to clean-up (refactoring and simplification) can create a growing debt of kludgy code, so apparent early speed gives way to growing impedance.

So there we have it: architecture, personified as Archman, all set to be agile, but stymied by growing technical debt. And what do the words say (like xkcd, you need to scroll over to get the extras)? "Tied by its shoelaces to the past." Agile is all about today--tomorrow will take care of itself. Right? But if we don't take care today, indiscretion builds a debt that ties the system to yesterday and last month and last year. So that increasingly we have less confidence that tomorrow will take care of itself, and we find ourselves needing to rebuild the system to be competitive tomorrow. All that in one little sketch with a few scroll-over words! So, you surely agree, it's a little ruffian genius, really. ;-)

Anyone need me to explain the coupling and bad smells one? Or Archman addressing The Kludge. It's funny, but where I've seen my sketches "borrowed" it's been by non-software people, and they've been some of my worst executions but the idea has been conveyed nonetheless... like archman carrying the weight of the world, or archman walking a tightrope holding a balance bar...

I'm teasing--mostly myself!

[But, yes, pretty soon I'll be sending out those chocolate frogs to remind you that the 4th birthday of this journal would be a good time to say something encouraging. Or not. After all, it doesn't serve anyone to encourage me! :-)] 

12/19/09 Leading Change

I'm collecting material on leadership and change. If you come across anything interesting, do please send me the link.

• The Little Book of Leadership, presentation on slideshare

12/19/09 Goodness Me!

I see that the CouchDB presentation that raised a ruckus after the Ruby Conference back in April is ploughing its way up the view rankings on Slideshare. It says a lot for "sex sells" and machismo doesn't it?  Well, at least Martin Fowler took a stand. I did, but I pulled my post, because, well, I'm a Q<= and not very drawn to wars, really. ;-)

There is much that is quite broken in our field around machismo behaviors that in part actively and quite intentionally repel women, and machismo behaviors that subtly and erosively create a field that is inhospitable to women--and gentle men who have managed to avoid the strong socializing forces that prioritize arrogance and dominance behaviors over facilitative networked behaviors. But Daniel Pink, Tom Peters, and others are looking at the world ahead, and seeing holistic thinking, empathy, collaboration and so forth being prominent attributes. I look at the world of now, and see those as attributes that play well in architects.

And no, having women in project and middle management doesn't mean we don't have a problem recruiting women into, and retaining women in technical roles.

12/19/09: Lessons for Architects from Scoobie Doo

Sara came chuckling into the office and repeated this snippet from a Scoobie Doo movie they'd just watched:

Fred: "Teamwork. I do a lot of teamwork."

... Sara: "Get it? I do a lot of teamwork."

I can't tell you how many architects "do" teamwork in the first person singular... Not you, of course. No, you've never written the team vision, or the architecture document, to give the team "something to react to" rather than allowing them to participate in setting direction and creating the strawman that becomes a self-fulfilling prophecy... and the prophecy goes like this: because the team isn't engaged in the creation they don't feel ownership so they don't invest in creating a better vision or architecture. No. But where they are empowered, that is when they are building the system, they figure out what they must do to make the system and hence themselves successful. The architecture is sidestepped and subverted. So the self-fulfilling prophecy is that ostensibly the architecture that is proposed is accepted... but as soon as the rubber hits the road it breaks up--just like everyone knew it would, because architectures are always wrong... Right? I mean look at our experience--the architecture of the built system never corresponds to what the architect wrote down. Indeed, why bother?

Ok, I put that a little strongly... but it happens. I can't tell you how many times I've had to intervene and say architecting by document review is not collaborative and is not going to serve the best interests of the project because good ideas will not be surfaced and the people involved will not be fully vested in the outcome. Even if they think they are--even if they are diligently reacting to the document! Collaboration actually involves working together! Not all the time, but some of the time--enough of the time. Working together. Not presenting at people. Not doing a stand-up. Not looping through cycles of edits and critical feedback, constructive or otherwise. Working. Together. Getting ideas and issues and approaches and alternatives into the shared mental workspace of the team, through models and words--and doing that mental messing around and probing and experimenting in the shared mental workspace, and then, as judgment directs, in code. Because something different happens when people work together. They bounce off each other. If the chemistry is bad, the bouncing is unproductive. If the chemistry--the goodwill--is right, the interaction surfaces new questions, new possibilities and new alternatives, and reveals otherwise hidden challenges and risks. Etc. All the good stuff of applying a multiplier not just because more brains are energetically engaged, but because the interaction sparks new discoveries. We're each on our own trajectories, have our own set of predilections and blind spots. And talents, insights and intuitions. Yeah, yeah, we know this. And still we think we can insert creativity and weed out the problems from an architecture by review!

Oh, I'm not arguing against the improvement/validation steps in VAP! Remember "and." Model out loud and "in pairs" or threes and fours. And have dynamic group reviews. Yes, as the architecture matures, we get to the formal review and high ceremony bit--if the complexity of the system or organization demands it. But early on, it is better to start with a small team working intensively together than a document created by one person working alone. Yes, one person may own and do most of the work of documenting the architecture to ensure integrity. But just as there is huge value in pair programming, there is huge value in architecting in a small team.

The architecture of the built system will differ from the first conception of the architecture. But if we do more of the architectural learning cheaply with collaborative, exploratory modeling, the second, third and n-th conceptions of the system and its architecture will differ from the first conception too! It may be hard to know when to switch the design medium from models (including sketches, diagrams and, where they serve deeper exploration or documentation, more formal models) to code, but it is harder, generally, to get people to seek out different alternatives and truly explore opportunity. We have such a strong need to make progress, we tend to go with the first ideas that seem workable and elaborate those. And this is why we have so many of those "failures of imagination" (Booch).

John mentioned a choice criteria for UML modeling tools that makes so much sense, now that it is voiced: "does the tool allow quick and dirty modeling or severely constrain one to work with (all) the formalisms of UML." It might seem like having the tool drive correct use of UML is a good thing, but during the rough sketching stage one wants to be able to work more... well, sketchily. "Chaordic" applies not just to systems, but to modeling them too, it seems.

12/20/09 Amazing Grace

Last night we watched Amazing Grace again (the first=last time was back in March 2007--I know when, because I wrote about it in this journal. :-)  It's a good time to be challenged--a good time to watch the story of a great leader and the team that chose him, and survey the epoch-defining ills of our day.

And isn't that interesting. Something I'd never noticed until the moment I wrote that! The team chose him to lead--the team came to William Wilberforce and opened his eyes and helped him see. Yes, he was a great leader, rightly attributed with leading change, but it is so interesting that his friend saw what he could and should accomplish for humanity, and brought a team to help him shape the vision and the course of action. Wilberforce led. He clearly led. But he had great follows who led him, too. Perhaps that is a mark of a great leader--being open to being led, though not by any means being an irresolute chameleon type...

It is so amazing what we don't see! Until we see it!

... well there is so much more in the movie that I could debrief... attributes of a leader, action, a diverse team, passion, strategy, the role of an astute "manager," ... uncertainty in the face of repeated setback and the role of love... but... I'm not entirely ready for Christmas...

The "gift of encouragement"--what a wonderful thought, because we all need that encouragement every now and then. It is a good right thing to work towards system integrity and balance.

The book is one by Christopher Alexander, of course.

Merry Christmas (thanks for the pointer Paul) to all of you who celebrate Christmas. And happiness to all!

12/26/09: It can be discouraging when organizational complexity is is managed along divide and conquer lines, so that local optimizing maps to the organizational power tree, and architects have to work through "persuasion and influence" to achieve holistic system goals--then, it can feel like we're Sisyphus. We can take a proactive approach, and see our role in part as culture changers, but it can be a hard path and we have to have some magical internal fountain of eternal enthusiasm to keep going back, day after day, to pushing that rock of cultural change! It can be more appreciated than people affected express, so it is nice when someone like Jane takes those all-important, significant minutes from her day before Christmas to thank us.