a Trace in the Sand

by Ruth Malan





February 2019


2019 OReilly Software Architecture Conference NYC: Visual Design and Architecture

[in process] Annotated Presentation Slides

Visual Design and Architecture Cover SLide

This talk is about visual design and architecture. That means that while there is a lot about architecture and software design that visual design doesn't address, we're focusing here on visual design, and exploring where and how it applies to architectural design in an Agile context. While Waterfall development relied heavily on visual design because design was, at least in theory, completed before the code was written, in Agile development, design is recast.

Code Is Design

Now, many people are saying that code is design. And this isn't new.

Jack Reeves: Code Is Design

In 1992, Jack Reeves wrote an essay published in the C++ Journal, in which he compellingly made the point that the code is the design that gets built, and, importantly, we design in code. When we're programming, we're designing.

But we still need high level design

This was at the time that object-oriented design in the form of the Booch method and OMT, then UML, was gaining sway. Jack's insights stuck a note of resonance, and unbeknownst to Jack Reeves, his essay was taken up on Ward Cunningham's C2 wiki, and was being discussed with some vigor.

Something else happened, that Jack Reeves didn't intend, and that was that some took him to be saying "'the source code is the design' means 'don't do design, just code'." So in 2005 he followed up with another essay, to rectify this take. And he highlighted his point from the earlier essay, that "In software engineering, we desperately need good design at all levels." And "Designers should use anything that helps."

Back to Jack Reeves: Code is the full design

And he returned to his point that our sketches or diagrams and models are not the design the compiler builds—the source code is. Noting further that since we have to do the "real design" in code, "we should not be afraid to code our designs as we derive them."

Agile Manigesto Continuous delivery

At the same time, the Agile Manifesto was quickly gaining following and causing a sea-change in how we develop software. The first Principle, "Our highest priority is to satisfy the customer through early and continuous delivery of valuable software," establishes the focus on continuous delivery of working software.

Two faces of design: software and code

What is meant by software design, generally depends on who we're talking to.

If we're talking to product owners and user experience designers, design means design of working software—as experienced by users of the system. And it encompasses determining the capabilities and properties of the system and how it works from a user point of view, determining the avenues, scope, and particulars of user interaction with the system.

In the context of conversations with developers, design typically refers to design of the code, how the code is structured and how the code works (to deliver the capabilities and properties of the system).

The other design surface, and one that we are increasingly paying more explicit attention to, is operations and design for operability and observability -- where we learn about the code (system behavior, under load, environmental stresses including component failure, and other demands) and the business (user behavior, patterns and anomalies, etc.).

Iterating acorss the faces of design

In Agile, we're advancing both faces of design as we iterate, deliver working increments of value, get feedback, respond, and adapt. Users adapt to the system, shifting work to the system or doing new things the system makes possible for them, and new possibilities for the system come into view. And using Agile practices like Test Driven Development and refactoring, we advance the design of the code.

In Agile, we're advancing both faces of design as we iterate, deliver working increments of value, get feedback, respond, and adapt. We're not just gaining a clearer understanding of the system; the system is evolving. Users adapt to the system, shifting work to the system or doing new things the system makes possible for them, and new possibilities for the system come into view. And using Agile practices like Test Driven Development and refactoring, we advance the design of the code.

[Aside: We have used "right system, built right" and it usefully points to the two (sur)faces. That said, I want to "loosen" or shade the notion of "right." As we design-evolve the system, we learn what the system is becoming, what users and others want from it; but also, the system changes its contexts, raises new challenges and opportunities as it comes to be used in new ways. And so forth. The point is, it's a responsive process -- generally imperfectly responsive. We don't have perfect understanding, the process is strewn with trade-offs (read compromises!) and judgment calls (clouded by complexity and uncertainty, in the fog change churns up), and the system and its contexts evolve, so what "right" is, needs to evolve.

'In his book “The Reflective Practitioner,” [Donald Schon] took issue with technical rationality, which in its focus on problem solving overlooked or even ignored the issue of problem setting. He understood design as a reflective conversation with a situation, in which the designer frames and re-frames the problem, the process spirals through stages of appreciation, action, and re-appreciation. The unique and uncertain situation comes to be understood through the attempt to change it, and changed through the attempt to understand it.' -- Jean-Pierre Protzen

That is, to further complexify the design matter, we're not just designing these "(sur)faces" of design, but redesigning, in effect, (at least part of) the containing systems. Suffice it to say, at this point, that the design of the development organization, per Conway's Law, is not independent of the system design; if ignored, the organization design will (regardless of architectural intent and expression) impact the system design. The design of the system impacts the responsibilities and actions and choice sets of the containing sociotechnical system. Whether we take this into account or not, we're changing the context, so better to be more intentional about how those impacted want things to be, to better shape the system for fit to purpose(s) and to context. We're designing not just the system, but the system-in-context, taking into account that the system is changing the world for users and others for its various contexts of use and impact (ecological impact, too). It makes new things possible which is why we're building it, but has potential negative impacts too, and we want to make more informed tradeoffs and design choices. More on these considerations later.]

The rest is cake, and the cake is a lie

So this "the code is the design" point is even more compelling in the light of Agile, than it was when Jack Reeves first made it. "The cake is a lie" meme is well played, when documentation is "obsolete the moment it's written."

Design Emerges

Returning to the Agile Manifesto for a moment, the penultimate Principle assures us that "The best architectures, requirements, and designs emerge from self-organizing teams."

Goode design doesn't just emerge

But, as Erik Dietrich points out, good design doesn't 'emerge' like a ray of sunshine breaking through the clouds.

Good design is work

"It comes coughing, sputtering, screaming and grunting from the mud, like a drowning man being pulled from quicksand" (Erik Dietrich)—good design is work. Real work.

Code is design and design is work

Still, as Tim Ottinger points out, when we say "the code is the design," we mean we've done the intentional and painstaking work that design takes. Ideally, on both design (sur)faces—we observe, learn, improve and adapt both the software (as experienced by users) and the code (as experienced by developers).

[Aside: Pressure to evolve and adapt the system doesn't just come from building out the system incrementally. The context evolves. The landscape changes, but the system also changes its context. As users and containing systems adapt to the new or extended capabilities of the system, new expectations and demands arise. New demands surface new forces and challenges, and new solutions are sought. With new solutions or capabilities, the system is put to new uses. These cycles of co-learning (symmathesy) and co-adaptation (even co-optation), mean that we need to be moving the design along both fronts—software and code. Not just adding to the code, but treating the code design as a design surface where we also need to observe, learn, and improve the design. When we don't feed design learning back into the code design too, tech debt mounts. When entropy sets in, we've lost agility. See also Lehman's Laws: 1. Continuing Change, and David Woods' Law of Stretched Systems.]

What is missing, if all we have is code?

"Code is design" is compelling, for it places emphasis on code as a design medium that turns directly into something we can put in the hands of users, to learn, probe for effectiveness, and improve or adapt. So the point here is to appreciate Agile, and where we are, but also to allow ourselves to reflect. And, indeed, the Agile Manifesto begins "We are uncovering better ways of developing software by doing it and helping others do it," inviting this reflection and disciplined openness to improvement.

Which brings us to the framing, orienting question for this presentation: if all we have is code (including tests), and from the perspective of design, what are we missing? What would help us design better, and/or better express and communicate our design, or facets thereof? Given what we're trying to learn, at this moment we're at, what is the quickest, cheapest, most effective way to explore, surface and address design challenges we face? It might be code, but what else? When and why?

As we do design, can we do more to help ourselves and our teams with the cognitive challenge of designing complex systems, or help us collaborate better on the design, bringing our different perspectives, experience sets, and ideas to the design effort.

We're looking only for a difference that will make substantive, consequential difference. Being fleet and responsive means avoiding gratuitous, low impact work. We want to only consider visual design to do what we don't do in code better, or anyway.

Code is good enough for the complier but is it good enough for us -- evolving and improving the design

The code is necessary and sufficient design expression to be built (by the compiler). And any other (design) documentation is neither necessary nor sufficient to be built. Still, the compiler isn't the only player. We might ask if code, alone, is sufficient for humans to design and evolve complex systems. And if not, what helps, and what helps enough to be worth it?

Clearly, given how much undocumented or poorly documented (perhaps because what documentation there was, is now obsolete) code there is out there, delivering value, we have an abundant (by code mass) existence proof. Of something? Something like, we can and do deliver value doing design in code, and expressing that design only in code (including tests, hopefully). Of course, when legacy systems become entangled and ensnared by coupling and poor design, agility peters, despite, or because of, the malleability of code. Returning us to the question: can we do better?


Design is making current situations into preferred ones

Better at what? When we talk about design, what are we talking about? In The Sciences of the Artificial," Herbert Simon ("ground zero in design discourse" — Jabe Bloom) notes "Everyone designs who devises courses of action aimed at changing existing situations into preferred ones." This characterization is profound, for all its straightforward simplicity. He also said design is concerned with "how things ought to be—how they ought to be in order to attain gaols, and to function."

Grady Booch: design decisions shape

And architecture? Architecture is design. Not all design, but importantly, architecture is design. In particular, "architecture represents the significant design decisions that shape a system" (Grady Booch).

Jeff BezosL Irreversible decisions

So really we're talking about decisions, and in a letter to Amazon shareholders, Jeff Bezos made this important distinction between irreversible and reversible decisions, emphasizing that consequential irreversible decisions need to be made with great deliberation and consultation,

Irreversible decisions are like one way doors

for they are like one way doors, and once you pass through, you can't get back.

Other decisions are reversible

But most decisions are changeable or reversible, and you don't have to live with the consequences of making the wrong decision that long.

Grady Booch: cost of change

Returning to Grady Booch's characterization: "architecture represents the significant design decisions" where "significant is measured by cost of change."

Reducing cost of change and high cost of change

These architectural decisions then, may be significant because they make changes easier (within some range of change). Or they may be consequential and irreversible and we need to pay close attention to them.

Woods If wrong, your project faiils

As Eoin Woods notes, they are the decisions that, if we get them wrong, cause the project to fail.

Make or break'

If a decision impacts identity, mission or defining value, and the very viability of the business, it is strategic. If it would cost substantial resources and time, defer value or require major organizational change to revert and rework, it is strategic—it impacts viability. Likewise, if a decision meaningfully reduces the cost of making changes, enabling our business to be more fleet, adapting and extending its services or product set as opportunities emerge or as the market shifts, it is strategic—it impacts responsiveness and value creation. Change is not a value in itself; its value is in value creation, in responsiveness to need, as our market evolves and expands in new directions. Which sheds light on the bigger distinguishing characteristic of architectural decisions—they are strategic. They have high impact. They determine what value we offer and how, and how we will contribute value to, and play a role in, the ecosystem, especially when we take a longer view. They impact system identity and integrity, and set and shape direction.

Also, architectural design is how we address demanding system challenges—enabling capabilities and system properties given forces and constraints impinging on the system. Challenges that require design attention. They impact structural and organizational integrity and resilience. They are make or break. Significant decisions that need, as Dawn Ahukanna puts it, deliberate deliberation.

No design is not an option

For these decisions, the no decision decision is a bad decision. Or, as Douglas Martin put it: "The alternative to good design is bad design, not no design at all." The question is not "does the system have an architecture?" but rather "how good is it?"

[Douglas Martin quote via Kenny Baas-Schwegler.]

"Prioritise design, and real agility will be your reward" @stuarthalloway

Constraints reduce alternatives

To be sure, decisions constrain—they eliminate options. Alicia Juarrero observes that this is what we commonly mean by constraint—this limiting or closing off of alternatives; this altering of the probability distribution of available alternatives. But! In so doing, Alicia notes, they make the system "diverge from chance, from randomness."

But constrinats create alternatives

Constraints close off avenues, restrict the degrees of freedom, but if this was all they did, systems, including organisms, would just do less and less, as they became more constrained (Alicia Juarrero).

Constaints enable properties to emerge

Instead, "constraints not only reduce alternatives—they also create alternatives." If we take (Alicia's example of) language, the constraints of syntax allow meaning to emerge.

Ground under the feet

Architecture decisions, at any point along our system's development path but especially early, "put ground under the feet" (Dana Bredemeyer) and enable us to "gain traction." They move us from potentiality to actuality—closing off options but enabling value to be created; to be able to move forward, we have to start to shape the space. To create a bit of clearing in the "great blooming, buzzing confusion" (James) or fog of uncertainty; to start to do something, and see how we're doing.

We build on these decisions, and they tend to become embedded within assumptions about what the system is, and does, as well as how it is built, we need to probe and test and improve these design ideas as quickly and cheaply as we can, while they are still reversible.

Return to our framing orienting question

Now, if we return to our framing question, namely "what is missing if all we have is code?" I have obviously primed the answer: architecture decisions. These are decisions that shape, that address cross-cutting concerns, that have non-localized impact on and so aren't typically visible just in terms of code. They are consequential, success and failure defining, constraining and enabling, and potentially, and over time tend to be increasingly, irreversible. Decisions we need to attend to—in making, and communicating, effectively.

Architecture Decision Template

Various architecture decision templates have been published, including by Jeff Tyree and Art Akerman then at Capital One (in IEEE Software, so this template and discussion gained exposure and influence), and Olaf Zimmerman at IBM. But Michael Nygard's simplified (yet well-described) template caught on as a just enough version for documenting architecture decisions in an Agile context. The Architecture Decision Record, then, documents decisions in terms of the statement of the decision, the outcome sought and the forces weighed in the making of the decision, along with consequences or implications of the decision. (The Tyree/Akerman and Zimmerman versions also keep track of alternatives considered but ruled out, and this is valuable too.)

We think the decision is our contribnution

Now, it is tempting to think, given that it requires our expertise, that the decision is where our focal contribution lies. The decision -- what we're going to do, the approach we're going to take, is the hard stuff. And it is important -- if it's architecturally significant, it's make or break. Or we wouldn't make the decision part of the architecture decision set, persisted in the Architecture Decision Record. The higher the cost and consequence of failure, the better we have to understand the problem, including the forces and tolerances.

Microservices tradeoffs

As architects, we're seeking to surface what tradeoffs we're making. In order to achieve the outcome we're targeting with the approach we're considering, what forces come into play, what are the downsides, what negative consequences do we need to seek to mitigate?

In this example, it is being pointed out that microservices help us achieve scalability and fault tolerance, but come at the cost of additional complexity in designing and managing distributed systems. Other gains include greater development team independence, which we see again in the next example.

Architect story time with Ruth

Let's spend a moment and read this (above) from Mattias Peter Johansson on Quora, about Spotify.


We see that allowing duplicate instances of different versions of various libraries enables Spotify squads (teams) considerable independence, removing the need to coordinate with other squads on libraries and versions. Because song size so dominates considerations that it generally falls beneath the threshold of sensitivity for the user, the tradeoff of team freedom for app size is easily (in their view) within the design acceptance envelope.

So in this case, a technical decision is being made for organizational gain (lowering team coordination costs and increasing team's degrees of freedom) at the expense of app size, which works as long as it's below the app user's tolerance threshold for resource consumption.

Decision value depends on context

That Spotify example illustrates this point that Diana Montalion makes: "the value of every decision depends on the context in which we make it." Diana illustrates with The Lord of the Rings and Frodo's journey to destroy the ring: his quest is meaningful within that context.

Sarah Mei: properties are joint

Sarah Mei makes a related, important point: system quality is a joint property of the system and the context. Emergent properties, or quality attributes of the system, aren't in this or that line, or lines, of code—we don't see them, when we look at lines of code. They emerge from interrelationships within the system, and with the environment or context of use, for various contexts of use, as well as context of operation, and/or development context(s).

Engineering is about tradeoffs considered in context

The choices we're making are technical, but the impacts don't remain neatly in the technical space. The tradeoff space isn't just about qualities that impact developer experience, or security properties or operational complexity, but user experience and partner experience through properties of the system in use. And more. So we investigate the upsides and downsides of our technical decisions, in these various contexts.

[Aside: I had a version of Cook and Rasmussen's Dynamic Safety Model at this point, but it was one of the things I eliminated during practice runs of the talk, to bring it within 90 minutes. It can be used to inform how we think about technical debt, for example—in that case, we're dealing (in this view or modeling frame) with developer workload boundary and we can think of the "performance boundary" as the changeability or adaptability boundary (the failure threshold has to do with coupling and other forms of debt, that thwart adding features and raise cost of change). In the operations case, the performance boundary has to do with scalability and resilience—failures that cause loss and/or distress for customers and news vans to line up outside headquarters.... Etc. ]

"We're trying to find habitable zones in a large multidimensional space, in which we're forced to make regrettable, but necessary, trade-offs." — Robert Smallshire"

The design envelope is multidimensional and we need to figure out what the dimensions of sensitivity are, what the thresholds are beyond which we fail, but which we may not even notice until we hit them.

Different mental models

That's a lot to bear in mind. Literally in mind. We all have mental models. They are all imperfect. And they are all different. But they are the basis for our decisions—consequential decisions; decisions we may not be able to back out of, especially as other decisions become layered upon, and even entangled with, them.

Decisions aren't just what was decided, but what made that decision make sense

So we have this composite structure for decisions, where the decision needs to be articulated in terms of the context, of the outcomes sought and the forces at play.

Not just about how we record and convey or communicate decisions, but about how we make better decisions

And making decisions, as well as conveying decisions, is as much about what is relevant in the context as it is about the decision we make in response. If we want to make better decisions, and convey them well,

What we need, to make better decisions

we need to have a (good enough) conception of the desired outcome(s), the forces we need to weigh and constraints we need to take into account, as well as the significant consequences and side-effects of our decision.

Developer contexts -- in contexts

Where these goals and forces arise in various contexts—development and operations, the contexts of use, and the broader value network. We're concerned with factors impacting developer experience, and forces arising from development constraints, capabilities, organizational forces like coordination mechanisms and costs (differing for collocated and distributed teams, say) and taking into account trends in the technology ecosystem. And so forth, also for operations engineers, management teams, security teams, as well as customers and users in various segments and at different points on the user path, as well as partners in the value network—channel partners, others adding value to our products, and more. Architects play a unique boundary spanning role, and we need to see (architecturally significant) impacts others would miss (because their locus of responsibility and experience is more focused).

formulating the problem is the problem

Horst Rittel, the design theorist who coined the term "wicked problem," also said "formulating the problem is the problem." Donald Schon, another pioneering system thinker, wrote (in The Reflective Practitioner): “problems do not present themselves to practitioners as givens. They must be constructed from the materials of problematic situations which are puzzling, troubling, and uncertain.” Design is fractal and emergent. We move (in both directions) between problem and solution,

Peter Naur: The theory of the problem and the theory of the solution

constructing a theory of the problem and theory of the solution (Peter Naur). It is a dialogic process, where interactions among the team, and with the system and users, inform and make more shared, the theories of what the system is and why it is that way, and how the code addresses what users need, and what its designer-developers need. But the team changes. The context changes—the system. Changes. What to do?

Saarinen: design in next larger context

In a book written for building architects, but which has translatable lessons for software architects, Eliel Saarinen is quoted: "Always design a thing by considering it in its next larger context." Design a chair to be used in conferences in that context, which differs from your office and designing a chair for those demands and that setting.

conext in context

So we design the parts of our system in the context of the system, the system in its various contexts—of use, operation and development. And that system-of-systems in the context of the ecosystem and its various value networks.

Ask not just what users need but also what developers and operations need and others in the value network

Alternately put, to develop our "theory of the problem," or to "load" the context into our mental models, so that we can uncover this multidimensional tradeoff space, we need to ask (not just) "what do users need?" but also "what do developers and testing need?" and "what do our operations and security teams need?" and "what do others in the value network need?"

We architect across—across boundaries: across not just the code and the teams involved, but across the internal system design and design of the system-in-use or system-of-systems design; across the different languages and concerns of these different spaces, the technical language of code and test and integration, deployment and operation, and the languages of the domains where the system is used; across the turfs and sense of ownership and decision responsibility; across views and perspectives; etc.

But of course, we can't attend to everything, at least not all at the same time, in detail. We "zoom out," as it were, to scan the ecosystem or value landscape, to identify opportunities and challenges that do warrant closer attention. To set framing for the problem, to understand the trends and forces that shape and constrain it. To get a bearing on the ecosystems that are or will be impacted.

Context Map

One way to get many perspectives represented in a (literal) "big picture" of the context, is to use the Context Map approach developed by David Sibbet at The Grove. To get a sense of how much comes together in a map like this, we're going to develop a Context Map or Competitive Landscape Map, from the ground up, as it were.

Start with a green line

So we start with a green line on the horizon. And the idea that there is an opportunity open to us, that is worth exploring. So here's the thing. We have these disasters like the fires in California and the hurricanes and flooding on the Gulf and East coasts, and other disasters. And while I'm commuting or cooking dinner and listening to the radio, I might get to hear one person's story, or some person with a lot of followers may retweet someone's GoFundMe. We know that others need help too, but we don't have access to their stories, we don't know who they are or what they need. And even when it comes to the stories that are told, we don't know what the response was—did it fill the need?

Add some structure(s)

Now, we have our horizon. First, we'll add some structure. Or structures..

Add some structure(s)

We're looking for who would use the system, and what their concerns are. In this case, people who need help, and people who want to help, are central, so we make space for them, and start to contribute what we know or sense their concerns, goals, frustrations, fears, are. So those impacted by a disaster want to rebuild their life, but are concerned about dignity and privacy. Those who want to help, want to figure out how to help in a way that makes an impact, a real difference, for someone. Helps recover and rebuild.

Idenify what others bring to the table; what differentiates them

Of course, there are other players who contribute in this space, and we don't want to duplicate what they already do well. They're part of the landscape we seek to understand at least enough to differentiate and partner. There are the early responders to large-scale disasters, like FEMA and Red Cross. They mobilize resources at scale. There's local organizations, like shelters, food pantries and Goodwill and Habitat for Humanity. These may be impacted by the disaster themselves, or overwhelmed following a larger scale disaster. The Human Utility (currently serving people in Detriot and Baltimore) deserves special mention—helping get overdue water bills paid, to help people with essential life needs. There's the likes of GoFundMe, that together with social media to get the word out, can be used to raise money, but these requests can be drowned out on general-purpose social media.

Add "winds of change" or trends, challenges, identify opportunities to add value

We add "winds of change" or trends in the market and technology space: increasing number and impact of climate change disasters, prevalence of mobile and social media, but with it increasing qualms about privacy. We identify challenges: spikes as media drives attention to a disaster, but rapid drop-off in attention thereafter; concerns about how to fund the system; and so on.

As we fill out more of the landscape, the "big picture" we're all helping to fill out, becomes more "our shared picture." Something we've heard different perspectives on, given different stakeholders in the room. And out of all of this, we're starting to glimpse a system worth building. And we're learning from what others in the room bring, in terms of experience and insight, but also their assumptions and goals or needs and positions.

But. Why architects? We have a contribution to make, and understanding to take, from being involved.

"When we take on architecture responsibilities, our center of gravity shifts from tech to delivering business value through technology." — Dana Bredemeyer

We're technical leaders, and

"Leading is helping people to make sense of the world." — Esko Kilpi

Now explore how the system might work

Rich pictures allow us to informally and quickly model socio-technical systems as (collaborative) interactions among people and systems, and systems-of-systems. We can "sketchprototype" to explore various ways we might go about the need we sense, to better understand it, and how we might go about addressing it. Asking (and exploring visually, just enough) "how do things work now?" And "how might they work?"—what would produce "preferred situations" (to refer back to Hebert Simon's characterization of design)?

Here, we've come up with the idea that we could support the creation of caring circles, where someone becomes a caring circle coordinator and sets up a caring circle, enrolling friends, or colleagues or some group, maybe their reading circle or friends in the local Papers We Love community, to help an individual, family, or organization affected by the disaster, that needs help. And we use rich pictures to explore how this might work. We can try out different ways this may play out, in different settings, and it's all as cheap as a sketch, and getting a few people to sketch-storm together on what they'd like to see.

Explore from different points of view, stepping into the experience

So we took in the landscape, at a high level but sufficient to see across the ecosystem(s) we expect to provide value within. Then we narrowed our focus just enough to explore the system, and how things work now, and how the system might play a role, within those systems of interacting systems. And now we focus on individuals, but we take care to include within our compass not just direct users (or would-be users) of the system, but others who are or would be impacted. Developers, testing and ops engineers, senior managers. Others in the value system who we want to partner with, or provide value to, who we don't want to disrupt but rather further enable. And so forth. But importantly users. The various design tools need to be in our kit to use just enough, just in time, including interviews and shadowing. Still, Empathy Maps (from Gamestorming) are a useful way to imaginatively enter the experience of these various stakeholders.

Explore goals

And Stakeholder Profiles can be a useful way to capture and summarize the goals (business/personal and system) of the various stakeholders, so that we keep in view the desired outcomes across the space that we need to weigh and balance as we make the tradeoffs that result in the decisions we make, rather than alternatives we decide not to pursue. It serves to understand the space, and then to remind ourselves and others, that architecture decisions require system perspective, weighing, resolving or balancing forces that arise from, and impinge upon, various stakeholders and systems.

Wardley maps and business model canvas

So we iterate, at broadest scope, informing the strategic direction for our system development, and at more narrow scope, informing and probing and improve-adapting the design work, as it unfolds.

At ecosystem scope (given the system or system of systems we're designing), we're exploring value and relationships, to determine how we will differentiate, creating and delivering unique and compelling value. We're looking at leverage points (Donella Meadows) and what else we need to do, strategically, to be sustainable. (For example, iTunes was essential to the success of the iPod, so digital rights management factored. For Tesla to be successful, a network of EV charging stations was needed. The key is looking for what else is needed, to make the value we're trying to add viable.) And we're investigating opportunities to partner with others in the ecosystem, to create a value network.

A Business Model Canvas is another "big picture" view, that pulls together how the business creates value for customers (value propositions delivered through channels to customers segments, activities and resources needed to create value and partners in value creation) and for the business (revenue streams and cost structure). While a Wardley Map shows a value chain where each component of value is shown on the evolutionary path of its ecosystem, in order to make strategic decisions about where to "play" in the value network.

Exploring the system design

A Customer Journey Map explores user experiences over time as they come in contact with different parts of the system (or system of systems, as in the case of a patient going through treatment in a hospital, interacting with different departments and services, and people/roles). This informs UX design (and information architecture), as well as capability design. EventStorming and DDD's domain models also have this quality of dual use — they help understand the domain/problem, but in a way that directly informs the design of how the system will be built (design of the system internals).

At any rate, we can do this in the early stages, as we explore how the system will work, in terms of interactions over time. Building, elaborating, nuancing our sense of how things work, and how the system will fit into and support or enable that. And what our system needs to be and do.

And we can return to these tools, as judgment determines, to explore context shifts, to explore how to serve new customer segments, and so forth.

Other ways to visually explore systems design

We explore assumptions, and relationships between outcomes or goals and what various players might do to achieve or impede them.

The idea isn't, by any means, to use all of these tools. It's useful to notice that different views draw out, or into view, different facets of "the problem." We do need to work some—just enough—at different "zoom levels" and switch our point of view. Work some at the ecosystem level, to set direction. Work some at the system level, and take into account various stakeholders including, but not only various users of the system. Just enough of this, to figure out what's most important to start with. And then we iterate and probe or test our assumptions, suppositions, and working theory of where value lies and how to build it. In sketches and models, just enough, where that is good enough, and in code, to deliver and assess and evolve the system.

Not trying to take over, but to partner and bring technology perspective to design table, and build mental models that will infoirm tradeoffs/decision making

Ideally, anyone on the team can be involved if they want to. Usually this is seen as the purview of Product Managers and Product Owners, and (at least lead) UX designers. The idea here is for software architects to partner well with product owners and user experience designers, to bring technology perspective to the design table, and to build mental models that will inform tradeoffs and decision making. It is about building just enough insight into the context (what we sometimes call the "problem space" but this is a rather unfortunate characterization; and yes, I do it too).

At any rate, we (architects, whatever our title) bring technology know-how/why/what/when/what-if and what else/what then to the strategy and design table. We can help answer: what's on the horizon; what's in the landscape; what does it mean in terms of how we can differentiate; can we build this capability, and should we? And by bringing developer and ops engineer experience explicitly into the design space together with user experience, we're making trade-offs that contend explicitly with the various outcomes and forces that shape sustainability in all these ecosystems. Sustainability in all the senses—technical, economic, social and environmental.

Use Case Diagram showing capabilities of CaringCircles System we've building loading into mind

But as much as being involved in this facet of design is about "loading" the context into our mental models to inform decisions that inherently involve tradeoffs, it is primarily about designing the system. Deciding what capabilities are identity-defining and critical, and what we should build now and soon, to lay foundations technically and in terms of value users and others in the value system care about, and want us to build on, creating pull for the design effort. Use Case Diagrams (top level, and drill-down views) show system capabilities we have decided to build (and external systems, for capabilities we will leverage or collaborate to offer). And Use Case Diagrams are useful as we add capabilities, for we can see where capabilities are not yet supported and new capabilities must be added, and where capabilities need to be modified or extended.

Use Case Diagrams may seem passé; they're a thing we tried but is now in the rear view mirror. I still like them for their "system-on-a-page" view, that allows drill-down into the high-level capabilities that matches our ability to drill-down in the components view (when we look inside the system, at its internal make-up of components and relationships). Capability Maps (showing capabilities and relationships) and User Story Maps can play the same role. The important thing is to have a way to represent the capabilities, to assess whether the system will do what is needed, to be fit for purpose. And then to take those capabilities and transform them into responsibilities of the system (informing architecture views).

Context Diagrams and Use Case Diagrams

Simon Brown's Context Diagram (the first C in Simon's C4 architectural modeling approach), gives us the same ability to summarize on a page the users and external systems, and high-level identity-defining and essential system capabilities. The drill-down in C4 from the Context Diagram is into the system, to the Containers (second C) view, and that is good too.

(I chose Simon's Tech Tribes example for illustration here, because it has some overlap with the Caring Circles system in that we have users who create tribes in the one, and circles in the other; both have content assembly and viewing, and so forth. And there's much divergence, obviously. Differences inform too.)

Architecture design is system design and system design is contextual design


Taking stock of where we are then: Architectural design is system design. And system design is contextual design. One way this expresses itself, is in decisions about what the system is and does, and how that reshapes what its containing or collaborating (sociotechnical) systems are responsible for, and do, and how. That is, design is not just about design of the system (and its internals), but its impact on other systems. Of course there are limits to what we can do, and hopefully we aren't redesigning users so much as offering them opportunities to redesign or redistribute their activities for themselves—but the point is to not ignore these effects, nor the forces they produce. And further, design of the system internals, needs to inform and be informed by this design of the system in context. The capabilities of the system, and the system properties.

"We shape our tools, and thereafter our tools shape us.” — Marshall McLuhan

"The iron law of tools—that which does for you also does to you" — Gene Hughson

Ackoff on systems and properties

So we've discussed using visual models to explore value, and hence system capabilities. But system design is also concerned with the properties, or characteristics of the system. In this roughly 10 minute (starting at 1:12) talk, Russ Ackoff covers and illustrates the key characteristics of systems. Notably, a system has properties that none of its parts have, on their own. When we take a system, decompose it into its parts, optimize the parts, and put them back together, we don't even necessarily get a working system. To see this, imagine you have the best automotive engineers in the world pick the best carburettor, the best fuel pump, distributor, and so on. Now ask them to assemble those best parts into a car. Most likely they can't because the parts don't fit, but even if they do, in all likelihood it doesn't work very well. And at any rate, we can't say anything about the properties, since they are emergent from interactions among the parts, and with the context (stopping on gravel versus pavement, etc.).

Rebecca Wirfs-Brock: Quality isn't by magic

To design for more the properties we want, we need to get a sense of what we want. For design to be intentional (towards preferred), we need to bring what we intend more into view.

Michael Keelings's quality attributes workshop

A lightweight way to do that, was developed by Michael Keeling based on the work of the Software Engineering Institute (SEI) on system quality attributes and quality attribute scenarios. It has the properties of many of these visual meeting/workshop style whiteboard/flipchart tools, in that is supports a group coming together to identify which characteristics or quality attributes of the system need to be prioritized (axes, and placement on the axis, of the qualities kiviat) and refined as attributes of the system that can be assessed.

Rebecca Wirfs Brock and Quality attributes Landing Zones

Rebecca Wirfs-Brock introduced another variant, leveraging ideas from Tom Gilb, which emphasizes that there is an acceptance range, not just a target. She likens this to landing zones on an aircraft carrier. At any rate, there is a minimum level (for qualities where more is better), below which we fail, and an outstanding level where we would delight users (and, potentially, customers, if these are distinct) but where the cost of reaching that level is such that our target is set lower.

Quality attributes shape our (multi-dimensional) design envelope, and having a range rather than just a specific target recognizes that we're making trade-offs to balance across various properties or qualities, and we may have to accept less than outstanding, or even the target, to be adequate along another dimension.

The team at the SEI, including Rick Kazman, Len Bass and Paul Clements, has done a lot of good work on quality attributes and articulating them unambiguously, so that we can evaluate our decisions, designs and systems for how well they do against them. If you like the idea of incrementing towards better fit to fitness functions, getting a clearer sense of target properties is important. And there is a good body of work behind quality attributes that is worth drawing upon.

Grady Booch: Forces

Decisions are about tradeoffs, but the design envelope is multidimensional and we need to figure out what the dimensions of sensitivity are, what the thresholds are beyond which we fail, but which we may not even notice until we hit them.

We seek to create a system that is fit to purpose and to context, for its various contexts of use, and contexts of development and operation. This fitness landscape is multivariate, and it takes weighing and resolving forces (where the forces are coming from every whichwhere), balancing and making tradeoffs. Constraints may be bounds in technology, physical limitations, or organizational and economic. To cope, we try to stay aware of what is shaping, detemining success (how we create value, and for whom, because it's not just about keeping customers engaged) and failure. We're dealing with complex, interacting matters of integrity -- structural integrity and robustness and resilience (adaptive capacity not just recovery capacity); organizational integrity and ethics; design integrity and cohesion, fit, and adapted affordances. It's not just that business goals, goals of different groups within the organization, and individual goals, and customer (and user) goals rarely line up so tradeoffs have to be made, but further, none of these may understand their goals well and more, they change, as contexts and needs and attention shifts. And then there's the question of limited resources and constraints on communication and co-ordination. We have to make it tractable.

Bucky Fuller: what's the most important thing to be paying attention to

We need to use judgment, to steer our discovery process, directing design attention where we need to, given the moment we're at. Disciplines like "You Ain't Gonna Need It" (YAGNI) and defering to the "Last Responsible Moment" need to held a little more loosely, as we decide what it architecturally significant and needs to be explored, probed, better understood and assessed because it changes what's ahead. Maybe it sets us up to be more responsive and agile. Maybe it shapes system or organizational identity, or is key to system integrity. Maybe it's critical to what kind of system this is. Each "maybe" signifies a judgment call, or set of them. There's the joke: "what answer distinguishes the architect?" "It depends." But a good architect tells you what it depends on.

So we look for defining moments, telling, shaping moments, and seek to identify what matters. In part, this is about "lifting our head" from the work, and identifying challenges ahead, and deciding how to explore and discover what is significant, and how to explore what our design options are. Writing code. But with an eye, sometimes, on the horizon, scanning for what's ahead, what's taking shape.

OODA is a loop

Agility is about sensing and responding—observe, orient, decide, act in quick cycles. We need to ask: what, at this extraordinary moment, do we need to be paying attention to, that no-one else is paying attention to, but which is significant. Recall, architecture is about what is make or break. What is setting up to cause us to fail, and how do we not do that? What opportunities do we have to add value, and how do we do that? What should we do now, and soon, and what can we defer. We're observing and orienting at different scopes -- tactical and near-term, sometimes, but staying alert, and maintining a discipline of scanning more broadly, across boundaries, across time, from time to time. As we accept the adaptive responsive nature of Agile and co-evolutionary development, we let go of the need to be done. Now, strategy, design and coding-testing, all become ongoing, but rely on a more dynamic process of observing and orienting, to decide and act. And probing-testing the actions we take, to reverse out of them before the beome "baked in" to assumptions and habits (our own, or users and partners in the value network), or workflows, or coupled into more structural entanglements and dependencies.

What is missing from code

Now, it's all very well talking about visual design in the domain we associated with, well, designers. When we're trying to understand the world of systems and people our software will fit in, and shape and be shaped by, it's bigger—so much bigger!—than our code and system. So models (representations, abstractions to skim for relevance and to understand, etc.), developing theories of "the problem," and experiments to test them, make sense. Including and reiterating for emphasis, the models or sketches part.

Up to this point, I've been arguing for the involvement of architects in design of what the system will be (the "problem" or "requirements" space). To help build understanding of the domain and the opportunities technology brings to it, to inform the technical design work, and to keep that understanding from becoming out of date, by remaining plugged in to design of system capabilities as the domain and the system evolves. And we need to design across what the system will be and how it will be built, because tradeoffs don't stay neatly compartmentalized. So there's that.

As we transition to talking about visual design and its role in designing how the code will be structured and how the system will be built to yield those capabilities and properties, let's take a step back and consider what we get from making design visual.


Table of Contents: Annotated Presentation Slides

Slides and Annotations. Note: broken into parts because with the slides as images accompanied by annotations, it makes for a big download. Even split into parts, it's a lot to download, so don't use cellular to access unless unlimited plan and lots of patience hitting refresh. :-)

Part i. Slides and Annotations: Introduction and Context setting around Design in Agile; Decisions and Constraints; Decisions and Tradeoffs; Getting to know the domains (contexts of use, development and operations, value partners and others); Design and expressions of system value, capabilities and properties

Part ii. Slides and Annotations: Why Visual Matters to Design, some exemplars we can learn from, and lessons we can draw about why we need to bring visual models back into our design toolkit (some already do, obviously, but why more of us need to)

Part ii. Slides: All the slides are here, but this section still needs to be annotated. It covers architectural design of the system (internals). More to follow.

All, Parts i, ii and iii. Warning: big file to download as there are lots of images due to all the slides. Parts i and ii, slides with annotations and Part iii, just slides so far.

Visual Design and Architecture Slides on Slideshare -- log in to like and download.

PDF here

Thank you to everyone for the mentions, likes and retweets and reshares on the various social media.


About My Work


I also write at:

- Bredemeyer Resources for  Architects



Architecture astronaut





Copyright © 2019 by Ruth Malan
Page Created: February 9, 2019
Last Modified: August 29, 2020