In my family, we have a tradition, that on the first day of school, my
wife and I drive or walk the kids to school or load them on the bus,
and spend some time together over breakfast remembering the summer, and
thinking about how they have grown, or any school concerns we have.
Usually my wife is concerned about being alone more.
Over the years it has been a precious tradition, and my wife
and I have come to rely on it. My kids have too.
That day my 8th grader was unhappy that I was not going to be
there to see him off to school.
I was scheduled to be off work the following day, as I was building a
storage shed in my
backyard. Since I arranged for a friend to come and help me
with that, I had already ordered the lumber and torn down the old shed,
when this meeting came up, I didn't want to try to reschedule the shed
project.
My attitude that morning was one of resentment. I had very
little confidence that this meeting would accomplish it's objective.
We had a specific technology problem that we had been
postponing solving for 6 months. Several times people had
offered suggestions and attempted to build consensus around a solution
and it had not happened. I was one of those and I got beaten
and bruised for my effort. Our project was critically late,
and at risk of failing it's political viability date. That is
the date at which if we haven't delivered, it is not politically viable
to be affiliated with the project.
Before the meeting I said this prayer:
In spite of my poor attitude, the meeting did go well and we did reach
consensus.
So why is this about priorities? Because I had a personal
conflict of priorities around this meeting. This meeting
became
critical, because of our lack of priority around resolving this issue,
which has now become the critical path of our project. As a
manager, I recognize that part of my job is to make sure that I (or
someone I assign) control
the critical path of each project by making sure that all the
appropriate decisions get made before they hit the critical path.
While there are always things that happen that are beyond our
control that change the critical path - resource issues, and external
forces that are unforseen, but the vast majority of things are well
within our control, if we just would exercise that control.
In a phone conversation with a liaison from a software consulting firm
that I employ, I lost my temper. I said things that threaten
the
relationship, that could make working with this person, and his firm
much more difficult.
In a phone conversation with a liaison from a software consulting firm
that I employ, I lost my temper. I said things that threaten
the
relationship, that could make working with this person, and his firm
much more difficult.
I let the frustration, and annoyance that I had been experiencing in
dealing with this group leak out in my words. Now, when the
project is in the red zone, is not the time for this. In any
case, the lack of maturity and self-control that I displayed could have
had dire consequences for the project, for the company, and for my
career.
I was lucky, there was little direct damage, and the collateral damage
was manageable. The biggest consequence was my management
realizing how volatile I can be when pushed past my limits.
In
twenty-four years of employment I have probably only had
two or three meltdowns like this.
--
What can I learn from this? How can I use this experience to
diagnose risks, to identify warning signs, so that from a project
perspective, I can find bombs like this before they go off.
How can I communicate to my management the conditions that I
was
asked to work under that allowed the bomb to form. Where are
the
raw materials, the timer, the detonator? In other words, how
can
I be prepared to dismantle this bomb before it goes off?
--
Raw Material
Bombs require raw material or explosive - TNT, Nitro, C4, or some
incendiary agent, benzene, gasoline, etc. In this case it was
the
combination of two ingredients: a vendor who expressed a certain
arrogance and unwillingness to work within the processes that constrain
our other resources. Even the smallest facets of their work
required adaptation by any person who needed to collaborate them.
Whatever the reasons they had for doing this, it created an
attitude of resentment in me, that I simply repressed and stuffed down.
It's not that that was my first response, my first response
was
to confront these issues, and negotiate some compromise that allowed me
to get more of what I need. That bringse me to the second
ingredient: a perceived lack of control or power to change things.
My manager(s) were very sold on this firm, and I inherited
the
relationship when I joined the project. As a manager, I have
had
varying levels of control and have had managers who were more or less
prone to direct me in specific details. I had never
had a
vendor with more juice with my own boss than I had before. It
was
a new experience, and not terribly pleasant. It seemed that
every
time I tried to bring some visibility or accountability to bear, I got
thrown under a bus and the vendor was allowed to skate. So
the
second ingredient could be a lack of support, or empowerment; or it
could be a lack of control; or it could simply be my not asserting
myself by presenting my manager with an ultimatum (I manage or you
manage, but I can't manage if you wont stop managing).
The Timer
Projects have built in timers. The schedule is
always
agressive, pressure builds during the project and as things slide,
crushing forces converge. As things move towards a state
where it
is obvious that we cannot succeed without superhuman effort there is a
building of pressure that allows events, comments, or circumstances to
trigger an explosion. In this case it was the
many accomodations that I and my team had made or tried to
make to
help this consulting vendor be successful, from pulling work back to
hiring a project manager and analyst, to hiring more resources to work
with their team. The harder I worked to help them, the more
they
expected me to do, and the more they attempted to identify me as the
excuse for their slippage.
The Detonator
Most explosive materials will not combust spontaneously, but require
some initiation or detonation to start the explosion - be that
electrical, chemical, or physical. In this case it was
chemical.
Consulting firm hired a new liaison who in his first week
started
to push expectations towards my team for work that we had agreed to do,
before he was on the scene. He didn't even know that when he
pushed the button that there would be a reation. However,
when I
started to get defensive, he just kept pushing the "BIG RED BUTTON".
Eventually, the detonation was effective.
Instructions for
dismantling
When I find my self in this situation, I need to make sure that I
disconnect the detonator before it gets pushed repeatedly.
Take a
walk. Hang up the phone. Tell the button pusher
that you
will get back to him later. That is a quick fix, but it
doesn't
dismantle the bomb.
I need to deal with the materials or the timer for that.
Behind
each of these was a conflict or a confrontation that needed to happen.
Each conflict or confrontation had consequences. I
needed
to take responsibility for allowing the ingredients to mix to create
the combustible mixture. By not facing up to the conflict, or
confronting the situation I had allowed my own attitude to deteriorate.
I had chosen to absorb all of this into my own
world, and
had not taken responsibilty for it. I allowed myself to move
into
the victim quadrant.
The truth is that my issue was not with my consultant, but my
management. I needed to identify the consequences of the
nature
of the relationship with this vendor, and allow them to mitigate those
consequences. After all, this wasn't personal, but business.
If management was not willing to acknowledge the
consequences,
shame on them. Managers (like me) are paid to do the right thing;
the right thing for the company, the project, and the staff.
When
they make decisions that have consequences that they are unwilling to
acknowledge or manage, they put the people who report to them in a
trick bag. That does not absolve me of the responsibility for
staying in the bag.
As I now see it, I had several opportunities to crawl out of the bag:
1) Confront my manager with the issues - give her an opportunity to
fold up the bag and move on.
2) Work through an alternate tactic or strategy that mitigates the risk
- perhaps my manager feels she has no alternative.
3) Expose the risk beyond my management circle... This is a
very
dangerous because the person you confide in needs to protect your
confidence.
4) Move away from the project - as a last resort, one can always simply
choose to walk away.
I actually contemplated 3 and 4 at various times, and thought that
"hanging in there" was a better strategy. I realize that I
discounted numbers 1 and 2 without sufficient consideration, believing
that I would not be heard or respected. I assumed that the
managers that had put me in the trick bag was unwilling to make a
different decision.
--
Finding It
A week later, things are different. My managers are now
aware, of
how much I was stomaching and how much resentment had built up.
The vendor is now aware of how close their relationship was
to
being fruitless. They have made some adjustments and somehow,
more progress is being made than before, and more people are happy than
before. I don't attribute all of the change to this event.
I think that some of it was planned or working itself out
before.
I thank God that things did not turn out much worse. This
certainly could have been a career limiting event, or even a ticket to
a worse position. I also recognize that a more controlled
confrontation can be very effective. I should consider being
more
confrontational when things are difficult and change is required and
slow in coming.
What you want is someone who will turn the handle and GRIND IT OUT!
You want someone who pulls out a machete and hacks his way
out of
the details and decisions to a solution! You want someone who
makes his own tools, proposes answers to his own questions, and rips
raw meat off the bone! You want someone who schedules a
meeting
to start the project and another to plan migration when it is complete!
In my experience, most contract employees get this. They are
mercenaries, hired guns who may have high opinions of themselves, lots
of pride, arrogance, and swagger, but in general, they know they have
to show value, fast! Full-time Employees may need to be
reminded
of this expectation, and when we hire them, we need to assess whether
they have this expectation of themself.
How can we assess this aspect of a candidate during an
interview? How fast do they like to go? Are they
able to
handle a sting of 60 hour weeks? Can they dig in and get
started
without direction? Are they like a tire with a slow leak -
you
have to keep pumping them up to keep going?
Ask them to describe their experience in situations where these
tendencies would come out.
"Can you tell me about a time when you were on a project that was behind
schedule?"
"How did the team handle it?"
"How well did that work?"
"Given what you know now, what would you do differently?"
What you are looking for along this path is a response that shows the
candidate wants to focus, work and deliver.
"Tell me about a time when you had to start working with little or no
direction?"
"How did you approach this?"
"How successful were
you - did you get the job done?"
"If you had to do that over, what
would you do differently?"
What you are looking for along this path is a response that shows
internal leadership, bootstrapping, and digging in, not asking for help.
"Can you tell me about a time about a time when you were on a project
where
team motivation or morale was low?"
"what did you personally do to overcome
this?"
"How well did that work?"
What you are looking for is a response that indicates that this person takes responsibility for their own motivation, morale, and drive...
OK Glen, so perhaps by your definition, I am not an agilist.
However, I suspect that your statement about agilist's
abandoning
planning, and scheduling are unfair.
When I started to investigate agile "practices", it was primarily a
reaction to the culture of the organization that I was in, and a
particular fixation with the schedule. My experience with
software development projects as a practitioner (software
developer/requirements engineer/analyst/designer) revealed that once
the schedule was proposed, all discussion of value delivery stopped.
As project lead, my job became to delay the proposal of
schedules
until all of the scope decisions had been made. Because the
schedule was fixed, we weren't free to re-sequence delivery, we
couldn't dynamically swap one valued feature for another. We
were
required to adopt a "change control" mentality that effectively blocked
out change at any time in the lifecycle after the schedule was
proposed, because any change would cause the schedule to move.
Management was
completely fixated on the schedule.
style="font-weight: bold;"> I have
blogged about this phenomenon before.
What I found with agile practices was not a solution to all of the
problems of software development (agile does not help you design a user
interface), nor a comprehensive solution to the problems associated
with software development projects. What I found was a
different
way of framing the value/scope/schedule discussion, so that I could
fix the schedule in smaller chunks (iterations) and the
business
could continue to evolve their priorities without imposing change on
the current fixed portion of the schedule.
So, now agile has devolved into XP+Scrum, and the rest of the practices
that were agile, have sort of gone their way, there are many people who
are "post-agile" - those that embraced the agile practices, understood
them for what they are, and adopted those that added value to their
work. As I work in the financial industry, I have had to
adopt a
more "enterprise-friendly" set of agile practices. But the
benefit of being able to have a continuous conversation about scope,
priorities, and value deliverywith your customer and user community, is
that one can deliver value in smaller chunks, much more frequently, and
this mitigates the risk of DAO (delivered already obsolete)
software, or having your project cancelled before you delivered
anything (I have lived through both).
Lemme rip into another statement you made casually in a previous post
called
href="http://herdingcats.typepad.com/my_weblog/2007/06/more-statistics.html">more
statistics.
The problem with these or any other metrics, is that when
there are no adjustments to be made on the race car, the metrics only
tell us that we are going to lose the race. When your budget
cannot accomodate any additional resources, your dates cannot slip,
there are two choices - reduce value delivery, reduce quality, or both.
What the agile practices do better for software projects is
to move initial value delivery (and the conversations that go with it)
earlier in the lifecycle.
Mercenaries
are concerned about weapon proficiency. They want to be
comfortable using any weapon, they know how to assemble and
disassemble any kind of gun, can set up any development environment in
a heartbeat. They are focused on firepower, and maximum
utility, pure take down. They don't necessarily know the
deepest secrets of the lore of the weapon manufacturer. They
are not gun collectors, they are marksmen. They
would select a weapon (software tool, language, compiler) for it's
ability to produce software. They will use whatever the
customer provides, whatever their employer has available, they will
become proficient on new tools quickly, because that is what they do.
That is how they survive and succeed. Mercenaries
are agnostic, or at least non-denominational. They will use
whatever is available and complain when it doesn't have sufficient
firepower.
Zealots are
a different breed. They are always looking for the ultimate,
the best, the most amazing, the most elegant, the most efficient, the
most pure. They have a need to believe, to believe that the
tool that they have chosen is the best. They are into the
lore, they know the name, and some personal history of the original
author of the language they have chosen. They are part of
the "community" for the tools that they choose. They want to
master, to tame, to understand how to wring, squeeze, or coax the last
ounce of capability out of a tool. They "go native", and
understand the tool, in the way that it was designed. They
are focused on the beauty, the elegance of a solution, and they give
credit to the tool, the community, and the original author for their
contribution to the solution. Zealots are often evangelistic,
trying to drum up support for their religion. They donate
money to build the temple, and if they are lucky, will get to be a part
of the priestly class (write a book, blog, or contribute to an open
source project).
Mercenaries take credit for their kills, whereas zealots bring the
kills back to the commune. Mercenaries share war stories,
zealots post code galleries. Zealots will spend time
mentoring, indoctrinating, coaching, and holding court for anyone who
will listen, mercenaries are either on a mission or in a bar.
Zealots retool out of frustration (with former tools),
mercenaries retool out of expedience. Mercenaries know how to
use a tool, zealots know how the tool wants to be used.
Most of you who have been in this industry for a while have interacted
with both breeds of software developers. It is not that one
is better than the other. Both can write extremely solid
code, very quickly, and under pressure. There are probably
other attributes that can be applied to each breed, but they would be
generalizations. Some developers can change breeds based on
the situation, although each would have a default breed.
Others would struggle to shift breeds except under extreme
duress.
As a manager, it is important to understand how your developers fit on
this continuum. Mercenaries and zealots are motivated
differently. Moreover, it is not just developers that follow
this continuum - managers, project managers and analysts also have
their ideologies. It is important to understand, because at
the extremes of the continuum there is very little understanding (or
tolerance) of the other extreme.
I myself lean a little toward the zealot. I spend a lot of
time in the world of ideas. Strength or weakness?
What both articles fail to recognize is that there are organizations
who have spent a significant amount of time documenting the difference
in productivity between languages, not so much as a means of selecting
a development tool, but as a way of validating estimates.
Back in the '70's the group who created Function Point
Analysis actually started ranking languages in terms of level, and
specifically levels were based on how many SLOC (source lines of code)
were required to complete a function point - with the idea that the
less lines of code the more function points a developer could build in
the same period of time, give or take.
There is a
href="http://www.cs.bsu.edu/homepages/dmz/cs697/langtbl.htm">language
level table
that has been published and revised over the years to include many
(hundreds)
languages. Unfortunately Ruby and Python are not on this
version. So, Ed, there is a community that is focused on
software metrics that does try to empirically measure the productivity
that each language brings - simply in terms of the number of lines of
code per some defined chunk (function point). Unfortunately,
they are also a religion, and it appears that one cannot get the
current table without a paid subscription to one of their sites.
I, not being a true believer in that religion, do not have
access.
That said, I agree with both Ed and Bob that it is unlikely
that any language gives a 10-fold (90% reduction in code) increase in
productivity to the developer. or even a 5-fold (80%
reduction in code) increase. What I have seen is that the
best developers, when working in their chosen paradigm, outproduce the
average by that same factor of 5 or 10. That is a smart
coder, working in a language that he has
style="font-weight: bold;">truly mastered,
style="font-weight: bold;">and likes, can
outproduce me and you by a factor of 5 or 10. When I find the
language that lets me code like that, I will convert to it's cult, and
try to become a high priest. Why, because it feels really,
really good, to be supremely productive. When your teammates
look at you as if you were a god, that is a form of programming
nirvana.
But Ed's beef was specifically around dynamic languages. I
want to speak to this, because the first (and only) language where I
approached true mastery is a dynamic language. Interestingly,
there are interpreted (normal) and compiled versions of this language,
and working in both, I didn't notice a tremendous productivity
difference, because the compiled version had an interpretive run-time
engine built in, so it "simulated" interpretation. This is
necessary to be able to execute code out of a string at run-time.
The power of this language came from it's underlying
paradigm, the dynamic typing was something that allowed one to
introduce errors unintentionally, and one actually had to
style="font-weight: bold;">write extra code to
test variables to ensure that they had the right properties.
The true power of this language was the ease with which one could build
data-driven applications. The language primitives contained
some powerful string manipulation functions that could be combined to
great advantage in dealing with complex string manipulation
applications like interfaces or messaging. The language
supported code and data indirection, so that one could hide the name of
a variable or database record in a variable and refer "through" like
pointers but not nearly as abstract. One could also drop
executable code in a variable and move it around then execute when
desirable. Is that self modifying code? No - that
would mean that the original program modified itself in memory.
This is merely an additional form of dynamism.
Slurp code out of the database, then execute was as simple as
locating the code, and a single line of code to execute.
Does that sort of power make programs complex? Difficult to maintain?
Hard to understand? Hell yes! Does it mean that we shouldn't
use those language features? Hell no! With this
caveat - you need to carefully control how these features are used in a
production system, and ensure that there is adequate documentation for
those who come behind to understand how you wrote an XML parser in 200
lines of code, or some other amazing feat of programming wizardry.
One thing about interpreted languages (I currently use python, and php
the most) is that they let you write code from the inside out and test
it without much hassle - that is, I write functions and test them long
before I write the main. Because of the interpreter, I can
get a command line from which I can call my function like a verb
and see the result:
print ( object->method('test1', 123, 345));
That allows me to get a considerable way down the path before I worry
about larger chunks of code than functions or methods.
Working with unit test harnesses gives me much the same ability but I
have to build first in a compiled language, which even if it takes a
minute, means I am going slower than interpreted mode. In
interpreted mode, I close the editor, execute the function, and see the
result. Bam! I have even baked a script where I
have the unit tests baked into the edit cycle, so that I close the
editor, and the editor script automatically runs the unit tests for the
file that I am editing when I close the editor, and will prompt to
re-open the editor if any of them fails. That can be done in
any interpreted language with a little workstation automation.
There is a tremendous power to the immediacy of this idiom,
code-test-repeat-until-done.
Each language brings with it certain primitives that were pre-existent
in the mind of it's author, that when the fully resonate in a
developer's mind, they create power and productivity. Any
language that one picks up, it pays well to sit at the feet of a high
priest or at least a true believer and understand deeply where the
power comes from. I had this experience with Python recently,
and took a simple desktop automation script to the local high priest
for review. After he looked at my crappy code, he recommended
some more efficient ways of doing the same thing that took more
advantage of Pythons capabilities. He asked me how long I had
been using Python, and when I told him that this was my first script,
he said - not bad for a first try. Supreme validation for a
novice from the high priest - maybe I can convert to this religion...
I want to poke at the word mastery a little bit. It is one of
my
favorite words because it describes something that we don't understand
today, as well as we understood it before. This is especially true in
information technology, where new ideas arrive at a rate where only
those delivering these ideas to the community have time to achieve true
mastery, but that is a different problem.
In the age when trades were regulated by guilds, rather than organized
labor, there were specific requirements for achievement of each rank.
As I understand it, one of the requirements for mastery in
most
trades was the ability to construct one's own tools. I
learned this from my 8th grade shop teacher, who showed me how to
sharpen wood chisels with a buffing wheel and jewelers rouge - he made
me promise not to tell anyone because this was his trade
secret. After doing some basic research, I realize that this
understanding is assumed in another - the master had to make their own
tools, because they owned the workshop - a master was allowed to set up
their own workshop. Each
master's tools were his competitive edge. These tools and his
mastery were his "trade secrets" and he guarded them zealously, because
they key to his financial success. At some point during the
industrial age, tools became much more standardized and toolmaking
became a trade in its own right, and tradesmen purchased their tools
rather than making them, or working for a master.
So what on earth does this have to do with agile, I can almost hear you
asking. Absolutely everything. I agree with Dave
when he
says that "agility is all about the journey". In fact, what
agile
practitioners are is navigators. Agile isn't about the
practices,
the practices are merely the vessel in which we take the journey.
Not every vessel is appropriate for every journey, the master
is
one who can decide which practices make sense for this particular
journey, and can adapt the vessel on the fly, tuning the practices to
the specifics of the team or project.
Those of us who are less experienced with agile practices, having only
implemented parts or aspects, or all of a single methodology on one or
two teams or projects may try to repeat our experience and be
frustrated when we don't repeat our original positive experience, are
not masters, but journeymen, who need to borrow our tools from a master
craftsman. We need the experience of an agile coach who can
see
where we are missing the boat, so to speak.
I am thankful for the masters who have gone before - the pioneers -
Beck, Highsmith, Cockburn, Beedle, etc. They have built ships
that I can sail, and I need to learn from their experience. I
think that there are many, many more who have learned from the these
masters, and have crafted their own vessels, sailed off to distant
shores, but have not shared their saga with the community.
I suppose that I disagree with Dave in his definition of mastery, that
mastery is the point when one can say, "Now I know it all" is what I
disagree with. That would be like saying that a master
carpenter
will never see a situation that requires him to be "creative".
In
fact, I would suggest that innovation comes from the fact that mastery
does not mean "I know it all", but it means I can try something when
what I have done before doesn't work. I submit that one is a
master of agile, when one can invent a practice that works where others
fail, while maintain the tenets of agility.
If features and functions are the soul of solution proposal, informing
us of what will be delivered,
style="font-weight: bold;">business rules
are the soul of detailed software requirements, informing us of how the
features and functions will work. They drive our ability to
deliver work flow, data and process validation, integration, process
risk mitigation, in general all of the key benefits of automation.
While it is certainly true that we cannot
style="font-style: italic;">complete a software
delivery project without contemplating detailed business rules,
style="font-weight: bold;">must
the inability of the customer to deliver detailed business rules
impede our ability to start design and construction of
the software?
To restate the question, can we design such that final business rules
decisions can be deferred until some time during the construction
process, without unnecessary risk of major obsolescence of the product?
Let's examine the reasons why we probably should design in
this
way to begin with, and see if we can't understand why this is a best
practice, rather than a necessary evil.
Reasons for designing to defer requirements
Modeling change in requirements
Rather than reacting to changes in requirements, why can't we simply
incorporate the need to contemplate change into the requirements.
Software engineers have devised design strategies like "data
driven", and "object oriented" specifically to deal with this problem,
yet we, as analysts, have not understood that we can leverage these
strategies. We continue to develop requirements as if we are
part
of a physical hardware, construction, engineering, project.
That
is, we start with an assumption that when the project is
completed, no change will
happen. We don't recognize that software can and does change
with
the business that we deploy it for. We gather
requirements
casting phased delivery as static targets, rather than points along a
continuum.
I have heard in recent discussions about various approaches to making
decisions about software design. One approach that
intrigued
me involves taking out "options" to hedge against change. In
this
approach "options" involve a more generalized design model that costs
more to deliver, but will support a greater range of motion within the
solution around a specific axis. In requirements engineering,
in
order to enable the design team to contemplate these options, we need
to deliver requirements that contemplate the possibility and
probability of changes to the solution along specific axes within the
problem space. If the requirements cannot predict the
probability
of change, then the design cannot accommodate it.
What if when we were gathering requirements, we contemplated probable
change within the business. Of course we don't want to assume
or
predict what will happen, but we certainly know what might happen. If
we required a software product to be designed to support a
style="font-weight: bold;">range of possibilities,
we could allow the designers to model the change into the software,
rather than merely reacting to it. Of course, there is some
cost
to model software this way. We could simply compare the cost
of building to model changeability versus the cost of
static
design,
plus subsequent reaction to changes. This is definitely a pay
me now or pay me later proposition, where the cost of pay me later can
be ten times the cost of pay me now.
Business rules as a target for modeling change
Business rules provide a great opportunity to model change.
Business rules are fairly granular, that is they can reveal
significant detail. When final rules cannot be agreed upon,
we
can propose some interim or prototype rules, to express the underlying
concepts that rules should be able to contemplate. Typically
systems use different "classes" of rules to solve different business
problems, so a purpose-based rule taxonomy is a
good starting point for rule
modeling. Each rule class can be modeled based on the data
elements that must be contemplated to execute the rules. Rule
classes can also be modeled in terms of the range of "possible
actions" that can be driven as the rules "fire". If
contemplated
data and possible actions for a given rule class are not likely to
change, even when the rules (specific conditions under which
actions are fired) are allowed to change dramatically, then the final
requirements can be deferred well into the construction phase of the
software development process. In fact, sample rules that test
the
contemplation of each data element and fire each of the
actions
can be developed, until the final rules are agreed.
So how can we model change when the data
contemplated by the rule class
has a high probability of change? In my experience, this is
often
driven by a need to acquire or cleanse data so that it is both
available and suitable for contemplation by rules.
Sometimes, new business strategies place greater emphasis on
different data elements. I am asking you to model the
business
requirements - that is, it is not our job to design the solution, only
to describe the problem - if the problem has strong future state
probabilities where business rules would change then we should model
that change in the requirements.
This would allow the solution proposal to
include an option to hedge against that change.
style="font-weight: bold;">
At the end, he asked a question about products.
At the end, he asked a question about products.
A few years ago I was a de facto product manager, running an application
team and
dealing with product decisions, I learned very quickly that about 40%
of our application support calls were actually usability issues.
Software usability as an issue is defined as any misalignment
between the
users expectation for how the software should work, and how it actually
works. You may think this is a strange definition for
usability,
but in reality, if a large percentage of your user base expects
software to work a certain way, you are better off building it that
way, than you are trying to change the mind of most of your users.
I also used to perform training for this software product.
(Wait,
this is really relevant) I worked with our HR based training folks to
get a basic structure for the training protocol (because I had never
built a training protocol before). We disagreed about how to
start the training. I felt that it was important to start
with an
overview of some of the basic assumptions that the software makes
(product worldview), that sets expectations and explains the work
paradigm that the software implements. I built the training
protocol this way, especially because I was rolling out to a new group
of corporate users for whom I was replacing an existing vended product,
that had a very different worldview or paradigm than my product.
I felt that I needed to get out there early to counteract
their
expectations. While I felt that it was really important to
set
the users up with an understanding of the paradigm before we entered
the "how-to" section. Since I didn't do the training any
other
way, I don't have a direct comparison to say that what I did was
better, but the training was successful, and I got mostly positive
feedback from users many of them recognizing that I explained how the
software "thinks" and that was helpful to them.
Some of the feedback I received was different though. I was
told
with some regularity that the software did not work the way some users
think. When we look at the logs for support calls, we can see
trends that identify these gaps in alignment. I am
not
advocating that we rearchitect the software to align with users
worldview. In a corporate environment that supports work
process diversity,
one could argue that there will always be some issues with alignment.
Just know that we can recognize these patterns from our
conversations
with users.
Sometimes we can use training or education to align expectations,
sometimes we should embrace the users expectations. One job
of
software product managers is to understand when to do one or the other.
Since I had been thinking about this topic all weekend, and had already
threatened to post about it, I am going for it.
My original thoughts came from my experience with the martial arts.
I started a program at the beginning of the year (for the
first
time) primarily out of my need to exercise. I wanted an
exercise
regimen that would occupy my mind as much as my body, because the
running, cycling, lifting, cardio-machining that I had been doing were
somewhat mindless.
Since I had been thinking about this topic all weekend, and had already
threatened to post about it, I am going for it.
My original thoughts came from my experience with the martial arts.
I started a program at the beginning of the year (for the
first
time) primarily out of my need to exercise. I wanted an
exercise
regimen that would occupy my mind as much as my body, because the
running, cycling, lifting, cardio-machining that I had been doing were
somewhat mindless.
So you see, I am not some black-belt master, but an absolute novice.
The particular system that I am studying places an emphasis
on
relaxation, natural movement, breathing, and reaction. The
method of study, always requires a partner, often several.
It is during these times when working with several partners,
that
is defending against multiple attackers or opponents that I have felt
the closest to a state of flow.
Flow occurs when we are not allowed to set-up or stage things in
advance. In self-defense, flow happens when the attacks come
in a
stream, and one cannot anticipate
or setup, one can only deal with each attack as it comes, reacting,
evading, countering, moving, breathing. Flow is a state of
heightened
efficiency that results from focusing in the now. Flow is the
state, where I am one with the art, and the principles flow naturally
from me, because they have been deeply embedded in me. It can
be
observed from the outside and the inside. Flow feels good.
In a work context flow
happens
in much the same way, when we are dealing with one task, issue,
project, thing at a time, but with sufficient focus that efficiency is
increased. I believe that most of us enter a state of flow
when
we are required to push our pace to the point where we don't have the
luxury of allowing ourselves to be distracted. I think that
we
can invoke a state of flow by increasing the pace of our work, as Lisa
has suggested. She has also suggested some ideas for
improving
the pace.
I also think that flow is a product of our inner discipline.
The
product that we control as we develop the principles, and art and
discipline around our work. As in the martial arts example,
the
deeper the art and principles are embedded are ingrained in me, the
greater the ease with which I attain a state of flow. My
self-discipline regarding focus and ability to ignore the many
thoughts, people, activities that are competing for my attention, is a
major contributor.
The enemy of flow is overwhelm.
Overwhelm comes when we cannot focus, when we are so
distracted
that we become inefficient, when we struggle to manage our interrupt
vectors. When I am overwhelmed, I thrash, I flail, I appear
to be out of control. Sometimes the best way to get out of
overwhelm is to walk away, retreat, and regroup. Overwhelm
can be accompanied with a sense of hopelessness, that no matter how
hard or long or diligently I work, I will not get done. When
this happens, the fatalistic and self-destructive thoughts come, and
the hole gets deeper. When in this state, most of us need
encouragement, external stimulus, strokes, to get out of it.
To echo something else that Lisa talked about, as managers, we often do
things that act as flow inhibitors for our staff. How can we
recognize when our actions, or intentions takes our staff out of flow
or worse, into overwhelm? When we are overwhelmed, should we
delegate more or should we absorb it and allow our staff to focus, to
stay in flow?
I
hereby declare that when it comes to project plans, I strongly prefer
HS to BS!!!
I have been going through a major planning exercise with my current
team. I have been using "pure" effort estimates and velocity
projections to produce project schedules for a while. The
basic
technique is simple, in that you take each milestone, summing up the
effort from the deliverables and project velocity in effort/time to
create a duration estimate (in weeks). Then you can lay out
the
schedule. Each velocity projection can also be turned into a
resource requirement (i.e. I need sufficient resources to deliver 18
effort days of work per week.)
This is the first time that I have been through this exercise with this
team, and they have struggled mightily with the concepts around
planning. Here is the list of planning approaches that I have
seen this time:
Given that I am laying out the schedule (towards the beginning of the
project) - I am not too concerned about detailed tasks. I
want to
capture in broad strokes what our deliverables are, and about how much
effort is required for each one. I have learned that planning
and
software design happen concurrently, not in sequence, and that each one
informs the other. My lead developers, on the other hand,
seem to
want to lay out tasks in order to get to deliverable estimates.
They are less comfortable with "SWAG" estimates.
They also
feel that they are totally "on the hook" for these estimates.
So
they use some of the approaches above to justify their estimates.
I really don't care, if I want to argue about estimates, I
will
argue. If they can't explain why it takes 12 man days of
heads
down effort to create a single web form, I will slash their estimate.
Regardless of the shitscreen that they throw up.
Building software is difficult and complicated, but some things cannot
be known ahead of time. I would rather baseline based on
knowing
that we don't know everything. This is what I call HS
planning, we know our plan is crap, but we have to start somewhere.
Plans with slack, or place holders to account for areas of
risk
fall into this category.
BS example:
The obvious issue here is that until we complete the object model, we
don't really know whether we have 1 or 5 objects to code, and whether
they all take the same amount of time or whether some are heavy and
some are light.
The trouble comes from trying to compensate for the fact that
we don't
know everything by fabricating plans that have details that appear to
justify or rationalize the slack or the size of our estimates around
unknowns. This is BS. It creates tasks that will be
replaced later when we know more. If we have a plan that has
the
following tasks:
HS example:
While that is utter HS, because it does not describe
specifically what needs to be done, it is not BS, in that it does
not mislead you into thinking that we actually know what to do.
We would never commit the "code and test" task. At
the
point where we needed to commit this task, we would already have
refined the plan, so the tasks we would be comitting would be the new
refined ones, and this HS one would have been replaced by
that time.
One must recognize that every plan, at the first baseline, has some
manure in it, the question we should ask is "what animal did it come
from?" Is it a recognition of unknowns and risks, or a
rationalization of our guessing.
Alright, I am done ranting now, no need to call security.
What I want to talk about is a model for organizing
requirements
so that the adequacy is somewhat self evident. To my eyes,
this
means that they have integrity (no obvious internal conflicts or
ambiguity) and they are of sufficient quality to allow us to move
forward. I am setting my scope around high level business
requirements for software development projects. There are
plenty
of issues here, and I think, broad applicability to other domains.
Adequate Software Requirements
What does it mean to have adequate software requirements?
Perhaps
this is the starting point. Adequacy in software requirements
should be defined in terms of whether they are sufficient to allow the
development of a solution
proposal.
Value Targets
When writing a solution proposal, I prefer to use the business
value request to define the target of my delivery. The
primary business value
target
is the answer to the question, "What's in it for me?" from the
perspective of the requester. When the analyst starts
sniffing
around, he or she may uncover other business value targets.
The primary target should also answer the essential
style="font-weight: bold;">WHY
question from the requester's point of view. Any other
targets
that are identified can be considered a bonus, but there achievement
should not reduce or mitigate the achievement of the primary target.
Solution Constraints or Criteria
After I have the business value target(s) defined, I need to constrain
my solution. This is not unlike the
style="font-weight: bold;">qualification process
that a salesperson uses to determine which product customer is most
likely do be satisfied with, or ultimately purchase.
Solution constraints or criteria define the
boundaries
around an acceptable
solution. These constraints can be cost, delivery schedule,
but
mostly should be functional, defining
style="font-weight: bold;">WHAT
should be in the box. Constraints say, if you can overcome
these
obstacles, I'll buy your product (implement your solution).
style="font-style: italic;">
style="font-style: italic;">
Context Description
It is important to define the environment in which the solution must
work.
That might include how the problem fits into the overall
business
process, interfaces with other business units or companies, character
of existing staff. Add to this any changes in context that
management is contemplating. Software delivery almost always
contemplates some change in business process - buried in the value
targets, there is usually some implied change. This needs to
be
teased
out and made explicit (a hedge against requirements that only
contemplate current context). Moreover, for a complex
software
development program, involving multiple concurrent and/or sequential
delivery cycles, the context is the part of the requirements that
allows us to see how each piece fits into the whole.
Separate Solution Proposal?
My primary point is that requirements are adequate without a solution
proposal. The practical implication of this is that the
solution proposal can (and should) exist separate from the
requirements. This is a point of contention with projects
that
are managed within a single organization, and a difference between a
consulting approach and typical in-house or monopolized approaches.
Why is it
different? I submit that this is because the consulting
approach
does not assume that it has won the business. A project that
is
managed in-house is free to assume that it is not competing against
other solutions, an assumption that is never valid for external
projects. If we were to prepare requirements as if we were
going
to farm the project out, letting the best consulting firms in the world
propose solutions, we certainly would have to separate the requirements
from the solution proposal. So isn't the combination of these
two
steps a short-cut that we are allowed to take in internally managed
projects?
A Valid Shortcut?
As I think about this, my experience tells me that for smaller, lower
risk projects, where the number of possible solutions is small, the
shortcut may be valid, and combining requirements and solution proposal
may save time and effort. I think for riskier projects, with
more
unknown domains, especially those that contemplate significant changes
to business process, that this shortcut leads to ossification that
inhibits the solution from evolving in response to progressive
elaboration of requirements (i.e. as we remove risk
factors, solve
unknowns, and the business process change becomes more apparent.)
I believe that the reasons for this are:
An Alternative!
So what can we do instead? I recognize that adding additional
steps in a sequential process necessarily causes delay. I
disagree with the need to have the process execute sequentially.
I simply believe that we can separate the documents, so that
the
solution proposal(s) can stand alone. In fact, I am convinced
that if we have the right people during the "discovery" phase of a
project, we can contemplate many aspects of design in parallel with the
elicitation of requirements. I believe that this parallelism
is
healthy, because it drives us to a deeper understanding of the problem
domains.
As a software developers, systems analysts, and engineers, our mission
is to
provide solutions. Our value proposition is always in the
solution. We probably can't help envisioning a series of
solutions while we are immersing ourselves in the problem domain(s).
It is part of how we learn, how we build our mental maps.
For the vast majority of analysts that I know (especially
those that grew out of software development), the solutions are always
rattling around in the background, and one merely need to open the tap,
for them to pour out. In fact, it can require significant
discipline to keep that tap closed.
Why we should
If we can refrain from embedding this in the requirements or
problem description, we can decrease the initial investment in a
specific solution, or solution concept. The longer we can
delay this investment, the longer we can continue to evolve the
solution before we have to get buy in. While we need to get
buy in, the longer period of adaptation and evolution we can
support during solution development the more likely we are to hit our
value targets. By fixing the solution early, during the
requirements, we compromise our ability to adapt and evolve the
solution.
There are several cool things about measuring velocity, separate from
effort and duration.
Perhaps the coolest thing you can do with velocity metrics is to
recognize how much time is spent on NON-DELIVERY
VECTORS. A non-delivery vector is a fancy word
for work that you do that does not align with a specific project goal.
Team meetings, staff interviews, staff training, coordination
and status meetings don't count towards delivery. They are
not "tasks" that should be captured in a project plan. My
team calls them "things". There are other types of "things":
helping a team mate solve a problem, having a design meeting,
one-off discussions and phone conversations, reading e-mail, responding
to e-mail. All "things".
The following ideas have been discerned by measuring in this
way:
More "things" means lower
velocity. The resources that I observed
having the highest velocity were those who had more repetitive
tasks: testers, report developers, database dml developers. I
believe that offshore developers working to specifications, should (in
theory) work at very high velocity because they are typically heads
down, low collaboration.
At first this was difficult for me to take and understand as a manager.
Here is why:
It is the same sort of reaction most people have when they first try to
have some discipline around their household budget. They are
incredulous about the way they actually spend money, as opposed to the
way that they perceived they were spending it. I had a
similar reaction to the way I was spending resources as a manager.
I had to stop looking at the individual velocity and focus on
the aggregate velocity. Having the brightest guy doing all
the work may give 2-3x more velocity for him personally, but having the
output of 4-7 people double because of his direction is better.
Being able to bring additional staff to bear because the
leadership structure and knowledge transfer capability is there is
worth more long-term, than a marginal increase in velocity, it means
that I can add resources efficiently (without significant
drag) because the team's infrastructure is able to support it.
Having the data to understand how your team really works means taking
the irrelevant data and fiction out of your estimates.
Duration based estimates always contain an element of
irrelevancy or fiction, because they can't factor out the effort that
doesn't contribute to the specific delivery vector.
Being prepared to
contribute involves two main things. Understanding
the content of your contribution, and being prepared to deliver that
content concisely.
In my team management work, I have deployed a daily "stand-up" where
people don't stand up but can get through a daily status with 20
contributors in less than 10 minutes. If you think that is
unbelievable, or sounds like a run-on Jimmy John's commercial, it is
because the team was prepared to deliver the necessary content
(delivery status for weekly iteration) as 3 numbers. Only if the
numbers revealed a problem was the contributor expected to reveal the
detail behind. Is this extreme? Yes! Effective? Yes? Did it take a
while to get used to? Hell Yes!!!
Can most meetings benefit from this approach - Only if you can dictate
(like a boss) what content each contributor will deliver, and provide a
concise template for them to use (3 numbers). As long as either of
these things are not deterministic, then contributors are free to make
it up as they go, and are much more likely to show up unprepared.
Then I kept thinking about it...
Could I improve the effectiveness of every meeting that I called?
Could I improve the effectiveness of every meeting that I attended?
How?
Improving Meeting Effectiveness
So if I were going to improve the effectiveness in meetings that I
called, I could use the principle above to achieve the goal!!!
I could do the following things:
What about meetings that we attend? Since we cannot often
insist on the meeting caller following these guidelines, what can we
do? We can ask the meeting chair the same questions
we would ask ourselves above.
Before you think I
am some meeting master, you must know that I am terribly
undisciplined about meetings, other than my team's daily
stand-up. Why, because the cost of time spent creeps up like
a productivity killing ninja, and my schedule gets booked with junk
meetings before I correctly prioritize my time. While I know
what to do, I don't practice with the right discipline.
Perhaps this will help me improve in this area.
Estimation -
my experience with software development, and construction projects
(church, home) though small, informs me of a key difference.
In construction projects,
style="font-style: italic;">Once the required work is known,
it can be estimated with some reasonable accuracy, because the vast
majority of construction tasks have been done exactly the same way by
the people who are doing them. The tool set doesn't change
every 2-5 years. Carpenters still use nails to fasten wood
together. Wood still has the same physical properties that it
did in 1965. In software, there is a programming language
every 6 weeks, and every 10-20 years the nature of raw
materials change - assembler to procedural to functional to
4gl to object oriented, etc. The new raw materials and tools
require estimation to be redone from scratch. The other
factor in estimation is the difference in productivity
between master, journeyman, and apprentice
practitioners. In construction the difference between
resources can be a factor of 2 or 3, but years of stable estimates for
building 40 linear feet of stud wall, provide a rational basis for
scheduling. In the software industry, only the practitioner
who is doing the work can estimate his or her rate of completion, and
the difference between practitioners can be a factor of 10 or 20.
Estimation -
my experience with software development, and construction projects
(church, home) though small, informs me of a key difference.
In construction projects,
style="font-style: italic;">Once the required work is known,
it can be estimated with some reasonable accuracy, because the vast
majority of construction tasks have been done exactly the same way by
the people who are doing them. The tool set doesn't change
every 2-5 years. Carpenters still use nails to fasten wood
together. Wood still has the same physical properties that it
did in 1965. In software, there is a programming language
every 6 weeks, and every 10-20 years the nature of raw
materials change - assembler to procedural to functional to
4gl to object oriented, etc. The new raw materials and tools
require estimation to be redone from scratch. The other
factor in estimation is the difference in productivity
between master, journeyman, and apprentice
practitioners. In construction the difference between
resources can be a factor of 2 or 3, but years of stable estimates for
building 40 linear feet of stud wall, provide a rational basis for
scheduling. In the software industry, only the practitioner
who is doing the work can estimate his or her rate of completion, and
the difference between practitioners can be a factor of 10 or 20.
When I hear that project management is different between construction
and software development, this is the only difference that
I think is real. For me, it is the reason that short
iterations are important, so that I can measure the output of each
practitioner against their estimates. That way I can predict
velocity or rate of completion. In software development, that
is the only way that I have found that works to manage the
style="font-weight: bold;">schedule, because
estimates are highly individualized.
In construction, the schedule challenges revolve around different sets
of constraints: availability of resources, availability of raw
materials, weather, and unanticipated work. Software
projects, thankfully, avoid the complexity of weather.