Martin's talk was about Domain Specific Languages, which will be the subject of his next book. He is well into this book, but claims that it will be still one year and a half before publication. I am not interested in DSL's per se, and I have long been out of the business of writing software for a living, and am mostly focused on requirements engineering, Software quality assurance, project management, and software team management.
This recent focus, does not exempt me from the need to stay current on the most vital abstractions and concepts used in software development. In order to interact with developers, I will continue to need to understand in some depth, the abstractions that they use to build, and the reasons and benefits for selecting one over another.
Moreover, learning new stuff makes my brain better. And that, my friends, is what this post is about. We all need to continually learn new stuff, regardless of how it applies to our job, or life. We need to try new ideas on like a coat. We need to experiment with things that we are learning or have become aware of. We need to adopt new practices (even if we think they are lame) and evaluate them from a practitioners perspective (rather than from a critical distance). We need to explore bodies of knowledge that are foreign to us, because it makes our brain better.
Regardless of that you do to earn a living, this is true. When someone throws you a new idea - rather than saying to yourself, "Why would I take the time to understand this, it is no better than what I am doing today.", we should periodically, put on our mental lab coat, and perform an experiment. We should borrow a page from the scientific method, and formulate a hypothesis and then conduct the experiment, to prove or disprove the hypothesis.
I have absolutely no reason for implementing DSL's on any of my projects. I will not be writing any code soon, but I wonder if any of the problems that I will need to solve over the next few years will be enhanced by my deeper understanding of some of the concepts presented to me.
This curiosity allows me to challenge the developers that I work with to be curious as well. It allows me to ask questions that can change the course of projects.
]]>Early and Continuous delivery of valuable software...
I take this to mean that we are going to structure a project
(or delivery stream), such that usable software "features" are
completed as frequently as possible. The shortens the
feedback loop and allows the customer to decide when sufficient value
has been delivered to justify a deployment or distribution event.
Different software development situations, call for different decisions
regarding deployment:
1) For a company whose primary presence is via software and/or the web,
especially where the revenue is driven by the software (think yahoo or
google), the frequent delivery of new or enhanced features is a way to
drive customer satisfaction. The product manager responsible
for these decisions, strike a balance between new ideas, and
enhancements or fixes to existing.
2) For a more enterprise software application, used by employees of a
company, the cost of deploying and retraining staff on new software
features, and the criticality of correctness. The business
process change management around delivery of new software features may
drive a very different roll-out cycle.
3) A software vendor, selling software to business units within an
enterprise, or small businesses, may have yet another model for making
decisions about deployment. They may be driven by demands
from new customers, and prospects, as well as a backlog of requests
from existing customers.
4) Software that is to be "embedded" in a physical device may have a
completely different decisions cycle, and a much more rigid schedule.
The delivery of the rest of the device into which the
software is to be embedded, and the revenue derived from the sales of
the devices, completely determine the schedule. In this
model, partial delivery has little or minimal value.
5) Software that is deployed in industries that are heavily regulated
(blood bank) or that have critical quality requirements
(military/aerospace) may have much more rigid schedules that are driven
by decisions made well before the development starts. In this
model, the ultra-high cost of quality assuranceand regulatory
certification reduces the value of partial delivery to near zero.
It is easy to see why the value of agile principles is readily apparent
to some, and a complete mystery to others. It is the decision
model that works in between need and deployment that determines the
value of early or frequent delivery. From a whole project
perspective, this is clear. But let's look from a
intraproject perspective.
If not, it's a waste of effort to show up early. And there is
likely a downside to doing so."
In Glen's post, he sounds like he equates
early delivery as equating to "ahead of schedule". This is
not, in fact, what the agile community means or intends.
Agilists who talk about early delivery are describing the
process of scheduling incremental delivery to expedite the most
valuable, risky, or juicy features of a software product so that they
can be deployed, tested, or sold, as early as possible. The
notion is that a - the shorter the timeline between business need and
delivery, the less likelihood that the business will have changed out
from under the requirements. The faster we can deliver
product, the faster we get paid. The faster we get something
working, the faster we can validate our design assumptions and
incorporate feedback into the remaining features.
Most of all early delivery is about arranging the sequence in
the schedule, not working out of sequence, or ahead of
shedule.
Not too long ago, I saw an e-book "Rocks
into Gold"
by Clark
Ching that ties this notion of early delivery to creative
solutions for the economic crisis.
The perceived evils that the concept of early delivery is
trying to address occur in projects with long timeframes between the
identification if business need and the delivery of value.
The evils that can be pervasive in this context are:
1) Your product is obsolete before it is delivered - this
occurs when the business need changes but your deliverable does not.
Risks around too little, too late.
2) Your project is canceled before it is complete - market or economic
conditions call for reduced spending, and the project's doesn't
show return on investment meaningful ROI until the company is
projected to be bankrupt.
3) Your product is delivered, and is not well received in the
market or user community, you are then required to make many
modifications that delay the delivery of additional value, and reduce
the return on investment. (Think windows vista).
I can't help but ask what is someone to do, to help an employee (or manager, supervisor, or executive) to improve their time span. There are many symptoms of a person who is expected to work beyond his time span. In my experience, most people, get used to working within a certain time span. The longer they are in a role, without specific challenges to increase that time span, the harder it will become for them to improve this capability.
Here are two tools that can be coached into a person, to help them become more effective at longer time span tasks:
1) Progressive Plan Elaboration - Start with a plan that is at a very high level, where the plan deliverables are each about the size of the person's normal time span without worrying about smaller details. Order these deliverables in a reasonable sequence without worrying about smaller details. Agree on a schedule for starting work on each of these deliverables, and make sure that the plan includes a task to plan the smaller details of each deliverable, before the work on that deliverable is scheduled to begin. This technique works because most of us get hung up around the many details, rather than the basic work structure. We worry about things that we won't know until we have done some of the work, and so we can't always plan in detail in advance. For some people who think from the details up, this can be paralyzing, and the top down approach, ignoring the details until we are close up can really help.
2) Iterative Measurement - There are some times when the work cannot be broken up into discrete deliverables, other times, when there are many deliverables being worked on in parallel. Project Plans with these features can be difficult to organize, so we can break things into discrete time boxes or iterations. This specific work (list of tasks) will get done in this time frame (use a iteration time span that is within your graps). Make sure that you are planning each time box in advance, enough to make work assignments and manage schedules.
How could Brad, in Tom Foster's post, use these two approaches to reduce the impact of procrastination? It sounds like Brad has a team of supervisors who can get the work done. He could probably work with them to set up a deliverable based plan, and have them work the schedule, and report the details of the plan for each deliverable before the work on that deliverable is stared. Alternatively, he could break the work into time boxes that he can manage (2 months) and set goals that are 2 months out and work toward those, re-establishing the goals, every months for a 2 month period. This approach could be called "rolling-wave" planning because there are overlaps between the iterations or time-boxes. If every month, I plan two months out, then I can respond to problems in the now, and adjust the plan, before a crisis becomes intense. By measuring my progress each month, I can feel myself starting to get behind, and I can adjust, rather than simply waiting for time to run out and making a mad dash to the goal.
]]>I have been working on software projects for some 20 years, and while I
am not a practitioner of any formal agile methodology either project
management or software development, I have been exposed to incredibly
ad hoc practices in managing software projects. I agree with
Glen
when he says that agile (spokesmen) railing against traditional project
management have never been exposed to good traditional software
management practices. Most project managers that I have
worked
with, take a couple classes, manage a couple projects, and maybe get a
PMP cert and then do the same thing over and over, with varying
results, because they personally suck, not because their methodology is
fundamentally flawed. I have been this project manager, and I
sucked as well. I think that most of this is because the
organizations that I have worked for did not sponsor, fund, or require
good project management practices.
Agile, and specifically scrum, are a reaction to the suckage.
It
is the project team reacting to the fact that project managers don't
use practices that help. What scrum and other agilish methods
and
practices around project management do is help the team remove two
specific smells from project management:
The thing that Glen is missing is that
style="font-weight: bold;">velocity is not a project
completion measure, but a team productivity
measure. Agile project management is more about
team than
project. Thus it makes sense that the most visible measure is
team productivity, not project progress. From a project
progress
perspective, velocity is exactly what Glen says - predicting the future
by evaluating the past. He calls this driving by rear view
mirror. Velocity, however, is good for the team. It
tells
them whether their current pace is likely to match the
published/committed schedule, and when measured frequently enough (like
every week), it tells them early enough, that they can
style="font-weight: bold;">decide to adjust
something to stay on schedule.
Velocity is a factor that is involved in the measurement of current
projected duration. Current projected duration is a simple
measure - remaining work / average velocity = projected duration.
This measure has a bunch of assumptions:
I think that Glen's characterization of velocity as a "rear view
mirror" and "yesterday's weather" metric is largely because of his
believe that past performance is not always a good indicator of future
results. While this is true, velocity and current projected
duration are useful metrics that can be applied in many ways to measure
productivity and schedule. Glen says that velocity is a level
of
effort measure, and I disagree. Here's why:
If calculated this way it is not, as Glen suggests, a LOE measure, but
much closer to EV, because it approximates the budgeted effort.
The thing about velocity that many project managers struggle
with
is that it is not precise, and it doesn't contemplate many aspects of
the project timeline, only team productivity. I believe this
is
because agile disciplines are mostly focused on the behaviors of the
team, and increasing/maximizing team productivity, and as a result, the
delivery of value by the team.
Here are some things that I have done with velocity (in combination
with other measures):
In my experience, velocity alone is insufficient for many
projects, as it doesn't really contemplate the complexity of external
dependencies, interactions between teams, etc. When these factors are
in play additional measures can be devised or adopted to present
conditions where the project is at risk. When my team is
largely
in control of the project and there are fewer connections and
dependencies, I have experienced the benefit of this simple measure
(velocity) because it is easy to understand, and difficult to fake out.
These two factors allow a development of trust between
project
manager and team and a focus on solutions (adjusting the plan).
The person who asked me was looking for my opinion because I am strongly opinionated, blunt, and have experiencing hiring and leading business analysts and requirements engineers. She wanted to understand the difference between an analyst and a senior analyst.
Other than the expression of leadership within a project or team context that I would expect of any senior contributor, my answer had to do with metaphors.
Metaphors are the solid business abstractions that software is designed around. Metaphors are the unambiguously defined concepts that ground the business process. Metaphors are the litmus test to see if requirements are cohesive and complete. If your metaphors suck, so do your requirements.
Back in the '90s when I was learning object oriented application design (OOAD) we were taught that each application or major feature had a "central object" that was the focus of it's existence. Microsoft word has a "document". Every e-mail client has a post or a message. This central object is the metaphor around which the application is designed, we just didn't call them metaphors back then.
When designing an application all of your actions are performed on metaphors, all of your business rules contemplate metaphors, and your data model expresses your knowledge of your metaphors. Your metaphors are the essence of your understanding and modeling of the REAL BUSINESS stuff that your users and customers have to deal with in the REAL WORLD as part of their job. The more your metaphors align with reality, and the more you can eliminate ambiguity among and between metaphors in your requirements, the easier it will be for software designers, architects, and developers to model and fashion a system around them.
That's my story and I am sticking to it. The senior analyst gets this, and carefully and thoughtfully identifies, defines, and clarifies essential metaphors within business requirements, and knows the the requirements aren't complete until all of the metaphors defined hold together with the business rules and actions required.
Metaphors work most effectively when you can get your user community to communicate (to you, and to each other) in terms of metaphors that you helped define. When you have defined useful metaphors that clarify the business process, your users will adopt your terminology because it helps them understand what is important. Then in that context your software (being true to those metaphors) will be intuitive.
An analyst can capture and document the business process. The senior analyst (one with mastery) can change the conversation casting metaphors that help the business community define its value proposition and supporting processes more effectively.
]]>In this role, I have been focused on analysis and have been interviewing more business analyst type resources than I have before. I don't call them business analysts, although my company has a job description for "senior analyst, business process". The reason is because their job is not to analyze business process, their job is to elicit, understand, organize and document requirements for software products. I prefer to use the term "Requirements engineers", and I like to talk about requirements engineering because requirements are not simply a description of the current or desired business practice, or a wish list. Requirements are a high cohesion document, describing capabilities that are required to deliver specific business value through software automation.
In a recent interview of a candidate for a contract "requirements engineer", I asked a question that I usually ask candidates of any skillset - "What the top three critical success factors for practitioners of ?"
My friend Johanna Rothman would say that this is not a very good behavior description question, because it does not give the interviewee an opportunity to tell how he has done this. I believe that it is a very good question, because it asks two things at once? Does this candidate see him or herself as a practitioner of a discipline, or as someone doing a job. It also forces them to describe how they practice this skill set. If this answer rolls off of their tongue, then they have spent some time thinking about how to do a better job. If they struggle with it, it is likely that they don't think about it much, they just do it.
Then there is the answer it self - this tells me what they think is important. I usually ask this question towards the end of the interview, after I have already asked the behavior description questions. I look for answers that are cohesive with the earlier responses, to see if they are spitting out what they think I want to hear, or they practice what they preach.
This candidate did pretty well - after he answered, he asked me what I thought the three critical success factors were.
My answer:
Semantic Clarity or Disambiguation - terms, and concepts, especially metaphors must be precisely defined.
Cohesion - the document must add up with mathematical precision.
Organization, especially abstraction or generalization - the bases of software is abstraction, and this must begin with requirements, classes of problems, value propositions must be clearly identified and categorized.
His answer:
He had cohesion and disambiguation or something close to it, but substituted scope management for organization.
I don't think he is wrong, but in my organization, scope is fixed after requirements, not before. This is because premature scope management inhibits value delivery, IMO.
I think I'll hire him...
-- Correction --
My candidate did not have cohesion, he said communication, and he talked about making sure that each person walks away from a conversation with the same understanding of the topics discussed. I agree that this is a success factor for gathering requirements. Certainly anyone who goes into a business to understand what is required, in order to add some specific value to the business must have ample communication skills, and most importantly, establish appropriate feedback mechanisms to ensure that the understandings are shared. This for me is a component of semantic clarity and disambiguation, call it a sub-factor.
]]>I just wanted to drop a post and let you know that I am back - I never really left, but every time I had something to post, it was pretty negative, so I just didn't.
I hope 2008 finds you well, and I will be able to get some new ideas out here soon, like tonight.
]]>In my family, we have a tradition, that on the first day of school, my
wife and I drive or walk the kids to school or load them on the bus,
and spend some time together over breakfast remembering the summer, and
thinking about how they have grown, or any school concerns we have.
Usually my wife is concerned about being alone more.
Over the years it has been a precious tradition, and my wife
and I have come to rely on it. My kids have too.
That day my 8th grader was unhappy that I was not going to be
there to see him off to school.
I was scheduled to be off work the following day, as I was building a
storage shed in my
backyard. Since I arranged for a friend to come and help me
with that, I had already ordered the lumber and torn down the old shed,
when this meeting came up, I didn't want to try to reschedule the shed
project.
My attitude that morning was one of resentment. I had very
little confidence that this meeting would accomplish it's objective.
We had a specific technology problem that we had been
postponing solving for 6 months. Several times people had
offered suggestions and attempted to build consensus around a solution
and it had not happened. I was one of those and I got beaten
and bruised for my effort. Our project was critically late,
and at risk of failing it's political viability date. That is
the date at which if we haven't delivered, it is not politically viable
to be affiliated with the project.
Before the meeting I said this prayer:
In spite of my poor attitude, the meeting did go well and we did reach
consensus.
So why is this about priorities? Because I had a personal
conflict of priorities around this meeting. This meeting
became
critical, because of our lack of priority around resolving this issue,
which has now become the critical path of our project. As a
manager, I recognize that part of my job is to make sure that I (or
someone I assign) control
the critical path of each project by making sure that all the
appropriate decisions get made before they hit the critical path.
While there are always things that happen that are beyond our
control that change the critical path - resource issues, and external
forces that are unforseen, but the vast majority of things are well
within our control, if we just would exercise that control.
In a phone conversation with a liaison from a software consulting firm
that I employ, I lost my temper. I said things that threaten
the
relationship, that could make working with this person, and his firm
much more difficult.
In a phone conversation with a liaison from a software consulting firm
that I employ, I lost my temper. I said things that threaten
the
relationship, that could make working with this person, and his firm
much more difficult.
I let the frustration, and annoyance that I had been experiencing in
dealing with this group leak out in my words. Now, when the
project is in the red zone, is not the time for this. In any
case, the lack of maturity and self-control that I displayed could have
had dire consequences for the project, for the company, and for my
career.
I was lucky, there was little direct damage, and the collateral damage
was manageable. The biggest consequence was my management
realizing how volatile I can be when pushed past my limits.
In
twenty-four years of employment I have probably only had
two or three meltdowns like this.
--
What can I learn from this? How can I use this experience to
diagnose risks, to identify warning signs, so that from a project
perspective, I can find bombs like this before they go off.
How can I communicate to my management the conditions that I
was
asked to work under that allowed the bomb to form. Where are
the
raw materials, the timer, the detonator? In other words, how
can
I be prepared to dismantle this bomb before it goes off?
--
Raw Material
Bombs require raw material or explosive - TNT, Nitro, C4, or some
incendiary agent, benzene, gasoline, etc. In this case it was
the
combination of two ingredients: a vendor who expressed a certain
arrogance and unwillingness to work within the processes that constrain
our other resources. Even the smallest facets of their work
required adaptation by any person who needed to collaborate them.
Whatever the reasons they had for doing this, it created an
attitude of resentment in me, that I simply repressed and stuffed down.
It's not that that was my first response, my first response
was
to confront these issues, and negotiate some compromise that allowed me
to get more of what I need. That bringse me to the second
ingredient: a perceived lack of control or power to change things.
My manager(s) were very sold on this firm, and I inherited
the
relationship when I joined the project. As a manager, I have
had
varying levels of control and have had managers who were more or less
prone to direct me in specific details. I had never
had a
vendor with more juice with my own boss than I had before. It
was
a new experience, and not terribly pleasant. It seemed that
every
time I tried to bring some visibility or accountability to bear, I got
thrown under a bus and the vendor was allowed to skate. So
the
second ingredient could be a lack of support, or empowerment; or it
could be a lack of control; or it could simply be my not asserting
myself by presenting my manager with an ultimatum (I manage or you
manage, but I can't manage if you wont stop managing).
The Timer
Projects have built in timers. The schedule is
always
agressive, pressure builds during the project and as things slide,
crushing forces converge. As things move towards a state
where it
is obvious that we cannot succeed without superhuman effort there is a
building of pressure that allows events, comments, or circumstances to
trigger an explosion. In this case it was the
many accomodations that I and my team had made or tried to
make to
help this consulting vendor be successful, from pulling work back to
hiring a project manager and analyst, to hiring more resources to work
with their team. The harder I worked to help them, the more
they
expected me to do, and the more they attempted to identify me as the
excuse for their slippage.
The Detonator
Most explosive materials will not combust spontaneously, but require
some initiation or detonation to start the explosion - be that
electrical, chemical, or physical. In this case it was
chemical.
Consulting firm hired a new liaison who in his first week
started
to push expectations towards my team for work that we had agreed to do,
before he was on the scene. He didn't even know that when he
pushed the button that there would be a reation. However,
when I
started to get defensive, he just kept pushing the "BIG RED BUTTON".
Eventually, the detonation was effective.
Instructions for
dismantling
When I find my self in this situation, I need to make sure that I
disconnect the detonator before it gets pushed repeatedly.
Take a
walk. Hang up the phone. Tell the button pusher
that you
will get back to him later. That is a quick fix, but it
doesn't
dismantle the bomb.
I need to deal with the materials or the timer for that.
Behind
each of these was a conflict or a confrontation that needed to happen.
Each conflict or confrontation had consequences. I
needed
to take responsibility for allowing the ingredients to mix to create
the combustible mixture. By not facing up to the conflict, or
confronting the situation I had allowed my own attitude to deteriorate.
I had chosen to absorb all of this into my own
world, and
had not taken responsibilty for it. I allowed myself to move
into
the victim quadrant.
The truth is that my issue was not with my consultant, but my
management. I needed to identify the consequences of the
nature
of the relationship with this vendor, and allow them to mitigate those
consequences. After all, this wasn't personal, but business.
If management was not willing to acknowledge the
consequences,
shame on them. Managers (like me) are paid to do the right thing;
the right thing for the company, the project, and the staff.
When
they make decisions that have consequences that they are unwilling to
acknowledge or manage, they put the people who report to them in a
trick bag. That does not absolve me of the responsibility for
staying in the bag.
As I now see it, I had several opportunities to crawl out of the bag:
1) Confront my manager with the issues - give her an opportunity to
fold up the bag and move on.
2) Work through an alternate tactic or strategy that mitigates the risk
- perhaps my manager feels she has no alternative.
3) Expose the risk beyond my management circle... This is a
very
dangerous because the person you confide in needs to protect your
confidence.
4) Move away from the project - as a last resort, one can always simply
choose to walk away.
I actually contemplated 3 and 4 at various times, and thought that
"hanging in there" was a better strategy. I realize that I
discounted numbers 1 and 2 without sufficient consideration, believing
that I would not be heard or respected. I assumed that the
managers that had put me in the trick bag was unwilling to make a
different decision.
--
Finding It
A week later, things are different. My managers are now
aware, of
how much I was stomaching and how much resentment had built up.
The vendor is now aware of how close their relationship was
to
being fruitless. They have made some adjustments and somehow,
more progress is being made than before, and more people are happy than
before. I don't attribute all of the change to this event.
I think that some of it was planned or working itself out
before.
I thank God that things did not turn out much worse. This
certainly could have been a career limiting event, or even a ticket to
a worse position. I also recognize that a more controlled
confrontation can be very effective. I should consider being
more
confrontational when things are difficult and change is required and
slow in coming.
What you want is someone who will turn the handle and GRIND IT OUT!
You want someone who pulls out a machete and hacks his way
out of
the details and decisions to a solution! You want someone who
makes his own tools, proposes answers to his own questions, and rips
raw meat off the bone! You want someone who schedules a
meeting
to start the project and another to plan migration when it is complete!
In my experience, most contract employees get this. They are
mercenaries, hired guns who may have high opinions of themselves, lots
of pride, arrogance, and swagger, but in general, they know they have
to show value, fast! Full-time Employees may need to be
reminded
of this expectation, and when we hire them, we need to assess whether
they have this expectation of themself.
How can we assess this aspect of a candidate during an
interview? How fast do they like to go? Are they
able to
handle a sting of 60 hour weeks? Can they dig in and get
started
without direction? Are they like a tire with a slow leak -
you
have to keep pumping them up to keep going?
Ask them to describe their experience in situations where these
tendencies would come out.
"Can you tell me about a time when you were on a project that was behind
schedule?"
"How did the team handle it?"
"How well did that work?"
"Given what you know now, what would you do differently?"
What you are looking for along this path is a response that shows the
candidate wants to focus, work and deliver.
"Tell me about a time when you had to start working with little or no
direction?"
"How did you approach this?"
"How successful were
you - did you get the job done?"
"If you had to do that over, what
would you do differently?"
What you are looking for along this path is a response that shows
internal leadership, bootstrapping, and digging in, not asking for help.
"Can you tell me about a time about a time when you were on a project
where
team motivation or morale was low?"
"what did you personally do to overcome
this?"
"How well did that work?"
What you are looking for is a response that indicates that this person takes responsibility for their own motivation, morale, and drive...
OK Glen, so perhaps by your definition, I am not an agilist.
However, I suspect that your statement about agilist's
abandoning
planning, and scheduling are unfair.
When I started to investigate agile "practices", it was primarily a
reaction to the culture of the organization that I was in, and a
particular fixation with the schedule. My experience with
software development projects as a practitioner (software
developer/requirements engineer/analyst/designer) revealed that once
the schedule was proposed, all discussion of value delivery stopped.
As project lead, my job became to delay the proposal of
schedules
until all of the scope decisions had been made. Because the
schedule was fixed, we weren't free to re-sequence delivery, we
couldn't dynamically swap one valued feature for another. We
were
required to adopt a "change control" mentality that effectively blocked
out change at any time in the lifecycle after the schedule was
proposed, because any change would cause the schedule to move.
Management was
completely fixated on the schedule.
style="font-weight: bold;"> I have
blogged about this phenomenon before.
What I found with agile practices was not a solution to all of the
problems of software development (agile does not help you design a user
interface), nor a comprehensive solution to the problems associated
with software development projects. What I found was a
different
way of framing the value/scope/schedule discussion, so that I could
fix the schedule in smaller chunks (iterations) and the
business
could continue to evolve their priorities without imposing change on
the current fixed portion of the schedule.
So, now agile has devolved into XP+Scrum, and the rest of the practices
that were agile, have sort of gone their way, there are many people who
are "post-agile" - those that embraced the agile practices, understood
them for what they are, and adopted those that added value to their
work. As I work in the financial industry, I have had to
adopt a
more "enterprise-friendly" set of agile practices. But the
benefit of being able to have a continuous conversation about scope,
priorities, and value deliverywith your customer and user community, is
that one can deliver value in smaller chunks, much more frequently, and
this mitigates the risk of DAO (delivered already obsolete)
software, or having your project cancelled before you delivered
anything (I have lived through both).
Lemme rip into another statement you made casually in a previous post
called
href="http://herdingcats.typepad.com/my_weblog/2007/06/more-statistics.html">more
statistics.
The problem with these or any other metrics, is that when
there are no adjustments to be made on the race car, the metrics only
tell us that we are going to lose the race. When your budget
cannot accomodate any additional resources, your dates cannot slip,
there are two choices - reduce value delivery, reduce quality, or both.
What the agile practices do better for software projects is
to move initial value delivery (and the conversations that go with it)
earlier in the lifecycle.
Mercenaries
are concerned about weapon proficiency. They want to be
comfortable using any weapon, they know how to assemble and
disassemble any kind of gun, can set up any development environment in
a heartbeat. They are focused on firepower, and maximum
utility, pure take down. They don't necessarily know the
deepest secrets of the lore of the weapon manufacturer. They
are not gun collectors, they are marksmen. They
would select a weapon (software tool, language, compiler) for it's
ability to produce software. They will use whatever the
customer provides, whatever their employer has available, they will
become proficient on new tools quickly, because that is what they do.
That is how they survive and succeed. Mercenaries
are agnostic, or at least non-denominational. They will use
whatever is available and complain when it doesn't have sufficient
firepower.
Zealots are
a different breed. They are always looking for the ultimate,
the best, the most amazing, the most elegant, the most efficient, the
most pure. They have a need to believe, to believe that the
tool that they have chosen is the best. They are into the
lore, they know the name, and some personal history of the original
author of the language they have chosen. They are part of
the "community" for the tools that they choose. They want to
master, to tame, to understand how to wring, squeeze, or coax the last
ounce of capability out of a tool. They "go native", and
understand the tool, in the way that it was designed. They
are focused on the beauty, the elegance of a solution, and they give
credit to the tool, the community, and the original author for their
contribution to the solution. Zealots are often evangelistic,
trying to drum up support for their religion. They donate
money to build the temple, and if they are lucky, will get to be a part
of the priestly class (write a book, blog, or contribute to an open
source project).
Mercenaries take credit for their kills, whereas zealots bring the
kills back to the commune. Mercenaries share war stories,
zealots post code galleries. Zealots will spend time
mentoring, indoctrinating, coaching, and holding court for anyone who
will listen, mercenaries are either on a mission or in a bar.
Zealots retool out of frustration (with former tools),
mercenaries retool out of expedience. Mercenaries know how to
use a tool, zealots know how the tool wants to be used.
Most of you who have been in this industry for a while have interacted
with both breeds of software developers. It is not that one
is better than the other. Both can write extremely solid
code, very quickly, and under pressure. There are probably
other attributes that can be applied to each breed, but they would be
generalizations. Some developers can change breeds based on
the situation, although each would have a default breed.
Others would struggle to shift breeds except under extreme
duress.
As a manager, it is important to understand how your developers fit on
this continuum. Mercenaries and zealots are motivated
differently. Moreover, it is not just developers that follow
this continuum - managers, project managers and analysts also have
their ideologies. It is important to understand, because at
the extremes of the continuum there is very little understanding (or
tolerance) of the other extreme.
I myself lean a little toward the zealot. I spend a lot of
time in the world of ideas. Strength or weakness?
What both articles fail to recognize is that there are organizations
who have spent a significant amount of time documenting the difference
in productivity between languages, not so much as a means of selecting
a development tool, but as a way of validating estimates.
Back in the '70's the group who created Function Point
Analysis actually started ranking languages in terms of level, and
specifically levels were based on how many SLOC (source lines of code)
were required to complete a function point - with the idea that the
less lines of code the more function points a developer could build in
the same period of time, give or take.
There is a
href="http://www.cs.bsu.edu/homepages/dmz/cs697/langtbl.htm">language
level table
that has been published and revised over the years to include many
(hundreds)
languages. Unfortunately Ruby and Python are not on this
version. So, Ed, there is a community that is focused on
software metrics that does try to empirically measure the productivity
that each language brings - simply in terms of the number of lines of
code per some defined chunk (function point). Unfortunately,
they are also a religion, and it appears that one cannot get the
current table without a paid subscription to one of their sites.
I, not being a true believer in that religion, do not have
access.
That said, I agree with both Ed and Bob that it is unlikely
that any language gives a 10-fold (90% reduction in code) increase in
productivity to the developer. or even a 5-fold (80%
reduction in code) increase. What I have seen is that the
best developers, when working in their chosen paradigm, outproduce the
average by that same factor of 5 or 10. That is a smart
coder, working in a language that he has
style="font-weight: bold;">truly mastered,
style="font-weight: bold;">and likes, can
outproduce me and you by a factor of 5 or 10. When I find the
language that lets me code like that, I will convert to it's cult, and
try to become a high priest. Why, because it feels really,
really good, to be supremely productive. When your teammates
look at you as if you were a god, that is a form of programming
nirvana.
But Ed's beef was specifically around dynamic languages. I
want to speak to this, because the first (and only) language where I
approached true mastery is a dynamic language. Interestingly,
there are interpreted (normal) and compiled versions of this language,
and working in both, I didn't notice a tremendous productivity
difference, because the compiled version had an interpretive run-time
engine built in, so it "simulated" interpretation. This is
necessary to be able to execute code out of a string at run-time.
The power of this language came from it's underlying
paradigm, the dynamic typing was something that allowed one to
introduce errors unintentionally, and one actually had to
style="font-weight: bold;">write extra code to
test variables to ensure that they had the right properties.
The true power of this language was the ease with which one could build
data-driven applications. The language primitives contained
some powerful string manipulation functions that could be combined to
great advantage in dealing with complex string manipulation
applications like interfaces or messaging. The language
supported code and data indirection, so that one could hide the name of
a variable or database record in a variable and refer "through" like
pointers but not nearly as abstract. One could also drop
executable code in a variable and move it around then execute when
desirable. Is that self modifying code? No - that
would mean that the original program modified itself in memory.
This is merely an additional form of dynamism.
Slurp code out of the database, then execute was as simple as
locating the code, and a single line of code to execute.
Does that sort of power make programs complex? Difficult to maintain?
Hard to understand? Hell yes! Does it mean that we shouldn't
use those language features? Hell no! With this
caveat - you need to carefully control how these features are used in a
production system, and ensure that there is adequate documentation for
those who come behind to understand how you wrote an XML parser in 200
lines of code, or some other amazing feat of programming wizardry.
One thing about interpreted languages (I currently use python, and php
the most) is that they let you write code from the inside out and test
it without much hassle - that is, I write functions and test them long
before I write the main. Because of the interpreter, I can
get a command line from which I can call my function like a verb
and see the result:
print ( object->method('test1', 123, 345));
That allows me to get a considerable way down the path before I worry
about larger chunks of code than functions or methods.
Working with unit test harnesses gives me much the same ability but I
have to build first in a compiled language, which even if it takes a
minute, means I am going slower than interpreted mode. In
interpreted mode, I close the editor, execute the function, and see the
result. Bam! I have even baked a script where I
have the unit tests baked into the edit cycle, so that I close the
editor, and the editor script automatically runs the unit tests for the
file that I am editing when I close the editor, and will prompt to
re-open the editor if any of them fails. That can be done in
any interpreted language with a little workstation automation.
There is a tremendous power to the immediacy of this idiom,
code-test-repeat-until-done.
Each language brings with it certain primitives that were pre-existent
in the mind of it's author, that when the fully resonate in a
developer's mind, they create power and productivity. Any
language that one picks up, it pays well to sit at the feet of a high
priest or at least a true believer and understand deeply where the
power comes from. I had this experience with Python recently,
and took a simple desktop automation script to the local high priest
for review. After he looked at my crappy code, he recommended
some more efficient ways of doing the same thing that took more
advantage of Pythons capabilities. He asked me how long I had
been using Python, and when I told him that this was my first script,
he said - not bad for a first try. Supreme validation for a
novice from the high priest - maybe I can convert to this religion...
I want to poke at the word mastery a little bit. It is one of
my
favorite words because it describes something that we don't understand
today, as well as we understood it before. This is especially true in
information technology, where new ideas arrive at a rate where only
those delivering these ideas to the community have time to achieve true
mastery, but that is a different problem.
In the age when trades were regulated by guilds, rather than organized
labor, there were specific requirements for achievement of each rank.
As I understand it, one of the requirements for mastery in
most
trades was the ability to construct one's own tools. I
learned this from my 8th grade shop teacher, who showed me how to
sharpen wood chisels with a buffing wheel and jewelers rouge - he made
me promise not to tell anyone because this was his trade
secret. After doing some basic research, I realize that this
understanding is assumed in another - the master had to make their own
tools, because they owned the workshop - a master was allowed to set up
their own workshop. Each
master's tools were his competitive edge. These tools and his
mastery were his "trade secrets" and he guarded them zealously, because
they key to his financial success. At some point during the
industrial age, tools became much more standardized and toolmaking
became a trade in its own right, and tradesmen purchased their tools
rather than making them, or working for a master.
So what on earth does this have to do with agile, I can almost hear you
asking. Absolutely everything. I agree with Dave
when he
says that "agility is all about the journey". In fact, what
agile
practitioners are is navigators. Agile isn't about the
practices,
the practices are merely the vessel in which we take the journey.
Not every vessel is appropriate for every journey, the master
is
one who can decide which practices make sense for this particular
journey, and can adapt the vessel on the fly, tuning the practices to
the specifics of the team or project.
Those of us who are less experienced with agile practices, having only
implemented parts or aspects, or all of a single methodology on one or
two teams or projects may try to repeat our experience and be
frustrated when we don't repeat our original positive experience, are
not masters, but journeymen, who need to borrow our tools from a master
craftsman. We need the experience of an agile coach who can
see
where we are missing the boat, so to speak.
I am thankful for the masters who have gone before - the pioneers -
Beck, Highsmith, Cockburn, Beedle, etc. They have built ships
that I can sail, and I need to learn from their experience. I
think that there are many, many more who have learned from the these
masters, and have crafted their own vessels, sailed off to distant
shores, but have not shared their saga with the community.
I suppose that I disagree with Dave in his definition of mastery, that
mastery is the point when one can say, "Now I know it all" is what I
disagree with. That would be like saying that a master
carpenter
will never see a situation that requires him to be "creative".
In
fact, I would suggest that innovation comes from the fact that mastery
does not mean "I know it all", but it means I can try something when
what I have done before doesn't work. I submit that one is a
master of agile, when one can invent a practice that works where others
fail, while maintain the tenets of agility.