Every few years a new methodology arrives for organizing the work of software engineers. Several years ago it was Extreme Programming; this year’s model seems to be Scrum.
A bunch of us were talking recently about Scrum and its benefits, and I was probably being overly negative and sceptical, especially since I know very little about it. But I didn’t like the answer I got when I asked the question “What kinds of domains is Scrum is especially good for, and what _doesn’t_ it work so well for?” The answer was, roughly: “You can apply Scrum to anything!” Uh-oh.
It makes me nervous when I hear that kind of claim, for a couple of reasons. First, it makes me worry that we’ve got a solution style that isn’t about listening to the problem (especially since I believe that different kinds of software dev can be deeply different, as I’ve argued before). Secondly, it makes me wonder what happens if Scrum is tried in some area and fails badly or doesn’t seem to apply; do we conclude that Scrum has limits, or do we say something like “Well, you see, they didn’t apply Scrum thoroughly enough!”. (At the risk of making a bad and cranky joke, this kind of response is truly “agile”.) There’s nothing so useful as a well-documented failure, or a falsifiable management theory.
Anyway, having already made it clear up front that I know nothing about the subject matter I’m spouting off about, it’s time to read a book. Recommendations for the single perfect Scrum book are welcomed.
Tim Converse 
The core of SCRUM is that it applies to managing any process that requires an empirical method, as opposed to a defined method. Most industrial processes are defined processes, all variables are understood from the outset, thus precise planning can be applied. The software development industry has made the mistake of trying to use a defined process where it is unsuitable.
Software development is complex, variables are not understood at the outset ( rarely do the ‘customers’ know what they want until they have tried a few version and discovered their requirements, also choices of which of the evolving technology should be used etc.)
So the breadth of application of SCRUM is where an empirical approach is required. This encompasses an iterative approach to producing deliverables with control over interruptions to the task of producing each iteration of those deliverables. After each iteration the set of delivered features/products is inspected by the customer and their feedback alters the deliverables and their priorities in the future iterations. There are other key control aspects to the methodology, but it is inherently simple.
Simply put, SCRUM is a management framework supporting an empirical method of product development. It has been used outside the software industry, although not widely. So possibly your colleagues could not articulate what they had gleaned from literature.
And yes, I am a certified ScrumMaster and possibly should be locked up. The only book I read was a prerequisite to becoming certifed, ‘Agile software development using SCRUM’. Combined with 25 years in IT, a lot of it as a project manager, it immeditately made sense.
Cheers
David
David —
Thanks, that was helpful. A couple of follow-up questions, one trivial and one deeper (I hope):
1) Your description of SCRUM is very abstract, and doesn’t commit to any particular problem domain. Yet in my contact with SCRUM masters I keep hearing statements like: “Sprints should be 1 month in duration. Two weeks is too short, and six weeks is too long.” Can anyone explain how it could be that the ideal sprint duration is independent of what people are working on?”
2) I understand and appreciate that SCRUM processes are designed for domains that are empirical rather than being defined. To the extent that’s true, it’s all to the good as far as I’m concerned. My worry about SCRUM is actually that it assumes too much definition rather than too little.
In my application domain, it’s often true that the biggest advances happen like this: a really smart person goes off into the metaphorical wilderness alone for two or three months, and comes back with stone tablets that say “We’ve been thinking about this the wrong way; instead of organizing by document, we should organize by word/person/concept/[etc.] (This example is completely made up, but preserves the flavor of what I’m talking about…). This 2-3 months of individual thinking leads to 6-12 months of calendar time of concerted effort by multiple people. At the end of that time, we get a measurably much better result. Until then, though, the customer has nothing to show for the work.
Now, can anyone tell me how the above process maps onto SCRUM? (And if SCRUM can’t support this, then I’m afraid I’m not a buyer.)
In answer to 1 I’d say that 1 month duration works for a software project. Likely to be different for a product where the time to produce something that is tangible and can be inspected by the ‘customer’ differs. Lets say a new menu for a Restuarant, might only take a week to produce the food on the menu. Then the ‘customer’ can try it, make suggestions and rate it. Then start interation 2. The idea being that if the ‘customer’ was in the kitchen during that week it could be chaos. The chef can approach the customer for clarification of requirements but the customer cannot interfere with the production of the menu.
On 2 you are running the risk of the smart person getting it wrong. Maybe a solution too clever for the intended ‘customer’. You make no mention of the ‘customer’ and when he/she gets the opportunity to give their opinions on what they have to use.
Also this type of development is rare, I have worked on one large project like this in 25 years. We replaced the humans in the process, so the ‘customer’ was less powerful in this scenario. But even then I’d say that SCRUM would have helped with credibility from the paymasters. Our 2 real smart guys (maybe IQ 150) could have produced pieces of the design in a month, probably aiding the design result in fact.
I am sure you are not talking about architectual design but conceptual approach to a solution. Design nowadays tends to be OO or component based, which suits the SCRUM process.
I’d rather take less risk with the design process and have more chance of an accepted result. I don’t believe that the smart person has to take 2-3 months, it can be broken up into chunks. A review of their thinking by their peers does no harm and reduces risk.
So my question to you is, how much time before the customer gets to vet the progress, possibly realising what they wanted was uninformed? In my experience the customers’ discovery of what they really want and subsequent influence to get it is one of the biggest risks to the project.
Of course if you are trying for the next quantum change, like the invention of the WWW, quite possible SCRUM will not be useful, but in 95% of software development I am certain it is.
I am trying to combine MDA techniques with SCRUM, forget savings from going offshore. Save much more and have higher success rates by staying close to your customer and being a great deal more responsive and efficient. The offshore trends makes me laugh, suffer the same miserable success rate (see Standish Group report on this), but more cheaply.
David
I was exaggerating a bit with my example (3 months in the wilderness, 6 months until any benefit), but not a lot. And of course, there’s no reason there can’t be interleaved peer review of ideas, etc., but that’s not identical to customer review of exposed benefits (unless we extend the notion of “customer” and “benefits” to near meaninglessness).
I like your chef/customer analogy, in part because it highlights _both_ the fact that the customer should periodically get to taste and suggest _and_ that there are timescales over which the customer will just get in the way, due to lack of expertise or cost of interaction.
My suggestion is that the right timescales vary much more widely even across the software development world than the Scrum story would have you believe.
The particular software dev space I’m coming from here is making web search engines more accurate and relevant. The reason I don’t mention the customer much is that the customer’s goals here are pretty well understood and agreed upon. The thing that makes it hard to scrummify is that, if our current state is A, and the goal state is Z, the customer is not going to have much insight or helpful suggestion about what B,C,D.. should be. Obviously there’s continual improvement of the real exposed artifact, with tracking and updates and metrics galore. But underneath the hood there are sizable projects that won’t affect that live service for very long periods of time. It’s those projects that I hope Scrum can support, and I will be suspicious of Scrum if it has to pretend that they don’t or shouldn’t exist.
I also like the fact that you said “in 95% of software development I am certain that [Scrum is useful]”. I could well believe that the type of task I work on is rare in that sense. But I’m reassured by willingness to at least talk about the possibility of cases that don’t fit. I could well believe there are software domains where, if you haven’t made highly customer-visible improvements in a month, you should be sacked; there are also domains where if you haven’t done some externally invisible work toward a one or two-year goal, then you should also be sacked.
By the way, Troutgirl made an interesting point in conversation about this: For software projects, at any rate, Scrum is depicted as a way to manage the project itself. Might it be better viewed as a way to manage the customer? So much of it is about not only getting customer input but improving transparency about where the money is going. Is it possible that this is _most_ of the benefit? I.E. maybe in some cases your software isn’t better as a result, but the customer feels more trusted, trusting, and clued-in?
I’d say that the definition of customer needs clarification here. It is the group that can judge the efficacy of the products being produced, in your case this is possibly your peers in the development group.
The reason for keeping the ‘customer’ out of the hair of the developer is mostly to stop them changing their mind about what they want until you have produced the agreed product artifacts for that sprint. Without SCRUM the interruption can end up being continuous, wiith nothing getting produced. SCRUM is largely about controlling the balance between what the customer wants and how he/she discovers this and the opportunity for develpers to produce useful product increments. The useful product increments give the customer the best chance of discovering their real requirements.
I am trying to distill it to the core of the methodology, as this is what applies to most projects. As you suggest if the methodology cannot hold up in cases other than the common scenario then it is no good to you. However this does not prove anything with regard to whether SCRUM is effective in the narrow but common project scenario.
As someone with a lot of experience, the idea of controlling the customer as a part of the project management role is not something I view at all as being outside the scope of project management. I like to think of SCRUM as a balanced approach to all influences on the successful outcome of a project. So troutgirl is right too. But the customer feels happy when what you produce is on budget and solves the problem they had, feeling trusted and clued-in is just a part of getting to that point. Think of them trusted and clued-in and the product not doing what they want/need.
David
OK, I think we’ve made progress. With these arguments about methodology it’s easy to slip into a mode where the advocate is trying to defend the idea that the methodogy is useful in some cases, and the skeptic is arguing that it’s not useful in all cases, and neither side realizes that there’s no inherent disagreement yet.
With regard to software development in general, I have no idea whether the subset where Scrum helps is 5%, 50%, or 95%, but our agreement that it’s unlikely to be 0% or 100% is a good thing.
I don’t understand why conditional application is so underused in this kind of methodology. The conditional branch is so crucial in software; why are there no software dev methodologies that say: “If you find yourself in this situation, use method A; otherwise, use (completely different) method B”? Instead, they all seem to take the following form: “Method A is great; if that doesn’t seem to work, then there’s something wrong.”
Even if Scrum is good for the average or most prevalent case, does it help Scrum advocacy to prescribe first, and find out what kind of development is being done second? (I am not accusing Dave of this, but I have had this experience with methodology advocates.) The only kind of advice that is definitionally good for everyone (without having listened first) is religious advice.
Can someone describe for me the aspects of SCRUM Methodology and the impacts it would have on holistic architecture development? What I am interested in is how does SCRUM add to, or detract from, an overall enterprise technical architecture. I see it as potentially being adverse in that the project teams will be more project-focussed in their delivery and will miss more corporate opportunities for enhancing the overall architecture. it seems that traditionally, project teams are focussed on their specific project deliverables without keeping their eyes open to a wider purpose….an example would be the support of a Service Oriented Architecture…is there an incentive for them to build towards such an architecture? Thanks…I don’t know enough about this mehtodology, but from what I’ve read thus far, I have some pretty big concerns
The architectural ‘products’ are a part of the product backlog. Since so many of the end products depend on the architecture being sketched out, tested a bit, refined and having different architectural decisions available at different stages of the overall project, they have a higher priority than most of the other products. This is reflected in their position in the product backlog.
The structure of the project team can change, to say, involve a senior (maybe experienced in architectural design) member from a couple of the other teams focussed on delivering the products tangible to the end user ( which architecture is not, generally). This way the teams all have a influence in the architecture, assuming that is desired. Of course a lot of this comes down to how experienced the ScrumMaster is in shaping things.
If you have doubts, do the ScrumMaster course, it is inexpensive and we discussed a lot of issues, incuding the one you raise.
Regards,
David
Having just been through a scrum sprint, I can definitely say that Timboy’s concerns are well founded and understated. It’s definitely oriented towards controlling management. In situations where mgt doesn’t need to be controlled, then it really has no useful purpose. I have been on many projects where it would make no sense whatsoever. For example, a 2 year project that couldn’t have been split up into 24 segments. I worked on a year long project that was very nicely split up into 5 interim releases. To have made that into 12 releases would have been stupid.
I speculate that the creators of scrum have had a limited software experience. They seem to be reinventing software project mgt, as if teamwork is a new concept. Specifically,
1) One can have a lot of Synergy without daily meetings.
2) too much importance is placed on the daily meeting, causing a shadow effect. In the half hour before the mtg, you really can’t start much, since you have to make the meeting.
3) A yardage goal has some positive effect, but it also distorts the decision making process.
4) An artificial deadline has a tendancy to result in hacking and a lack of considered thoughtful design.
5) the process is not scalable.
Having recently read the book Domain Driven Design and also having an acute interest in combining MDA with SCRUM to satisfy an FDA compliance process illustrating that our company is at all times in a “state of control” with respect to our development, I wanted to make a couple of observations. It appears that:
1.
SCRUM is more about unleashing human potential by allowing people to fail in an environment by keeping the cost of failure low.
The typical corporate mindset is that failure is to be avoided, we have to micromanage our people and if we don’t we will never get what we want – trying is to be avoided and only success is rewarded. This kind of thinking is absurd (think of Thomas Jefferson).
Everyone knows that any piece of software goes through many “trials” until we get it right. Ken Schwaber himself stated that he had a customer describe SCRUM as:
“Getting What You Don’t Want More Often”
This kind of thinking is a powerful paradigm shift because it allows potential to be unleashed within a team to accomplish a sprint goal. Instead of developers or anyone else for that matter, walking on eggshells around each other deathly afraid to commit the wrong thing to paper (and find out about it 6 months later upon customer review) or so concerned about saying the wrong thing in the efforts of managing perceptions, it restores one basic idea that we all seem to forget — We are all Human and humans make mistakes because that’s how we learn.
2.
SCRUM is a way to manage creative people by allowing them to make mistakes. Creative people have all kinds of ideas – all kinds of ways to solve problems, but if they cannot fail they won’t try and if they are left to their own devices they will usually fail (unless they adopt complex process frameworks to regulate their behavior or have an abundance of sensitivity which most developers do not). Smart creative people will learn to “play it safe” over time – knowing what to say and do and when; but it is a collosal waste of time.
Having all stakeholders and team members organized around a collective goal is more consistent with problem solving anyway and a lot less painful.
3.
The current paradigm of business is the military one of “Compartmentalization” – know only what you need to so that you can do your job effectively. SCRUM tears down these artificial walls and in the process restores the dignity of people (which the latter denies). This results in huge leaps in productivity as people are no longer concerned with what they can and cannot say or do or what role they were hired for – they get back to being concerned about being effective. Spiritually, it restores people from being resources to humans. It is a win/win. This is at least as significant as the move from processes and functions to objects.
4.
Now, the problem becomes not how
to manage that creativity firehose – that is addressed by SCRUM – the problem becomes how do we record the results of that creativity and use it to manage complex ideas and build software that people want to use. MDA and Domain Driven Design solves that.
You are exactly right, scrum is just another bullshit idea. It fogets the human factor. I hate to break it to you scrummers, us hackers run this world and your best bet is to find management that can locate and work with good hackers. simple as that. hackers run this industy. any offense against your resident hackers will blow you out of the compedative market.