The Water-Scrum-Fall Guide to Requirements

 Recently I have been involved in a number of conversations in relation to more efficient and visible requirements gathering. The issues relate to Business Analysts and the length of time it takes them to complete requirements and when are finally complete, the high level of detail and complexity that is captured - way beyond the scope of the original request. It’s a very easy trap to fall in but there is a way out. If you are in a Waterfall type project then there are some agile principles that can help.

Shut the Gate

One suggested approach to resolve this issue is to have more weekly reports and gate reviews. Fill in a template, email it off, give a weekly update and so on. To infrequent, too impersonal and too easy to paint a picture that is different from what is actually happening. This won’t work. It actually slows the process down. Which in turn means more Gate Reviews, more formal “template” updates and more smoke when you are trying to find the fire.

Think!

What do I need for day one production, use it to refine your scope. This is rapidly becoming my mantra. If I had to drive between Auckland and Hamilton what would I need? A car, licensed, registered, roadworthy car and petrol. Do I need a car radio? No. Do I need Air Conditioner? No. Do I need a Coffee Holder? No.  Think the same way when capturing requirements.

Manage

The second part of the solution is being managed, both the person and the tasks. No one should be allowed to work without reporting progress and having visibility of what they are doing. This is not an SDLC problem, it is something that Agile will not resolve, this is a person management problem. 

Be Agile-ish

What I have recommended is to add some good Agile practices to the Requirements phase. You don’t need  to wait for the coding to start to use some of the good agile practices.

• Cross functional team, involve the functional, technical and test experts.
There is often a surprising amount of business logic in the heads of the Developers and other technical experts. Their input into the requirements is vital to ensuring they are sensible, manageable and realistic.

• Daily stand-ups
Discuss progress, issues, upcoming events each day, do not wait for weekly reports, gate reviews, etc. Surface and deal with issues on the day.

• Visible tickets
What ever you are working on can go into a card, sticky note, JIRA ticket. Define the task and whom it is assigned to and hence whom is responsible. Completing requirements still means working through a series of tasks and having them visible means progress can be tracked.

• Define done before you start
Decide on what you need for day one production readiness. Think the basics. Keep the requirements you capture as simple as possible and for each one define what is done. This makes life significantly easier for the developer, tester and those doing the user acceptance testing. Also it helps prevent scope creep as it prevents different uninformed and undefined perceptions of what is done.

Build Frequently & Demo Often

Once the requirements are signed off, sprint (literally) to get the first build in front of the customer. At the completion of the first demonstration the customer(s) will provide feedback which will mean changes. Don’t think of these as scope creep or changes, but the natural evolution of the software. 

Rinse & Repeat

Again capture this feedback, discuss, socialise, make it visible, define done and track it. Be pragmatic, if you find a method or approach that works then use it. Mix methods to find the best results. Never stop trying something new to solve a problem. 

Why Projects Succeed

Over the past year I have worked on or managed a number of projects which have been quite successful, they have complete on time, close to, just over or under budged and with happy stakeholders.  Good luck or good project management, here are my thoughts:

Project Management

Defining bad project managers is easy, they are the ones that ask you how their project is going cause they don’t know and/or don’t care, or ask you if they should turn up to their own project meetings. Disengaged, disorganised, no milestones, no updates, no idea of issues and risks, no chance of success. Go too far the other way and you can have a micro managerial fanatic bogged down in detail and making no progress.

A good PM has a firm, but subtle hand on the tiller. They guide the project with informed decisions, communicate clearly and often, build and use relationships to get work done, they delegate, trust, monitor, measure and have contingency plans. They do not seek to offload their responsibilities or hide from the difficult projects. They know how to get things done and who to go to.

Managing Changing Requirements

Unless you are running a very short, very limited scope project, requirements will change. It’s how you manage this change that is important. You can push back and remind the senior user whom signed the high level requirements that they cannot change their minds. Or you can expect changes, the first few demonstrations will also reveal changes to the original requirements and design. As Senior Users begin to use the iteratively built features of your software they will give you feedback which may invalid or ideally fine tune the original requirements. This is great, this is what you want your users to tell you. No point sticking to the original requirements and design if no one wants the end product.

Roles & Responsibilities

I have sat in several meetings and asked “Who is the Project Manager”, only to receive silence or “Who is the Scrum master?”  and even once “Who is the customers/senior user?”. Who is doing what and from that when. This forms the basis of a plan. Otherwise you’re like a group of outfielders running around under a fly ball all yelling “yours”.  Without R&R you cannot have a plan and without a plan, even a high level one, you chances of succeeding are low.

Define whom you and what you need to do, add in a timeframe and bingo, the elements of a good project start coming together.

Communication (previously known as Co-location)

About a year ago I would have sworn that co-location was a key element in a project’s success. But since then I have completed 5 projects working with a company based in another city, doing all the work remotely.  Having clear R&R (as above) and communications is the catalyst to co-location success. Since the person (or persons) is not sitting next to you, you have to put extra effort into communicating in a clear and concise manner. Start with a good plan and work from there, ensuring that everyone knows what they are doing when. Actually I believe that Non-Location maybe better for a project as it prevents you from assuming good communications will come from simply sitting together.

Technical Factors (infrastructure & environments)

This used to be such a headache. I know I have an Applications basis, but how many times has a project been delayed by waiting for infrastructure to be built, configured, ports opened, firewalls changed and soon on. Even in the days of virtual environments and clouds it can still take time. Then there is the time and frustration cost of a failed migration from a System Admin whom cannot follow instructions. Having a good Operations person on the project is invaluable, days of waiting for database and file access turn into minutes and migrations become seamless and hassle free. Frequently – and often without realising it – they become one of the key project resources and unsung hero. I can think of several where I currently work.

Human Factors (context switching & resource planning)

Managing a team of 20+ people includes a strong element resource planning and management. Projects start, stop, pause and take off suddenly. Though a variety of formal and informal communications channels we are made aware of current or planned projects. We plan and resource accordingly. We also had contingency based on project process. The aim of this is to ensure a Developer(s) can stay on a project until completion and avoid distractions. Which leads into context switching. In simple terms it’s an overload of tasks (be it project or incidents) that prevents an individual focusing on completing their work in quality and timely manner. Carefully resource planning and project priorities  will avoid this from happening. The culture of the organisation you work for can assist or hinder this process. I have heard senior managers tell me it’s ok to have a Developer assigned to three active projects – maybe they should learn/remember how to code and try it.

Catalyst & Gelling (Soft Human Factors)

Have you ever notice that there are certain individuals on projects that others will listen to, whom are knowledgeable, willing to support and help others and though not team leaders or managers, others will naturally follow their instruction.  Project teams will naturally bond and gel around these people and their wisdom and knowledge becomes invaluable. I have worked with a few people like this, they are naturally intelligent, calm, patient and helpful. They will also often be very modest and step back from the limelight and move onto the next project. They know who they are, good project managers want them and good people managers retain them.

Conclusion

Go through all the above factors and you come to the conclusion that a successful project is not the result of good luck, but one of good management . Good communication and the skills of an air traffic controller juggling several planes in and outgoing also helps. that it all involves people – whom understand technology  - but not technology itself. I will leave you with my favourite quote of the year so far.

The major problems of our work are not so much technological as sociological in nature. (PeopleWare Productive Projects and Teams (DeMacro & Lister).

The Basics of Good Application Management

 

  Recently I completed the corporate Wiki upgrade, bang on time and at one-third of the planned budget.  When I presented the closure report I was asked by our Accountant  why I was so under budget. I  replied “I am a really really really good Project Manager. [Happy Face]”, followed by a humorous exchange.  I had a similar result   one month ago with the JIRA upgrade, on time and only half the budget.

However it did get me thinking why it was such a successful project. It all went to plan and the final production upgrade was uneventful.  It comes down to the basics of good Application Management. I have read many application management articles, some are too high level and many can be overly complex,  so based on my experience and the theme of “simple is good” here are the basics:

Strategy & Processes

Have a vision for your application, have plans to get it there. Measure those plans and change them if they are not working. After the upgrade to the newest version last year I had a vision of the Wiki usage increasing, more spaces, more content, more views and more users. The application Wiki statistics have proven this.

Process, the how, what and why – in detail. During the our big upgrade we realised that a lot of processes we needed existed in email folders or did not exist. Discussing, proposing, socialising and finalising processes around support, upgrade paths, plugins, macros, etc. meant we can move from project to operational mode without losing momentum and knowledge.

Treat DEV and TEST like PRODUCTION

One of the issues we faced with our three year catch-up upgrade project last year was the mess that the Development and Test environments were in. Development was a partial failed install of an old version and was dead, Test was running, but was a failed install of yet of another version and no one could log on and Production was stable, running and out of date and was been used like development and test. Plus there were issues with disk space, missing files, missing images, permissions, firewalls, ports, backups, user numbers, licensing, plugin versions, performance all of which needed to be fixed.

After considerable effort to align all three environments it was made very clear to those whom access these environments they are not a play pen for bored Developers or Operations staff. All changes on all environments need my approval. I regularly check them and cycle through changes. If anything is broken I want to know why and I want to know whom.  A discussion ensures followed by corrective action.

We now have an ecosystem which is  easy to cycle your upgrades and changes through.

Use Experts

The corporate Wiki is not our core business, we are not interested in customising the application or developing any form of intellectual property or specialist knowledge. We are happy to outsource.

So when we commenced planning last year we sought the help of experts in this particular field. Especially as previous efforts had failed through lack of internal resource availability and expertise. Our external partners are experts, they  are also fast, efficient, effective and  thorough.

Partners not Vendors

This is a deliberate choice of words. I want to work with another company that sees our working relationship as long term and beneficial to all involved. They go beyond the boundaries of our agreement to assist and we do likewise. This is a benefit to both of use. It still amazes me how many companies treat their clients as an invoice recipient and  if you are lucky - a biannual coffee visit.

Learn from Your Lessons Learnt

This sounds obvious, but how often is it done. There can often be a “Groundhog Day”  affect with IT Projects. Understand how and why previous projects did not work and plan differently. In our case, we used an external expert, listened to their advice and respected their expertise. This made a huge difference in the quality of technical work done. One of the weak points in our previous project was communication, during the planning of the most recent upgrade we developed a communications strategy, a communications log and followed the communications plan. The outcome was a much better informed and happier user community.  The lessons from our last upgrade will go into the next upgrade,  this is to prevent the good points for being forgotten as much as to remediate the bad points.

Understand Your User Community

You don’t want to release “New Coke” thinking it’s the next big thing.  Find your experts and champions and get them on-board, work with them and get close to them. Listen and learn from their experiences. They will offer good advice and make great user acceptance testing Testers.  They will championing your application to others and spread the word.

Benefits

Define and measure. If you don’t have a good reason to do something, then don’t do it. This comes back to your strategy. If you are doing something and planning to get a benefit and do not, go back and think again, you are doing it wrong.

Closing

All the above can take some hard work and some good planning (let’s not confuse simple with easy).  Many of the above areas overlap, lessons learnt with communication, expertise and change control on all your environments. Above all – have a plan. Be realistic, define, measure and realise your benefits. It gets easier when the elements fall into place.

Prince2: The Simple Approach is Best

 

After years of informal project management – running projects with no formal qualifications or more often, being asked to "assist" after the project is stalled, struck, and in deep do-do. I decided to undertake some formal PM training, namely Prince2. Having some formal training and an understanding of a project methodology would help round out my knowledge in this space. Part of a Development Manager role is to be heavily involved in projects so it’s good to know.

So here are my thoughts.

Roles and Responsibilities. One of key decisions in the earlier stages (starting up or initiating) of a project phases. Yet so easily often forgotten. How can you have a project without a project manager? The answer is you cannot (if only this was a Prince2 exam question). But also whom is the technical lead, BA, Tester, who is doing what. This is a common failing of many projects. Not having clearly defined R&R. So you don’t know who is doing what, what is being delivered when? Whom is signing off of and so on. Confusion reigns.

Planning.  One of the overriding themes of Prince2  was – be prepared.  You will need to setup a lot of management products and they are there for a reason. Issue Register, Daily Log, Lessons Learnt, Risk Register and so on, then there are the governances, Risk Management Strategy, Communications Strategy and so on. But a well prepared project, is a ready for anything project, when risks and issues occur, you are respond in an organized way. Like a well drilled ship’s crew you response to each alert and emergency as per the plan. The plan gives you confidence it what you are doing and that  you can resolve the situation. Everyone knows what they’re doing and when they’re doing it.

But is it Simple, Tailor it to what you need.

What struck me as important was how simple many of these management products where. The project briefs were just that, brief, one to two pages. Many of the strategies were one to two pages for simple projects. They were concise, to the point and useful. You went to your Risk Strategy when a risk occurs, you use your Communications Strategy to guide your communications to who, when and what message. Keep it simple.

 How many times do you work with a Program or Project office and when you go to prepare these products, they send you long, painful, tortuous templates with equally useless examples of ones already done. By the time you fill in there SO much detail the purpose is lost. The document becomes unusable, unused, useless and you manage off the seat of your pants. Ever wonder why an aircraft evacuation  card is simple? Do you want a 10 page document on how to get off the plane after it crash lands? No, where’s the door and how do I get there. Done.

Takeaways form Inspiring Project Managers or Middle Aged Development Managers.

So going forward. I like the structure, preparedness and planning that Prince2 dictates. Be ready for the expected and unexpected and have a plan to deal with it. Keep the plans simple, keep the management products simple. There are some nice and simple learnings, keep Lessons Learnt Logs, future projects will praise your foresight and refer back to them for wisdom and learnings. Define, measure and capture your benefits, years down the track and project is still delivering, state it, measure it, publicize and celebrate it.

Most of all if you’re the stick in the mud that practices planning and preparedness, your successes will stand out from other stuck in the mud.

The Inherent Quality of Quality

There is a particular section – especially two sentences – from the PeopleWare eBook that has become quite literally lodged in my head.

“In some Japanese companies, notably Hitachi Software and parts of Fujitsu, the project team has an effective power of veto over delivery of what they believe to be a not-yet-ready product.”

And

“Enough of a quality culture has been built up so that these Japanese managers know better than to bully their workers into settling for lower quality. Could you give your people power of veto over delivery?”

My Japanese wife will simply point out that being a hairy white barbarian I naturally have no concept of good quality or culture, hence I cannot marry the two concepts together. Going beyond the humor of our cross cultural marriage these are quite profound statements.

Imagine being on a project team and having the final say. No pressure from the project manager, your manager, senior managers, directors, they all understand that quality is first and if the product is not a quality product you do not deliver. 

Quality initiatives can come in any forms. Often they are driven from the ground up. The truly professional motivated IT people will brand together, set standards, establish principles and seek to eliminate poor practices. However without management support these ideas are likely to last more than a week, maybe a month if you are really keen. The constant and often overbearing demand for an output – be it any output – overrides the ideal of quality. How many times has a senior manager pursed their lips and said “I don’t care, it needs to be done”.

Managers themselves – often aware of the technical debt or far from complimentary feedback on their teams’ products – may also embark on quality initiatives. The more senior ones will have enough authority and budget to essential to drive these initiatives. Teams will respond to this. Managers may lack the details of implementing quality. At a detail level they will need to trust and empower their team members. However such great ideas will get derailed by others whom will politic, escalate and literally yell to get things done.

Therein lays the obvious yet challenging secret to the Japanese success. Culture. They all accept it. It is an establish professional protocol. Not a fad, not a CIP, not a fancy presentation from a suite wearing smooth latté slipping consultant. You don’t need to sell an ideal to a group that have in established from birth. The entire organization buys, they all accept it. And therein lies the reason for sustainable success – the entire organization buying in.

It makes you think of John Lennon’s famous lyrics “imagine all the projects, with quality outcomes…”

Agile (and we didn’t even know it)

Back in 2003 I was involved in the resurrection of the building consents project. This was the third bite in the fourth year of a very prolonged and unsuccessful series of projects to replace a legacy system. The first failed as it placed a poorly coded ASP application on top of the legacy system –which would not accept multiple web users and crashed on go live day, the second failed largely due to the PM being so disinterested they did not attend their own project meetings and the technical lead going off at angles meaning that most of the code was half done. So it was my turn, using the leftover code form the second attempt. There was me, Mickey the band new Developer and Ruth, the expert from the business, whom became the ultimate champion and unsung hero of the project.

So we kicked off. The list of current defects was extensive. Virtually most of the application did not work and you could not login. We decided to focus on each functional area, get it working before moving to the next. So we started with the login screen, security, administration options and the basic consent summary landing page. Each day Mickey would code away,  while I would peer review and test (between being the BA, Trainer and PM). On a daily basis we would pop up and see Ruth in her quiet little office and spend an hour having her test the application and get feedback. We would then do a bit coding in the afternoon and so on.

After the consent summary it was onto the name and address module, building inspections, consent tracking, fees, conditions and finally notes. Each section was focused on and worked on until Ruth was happy that it was working as required. We then demonstrated and tested the code with a smaller group of key users. We then worked on creating a consent, printing and reports (which were really basic). Finally after two months of daily code, demo, test, code and repeat we were ready to begin more formal testing.

 We made some very pragmatic decisions as we went. The layout and logic of the consent tracking was not ideal, but we did not have the time to re-write it, the name and address register was unnecessarily complex (and ironically probably still is 10 years later),  the inspection business logic was too complex to code within the time frame. Each of these issues was prioritized into post-production releases – of which we had planned three before we finished.

Finally into UAT, training and release. Go live day was entertaining. As the previous aforementioned version 2 had died on go live day all the users jumped on and started searching and looking up consents, waiting for it too fall over. After about half an hour I popped up and asked if some would like to create a new consent – and actually use it. Off they went, no issues.

There are many agile-ish elements to this project. Sprints (off sorts), multi-functional closely located team, frequent demos, peer programming, retrospectives and so on.  At the time (and for many years later) the waterfall approach was standard at Council – despite efforts to introduce more agile practices. There were many common sense elements to why we succeeded - a dedicated team, engaged focused and extensive business knowledge, committed senior management whom wanted a successful outcome, a wider group of enthusiastic  users, limited but visible project governance. Also at the time using the corporate property system as the sole and automated source of property data as opposed to manual entry was new, but lead to a vast improvement in accuracy i.e. we went from having 7% of consents not matching to valid properties, to 0%.

The application is still in use, 10 years plus, parts of the original code and a lot of the original data model are still in use. We never did get around to upgrading to .NET and it’s still an ASP application. Due for decommission shortly as a new corporate wide solution is adopted.  Mickey left council two years later and two years after that Ruth retired. Over those four years myself and Ruth worked through all the backlog and made numerous and regular changes based on the feedback she received. Every year we would have a cake to celebrate the birthday of the application, this continued right until 2010 when I left, interestingly enough it was often over a mouthful of cake that some great ideas came out. This project remains the one I am most proud of as a Developer and I do miss working with Ruth and Mickey.

Standup My Friend

Recently I was asked what were some ways  to create a positive workplace culture, the organization in question had established a social club and it was not working. No one was showing up at the events and it was about to be axed.  I had a chat with my peer and told him social clubs only work when people work. Let me explain:

Organizations that are challenged with the need to develop or enhance culture will decide that one of the best ways is to organize a social club. In the ideal classless society such as New Zealand this is a seemingly great idea, grab the BBQ, butter the bread, squeeze the sauce and by virtue of wrapping a sausage around a piece of bread we are great mates. How many times at the social club BBQ do people come, eat, say hi and vanish with the sausage in a napkin back to their desk? How many times to they sit down and chat long after the sausages have slowly brunt or gone cold, the flies have landed on the bread and the salad remains untouched?

To want to socialise together you first need to work together. You come to work (noun) to work (verb). Simple. You need to complete tasks that are assigned to you. So far so good. Nobody said you needed to like your colleagues, this is assumed that it will happen. So your neighbour three desks down the row and on the same project team. What do you know about them?

Daily standups are an effective communications mechanism for projects. The team get to know what’s going on every 24 hrs. But there are also a great social mechanism. Inevitably small and occasionally candid pieces of personal information slip into conversations. Suddenly the annoying tester three desks down isn’t the jerk who seems to enjoy failing your code, but a guy with a pregnant wife who spent the weekend shopping for baby clothes. He seems more interested in his wife’s blood pressure than yours.  But that’s understandable, after all you have two small children. You were once that naive person that though antenatal classes taught you everything you needed to know about childbirth. He also seems like an okay kind of guy, so why not go and have a chat to him – offline – about his testing approach, expectations, ask for his opinion and feedback before submitting the code.

I am not suggest daily standups become a substitute social club, but they are a good start to social interaction. Communication is why humans evolve to where we are on the food chain, we can co-ordinate, retain and spread oral knowledge, entertain, empathize and so on with these skills. Otherwise we would be another struggling homo- species and cat food on the plains of Africa.

And antenatal classes, you forget everything at the first contraction.