Tactical Documentation During Modernization by Ed Lyons
(Generic whiteboard diagram)
The widespread adoption of Agile technology resulted in a lot less documentation being written, and a devaluing of that kind of work, in general. However, focused, useful documentation is crucial for project success. This is especially true in modernization efforts, where a new perspective is needed across many systems. In modernization engagements, tactical documentation efforts lower risk, increase velocity, and aid the buy-in of team members.
Here are the documentation challenges that modernization efforts face:
Some systems have become “black boxes” and the people who know exactly what was inside some of them are gone
Existing documentation is out of date, or is missing
Existing documents and diagrams are not useful for the modernization effort
Documenting systems that will be retired seems like a waste of resources
Long-time organization members do not have clarity about the modernization effort and the steps involved
Documentation is often written to describe what a system is, not how it fits into a modernization effort, and lacks urgency and tactical value
Here are some best practices we use at EQengineered:
Modernization is a perspective change, and documentation must support that.
Modernization engagements often involve systems that are no longer well-understood. Perhaps the documentation for those systems is outdated or entirely missing.
But even if documentation or system diagrams are there and are relatively recent, they might not be that useful to the modernization effort. Modernization requires a new perspective and a refactoring of priorities. Gathering existing documentation is useful, but is just the first step. Client team members who have maintained those systems might not understand how modernization affects those systems, and how risk models will change.
There is a need for high-level documentation that shows how every system will be affected by modernization. One example would be to identify which systems are remaining in place, which will be changed, and which will be retired. There should also be prioritization. This will help all team members make better decisions. I once saw a large engagement where there was confusion among teams about priorities and targets. The lead architect created a single, large diagram that showed how the systems will be affected over time, and asked everyone to put it up at their desks. Just one diagram reduced confusion and helped everyone be more productive.
There is also a need for tactical diagrams at the system level. Teams need to see how the higher-level changes are going to affect the system they are working with. Once all team members see the big picture, and also how their systems will change, there will be greater velocity.
Documenting systems that will be retired can be useful
In most modernization engagements, there are systems that are identified for replacement. However, this does not mean documentation for these systems is a waste of resources. As long as a system is in operation, and will undergo changes or new stress, it is important to understand it.
An example would be that a new component will be put between two to-be-retired components that were previously closely tied together. The interface between those two systems, which may have been previously undocumented and not ready for these changes, now needs to be understood completely so that there is less risk in separating them.
Also, old systems may be put under new stresses, such as different types of traffic, or more traffic per instance, or a new security model. In these cases, it is important to document the areas affected by the changes.
Note that simply referencing a large set of existing documentation for a to-be-retired system may not be useful. It can be large, overwhelming, and have nothing to do with the changes required by a modernization effort. It is better to create a new document that has duplication and some links that will make it more clear how to accomplish the changes.
Modernization documentation must be tactical, informal, iterative, and conversational
Many organizations make documentation too formal, or have a small set of gatekeepers create the text and diagrams. This documentation can be uninspiring, or leave developers confused. A modernization effort needs energy, velocity, and adaptation.
Some diagrams and documentation must be written by architects to keep all team members on the same page, but most diagrams should be done quickly, and be less formal. They must be targeted specifically at the people who will use them, and be in a conversational style that will be immediately understood. Content must aid the modernization effort and not be cluttered with details that aren’t needed. Formatting should be minimal. This is not the time for charts, or trying to get bullets to line up, or getting frustrated with the “table of contents”! So much time can be wasted struggling with tools. Especially for documentation that will not be used for very long, something much faster and more iterative is required.
Wiki pages that anyone can edit are far better than alternatives. Get out of diagramming tools and start taking pictures of whiteboards on your phone and upload it to your wiki. I have sometimes used a picture of a whiteboard in a document and team members think it is amusing, as they have never seen that. (A generic example is at the top of this post.) But these diagrams, especially if you use colors and fine-point markers to allow detailed work, can be illuminating. I once showed one to a team member who only had a partial understanding of an old system. He saw it and laughed at the colors. But then said, “So that is how it works!” He then said, “It’s only a whiteboard, but I think we should use that in the documentation.” Only a whiteboard? Any diagram that helps team members understand a system is always welcome. Usefulness is the goal, not how nice a graphic designer could make it look. And anyone can quickly draw on a whiteboard and take a picture of it. Also, if you learn something new, draw it again with the new knowledge. I often go through several versions during a phase of a modernization project.
Conclusion
A modernization project needs lots of new documentation to keep everyone working productively and with good velocity. It will not be the same as the documentation that is there for existing systems. But if it can be done quickly, iteratively, and with purpose, the effort is far more likely to achieve its goals.