Developer documentation: the necessary evil?

Corresponding service: Developer documentation 

Anna M. is a software engineer. She has just started working for a company that develops custom add-ons for an out-of-the-box financial software solution. For developing the add-ons, Anna is supposed to use the API of the software. Little does Anna know about the API's structure, its interfaces, and its functionality. She looks up the documentation of the API and realizes that she's got a problem.

The API is poorly documented. All that Anna finds is a short overview of the interfaces and some sparse code comments in the classes.

Not only end users need well-documented software. Software engineers too require documentation to accomplish their tasks. When a software system is developed, it usually gets documented. Yet the quality of documentation varies from product to product. Sometimes, the documentation is incomplete, as Anna has just found out. Sometimes, it does not grow with the developed system. And sometimes, it is simply cut out because of pressing deadlines.

This article summarizes our experience and know-how from various software documentation projects. It presents an opinion on how to create good developer documentation. We would like to thank Stefan Zörner for reviewing this article and for his valuable input.

The incomplete puzzle

Developer documentation comes in different forms. Imagine these forms as the pieces of a puzzle. If the pieces fit together, we get a complete picture of the software. In most software development projects, we find the following puzzle pieces:

• UML diagrams
• Technical concepts
• Descriptions of use cases and scenarios
• Code comments
• Requirements
• Tasks and issues in the bug tracker

Sometimes, the pieces fit, sometimes they don't. They exist in multiple file formats and are spread across several IT systems, for which you need dedicated access rights. In many cases, their contents overlap or are contradictory. They can also get out of synch when they are not updated with the development of the software. The result: The documentation becomes useless.

In addition to the pieces that are spread across IT systems, some may be missing altogether:

• An architectural overview
• Development conditions, for example, regarding integration of and connections to external systems, budget constraints or time schedules.
• Important design decisions
• Technical and business processes and interaction between components

Agile development = no documentation?

Many software engineering teams successfully employ agile methods. Some, however, take the agile principle "working software over comprehensive documentation" too literally (https://agilemanifesto.org/). They reduce documentation even more. That is a misunderstanding. The agile principle does not mean that there should be no documentation. It means documentation should be reduced to what is important and necessary. For example, avoid long specifications before you start developing. But here is what it often looks like in practice:

• Software is developed but not documented.
• Documentation is not part of the iteration and the definition of done.
• Code is documented but concepts are not.

Consequences of poor documentation

All the problems we have mentioned so far lead to less documentation. And the little documentation that is produced, often is unfit for enhancing and maintaining the software. What does that mean exactly?

• There is no documentation that experienced team members can rely on when training new staff.
• Undocumented knowledge about system architecture or implementation details gets lost if an important member leaves the team.
• Enhancing the system by new functions or maintaining it becomes more costly. Each modification requires the software engineering team to research how components interact or how interfaces work.
• Impacts of change requests are hard to assess as it is not immediately obvious which components are affected.

Writing good developer documentation in four steps

What is required to document a software system both comprehensively and with as little effort as possible? We recommend to follow these four steps:

1. Define the target groups for your documentation.
2. Define documentation types and methods.
3. Form an interdisciplinary documentation team.
4. Write separately, publish jointly.

Step 1: define the target groups for your documentation

Before we start writing, we need to study the readers of our documentation. If we know the readers and their demands, we can start working out contents and types of our documentation. This way, we can limit documentation to an amount that is actually required.

In this article, we distinguish between two target groups: (1) software engineers and software architects and (2) software testers.

Software engineers and software architects

Software engineers and software architects who use a development framework or maintain an existing software system require the following information:

• System requirements and system functionalities
• Use cases (who uses the system when, under which conditions, and why)
• Software architecture (systems, components, layers)
• Technical concepts, such as transaction security, scalability, protection against unauthorized access, license management
• Documentation of interfaces
• Reference documentation of classes and methods
• Design decisions (why is the system the way it is), context and conditions for development

Software testers

Those who test a software system and its extensions and modifications, require similar information as software engineers and software architects. Additionally, they need:

• Documentation of test cases
• Documentation about the architecture of the test system

Step 2: define documentation types and methods

Software engineers require different types and levels of information – impossible to pack all of it into one document. We need to split the content into several documents. Before we do that, we need to answer a few more questions:

• Who reads what (personas)?
• What is read and when (application context)?
• How frequently will the documentation be changed?

Who reads what (personas)?

We distinguish between three learning and reading types, so-called personas:

Software engineers who start coding immediately. They read code comments and use sample code or existing code as template.

Software engineers who first read the basics. They read the architecture documentation and technical concepts.

Software engineers who read on demand. They study a specific use case (how do I solve this problem) and search for topics to find use case descriptions and sample code.

Anna belongs to the second type. She studies the architecture documentation and the technical concepts before she starts working with the framework. Later, she will read use cases or code comments, if necessary.

What is read and when (application context)

We distinguish between information that should be available in the respective context and information that the user can read outside the development context and sequentially. Examples are:

• Reference documentation for methods and classes (code comments). They must be available directly and in the context of the IDE.
• Technical concepts and software architecture documentation. They can be read outside the development environment and sequentially, like a document.

How often will the documentation be changed (change rate)

The change rate of documentation depends on the documentation type:

• Architectural descriptions and technical concepts are mainly created while the software is being developed. They rarely change in late stages of software development or during maintenance.
• Use cases, scenarios, and sample code are modified more frequently. They have to be designed to be easily extendable in case new functions and use cases are added.
• Reference documentation of classes and methods (unless it refers to published API interfaces) change most often and are best written as code comments.

Types of developer documentation

Depending on the target group, the application context, and the change rate, developer documentation can be provided as different types:

• Architecture documentation
• Developer guides
• Reference documentation for classes and methods

Not all of these types are always required. Which developer documentation type you create depends on your requirements and development processes. Writing and maintaining documentation is expensive. You want to make sure that you create the type and amount of documentation that your target group really needs.

Architecture documentation

This type of documentation describes the system structure. It summarizes design decisions that were made when the system and its basic structure is developed. This type rarely changes and can be read outside the development environment. It may include the following contents:

• Overview of the software
• Definition of important terms (glossary)
• Purpose of the software
• Development context and conditions
• Quality features
• Stakeholders and personas
• Risks
• Design decisions
• General technical concepts
• Different views of the software architecture, such as subsystems and components and sequence diagrams
• Information about deployment and operation

The arch42 template (https://arc42.de/) and Stefan Zöllner's book "Softwarearchitekturen dokumentieren und kommunizieren" (in German) provide a useful structure for architecture documentation. See also http://swadok.de/.

Developer guides and API guides

Developer and API guides present use cases, troubleshooting, and general tasks. They enable software engineers to develop modules for a software system. They should be designed to be easily extendable in case the system changes. They can consist of:

• Descriptions of how to implement specific use cases or business processes using the existing interfaces. They are useful for software engineers, for example, who can name the problem, but do not know which interface or class to use for solving the problem.
• Descriptions of how problems are solved.
• Basic information about how to extend and modify the system.
• Descriptions of general tasks, such as event handling, logging, localization, and access management.
• Sample programs, sample code, and links to the reference documentation.

Reference documentation of classes and methods

This documentation type describes the interfaces, classes, and methods of a software. It is easily changeable and can be integrated in the IDE. It usually comprises:

• Detailed documentation of interfaces, classes, and methods as code comments.
• References to the developer guide and the architecture documentation.

All these documentation types interrelate. The contents should be published in different documents, but must be synchronized and fit together. This only works out if all documentation types are considered and treated as a whole, not as individual documents.

Document types, unite

Chunks of information that cover the same subject but are spread across several types of the developer documentation should be linked. This way, the reader is able to find all the information that relates to the same subject. Ideally, readers can access all types of developer documentation on one platform offering full-text and semantic search functions.

Step 3: form an cross-functional documentation team

A software system should not be documented by software engineers alone. Using only technical writers is not a valid alternative either, because writers usually do not know all the implementation details and design decisions. Many of them do not have a background in computer science.

Good developer documentation is created in an interdisciplinary team that benefits from the skills of its members:

Product managers are familiar with the requirements on the system, the development context and constraints. They know what the system is used for. Often, they define software requirements and know the end-users of the system best.

Software developers and software architects know why certain design decisions were made and are familiar with the system architecture. They know what information other software engineers require if they want to learn about the system or if they need to maintain and enhance it. They know the programming language, the development environment, the build processes, and the deployment.

Technical writers can structure documents, analyze target groups, ask developers the right questions and take a cross-component view. They can use typical user tasks described in the end-user documentation as a basis for developer guides. They also have the required writing and didactic skills.

The skills in an interdisciplinary team can be bundled to create documentation that is useful and suitable for enhancing and maintaining your software. Don't forget: The maintenance phase of a software is usually much longer than its development phase.

Step 4: write separately, publish jointly, with the proper tool

What happens if these types of developer documentation are written with different tools? And what if user documentation and developer documentation are created in different departments, if there is no interdisciplinary documentation team? The documentation drifts apart. Or, to pick up the example from above again: The pieces of the puzzle won't fit.

Let's take a look at the different tools that are currently used for writing documentation.

Tools for developer documentation

• UML tools
• IDE for code comments
• Wikis

Tools for end-user documentation

• XML editors or content management systems for XML-based documentation
• Desktop publishing software, for example, FrameMaker or InDesign
• Wikis

The common ground

To document software in a team, your tools and your documentation should meet the following requirements:

• Version management for documentation and its modules
• Modular/topic-based structure
• Documentation is accessible online
• Functions for collaborative writing and access management
• Integration of code comments and UML diagrams

XML-based documentation solutions meet most of these requirements. They offer version control and topic-based authoring. And with XSLT you can generate online formats like HTML or WebHelp.

Wikis too are used for collaborative writing. They are perfectly suited for interdisciplinary work.

However, XML-based documentation solutions and wikis do not fulfill all requirements equally well. Sometimes, the solutions have to be customized. Integrating UML diagrams, combining architecture documentation and code comments as well as version management can be challenging. These issues must often be solved according to the specific individual situation.

Ideally, all types of documentation should be published on a common platform. That can be a website or a wiki. This way you make sure that your readers find all the information they need in one place.

Conclusion

Let's summarize. What makes good developer documentation?

• It is consistent and comprehensive and does not contain redundant or outdated content.
• It is modular, topic-oriented, links related topics, and offers full-text or semantic search.
• It includes all the required types of documentation, such as architecture documentation, developer guide, and code comments.
• It is suited for the target groups.
• It considers the needs of the end users.
• An interdisciplinary documentation team creates it.
• It is created with tools that are suited for separate authoring and joined publishing.

A software engineer like Anna will judge the quality of an API or of a framework also by its documentation. If she requires more time because of inadequate documentation, she will not accomplish her tasks in time. Perhaps she will make mistakes that compromise the product. A well-documented API or framework shortens the learning process and increases quality.

References

• Knowledge base article by Ulrike Parson: Documentation for Software Engineers
• Standard structure for architecture documentation: https://arc42.de/
• What's an API writer – Wikipedia

Sources

• Knowledge base article by Ulrike Parson: Documentation for Software Engineers
• https://agilemanifesto.org/
• https://arc42.de/
• swadok.de
• Wikipedia