How Do You Know Your Developer Documentation is Successful? — Metrics!
How to gage the success of your developer documentation with good metrics
When it comes to public facing developer documentation, there are many ways to measure its success. But not all metrics are created equal. By the end of this article, you should have a good idea of what metrics are useful and what metrics to avoid.
Pitfalls to Avoid When Choosing Documentation Metrics
Before we dive into the details of great documentation metrics, let’s get some common metrics mistakes out of the way. In general, you should avoid focusing on the following:
- Word count — similar to coding, more written does not necessarily mean that you’ve created something of higher quality. I’d argue that most people want to learn something as quickly as possible so the more concise you can make your content, the better.
- Number of pages — if you’re going to use this, you’ll have to ask yourself what counts as a page? And once you do that, you end up assigning a certain number of words to a page, and you’re back at word count again.
If you are looking for productivity metrics, you’re veering into evaluating the success of your writer vs. your documentation. We’ll go over ways to measure productivity of writers in a future article. If you want to determine if a document is high quality, we’ll cover some good options here.
Easy Documentation Metrics You Should Always Have
The easiest way to create successful documentation is to tie it to company goals. If your company is building a new feature or product, then plan to create documents that describe this. You can track success internally with a series of questions that act as your metrics. The questions to use are:
- Did you create content that is aligned with company goals?
Example: You’re releasing a new API, so you wrote the reference documentation describing the endpoint and parameters, and a tutorial with two code samples demonstrating a basic implementation.
- Is the content complete?
Example: This is actually a difficult question to answer. It will be based on what you decide is required for a complete set of technical documentation. We’ll cover more details on content completeness in a future article. For now, define it as creating all the documents you agreed to create in a set amount of time.
- Is the content technically accurate?
Example: You correctly describe all the parameters for the new API you’re releasing.
- Is the content free of typos?
Example: No misspellings, grammar mistakes, etc.
- Does the content maintain a consistent, repeatable use of style?
Example: You decide to use the Chicago Manual of Style for your writing, and check your work to make sure it matches standards. You document anything you do differently from what’s in the manual.
- Are images clear? Do they explain the technical concepts clearly and correctly?
Example: The image is appropriately sized in the documentation, crisp, easy-to-read, and typo free. It correctly explains the technical concept it illustrates.
- Was the documentation produced in a timely fashion?
Example: The documentation was released the same day as the new product, meaning everything was available to users at the same time.
- Did stakeholders who requested the documentation say that it met their expectations?
Example: The product manager releasing the new API requested API reference material and agreed that the documentation you delivered was good.
If you don’t have any metrics tools or processes in place for working with other teams or customers to collect data, these are great, affordable metrics that will always deliver value.
Public Facing Measures of Successful Documentation
While internal metrics for measuring the success of documentation are helpful, you also want to find out what your customers think of your content. For developer focused documentation, you have two general categories to consider:
- Developer focused blogs, tools, and forum content
- Technical documentation (API reference, tutorials, code samples)
Metrics for Developer Blogs, Tools, and Forum Content
If you’re running a developer focused blog, using a tool like GitHub or Glitch, or posting in forums where developers hang out, you’re in luck. The tools for measuring success are somewhat built in. You can measure:
- Engagement — how many people look at your work on GitHub or Glitch? If you have a forum, do people write comments or submit their own questions?
- Number of followers — in places like GitHub, Stack Overflow, or a company forum, you can measure how many followers you have and when they start following you.
- Number of views — how many people read your code sample or blog post?
- Conversions — for a blog post you can include a call-to-action to get people to sign up for a free trial, download a particular asset, or read more content on your site.
It’s important to tie these metrics to company goals. For example, if you want more users in your developer forum, the goal could be because you want developers to start helping one another troubleshoot. This would cut down on time your team needs to spend answering questions. Or another example, you might want a lot of views on a blog post because more views provides more opportunities to get a conversion to a trial account. Always tie your metrics to an approved department or company goal.
Metrics for Technical Documentation
Technical documentation metrics can help you find out what content is performing well and what isn’t. While internal measures of success are useful, it’s important to offer ways for customers to provide feedback. Here are a few ways you can collect data about documentation from customers:
- Surveys — Ask customers what they like and don’t like about your content, what was easy or difficult to find, and what they want to see more of.
- Ratings — Offer a way for people to rate each page of content on a scale of 1–5. You can see what pages work well, and what pages could use improvement.
- Page views — Find out how often different pages are viewed and when page views change. If you hold a hackathon, do you get more people viewing your docs? What did they look at most? Page views can help you determine what kind of content is most useful. When using this metric, be aware that results are partially dependent on how popular your product is.
- Bounce rate for landing pages — How often do people reach a landing page in your technical documentation and leave when you want them to click through to a topic? Improving bounce rate can improve the SEO rank of your site.
- SEO rank — How well does your documentation site do in SEO rankings? Do your pages turn up in results that turn up competitor results? There are a couple of caveats for this metric- if you use an out-of-the-box tool to create technical documentation, it may not be set up to do SEO competitively. You should also take into account whether other companies are paying for good search results before using this metric.
- Customer Support use cases — You can check if Customer Support uses documentation to answer questions. If they provide links to content, you can implement a process where Customer Support uses trackable links to share content. Then you can see how often documentation is used for that purpose.
There are many other ways to collect metrics about your content. These are a few ideas to get you started. Remember that for any metric you use, it will become something you think about regularly. Make sure you choose metrics that add the most value for your company and department goals.
Developer Advocate and Comedian