Open source software development teams often face challenges around their long-term sustainability. In service of supporting strategic planning and needs identification for these open source tool teams, Internews has developed the Open Source Software Needs Assessment, a web-based toolkit that open source project teams can self-administer.

The toolkit contains instructions that walk teams through the self-assessment process. It consists of 4 parts:

1. Assess – follow a rubric to rate the health of your open source project in 10 key areas
2. Strategize – Strategic Planning & a SWOT Analysis (Strengths, Weaknesses, Opportunities, Threats) to clarify the “big picture” for the project
3. Review – Choose the 5 key areas most important to the project team
4. Discuss – Talk through detailed questions to identify needs, opportunities, and the team vision for each of the 5 key areas

Tool teams who undergo the Open Source Software (OSS) Needs Assessment should walk away with a robust plan to help guide priorities for the project team over approximately the next year.

The output of the OSS Needs Assessment is a clear baseline picture of the health of the tool, the top 5 areas of concern for the tool team, and concrete, specific steps the team can take to achieve their vision and goals.

The plan is made available as a printable document at the end of the self-assessment. This document can then be leveraged to develop grant proposals and a clear rationale supporting the project’s need for various resources.

How to Conduct the Self-Assessment

Maintainers, project leaders, and core developers for the open source project should set aside 1-2 hours to work through the assessment together. This can be done over a virtual video call, for example, where one person shares their screen and fills out the form on behalf of the team.

In this blog post, we’ll discuss each of the 4 parts of the assessment in detail.

---

Assess

The introductory activity for the OSS Needs Assessment is a self-scoring exercise. Teams consider 10 key areas of open source project health and rank themselves from 0-5 for each area. Each Score provides a description of what it means.

For example, under the key health area of “Community,” here are the scores:

• Score 0: Does not meet Score 1 criteria
• Score 1: The project has a homepage that points to all the information required to participate in the community and contribute to the project.
• Score 2: The community welcomes contributions from anyone who acts in good faith and in a respectful manner and adds value to the project. New contributors receive welcoming messages/comments and thanks for their contributions.
• Score 3: Contributions include not only source code, but also documentation, constructive bug reports, constructive discussions, marketing and generally anything that adds value to the project. Code reviews are undertaken with the goal of accepting as many contributions as possible; comments on code contributions are constructive and show appreciation.
• Score 4: The community strives to be meritocratic and over time aims to give more rights and responsibilities to contributors who add value to the project. There is a defined and consistent process for acknowledging contributions of all kinds. The project attempts to make code reviews efficient and reduce the time between starting review and accepting the contribution.
• Score 5: The way in which contributors can be granted more rights such as commit access or decision power is clearly documented and is the same for all contributors. Most code contributions of reasonable quality are accepted.

Each score is progressive. For example, a project that scores itself “2” also has the features described in “1.” A project at a “5” also has all of the features described in 1-4. If an open source project has some capabilities in a higher score but not all of the capabilities in the scores below it, teams can use their best judgment of what best represents their status quo.

Alongside each of the 10 key area scores, there is space to indicate how important that area is to the team: Not Important, Somewhat Important, or Very Important.

There is also a section called “Justification” with a text box underneath; in this space, teams can provide a few sentences of explanation for why they decided to self-score as they did for that particular area. Taking good notes as you go through the process is helpful in the long run by providing a record of why the team ranked itself a certain score in a particular area, and can provide a quick reference for revisiting that area at a later date.

Strategize

his section focuses on posing questions to help the tool team describe their “big picture” goals and challenges. For this section, the tool team should talk through the questions for each section and note down their answers in the associated text box.

For more open-ended questions, try to steer the conversation towards answers that will ultimately prove useful in designing a capacity action plan. (In other words, focus on the practicalities at stake.)

Take good notes in the text boxes provided, so that you’re able to refer back later to understand your answers to the questions and the background context for how you arrived at those answers.

Review

This is a relatively quick section that asks the tool team to prioritize 5 areas that are of greatest importance to the team. The toolkit will auto-include any area that was marked “Somewhat Important” or “Very Important” in the Assess step of the process as a potential choice for the Top 5.

By clicking on the name of each area, a drop-down will appear where you can review the score it was given, its importance, and the justification for the score.

Discuss

Now that the team has chosen the 5 most important areas of focus, this section poses a number of detailed questions about each of those areas. By clicking on the name of the focus area, a drop-down will appear showing the guiding questions for that assessment area.

Use the empty text-box to take detailed notes, so that you’re able to refer back later to understand your answers to the questions and the background context for how you arrived at those answers.

Each drop-down area will also ask the following questions:

• What are the biggest challenges or blockers for the project in this area?
• List any quick or easy fixes the project could make in this area:
• What steps could the project take in the next 6-12 months to improve in this area? What would you like to see the project accomplish?

The preceding guiding questions are designed to provoke discussion that will lead to productive answers to these three questions for each area. In deciding what steps the project could take to improve, consider the biggest challenges your team is facing and whether there is any “low-hanging fruit” where the team could make a quick and easy fix that would have a positive impact.

Finally, as your team discusses each area, consider whether you may want to revise the score you selected for that area as a result of the conversation. To change your score, navigate to the “Assess” tab and update the score (and justification, if needed) for that area. It is in your interest to score each area as accurately as possible, so that you have a clear rationale for support in the areas where you most need help.

Capacity Action Planning

Upon completion, the self-assessment will offer to provide a downloadable pdf of your notes and answers.

Note: The answers and notes you submitted won’t be erased unless you click the “new submission” button or clear the browser cache. So even if you close the tab or the browser window, answers are still saved.

The print-out will be a copy of the web form and your answers to each section. This is your “Capacity Action Plan.” Pay particular attention to your “Strategize” answers and to the 3 key questions for each area in the “Discuss” section. By referencing these, your team can build out a plan to increase the project team’s capacity over the next 6-12 months. You can also use these answers to develop funding proposals and a supporting rationale.

At a later date, once your team has had time to put effort into the areas you identified through the self-assessment, you can take the assessment again (be sure to click “new submission” to clear your prior answers) and re-score the project in each of the 10 areas to see where there have been improvements.

---

Additional information about the Open Source Software Needs Assessment, including licensing, is available here.
Information on the BASICS program, which funded the assessment, is here.