Qualitative research on QuantumByte Developer Documentation
UX evaluation of the QuantumByte developer documentation to understand developer needs and pain points, assessing usability and interaction, gathering contextual feedback, informing future improvements & recommendations
My Role
UX Research . Interview . Solution Mapping . User Journey . Ideation .
Time
Team
40 Days
2 UX Researchers . 1 Design Manager
Industry
Customer Segment
Tech & I.T.
B2B
ABOUT
QuantumByte
QuantumByte is a global technology company known for its software products, including its widely used operating system and productivity tools. It also develops cloud computing services and enterprise solutions, catering to a diverse range of industries.
ABOUT
QuantumByte Developer Documentation
Developer Documentation refers to the official guides, tutorials, and resources provided by QuantumByte for developers who want to build applications or customizations for the QuantumByte communication channel.
EXPLORING
Why is it used?
It's used to help developers understand the APIs, SDKs, and capabilities of Teams, enabling them to use the functionalities of the Teams platform.
EXPLORING
Why was the research required?
After developers submit their applications for approval and receive feedback, many do not return to the platform, leading to a drop-off and them moving to a different platform to develop, resulting in a loss of business for QuantumByte.
ISSUE FACED
“How to improve documentation so there are fewer errors and feedback necessary for developers thus making the process streamlined and able to retain developers.”
BUSINESS GOALS
Reduce drop-off rates
Make onboarding simple, personalize content, keep the experience smooth, gather feedback, and use analytics to spot and fix where users are dropping off.
Increase time efficiency
Automate repetitive tasks, make navigation and search easy, put key features front and center, speed up load times, and offer helpful tips and shortcuts to boost efficiency.
​
Reduce dependency on external Documents
Centralize info within the platform, create self-contained workflows, integrate a searchable knowledge base, offer real-time help with tooltips, and keep content up-to-date to reduce reliance on external documents.
OUR PROCESS
Let's understand the issues
Heuristic Analysis & Competitve analysis
We conducted a detailed heuristic analysis of the QuantumByte documentation, examining it against established usability principles to identify any issues.
This process helped us understand the overall structure of the documentation and allowed us to scan through various sections efficiently.
As a result, we uncovered 53 usability issues, which were categorized based on Nielsen’s heuristics. After discussing these findings with the client, we proceeded to the next stage of the project.
​
​In parallel, we also performed a competitor analysis by researching 6 companies that directly compete with QuantumByte. We analyzed their documentation sections across 5 key features, evaluating how they showcase their content.
This competitive analysis provided us with valuable insights into how others in the industry present their documentation, helping us benchmark QuantumByte against its competitors and identify areas for improvement.
Companies were analysed with 5 main features
Visibility of Navigation structure and content filter
Tutorials and Learning Resources
Capture user’s feedback on documentation and product
Support Community forum
Validation Guideline
Confusing Navigation
Users struggled with navigation, unsure of what actions were possible and how to find expected results, leading to frustration and confusion.
Minor Issues Needing Validation
We identified several smaller usability issues that may impact user experience, but they need validation through user testing to confirm if they’re common problems.
Inconsistencies & Lack of Feedback
Pages had inconsistencies, missing feedback, and poor affordance, making it harder for users to interact smoothly and understand what actions were successful.
Strong Features, Weak Support
While the features are competitive, users still face difficulty finding solutions, compounded by the lack of strong community support for quick, tailored help.
What does users have to say?
Understanding the issue from users by survey
We began our research by designing a survey, starting with a brainstorming session to develop the questionnaire alongside our team.
The initial findings from our heuristic evaluation, combined with the documents provided by stakeholders, gave us a clear understanding of the existing usability issues. These insights were instrumental in shaping the questions we included in the survey.
​
The primary goal of the survey was twofold: first, to validate the issues we had already uncovered during our earlier analysis, and second, to dive deeper into the users’ experiences to identify any underlying or more complex challenges they might be facing.
By using this approach, we aimed to gain a more comprehensive view of user pain points, ensuring that our solutions would effectively address both the obvious and subtle usability issues.
Overall survey said users didn't find much issue with documentation and they found it useful.
Few survey result
“If users find it useful, then why are they dropping off? That was the big question we were trying to figure out ourselves.”
​Given the limited insights we gathered from the surveys, we decided to shift our focus to conducting in-depth interviews.
We believed this approach would provide a deeper and more nuanced understanding of the issues users were facing.
The interviewees were carefully selected based on their seniority and the specific technologies they worked with. By choosing participants from different experience levels, we aimed to capture a wide range of perspectives, from simpler issues faced by less experienced users to more complex challenges encountered by advanced users.
This approach helped ensure we covered the full spectrum of usability concerns, providing a clearer and more comprehensive view of the problems.
Understanding the users response
Survey Results vs. Reality
While 80% of users said they were happy with the documentation, the dropoff raised questions about whether the surveys reflected true user satisfaction.
Questioning the Value of Surveys
We started to wonder if users were just completing the surveys for the sake of it, as the data didn't give us much to work with or explain the real issues.
Interviews Uncovered Deeper Issues
In contrast, the interviews revealed much more. Users were open to sharing how they actually use the documentation and the challenges they face daily.
Clear Insights for Improvement
Through interviews, we got a better understanding of user flows, why they’re dropping off, and what helps them use the documentation more effectively.
Ideating to find solutions
Persona & UserJourney
Create detailed, fictional profiles representing different types of developers who use the documentation, ensuring it meets the diverse needs of the audience. Build empathy for their experiences, challenges, and goals, fostering a user-centric mindset within the team.
Map out the entire journey developers go through when using the documentation, gaining a clear understanding of their actions, interactions, and pain points along the way.
Images are blurred for confidentiality. Contact to get a complete walkthrough
Creating a User Persona
From the interviews, we developed a user persona that gave us a clear vision of the ideal user and helped us stay focused on their needs throughout the process.
Mapping the User Journey
We mapped out the user journey, highlighting key moments and emotions, and identified opportunities to improve their overall experience.
Pinpointing Major Pain Points
We identified the main pain points, as well as what’s working well, giving us a clear view of where users struggle and where we can intervene.
Highest Drop-off During Build Phase
The journey map revealed that the most significant drop-off happens during the build phase, allowing us to prioritize impactful solutions there.
SOLUTION
Final Findings
Information Architecture Challenges
The current information architecture and content organization do not effectively cater to developers' natural mental models and task-oriented workflows.
Suboptimal Search Functionality
The documentation's internal search engine produces inconsistent and sometimes irrelevant results, forcing developers to rely on manual browsing and navigation to find the desired information.
Lack of Contextual Guidance
The documentation lacks sufficient contextual cues, examples, and troubleshooting guidance, leaving developers feeling lost and unable to apply the concepts to their specific use cases.
Inconsistent Content Quality
While some sections of the documentation are comprehensive and well-written, others suffer from outdated information, incomplete explanations, and a lack of visual aids to support complex technical concepts.
