Our Design Researcher Fiona Yasmine shares her top 6 lesson-learned about conducting seamless usability testing session with rural users.
Following our discussion on Jiva’s Remote Usability Testing (UT) Day in the previous blog post, let’s now dive into why UT is critical to do before releasing any app, and what a researcher must do to ensure they conduct seamless UT sessions.
It’s not uncommon for people in product development to overlook small, perhaps insignificant issues they initially deem unproblematic. But, such issues could turn out to be a problem when users start to interact with the product. This is why UT is important! UT helps uncover these problems early, allowing for quick iterations and improvements before the product launch.
Having spent hundreds of hours doing interviews, I’d like to share some learnings from conducting remote UT sessions with my fellow researchers.
Here are my top 6 things to keep in mind when implementing remote UT – from research preparation to execution.
When you notice your participant starts to become uncooperative and shows signs of fatigue, it is best to not push the interview to continue.
I noticed from my experiences that during the interview stage, participants often start off excited but lose interest and stop giving effort once they realise they have to complete several tasks for the usability testing, which could take time.
When this happens and you decide to push through, it could be challenging for your participants and you as the researcher. Participants could feel tired and frustrated, and you would not obtain any valuable insights when in such situations.
This is common especially with rural users, who often get easily tired or are not as familiar with using gadgets.
Is interviewing participants as simple as asking them questions? Think again. Trust me, it’s not that easy.
As a researcher, you often expect participants to give their answers exactly like you expect them to. But this is exactly what you need to avoid.
When doing UT, you want to discover overlooked issues, so the one thing you need to avoid is leading questions. You don’t want participants to align their answers with your expectations - this will lead to biased results.
Example:
Leading question → Do you think this new feature makes the app easier?
Open-ended question → How would you describe your experience navigating through the new feature on the app?
When trying out a product for the first time, users need time to understand it. Make sure they take their time before responding - it will allow them to think about your questions and their interactions with the app. Your participant’s initial responses during the UT may not fully convey their thoughts or experiences. So, ask follow-up questions to gain deeper insights and understand your participant’s perspectives better.
Without defined success metrics, there is a chance that your research will end up evaluating aspects beyond the intended scope. There are a lot of components and potential user scenarios in the app, so doing UT without clear metrics can lead to confusion about the evaluation criteria and outcomes.
It's important to stay open-minded especially when you receive negative feedback. It might be too much for you to handle at first, but it’s valuable. Negative feedback often pinpoints areas for improvement, enhancing the user experience significantly.
6. Be flexible! Always prepare a backup plan.
In my experience, users often bail out at the last minute. So, to ensure we have enough participants who meet our criteria, Jiva’s Research Coordinators (on-ground research team) and I always keep an additional list of available participants.
Even when we've made a structured timeline, things often unexpectedly change with rural participants due to factors beyond our control. Therefore we need to be always prepared for such things!
Another challenge I frequently encounter is the unstable and unpredictable internet connections in rural areas. To mitigate this issue, I maintain regular communication with the research coordinators. This helps me stay informed about locations with better connectivity, ensuring sessions are not delayed.
–
Fiona is one of Jiva’s youngest team members in the Design Research team. She specialises in remote Usability Testing and has spent hundreds of hours speaking to rural users in search of ways to make our app more accessible and easier to use.
Want to read more specialised tips and work-related content? Follow Jiva on Medium, LinkedIn, and visit our website jiva.ag!
Being the Senior Service Designer, Karan Patel intends to discover how Jiva can deliver the best experiences for its users, be it from Jiva or other people. So many charts and journeys are created and innovated to serve the users. In this case, service blueprinting is the way to go. In this article taken from the latest #WorkWonders episode, Karan talks with Nea Ningtyas & Bintang Lestada, our Sr. Content Specialists about his experiences as Jiva’s Senior Service Designer in Indonesia and the process of his job.
As a Lead Product Manager, Nilesh Gaurav knows the in and out of Jiva’s platform. But sometimes you might be wondering, what does Jiva’s product actually do? And how does it work with users who live in rural areas? In this article taken from the latest #WorkWonders episode, Nilesh talks with Nea Ningtyas & Bintang Lestada, our Sr. Content Specialists about his role as Lead Product Manager and the challenges that he faced in his career in Jiva, be it socially or logistically.
.png)
Fathia Ramadina has been in user research for years prior to joining Jiva - but there are things about rural users so distinct that uncovers rich layers of enlightening insights on doing research. In this blog post, she reflects on how she found that after doing many research projects and hundreds of interviews with Sahabat Jiva, a meaningful rural user research goes beyond techniques.
Our Design Researcher Fiona Yasmine shares her top 6 lesson-learned about conducting seamless usability testing session with rural users.
May 22, 2024
Following our discussion on Jiva’s Remote Usability Testing (UT) Day in the previous blog post, let’s now dive into why UT is critical to do before releasing any app, and what a researcher must do to ensure they conduct seamless UT sessions.
It’s not uncommon for people in product development to overlook small, perhaps insignificant issues they initially deem unproblematic. But, such issues could turn out to be a problem when users start to interact with the product. This is why UT is important! UT helps uncover these problems early, allowing for quick iterations and improvements before the product launch.
Having spent hundreds of hours doing interviews, I’d like to share some learnings from conducting remote UT sessions with my fellow researchers.
Here are my top 6 things to keep in mind when implementing remote UT – from research preparation to execution.
When you notice your participant starts to become uncooperative and shows signs of fatigue, it is best to not push the interview to continue.
I noticed from my experiences that during the interview stage, participants often start off excited but lose interest and stop giving effort once they realise they have to complete several tasks for the usability testing, which could take time.
When this happens and you decide to push through, it could be challenging for your participants and you as the researcher. Participants could feel tired and frustrated, and you would not obtain any valuable insights when in such situations.
This is common especially with rural users, who often get easily tired or are not as familiar with using gadgets.
Is interviewing participants as simple as asking them questions? Think again. Trust me, it’s not that easy.
As a researcher, you often expect participants to give their answers exactly like you expect them to. But this is exactly what you need to avoid.
When doing UT, you want to discover overlooked issues, so the one thing you need to avoid is leading questions. You don’t want participants to align their answers with your expectations - this will lead to biased results.
Example:
Leading question → Do you think this new feature makes the app easier?
Open-ended question → How would you describe your experience navigating through the new feature on the app?
When trying out a product for the first time, users need time to understand it. Make sure they take their time before responding - it will allow them to think about your questions and their interactions with the app. Your participant’s initial responses during the UT may not fully convey their thoughts or experiences. So, ask follow-up questions to gain deeper insights and understand your participant’s perspectives better.
Without defined success metrics, there is a chance that your research will end up evaluating aspects beyond the intended scope. There are a lot of components and potential user scenarios in the app, so doing UT without clear metrics can lead to confusion about the evaluation criteria and outcomes.
It's important to stay open-minded especially when you receive negative feedback. It might be too much for you to handle at first, but it’s valuable. Negative feedback often pinpoints areas for improvement, enhancing the user experience significantly.
6. Be flexible! Always prepare a backup plan.
In my experience, users often bail out at the last minute. So, to ensure we have enough participants who meet our criteria, Jiva’s Research Coordinators (on-ground research team) and I always keep an additional list of available participants.
Even when we've made a structured timeline, things often unexpectedly change with rural participants due to factors beyond our control. Therefore we need to be always prepared for such things!
Another challenge I frequently encounter is the unstable and unpredictable internet connections in rural areas. To mitigate this issue, I maintain regular communication with the research coordinators. This helps me stay informed about locations with better connectivity, ensuring sessions are not delayed.
–
Fiona is one of Jiva’s youngest team members in the Design Research team. She specialises in remote Usability Testing and has spent hundreds of hours speaking to rural users in search of ways to make our app more accessible and easier to use.
Want to read more specialised tips and work-related content? Follow Jiva on Medium, LinkedIn, and visit our website jiva.ag!
