Generating an opportunity backlog and discovering users' sentiments for MetaMask and Trust Wallet apps
Embedded Files
Role
UX Researcher
Team
UX Researcher
Software engineer
Tools
Excel, Google Doc & Sheets, Orange Data Mining, Figma, VSC, Microsoft Teams
Context
Blockchain wallet usage hit 68 million in February 2021 and mobile wallets are second in popularity. Based on Google Play Store downloads, MetaMask and Trust Wallet are two popular decentralised crypto wallets. Based on Google Play Store, Trust Wallet had over 10 million downloads and 1.11 million reviews by August 2022, and MetaMask had over 10 million downloads and 117K reviews with over 30 million active monthly users.
Opportunity
The MetaMask and Trust Wallet apps' reviews on Google Play Store showed numerous repeated negative user comments, highlighting problems and users' needs.
Result
Generated the opportunity backlog—a list of opportunities (list of the most frequent user problems, their sizes and causes) to enable product trios to prioritise the opportunities that drive most the business value.
Discovered users' sentiments for better future product planning.
Identify opportunity space
Identify opportunity space
After reviewing user feedback on the Google Play Store for Trust Wallet and MetaMask apps, I found a significant number of negative comments, underlining various problems and users' needs. My findings showed that there were a lot of opportunities that if addressed would drive business values.
Examples of negative user feedback for MetaMask and Trust Wallet
Customer feedback provides businesses with opportunities to act upon. Research showed customer feedback is a decisive factor in choosing which app is worth downloading, as...
Data showing importance of addressing negative user reviews
Leveraging this information, product trios can plan a better roadmap, have an impactful opportunity backlog, address users' problems, and drive business outcomes.
Define research strategy & outcomes
Define research strategy & outcomes
🏆At the end of the research, my goal is to see ...
Whether the result of user feedback analysis would reflect the most important problems users faced in everyday use of MetaMask and Trust Wallet.
Whether the result of user feedback content analysis would uncover the underlying reason behind those problems?
The UX research process
The UX research process
The research process involves
Collecting user reviews
Discovering users' emotions
Identifying frequent user problems
Discovering the root causes of problems
Research outcomes
Collect user feedback
Collect user feedback
We collected MetaMask and Trust Wallet’s user feedback from the Google Play Store with the help of the software engineer using the Google-Play-Scraper library. The result was two Excel sheets of low-rating reviews (scores 1 and 2) for two applications.
MetaMask and Trust Wallet user reviews
Challenge
Challenge
Reviewing and analysing this large number of comments one by one to find the main problems and their reasons was a challenge. I chose to use an AI tool called "Orange Data Mining" to speed up the research process.
The AI tool leveraged me to...
Remove redundant terms from user reviews
Discover the user's emotions
Find the most repeatedly used words
Identify the main problems in negative feedback
Discover the root causes of problems
Orange Data Mining tool environment and documentation
Discover users' emotions
Discover users' emotions
First, I removed redundant terms such as stop words, URLs, special characters, tags, and symbols from reviews to increase result accuracy. The Orange Data Mining tool allowed me to preprocess Excel data sheets.
Next, to determine users' feelings about MetaMask and Trust Wallet, I used the Sentiment Analysis widget of the Orange Data Mining tool. Finally, using the Distributions widget, I mapped the emotion results to bar charts.
The chart represents the frequency of sentiments across scores
ranging from -100 (most negative) to 100 (most positive)
MetaMask
52.45% of reviews rated above zero and 47.55% scored below zero.
Trust Wallet
29.87% of low-score comments scored below zero and 70.13% scored over 0.
Analysing users' feedback showed they were mostly complaining about problems but scored neutral in sentiment analysis.
⚠️ Reasons why negative comments scored neutral
When users are complaining, they frequently use polite or positive words that may outweigh the negativity in the comment.
Sarcasm or irony in comments may sometimes be misinterpreted by sentiment analysis tools as positive and outweigh the negative.
Some comments have both good and bad emotions, which could make the end scores neutral.
That's why it's important to look at user feedback in more depth; this shows that sentiment results should never be read on their own.
Identify main problems
Identify main problems
I applied the Word Cloud widget to user feedback and identified the most repeated words. The word frequency was indicated by the Word Cloud's size. Having the most frequently used words, I defined topics that the main problems were related to.
Main problems' topics
Discover the root causes of main problems
Discover the root causes of main problems
I analysed comments that contained the most frequent words to find out the underlying reasons for every main problem users experienced.
Part of coded user reviews
Main problems' root causes
5 research outcomes
5 research outcomes
Based on the research findings
MetaMask
MetaMask’s users mostly complained about the recurrence of issues including invalid password messages, hacks, crashes, slow performance, and failed transactions, as well as inefficiency in UI elements functionalities, such as QR code, backspace key, keyboard, buttons, and more.
They mainly worried about account loss and login issues due to invalid password/SRP and hang/crash/lag in password/SRP verification.
Trust Wallet
Trust Wallet’s users needed fixing update bugs that led to failed transactions, login issues, slow performance, poor security, issues in DApps connection, and money withdrawal/bank account transfer issues.
They cared about receiving support to resolve their problem, but the support service was disappointing.
They were highly worried about login and access account issues, delays in reflecting tokens/NFTs in the account, and no balance update after transactions.
Research outcomes help the product trio members take actionable insights to resolve the most critical user problems, that drive business values.
For example:
Addressing account access problems can lower the risk of asset loss, promoting trust
Giving useful information such as the duration of token/asset reflection in wallets promotes transparency
Providing efficient customer service and tutorials strengthens trust
In due course, these enhancements can drive business values such as user satisfaction and retention.
Learnings and next steps
Learnings and next steps
💡 What I learned
The value of AI tools in user research, particularly with large volumes of data, since they improve accuracy and efficiency, and are time and cost-savings.
🎯 My next step
In the continuation of the research, I intend to:
First
Interview users of Trust Wallet and MetaMask to learn about context and get their first-hand detailed feedback on the problems, therefore validating the findings.
Second
Ideate solutions to address the first critical problem and rank solutions using the impact/effort matrix in a team.
Page updated
Google Sites
Report abuse