NewsNow: User Research Initiative
During my role at NewsNow, I initiated and led a long-running user research initiative. Design methodology at NewsNow had traditionally been informed by engineering, along with A/B testing paradigms and voluntary user feedback. Existing understanding of the makeup of NewsNow’s userbase and their motivations had been inferred from A/B test results for features over a span of many years, and while this served as a valuable starting point, it was far from concrete.
There was, however, a large appetite within the team, including senior management, editorial and development team members, to better understand our users. Given this appetite, and a product roadmap brimming with large-scale design and development projects, it seemed right to make strides towards uncovering our userbase and giving our users a seat at the table. I put together a plan of action and advocated for human-centred design principles as part of a more efficient and effective product design and development process.
The initiative consisted of four components: “Identify”, “analyse”, “empathise", and “realise”. Work on these components could be undertaken ad-hoc, spanning multiple projects. Our initial work on user research, as part of NewsNow’s redesign, helped to form the basis for this structure, and the subsequent steps were taken over the course of a project to design and develop a new core product.
This initial phase was primarily a data-gathering effort. We launched a site-wide demographic survey to a randomly selected subset of users, asking broad questions about people’s interests, usage habits, age, gender, location, and general news reading habits, with the goal of identifying both the general demographic makeup of our existing userbase, as well as our place within their news diet. The survey was implemented in Hotjar, and thus we were able to gather additional metadata for each submission, such as from which pages users submitted the survey and information about their device. In addition, Hotjar enabled us to collect responses from users who did not complete the full survey, which proved especially useful in looking for selection/response bias.
In addition to the survey, I also used Hotjar to generate a wide array of heatmaps, to look more specifically at how users on different topic categories might behave differently to one another.
Finally, I was able to gather quantitative data from Google Analytics to further understand device/browser/OS distribution, usage habits, and behaviour.
The demographic survey generated approximately 6000 responses, which I exported as a CSV and processed in Google Sheets. I spent a long time sifting through all the responses, slicing the data in various ways and looking for patterns that might help me to uncover segments within the userbase. As part of this process, I weighted the responses by geography, and device category, using the Google Analytics data from the period of time in which the survey was active as a source of truth.
Upon applying weights to the data and segmenting by country, I discovered a number of very clear segments within our userbase. Focusing on what were, by far, the three largest segments; UK, US and Nigeria, I was able to identify vastly different distributions of age, topical interests, and device usage between the three groups. What was particularly interesting about this was that the demographics in each segment appeared somewhat narrow; we were appealing to specific age-ranges and categories of interest in each country that were distinct from one another, and that appeared unlikely to be representative of online news readership demographics as a whole. To validate this, I checked some publicly available data regarding online news readership from Ofcom, Pew Research, Statista, and SimilarWeb. By checking our own data against these broader demographic insights, I was not only able to confirm the boundaries of our users’ segmentation but was also able to identify prominent demographics in the wider population, both in the UK and the US, that were underrepresented in the NewsNow userbase.
Based on my analysis of the survey results, analytics data, and third-party data, I was able to craft a handful of personas, representative of our core users. In addition, based on the demographics I had found to be under-represented, I created additional personas, representative of our target demographics, with the goal in mind of better serving these demographics and subsequently growing and diversifying the NewsNow userbase. I created a full writeup of my findings, including my methodology, data and charts, sources, and a summary of the results, which I presented to the wider team.
The next step was to engage with real people, in-person, and begin to build a rapport with the people behind our personas. Through our earlier demographic survey, and through dedicated research participant recruitment services such as UserInterviews.com, we were able to work with users that were representative of all our user personas, from a range of backgrounds; from the UK, US and Nigeria.
The methodology for the interviews was based on a semi-structured interview format, which offered a relaxed atmosphere and an opportunity to build a rapport with participants, whilst still ensuring that all our talking points were covered. For the purposes of this round of interviews we had created a hi-fi Figma prototype of our new product, including multiple pages with varying degrees of interactivity, for mobile and desktop. In addition to the Figma prototype, we also had a working prototype running on a staging container, so users could experience the new product with real-world data.
Interviews were conducted on Lookback.io, recording video, audio, and notes, and lasting approximately 30 minutes, for which users were incentivised with Amazon vouchers. I lead all the interviews, with our UI designer observing and keeping notes. The process for each interview was to first understand a bit more about the participant; their interests, hobbies, news-reading habits, and a bit about their day-to-day life, to understand how our product might fit into their day. I would then show them the product, and ask for their first impressions and gut-feelings about the overall look, feel, and utility. Beyond that, participants were free to use and explore the product and were asked to try and vocalise their thoughts and actions as much as possible while they did so. I would answer questions, make notes of potential pain points, and occasionally offer some steer, either to ensure participants were able to cover all the talking points, or to help on the occasion that they got stuck with the prototype or the UX. At the end of each interview, I would sit down with our UI designer for a general recap, to discuss any points of interest, collate notes, and tag certain points of note in the recording.
Once all the interviews were complete, I worked with the design team to create a summary of our findings, as well as a highlights reel, containing the most interesting and useful clips of our interview recordings, grouped into categories, to offer colleagues and management a quick glimpse into the responses of our participants.
The overall results of this initiative were enlightening, challenging, and extremely positive. For the first time, we were able to understand and empathise with the NewsNow userbase in a personal way. We were shown which parts of our new product were working well, and in some cases even better than expected, as well as parts of the product that needed improvement. We were given a better understanding of how our product fit into the wider industry, how our users interpreted the brand against competitors, and how we might better serve a more diverse range of people in the future. We also found that some of our initial expectations were challenged: There were certain areas of the new product which we had been intending to make personalisable, or using algorithms to modulate aspects of the UI based on the user segment being served, but in fact we found that our users harmonised on certain aspects of the design, and that by executing certain aspects of the design well, we would negate the need for additional complexity. We were also able to re-prioritise our product roadmap, iron out some rough plans and add more specificity to design tasks in the backlog based on all the new insight we had attained.