Designing Textio’s Next Gen In-App Reports
The Challenge
Textio’s in-app reporting had lagged behind the development of our other features for years. Our first generation of reports were jokingly described internally as “Textio is good at counting things.” Our original suite of reports were very high level; mostly focused on the things we highlighted in our guidance (both good and bad). Usage reporting was also minimal, and our Customer Success (CS) team often had to supplement in-app reports by manually generating more detailed reports for individual customers.
Defining a Solution
Our Product Manager for in-app reporting regularly conducted customer calls and brought an initial list of potential report types that we brainstormed from. We also leveraged feature-request data in Productboard and conversations with our CS team.
- From that, we developed initial concepts from that data;
- Product and Design held research calls with customers, reviewing early sketches and concepts;
- Gathered tons of feedback on what customers liked (and what was missing);
- Continued to iterate through this process (repeated rounds of design concepting and user testing).
We settled on three main areas of focus:
- User Adoption - How could we help team leads understand the current state of their teams Textio adoption (and act on that)?
- Impact and Live Job Post reports - Help team leads see the impact (and ROI) that writing with Textio’s guidance was making on their recruiting messaging, and allow them to audit/monitor live job posts.
- Key Takeaways - Distilling dense reporting in our performance feedback product by identifying significant group differences.
User Adoption
Textio’s initial user activity reports were very basic – we counted monthly logins. Our goal was to help team leaders get a more accurate picture of adoption across their company – and give them tools to help increase adoption. That boiled down to two steps: 1) helping the identify differential patterns of adoption (by team, department, location, etc.) across their company, and 2) providing actionable next steps, ideally one-click actions, that a customer lead could take to improve adoption.
One example of the latter was our "adoption funnel" report, which identified what stage of the adoption process members of an organization had reached, then offered easy actions team admins could take.
Impact & Live Job Posts Reports
A key goal for many teams using our recruiting product was to monitor “live” job posts (those currently posted on career sites and job boards). Teams could identify low performing posts and could identify those in need of immediate updates (e.g., if a post contained problematic or offensive language).
Understanding the impact of Textio’s guidance on pipeline and hiring was a frequent request from customers. Our impact reports looked at how higher Textio Scores correlated with a job post’s performance, as well as providing measures of other important factors, such as pay transparency, degree requirements, and more.
Key Takeaways Report
The launch of Textio’s performance feedback guidance introduced our first equity reports – breakdowns of differences in feedback quality and amount between different groups of employees (by department, by gender, and by race/ethnicity). These reports provided a lot of data, and customers needed a way to identify which differences were significant, and which were likely the result of normal variance.
In addition to our equity reports, we introduced a Key Takeaways report. This highlighted statistically significant differences in feedback quality or amount of feedback between groups.
We selected a subset of factors to test (e.g., personality feedback, non-actionable feedback, and overall amount of feedback received) and developed a set of editorial-style cards to address each significant difference identified.
Additional Notes
We shipped Key Takeaways first, as a part of our performance feedback product launch; it well received by customers. Adoption reporting followed, with Impact and Live Job Posts slated after.
Project challenges included prioritizing and staffing: Our Insights team was a small one, and regularly pulled into multiple other initiatives. This led us to ship new reports at a slower cadence than would be ideal. We also ended up backfilling some need by providing automated data downloads (CSV) until full in-app reports shipped.