Case Study:
Developing a 360 Leadership Assessment
The development of a custom individual assessment, closely tied to our workshop content, enabling participants to evaluate their inclusive behaviors.
Product Overview:
The Inclusive Leadership Assessment (ILA) aims to help leaders, both individuals and organizations, create more inclusive and diverse work environments. Through the assessment, leaders are provided with valuable feedback and actionable insights specifically focused on understanding and enhancing individuals’ inclusive leadership capabilities, while also assisting organizations in refining their diversity, equity, and inclusion (DEI) strategies.
The Inclusive Leadership Assessment (ILA) is now a comprehensive platform offering the following key features:
- Individual Assessment & Reports: Participants receive specific feedback on their inclusive behaviors, allowing them to identify gaps between self-perception and how others experience them. The included report provides actionable data, and highlights strengths as well as opportunities for growth.
- Aggregate Reports: Organizational leaders gain actionable insights into the overall organizational climate and demographic-specific trends. These reports help pinpoint strengths and learning opportunities, provide guided reflection, track progress over time, and offer heat maps for demographic comparisons.
- 1:1 or Group Coaching: The ILA supports individual employee development through 1:1 or group coaching, while aggregate reports facilitate organizational consulting conversations and considerations.
The Challenge:
The CEO and I shared a vision: to develop a high-quality multi-rater assessment that would enhance our workshop learning series and offer individual leaders insights into the inclusivity of their behaviors. Despite a plethora of personality and leadership assessments available (StrengthsFinder, DiSC, etc.), none were dedicated to the domain of inclusion and inclusive behaviors. This gap presented an opportunity for us to carve a niche in the market. Given our limited resources as a small, unfunded company, I undertook the development in phases, working within budget constraints over time.
The Solution:
We developed the assessment in two main phases.
Phase 1: Assessment Development and Refinement
In the inaugural phase of our journey to create the Inclusive Leadership Assessment (ILA), we focused on building and testing the initial assessment. This involved the extensive process of researching and crafting the survey and conducting thorough testing with focus groups, culminating in a pivotal pilot phase with one of our clients. Over the course of a year, we conducted research into user experiences, leading to careful adjustments to fine-tune the survey questions. The individualized report started as a simple PDF produced by placing key graphs and tables into a PPT layout to a more robust report booklet produced using R.
Phase 2: Online Platform Integration
With two years of in-house utilization under our belt and a steadily growing base of customer support, it was evident that the ILA was ready to take the next leap. Phase 2 marked the decision to invest in transforming the ILA into an online platform. This transition was instrumental in boosting our capacity to host assessments efficiently and improved the user experience for all participants. We introduced a participant portal, granting individuals the convenience of self-assessment, the ability to invite reviewers, and the seamless retrieval of their report at the close of the assessment period all in one place. In preparation for the online build-out we also refined the report design, re-organizing the page order in response to user feedback.
Following the build-out I continued to collaborate closely with our technology partners to continuously refine and enhance the online environment via feedback from user testing.
Through a four-year iterative development process, we self-funded the entire project. This was made possible by two years of prior user testing, which minimized the complexity and cost of edits when transitioning to an online platform with our technology partners.
Company
Greatheart Consulting
Position
Director of Learning, Research, Analytics & Development
My Roles
Product Manager, Project Manager, Assessment and Survey Methodology Designer, Page Layout and Graphic Designer, Content Creator
Technology Used
Adobe InDesign, Adobe Illustrator, Adobe Acrobat, PowerPoint, Excel, Envisia Analytics Platform
The Assessment Design Philosophy
I believe that meaningful growth starts with self-awareness and understanding the impact of one’s actions. Therefore my approach to this assessment reflects a commitment to research-backed, user-friendly tools that empower individuals to become better leaders.
The Assessment:
My goal was to create a survey firmly grounded in research and that could accurately measure key behaviors experienced by colleagues in the workplace. We started with research: delving into the intricacies of measuring qualities like effective listening, trust levels, feedback dynamics, and more. This research laid the foundation for our unique assessment, which comprises 4-6 carefully crafted questions for each behavior. The behaviors measured aligned with the curriculum taught in the inclusive leaders learning journey. Questions and question wording were rigorously tested through focus groups and pilot programs, to ensure accuracy and relevance.
The Individual Report:
On the report side I was committed to creating an individual report which would be user-friendly. I wanted the data to be displayed in a way that participants could easily understand, without the need for specialized training or expensive support. We meticulously fine-tuned the report’s content, presentation, and structure to provide a seamless experience. Each report integrates crucial data while omitting unnecessary complexity, making it a valuable tool for personal growth. Additionally, I included thought-provoking questions alongside each skills data page. These questions help guide participants in reflecting on the data’s significance and applying the insights to their own lives.
The Aggregate Report:
The Aggregate Reports were crafted to deliver actionable insights to organizational leaders and decision-makers. It was designed to provide a comprehensive view of the organization’s climate across demographics, and the question by question breakdowns offer insights into specific strengths, improvement opportunities, and group comparisons. Just like the Individual Reports, my goal was to ensure that these reports were user-friendly and could be effectively utilized without the need for a specialist, empowering organizations to make informed decisions and drive positive change.
A Look at the Assessment Elements
The Platform
Upon launching the assessment, participants are welcomed into their dedicated portal, designed to streamline their engagement. This homepage offers all the necessary activities, from taking their self-assessment to nominating reviewers and facilitating the download of their individual report at the assessment’s conclusion.
In the online assessment design I focused on clarity and ease of use. The platform features clearly defined Likert response options with selected choices highlighted in green for quick identification. Questions are thoughtfully framed around specific behaviors, enabling participants and reviewers to discern precisely which aspect of a broader concept, such as listening or providing feedback, is being assessed.
Skill Data Pages
To enhance the report’s usability, I opted to organize the data according to the relevant skills. Color-coding elements such as the Level of Agreement, and the Strengths & Weaknesses allows for quick, at-a-glance comprehension and simplifies the process of digesting a large volume of data. The overall goal is for the participant to take action on the feedback they have received—so making the feedback easy to understand was a priority.
Open-Text Responses
Pure data and Likert scales can only go so far. Therefore, I incorporated an open-text response option for both participants and reviewers at the conclusion of the assessment. These optional questions encourage participants and reviewers to reflect on the participant’s strengths as an inclusive leader and areas where they can improve their inclusive leadership behaviors. These qualitative responses breathe life into the data, offering valuable context to complement the quantitative findings.
Furthermore, this self-reflection element for the participant serves as a primer to facilitate them thinking about their strengths and weaknesses even before the self-assessment portion is complete.
Time-Over-Time Reporting
An essential feature of the assessment is its ability to compare data from two assessment rounds taken over time. Ideally, the assessment becomes a routine part of a leader’s annual process, providing regular opportunities for personal reflection and feedback on their inclusive behaviors. Leaders can effortlessly compare their most recent assessment data with the previous one to track changes. Likewise, in the Aggregate Report, organizations can compare the current year’s data with that of the previous year.
Translating Data into Action
For the average person, it can be difficult to translate data into action. While Greatheart did offer 1:1 coaching in connection to the assessment, I did not want people to be required to engage in coaching simply to understand their results. Including ‘Questions to Consider’ was one way to support participants in thoughtful reflection and action planning. Here’s an example of these questions within the ‘Hiring & Progression’ category.”
Results
- A Robust Assessment Tool: The successful creation of a comprehensive multi-rater assessment tool that accurately measures key behaviors related to inclusive leadership in the workplace and provides valuable insights into leadership capabilities.
- Data-Backed Validation: Rigorous testing with focus groups and a successful pilot with a client confirmed the survey’s construct validity and relevance as an inclusive leadership measurement tool.
- User-Friendly Individual Reports: An easily comprehensible Individual Report that can be utilized without the need for specialized training or support.
- Online Platform Implementation: The successful migration of the assessment onto an online platform improved accessibility, capacity, and the overall user experience for both individuals and organizations.
- Revenue Growth: The online build-out generated a low-lift income stream for the organization, and opened doors for future growth opportunities. Shifting the ILA from in-house hosting to a web-based platform allowed for 600% growth in the first two years.
- A Data-Based Learning Component: The assessment seamlessly complements the existing learning series by providing a data-driven dimension to the workshop offerings. This integration has enabled organizations to align leadership development with tangible behavioral insights, resulting in more effective and impactful leadership training.
✓
600%
Sales growth after investing in an online build-out.
✓
1,000
1,000 Leaders participated in the ILA to date.
✓
12
The ILA has been used at 12 Fortune 100, SMB, and non-profit organizations.
Janai Wallace 