This is where we examine the successes of our designs from the perspective of the community and identify areas for immediate or future improvement. By reflecting at this phase, we're refining our methods for collecting feedback to be more inclusive and to surface any oversights.
Consider who within your organization and from the community should be involved in answering questions in this phase.
Think about the factors and forces that have the greatest impact. Have we gotten feedback from multiple perspectives for each of these factors?
Do the people providing feedback feel heard and respected in this process?
Is there anyone else we should get feedback from? What are the risks of not hearing from them?
To what degree do our designs reflect those we have engaged in evaluation?
Map of research insights, factors and forces, and community decisions that fed into designs
Assessment of commitments to responsible practices made during earlier phases
Risks of not hearing additional voices
Does the testing environment reflect how available attention and resources may vary within the communities that will use this solution?
Does the testing environment feel appropriate and safe to community members?
What might we change about the testing environment or scripts to make testing closer to how people would actually use the solution?
Are there multiple environments that would be valuable for us to test in?
Assessment and improvement of the testing environment(s) and methods
How might those power dynamics be affecting the feedback we are getting?
Can we make any changes to minimize these effects and get more authentic feedback?
Should we consider other methods, such as asynchronous testing or community members collecting feedback? Why or why not?
Strategies to mitigate the power dynamics
Power dynamics in testing environments
Are we aligned with community members on the success of our designs?
Are we aligned with our client / leadership on the success of our designs?
If there is misalignment amongst groups about the success of our designs, how will we address it?
How are our biases or project pressures influencing how we’re thinking about the success of our designs?
Threats to successful equity outcomes
Consensus on successes and any future actions that are needed
When we uncover shortcomings in our designs that affect only certain community members, how will we respond?
How might we adjust our design solutions to benefit secondary and tertiary community stakeholders as well?
How might we involve community members in our iterative process?
Plan to involve community members in further iteration
Alignment on how we will respond to feedback in an equitable way
Who stands to gain the most through our solution? Who will benefit the least, or be disadvantaged?
In what ways does our solution challenge the way things are currently or differ from other solutions in this space?
What impact might our design have on non-users?
What might a solution look like that heals inequities and empowers
communities rather than harms?
Targeted design exploration to maximize equity outcomes
Unintended consequences of our designs
Within Kaleidoscope, we're referring to inequities as disparities stemming from social and systemic oppression that lead to worse experiences and outcomes for certain individuals and groups.
Within Kaleidoscope, we're referring to power dynamics as the ways authority and influence are distributed in relationships between people or groups, which has an effect on people’s ability to act, behaviors, and beliefs. These dynamics include influence over others and can affect how individuals or groups perceive their own power (or lack thereof.)
Within Kaleidoscope, we're referring to factors and forces as the conditions or systemic structures that influence people’s experiences; these can include institutions, social structures, policies, people, technology, environmental and political factors, as well as individual circumstances that impact one's agency and access.
Within Kaleidoscope, we're referring to engage/engagements as inviting individuals or groups external to the immediate project team into the research and/or design process; types of engagement can range from consultation to methods that prioritize collaboration and sharing power (i.e., intentional actions to shift, grow, or rebalance power).
Within Kaleidoscope, we're referring to communities/community members primarily as the people and groups who will be served or impacted by the product or service you are creating. Although people can share common needs, values, or goals, applying an intersectional lens helps us avoid oversimplification, homogenous labels, and limiting assumptions when we think about groups. (For example, in a group of parents, we would benefit from looking deeper to explore the experiences of mothers, and deeper still to learn about the experiences of mothers of color.) Sometimes you may work directly with community members, and other times you might work with community leaders and advocates who have established trust with the communities your solution is intended to serve.
Within Kaleidoscope, we're referring to impact as the emotional, social, and material results of one’s interaction with products, services, and systems. These effects can be planned, anticipated, or unintended.