overview

This is where we examine the successes of our designs from the perspective of the community and identify areas for immediate or future improvement. By reflecting at this phase, we're refining our methods for collecting feedback to be more inclusive and to surface any oversights.

Consider who within your organization and from the community should be involved in answering questions in this phase.

voices to engage

Who have we engaged for feedback and testing?

additional questions

Think about the factors and forces that have the greatest impact. Have we gotten feedback from multiple perspectives for each of these factors?

Do the people providing feedback feel heard and respected in this process?

Is there anyone else we should get feedback from? What are the risks of not hearing from them?

To what degree do our designs reflect those we have engaged in evaluation?

potential outputs

Create

Map of research insights, factors and forces, and community decisions that fed into designs

Understand

Assessment of commitments to responsible practices made during earlier phases

Risks of not hearing additional voices

resources
An arrow pointing downwards.

testing environments

Is the testing environment similar enough to where and how people would actually use the solution?

additional questions

Does the testing environment reflect how available attention and resources may vary within the communities that will use this solution?

Does the testing environment feel appropriate and safe to community members?

What might we change about the testing environment or scripts to make testing closer to how people would actually use the solution?

Are there multiple environments that would be valuable for us to test in?

potential outputs

Understand

Assessment and improvement of the testing environment(s) and methods

resources
An arrow pointing downwards.

power dynamics

Who is collecting feedback? What are the power dynamics between them and the people giving feedback?

additional questions

How might those power dynamics be affecting the feedback we are getting?

Can we make any changes to minimize these effects and get more authentic feedback?

Should we consider other methods, such as asynchronous testing or community members collecting feedback? Why or why not?

potential outputs

Create

Strategies to mitigate the power dynamics

Understand

Power dynamics in testing environments

resources
An arrow pointing downwards.

alignment around success

How successful are our designs in the eyes of different stakeholders in this process?

additional questions

Are we aligned with community members on the success of our designs?

Are we aligned with our client / leadership on the success of our designs?

If there is misalignment amongst groups about the success of our designs, how will we address it?

How are our biases or project pressures influencing how we’re thinking about the success of our designs?

potential outputs

LIST

Threats to successful equity outcomes

Understand

Consensus on successes and any future actions that are needed

resources
An arrow pointing downwards.

further iteration

How might addressing the shortcomings in our designs affect communities differently? What are potential tradeoffs?

additional questions

When we uncover shortcomings in our designs that affect only certain community members, how will we respond?

How might we adjust our design solutions to benefit secondary and tertiary community stakeholders as well?

How might we involve community members in our iterative process?

potential outputs

Create

Plan to involve community members in further iteration

Understand

Alignment on how we will respond to feedback in an equitable way

resources
An arrow pointing downwards.

exploring impacts

What biases or inequities may be reinforced by our design?

additional questions

Who stands to gain the most through our solution? Who will benefit the least, or be disadvantaged?

In what ways does our solution challenge the way things are currently or differ from other solutions in this space?

What impact might our design have on non-users?

What might a solution look like that heals inequities and empowers communities rather than harms?

potential outputs

Create

Targeted design exploration to maximize equity outcomes

Understand

Unintended consequences of our designs

resources
An arrow pointing downwards.

Within Kaleidoscope, we're referring to inequities as disparities stemming from social and systemic oppression that lead to worse experiences and outcomes for certain individuals and groups.

Within Kaleidoscope, we're referring to power dynamics as the ways authority and influence are distributed in relationships between people or groups, which has an effect on people’s ability to act, behaviors, and beliefs. These dynamics include influence over others and can affect how individuals or groups perceive their own power (or lack thereof.)

Within Kaleidoscope, we're referring to factors and forces as the conditions or systemic structures that influence people’s experiences; these can include institutions, social structures, policies, people, technology, environmental and political factors, as well as individual circumstances that impact one's agency and access.

Within Kaleidoscope, we're referring to engage/engagements as inviting individuals or groups external to the immediate project team into the research and/or design process; types of engagement can range from consultation to methods that prioritize collaboration and sharing power (i.e., intentional actions to shift, grow, or rebalance power).

Within Kaleidoscope, we're referring to communities/community members primarily as the people and groups who will be served or impacted by the product or service you are creating. Although people can share common needs, values, or goals, applying an intersectional lens helps us avoid oversimplification, homogenous labels, and limiting assumptions when we think about groups. (For example, in a group of parents, we would benefit from looking deeper to explore the experiences of mothers, and deeper still to learn about the experiences of mothers of color.) Sometimes you may work directly with community members, and other times you might work with community leaders and advocates who have established trust with the communities your solution is intended to serve.

Within Kaleidoscope, we're referring to impact as the emotional, social, and material results of one’s interaction with products, services, and systems. These effects can be planned, anticipated, or unintended.