Blog

I'm a Top Writer in Diversity on Medium! Follow me on Medium

Posts in Entrepreneurship
How to measure “inclusion” quantitatively : free resources and research data for D&I advocates

What do you look for when you’re looking for a new job? 

Good pay, no commute, better title, opportunities to advance, a rocket ship company… oh, and culture. Culture is important. 

Well, I believed (and still do believe) there is a group of people who would put inclusive culture at the top of the list of things they look for in a new job. Because I was one of them. I still believe there are people like me, who would choose a lower paying job over a company run by CE-bros with a toxic culture. 

People who want to work at a place where they can bring their whole selves to work, where they don’t have to merely survive while suffering from daily microaggressions and other oppressive BS. Call me naive, but I thought such places can exist, but surely, it would take a lot of research and validation. 

So how do you go about finding such a place? 

How do you *really* know if a company has an “inclusive culture?” 

Well, my business partner and I wanted to find out. 

Clue #1: Diversity Reports

In recent years, more and more companies have been eager (and pressured) to share “diversity reports,” sharing their demographic data. Still, the number of companies that have been releasing diversity reports consistently is limited. And, because the format and contents of these reports are so varied, it puts the onus on the job seekers to decipher and compare. Most of these reports breakdown demographic information into race and gender categories, vs. reflecting people with overlapping / intersecting identities, making it difficult to understand the true population of the company.

And as we know, diversity does not guarantee inclusion. In fact, some companies have become better at attracting more diverse talent (“pipeline problem”) that they can quickly replace the folks they lost. The problem with most diversity reports is that they are a snapshot — like a balance sheet — that only gives you a partial picture of reality.

Just like you need a lot more than just a balance sheet to understand a company’s health, you need a whole lot more information than just the current population information to understand a company’s actual culture.

In order to understand the actual culture of a company, through the lens of inclusivity, you need way more than a snapshot of who’s surviving at the company today. But keep those reports coming, we still want to know! 

Clue #2: Glassdoor, et al. 

Ah, Glassdoor. 

I just have three points. 

  1. People don’t trust Glassdoor in general
  2. It’s hard to find inclusion-specific information on Glassdoor 
  3. See screenshot below 
 Uber’s Glassdoor rating in April, 2017 (Note — Susan Fowler’s blog post went viral in February, 2017)

Uber’s Glassdoor rating in April, 2017 (Note — Susan Fowler’s blog post went viral in February, 2017)

 Uber’s Glassdoor rating on Aug. 11, 2017

Uber’s Glassdoor rating on Aug. 11, 2017

Need I say more? 

Clue #3: Company website

HAHAHAHAHAHAHHAHAHHHAHAHAHAHAHAHA

Ok, I do like studying the company’s leadership and board members, though. And their pretty marketing language on how much they value diversity and inclusion. I also think it does count for something (maybe, like, half a sticker) if a company is vocal about valuing D&I vs. not. 

Those are just a few resources that are available online. Unfortunately, finding inclusive culture information that is reliable and comprehensive is not easy. Plus, culture varies not only by company, but by team within a company. Getting this data online is near impossible. 

Some forward thinking companies have gone ahead and begun the process of measuring inclusion using tools like Culture Amp’s Inclusion Survey, but the information they gather is often kept private and is shared only if the data looks favorable. After all, what incentives do companies have to air their dirty laundry? 

So what do people trust? 

People trust unaltered feedback from current or former employees of the company. Resourceful people who have access to people at companies they are interested in prefer to talk to current or ex-employees to get the real scoop.

“Tell me the real deal. Should I apply?”

No surprise here, right? 

But unless you have this social capital, how are you supposed to access this information? Without access, you have to rely on your own judgment based on limited information available publicly and roll the dice. Just keep a low expectation, is that it? 

We wanted to change this. 

We wanted to redistribute power to job seekers by providing them with discoverable, reliable, comprehensive, relevant, and comparable data around inclusion.

We wanted to provide a safe platform for employees experiencing toxic or inclusive cultures to speak up. We wanted them to be able to tell their side of the story without the company’s interference. 

We wanted to standardize how we talk about and measure “inclusion” and provide a useful tool so companies can improve against concrete metrics. We understand the idea of capturing “inclusion” in one quantitative score may seem simplistic. But what we see over and over again is that businesses don’t prioritize what cannot be measured

While we would never claim one quantitative metric alone can paint the most accurate and comprehensive picture of a company’s inclusive culture, we did believe it can be a tool used to supplement existing efforts to get a pulse on a company’s culture. Or at least serve as a starting point. 

Ultimately, we believed that with better information about companies’ culture, inclusion-seeking job seekers would be able to make better decisions. On the flip side, we hoped this would put pressure on companies to improve their culture to be more inclusive, and measure how they’re doing via real-time, candid feedback from their workforce (most companies we talked to did not like this idea). 

So the idea of “inclusion dashboard” was born. We did a lot of research around how one would define and measure inclusion. We interviewed a bunch of people who were exploring new opportunities to understand their research methods and priorities. We talked to D&I experts, companies at the forefront of leading D&I, researchers, professors, Venture Capitalists, and spent months thinking about how we can close this information gap. 

We learned a lot. 

Here are some interesting, but not surprising, findings / validations from our interviews: 

  • People who have experienced harmful culture rank inclusive culture higher on the list of priorities when looking for a new job vs. people who have not experienced harm.
  • Ensured anonymity was the #1 requirement for making people feel comfortable leaving a company review.
  • 4 common factors were identified as a requirement for making information trustworthy: Relevancy of the data to the individual, credibility of the information source, balance of good and bad data (goes back to credibility), connecting directly with an individual who works at the company.

There’s a load more data we gathered that you can check out in our now archived “pitch” deck. Note we never actually wanted to raise capital, we just wanted to get advice from smart people and launch the survey to see where it goes. We had some ideas for how we would monetize, but we just wanted gather and make inclusion data public and iterate from there. 

Anyway, after conducting our own research and interviews, we attempted to develop a way to define and measure inclusion. It contained three major parts:

  1. Psychological Safety: Do people feel belong and empowered? 
  2. Trust in Leadership: Do people believe their leaders actually care about inclusion? Do they trust their leaders will do the right thing, especially when shit hits the fan? Do they believe their HR team will address complaints swiftly and competently? 
  3. Incidents: Have people experienced, witnessed, or overheard harassment, assault, or other forms of harm based on identity?
 Our fancy shmancy diagram breaking down “inclusion.”

Our fancy shmancy diagram breaking down “inclusion.”

 

We then created a survey to measure this. 

A quick note — big props to the team at Culture Amp and Paradigm IQ for releasing the Inclusion Survey — we reviewed their survey and adapted some of the questions (particularly helpful for the Psychological Safety section) and added additional ones that we thought were important for assessing inclusion. We also love all the work Project Include has done — we kept in mind a lot of their recommendations around measuring progress

We tried to make the survey as concise as possible, especially given the fact that it was intended to be done publicly (aka voluntarily). We needed the process to be as easy as possible. 

Our survey contained 30 questions in addition to gathering people’s demographic information, tenure and department at the company, and their level. This would allow us to breakdown the data in meaningful ways, showcasing how different folks at different levels, in different teams, with different identities felt about the company’s culture. 

We added important questions like, 

  • “I feel comfortable going to my HR with complaints (e.g., harassment, bullying, microaggression, etc.)” and 
  • “I was pressured or incentivized by my company to leave a positive review” 

that we don’t usually see in company-run employee engagement surveys. 

Here’s the link to receive the full survey, if you want to adapt it for your company and use it, feel free. Note the survey was still in the process of being polished with help text, definitions, and more thorough answer options. 

We were also in the process of finalizing our scoring algorithm based on the survey results and demographic data of the survey respondents. We were toying with some controversial ideas, too, weighting scores more heavily if the responses were submitted by folks in marginalized social identity groups (e.g., people of color, women, queer, trans, disabled, etc.) vs. those in privileged identity groups (e.g., white, straight, man, able-bodied, etc.). This was based on our belief that in order to achieve true inclusion, we need to ensure the most marginalized group feels included. 

We were going to launch the survey and gather data publicly, and publish the data in a comprehensive and digestable way using an interactive dashboard. People would be able to slice and dice the data to fit their needs, so if I’m looking for inclusion data from the perspective of queer women of color in Customer Success, for example, then I’d be able to. 

We worked with a product manager to scope out the platform dashboard features, prioritized business requirements. For example, we wanted to create a way to verify someone’s identity and employment history via LinkedIn (we know it’s not the perfect way) to solve for credibility, and put a threshold for when we would release a company’s data so we can solve for non-traceability based on demographic and reliability in terms of the data volume (e.g., we wouldn’t release the survey results unless a company had X number of responses, so it’s not easy to trace back who filled it out based on their identity markers). 

 Business and Product Requirements List

Business and Product Requirements List

We looked for advisors. We had partners ready to help us develop the site. 

So why did we stop? 3 major challenges surfaced.

  • Challenge #1: Gathering a large volume of data to make the site useful 
  • Challenge #2: Refreshing the data often to ensure it stays up-to-date 
  • Challenge #3: People don’t make decisions based on information

We believed we could overcome Challenges #1 and 2 eventually. But #3? That was a tough one for us to swallow. What’s the point if we gather and disseminate data but it doesn’t influence people’s decisions? How would we actually influence change? What were we solving for? 

“Since the Susan Fowler story broke out, the number of female engineer applicants has not decreased. There also wasn’t a mass exodus from Uber” —very reliable Uber insider

This crushed us. Sure, you can argue “but Uber is different — not every company has the leverage Uber has” or “no, but I really care!” 

But for us, it seemed like the ultimate impact of closing this gap was unclear especially when the road to building a useful product was a complex one. 

The truth is, unless enough people vote with their employment status, companies are not going to fundamentally shift their internal culture. 

It was a tough decision for us to scrap this “Glassdoor for inclusion” idea. We were so passionate about solving this problem, and we still are. We ultimately decided, though, this is not going to be the dream we make reality. We are still using bits and pieces of our learnings today, and have utmost respect for folks trying to tackle this gap. 

We don’t regret having done the research. 

And we want to share what we learned with anyone who cares. Because we are all in this together, and we have to build solidarity and coalition to move the needle a millimeter at a time. 

So we’re open sourcing our materials here:

You’ll find flaws in our stuff, no doubt, and maybe even judge us for being naive (but without it, can change really happen?). Basically, please don’t troll us. But do share your thoughts and ideas so we can all learn from each other and be better

The following companies are also trying to bring inclusive culture data to the forefront to help jobseekers. Check them out and support them: 

So what are we doing instead? We are providing a new kind of “diversity and inclusion” workshops to develop inclusive leaders. It’s hard work that requires a lot of emotional labor. It’s not very scalable. But we truly believe in the impact we can have on people with our approach, one rooted in social justice and the idea that change happens incrementally and with individuals. It may not be as scalable or sexy as running a software company to some, but we love connecting with real people, holding space for tough conversations, and seeing lightbulbs go off in people’s hearts. We’ve been getting raving reviews and feedback from folks who say we’ve completely changed their perspective on diversity and inclusion. Now that means something. 

Have ideas on how to measure inclusion better? Share in comments or message me directly! I’d love to hear from you.


About Michelle Kim

Michelle is an entrepreneur, activist, speaker, and a coach passionate about empowering individuals and organizations to create positive change. She is the co-founder of Awaken and owner of Michelle Kim Consulting.

Follow Michelle’s continued journey to create change in this world:

Subscribe | Website | LinkedIn | Twitter | Facebook | Awaken

 

Why I'm tired of "Diversity" Workshops

I always shuddered at the thought of building a services company. I had always envisioned starting my own company one day, perhaps a product company with rapidly growing recurring annual revenue and a kickass margin. Super scalable. All the sexy buzzwords, throw them in there, I wanted them.

Never did I imagine myself becoming a workshop provider (or a career coach, for that matter). 

Well, life happens. And like Oprah said, you just have to lean in to life.

Ever since I was a student, I wanted to start my own business. Not because I wanted to be “my own boss” or because I wanted to build the next billion dollar unicorn (btw, did you hear unicorns are falling?).

I wanted to build a company I can be proud of. A company with my values and principles deeply embedded throughout. I wanted to create an alternative reality. I wanted to build a company that:

  • Treats all people with respect and dignity
  • Is radically transparent
  • Pays people well unapologetically and equitably
  • Hires and rewards people with integrity, grit, and empathy
  • Fires jerks and bros (or don’t hire them to begin with)
  • Is truly diverse (not some “diversity of thought” bs)
  • Allows people to be their whole selves
  • Is unafraid to take a stand on political issues no matter how risky
  • Roots for the underdog
  • Wants to do good, for the sake of doing good, not for ROI
  • Cares about social justice

When I imagine my perfect “company,” I remember the time I co-led a queer student organization at UC Berkeley. We were made up of majority queer people of color, and had members from all identities and intersections. Our mission was to create an inclusive space for all queer people — folks of color, folks with disabilities, undocumented folks, truly.. all people on the margin who wanted to come together and build community, participate in developing youth leaders and empowering ourselves.

I thought I could one day achieve this vision by starting a sexy, scalable product company. Well, maybe I still could one day. But for now, I’m doing workshops.

So why did I decide to start a company providing “D&I” (Diversity & Inclusion) workshops?

I got tired.

I got tired of sitting in so-called “diversity workshops” that barely scratched the surface.

I got tired of seeing old white people dominate conversations around race and gender, “diversity,” and what it means to be an inclusive leader.

I got tired of corporate-bred D&I workshop facilitators (again, most of them old white people) diluting critical social justice concepts into palatable talking points for straight white men.

I got tired of seeing white, cis, hetero people never once feeling uncomfortable when being educated on D&I, but feeling absolved after having “checked the box.”

I got tired of seeing my friends and mentors not get paid for their social justice work. Being discounted to “soft skills” facilitators, not warriors, activists, and mission-critical educators.

I got tired of feeling the only reason why companies tolerated my outspokenness was because I was a high performer (and that I was a less threatening East Asian woman) and I had to continue to earn my right to call shit out .

I got tired of seeing companies using “Diversity and Inclusion” as marketing catchphrases to gain public validation, yet never wanting to dig deeper or put money where their mouth is.

I got tired of talking about metrics I didn’t care bout, I got tired of losing myself, I got tired of covering.

I got tired of doing extra emotional labor around D&I issues because no one else would.

I got tired of dealing with “brilliant jerks.”

I got tired of feeling like dying a slow death by a million paper cuts made by daily microaggressions.

I got tired of seeing my peers be mistreated.

I got tired of being let down by people.

I got tired of losing faith in humanity.

I got tired of never feeling free.

Every time I sat through a divershitty training (yeah I just made that up), I wished someone would come in and do a REAL workshop. Encourage REAL TALK. Make me and others feel uncomfortable, because without discomfort there is no real learning when it comes to understanding systemic and institutional oppression.

I wished someone would bring in critical social justice concepts into the workplace, and not be afraid to talk about structural racism, misogyny, and homophobia, and how unconscious bias stems from our deeply socialized identities that are perpetuated systematically.

I wished someone would actually name white privilege, misogyny, heterosexism, ableism, and gender binary. I wished someone would actually say the word queer or trans. I wished someone would acknowledge the mass incarceration and killings of black people by the criminal justice system.

When the time came when I no longer could stay in the toxic tech industry as an employee, I, along with thousands of women who have left before me, left.

So now I’m trying to make my distant dream and wishes a reality. I’m trying to unlearn the toxic shit I had to pick up in the corporate world, and bring back the old, authentic me. The old me who was unafraid to call shit out, who was passionate about building solidarity and coalition, who took risks and used privileges to provide access to others.

 Awaken Team facilitating a workshop at General Assembly SF

Awaken Team facilitating a workshop at General Assembly SF

 

I’m rolling up my sleeves and applying everything I’ve learned from my social justice activism and surviving the corporate / tech world to redefine “D&I workshops.”

I’m working to bridge the gap between “Diversity and Inclusion” and social justice activism. I’m working my ass off to get well-deserving, non-corporate-bred folks paid.

In order to create change, we need to embrace discomfort. We need to create a compassionate space for uncomfortable dialogues, where we allow each other to fuck up, but also hold each other accountable. We need to acknowledge that change doesn’t happen overnight, but it can happen incrementally.

While I would never claim “D&I Workshops” will solve all your companies’ toxic culture problems, it can help begin the conversation. It’s a starting point.

There are so many amazing people trying to do different things to move the needle a smidge on creating a truly inclusive culture. And we need all of them. We need all of the process changes, policies, culture shifts, engagement surveys, ERGs, D&I consulting, anti-sexual harassment training, offsites, Artificial Intelligence based recruiting, VR training, VC accountability… we need everyone and we need all of them.

The problems we are trying to solve are so massive and so ingrained. We need all the help we can get to have a fighting chance at moving the needle.

So here’s me, choosing to do what most software junkies call “unsexy” work (but you just wait). And you can help me by spreading the word about Awaken (and our upcoming workshop series).

Come on, let’s wake people up.