Background
The Employment Equity Form is a part of the online application process used by the client in which an applicant can choose to self-disclose among four groups: aboriginal peoples, women, people with disabilities, and visible minorities under the Employment Equity Act. The disclosure is completely voluntary.
Challenge
The current design of the form prevents applicant from disclosing any information under the act which poses challenge for our client who is trying to track the progress of diversity initiatives taken by it to improve the application rate of diverse applicants and their absorption.
My Role (In a team of three)
Employment Equity Research: Managed research on the Employment Equity Act and explored best practices in self-disclosure.
Form Design: Designed new forms to improve data collection and compliance.
Internal Feedback Sessions: Organized and facilitated paper testing to gather feedback on the new designs.
Usability Study Design: Reviewed the Mechanical Turk (M-turk) study designed to test the final prototypes.
Results Analysis and Presentation: Analyzed the results from the M-turk study and presented the findings to the client.
Here are the issues we identified with the design of the current form:
Confusing Information Disclosure: The explanation about how the information collected will be used, which is crucial for applicant disclosure, is placed in a box that is color-coded similarly to other general instructions. This can lead applicants to overlook the information's use as defined by the EE Act, causing them to skip this disclosure section entirely.
Lack of Emphasis on Diversity: The current wording doesn't sufficiently highlight the organization's commitment to diversity, giving the impression that diversity is merely a statistic or a checkbox. This lack of emphasis on the value of diversity can undermine the organization's efforts to attract a more diverse applicant pool.
Masculine Language: The language used in the form has a masculine bias, as revealed by a test with the Gender Decoder tool (http://gender-decoder.katmatfield.com/). This can make the job less appealing to women, contributing to lower female application rates (as discussed in Gaucher, Friesen, & Kay, 2011).
Find and Frame
We conducted audit of the form and documented our findings.
Part A of the form:
Part B of the form:
Additionally, once an applicant decides to disclose, there are further obstacles in the form that could lead to dropouts:
Purpose Phrasing: The stated purpose of disclosure, highlighted in red, gives the impression that this information might be used against the candidate in the application process. This can create a sense of mistrust and discourage applicants from disclosing sensitive information.
Limited Gender Options: The form only offers two gender options, potentially excluding non-binary or gender-diverse individuals. This limitation can prevent these candidates from feeling included and respected, thereby leading to form dropouts.
Prototyping and Paper Testing
With this assessment in mind, we set out to re-design the above form. The below images showcase the prototypes and the feedback collected from behavioral scientists and students interning at BEAR(Behavioral Economics in Action at Rotman) when we conducting hall testing.
Designing Part A of the form:
Version A
Version B
The feedback collected highlighted that we should go with Form 2 as our main design.
Designing Part B of the form:
We researched best practices for self-disclosure and discovered that providing open-ended options instead of a limited set of choices, along with an additional "prefer not to disclose" option, offers more flexibility and choice for individuals sharing personal information. This approach allows people to identify themselves in a way that aligns with their personal identity.
Following this research, we incorporated these best practices into our form, as depicted in the image below:
Unmoderated Testing and Results
We tested the newly designed Parts A and B of the form separately on Amazon's Mechanical Turk (M-Turk). A total of 260 participants were randomly assigned to evaluate either the newly designed form or the old form.
Results for Part A design:
58% of the study participants preferred the newly designed form and were willing to proceed to the next step of self-disclosure.
Results for Part B design:
There wasn't a significant difference in self-disclosure rates between the old and new forms when it came to Part B.
However, it was interesting to find that participants entered ethnicity information in the visible minority status that wasn't part of our client's existing drop-down menu. The ethnicities included Asian, Hispanic, Vietnamese, Mexican, Thai, and Latino. This suggests that the original form design, with its limited drop-down options, might have discouraged some candidates from disclosing their identity accurately.
Impact
We were able to positively influence the client in two key ways:
Validation of Results: The client was pleased with these findings and asked us to suggest other studies that could be conducted with the EE form to ensure the final results are robust before implementing changes to the existing form.
Embracing Experimentation: The client became enthusiastic about fostering a culture of experimentation. They expressed interest in trying out new approaches and methods within their internal processes as well.