Evaluation

University of Washington's Center for Evaluation and Research for STEM Equity (CERSE) serves as the evaluator for BRAINS. The evaluation team collects data via surveys of participants and non-participants, observations of meetings and events, and tracking career outcomes. "Learning to Thrive: Building Diverse Scientists' Access to Community and Resources through the BRAINS Program," (CBE - Life Sciences Education, Sept. 2016) reports early data on near-term individual impacts on career-advancing behaviors and career experiences of BRAINS participants from the 2013 and 2014 cohorts. Also, a PDF iconposter about the impact of BRAINS was presented at the 2016 Society for Neuroscience annual meeting.
  1. Early Program Impacts
  2. Anticipated Long-Term Impacts

 

1. Early Program Impacts

BRAINS has had three cohorts to date, one starting in January 2013, the second in September 2014, and the third in September 2017. Total participants: N = 81. The majority (43%) of these participants have identified as Black or Black Hispanic, while 21% have identified as Hispanic, and 20% as White Hispanic (Figure 1).

 

race and ethnic diversity of BRAINS participants

Figure 1: Race and Ethnic Diversity of BRAINS Participants, All Three Cohorts.

Thus far, the first two cohorts have completed four and three follow-up surveys, respectively; as the third cohort only recently began the program, they have not yet had time to complete any follow-up surveys. Evaluation data show that participation in BRAINS leads to near-term impacts on career advancing behaviors and experiences.

 

Career Advancing Behaviors

BRAINS participants from the first two cohorts reported successful career progression, research productivity and increased connection with other scientists. In 2017, 51.9% of participants were in tenure-track positions (versus 25.5% at application)(Figure 2). Participants engaged in more networking and mentoring activities after their BRAINS experience (Figure 3).

Tenure-Track Comparison

Figure 2: Tenure-track positions of participants and non-selected applicants at the time of application and in May 2017.

 

Table showing difference in mean career behavior index scores from application to first annual survey

Figure 3: Participants engagement in career connecting activities such as mentoring and networking increased from time of application to the BRAINS program to one year later.

 

Career Experiences+

As illustrated in Figure 4, the first two cohorts of BRAINS paticipants reported an increased sense of belonging to neuroscience and increased self efficacy.

Graph showing increase in mean index scores from application to first annual survey

Figure 4: Increases in sense of belonging and career self-efficacy mean index scores from time of application to the BRAIS Program to one year later.

 

2. Anticipated Long-Term Impacts

We expected that near-term individual impacts will both lead to long term career impacts on participants, as well as impact diversity and inclusion in neuroscience at the national scale. Figure 5 illustrates the BRAINS program theory and impacts.

Table showing difference in mean career behavior index scores from application to first annual survey

Figure 5: The BRAINS program theory suggests that program features will lead to short-term impacts on career behavior and experience. We anticipate that these individual impacts will lead to long-term impacts on the careers of BRAINS participants and serve to impact diversity in neuroscience at the national scale.