Our Impact

MHA Labs’ is a skills development strategy that works concurrently with investments in academic excellence, technical expertise and college/career navigation to increase the number of youth achieving college, career and life success.

Over the the last two years, the research on the impact of non-academic factors such as “grit” and persistence has reached a tipping point creating national focus on the social, emotional and cognitive skills of youth. While some have interpreted “persistence” as the next silver bullet solution,  MHA Labs makes no claim that 21st century skills are sufficient on their own to produce a college and career ready youth. 21st century skills are nonnegotiable to success but are not solely determinant. Youth live in complex social and economic conditions that regulate the successful outcomes of even the most college and career ready youth.

To maintain this holistic readiness focus, MHA Labs developed a logic model where 21st century skills are authentically embedded into all learning settings. These settings include home, school, enrichment programming, and employment. Each phase of the logic model drive performance towards 18 measurable cradle to career milestones. MHA Labs has included satisfaction outcomes to ensure that youth are not driven towards college and career readiness at the expense of aspirations that will bring personal, family and life fulfillment.

College & Career Success Youth Development Milestones

Metrics System Aligned to Logic Model

Section 1: Source Stakeholders

All stakeholders across cradle to career share common 21st century skills standards…

MORE +

  1. Goal: Build collective impact strategies where youth development stakeholders agree to align to and use MHA Labs Building Blocks and Assessment Tools as common 21st century skills learning and performance standards.
  2. Targets in Year 1: Collective Impact Communities
    • Chicago, Illinois
    • Minneapolis – St. Paul, Minnesota
  3. Impact Metric Overview: MHA Labs uses “Impact Impressions” to track the number of stakeholders that are building awareness the Skill Building Block framework. Managing the exposure process is highly critical for a tool to meet its expectations in the market. In preparation for launch, MHA Labs worked with professional development subject matter experts to design a user adoption model. After careful analysis, it was determined that how a user was first exposed to the tools would be the most significant driver of future successful implementation. This is particularly true for sectors such as public education where skill standard and assessment reforms are highly polarized topics. Note, when user impressions convert to user practice or program activity this represents the metric of adoption.  This will be reported in Section Two.
  4. Impact Metric Rationale
    • Individual exposure builds awareness
    • Individual exposure plus sense of value creates buy-in
    • Buy-in and awareness drives exploration
    • Continuous exposure and exploration builds retention
    • Retention drives daily implementation
    • Exposure across stakeholders builds common language
    • Common language build community
    • Community builds productivity
  5. Impact Metric Tracking Items
    • Receipt and Review of Skills Framework Tools
    • Peer Exposure: User to User Transmission
    • Participation in an MHA Labs Orientation
    • Participation in an MHA Labs Training
    • Participation in an MHA Aligned Class or Program
    • Receipt of MHA Labs Aligned Skills Feedback
    • Use of an MHA Labs Assessment
    • Conference, Meeting or Committee Participation

Section 2: Improve Inputs

All stakeholders use 21st century skill aligned practices and programs to increase skills application and performance…

MORE +

  1. Goal: Work with users in MHA Labs collective impact communities to introduce and use engaging practices and programs to build youths’ 21st century skills. Note that programming includes but is not limited to lesson planning, program planning, activity planning, job descriptions, job tasks and job project plans.
  2. Targets in Year 1: Collective Impact Communities
    • Chicago, Illinois
    • Minneapolis – St. Paul, Minnesota
  3. Impact Metric Overview: After a user is exposed to the skills framework, they move towards practice and program implementation. Similar to the common core implementation, MHA Labs does not mandate a specific curricular intervention but promotes authentic, contextual integration into the subject matter of the class, program or employment setting. This approach is even more critical in the field of 21st century skills with are inherently subjective and contextual in nature. Instead of attempting to regulate human behavior, MHA Labs promotes a model where youth apply skills across the wide range of learning contexts where they are provided goals, expectation and feedback based on their performance in context. A youth’s success in this type of cross-context model drives adaptability and social awareness, where a youth learns to decode and adapt to the success requirements of a setting rather than relying only on one set of standardized behavior.
  4. Impact Metric Rationale
    • Programs begin to intentionally include 21st century skills in program design
    • Programs standardize 21st century skills in program design
    • Practice begins to intentionally include 21st century skills application
    • Practice habituates 21st century skills application
    • Practice beings to give youth real-time, explicit feedback on skills performance
    • Practice habituates real-time, explicit feedback on skills performance
    • Practice beings to provide youth with summative skills performance reviews with or without MHA Labs assessments
    • Practice habituates summative skills performance reviews with or without MHA Labs assessments
    • Programs begin to use qualitative and quantitative data to support youth skills development interventions as well as practice and program changes
    • Programs habituate the use qualitative and quantitative data to support youth skills development interventions as well as practice and program changes
  5. Impact Metric Tracking Items
    • Planning: Intentionally embedding 21st century skills into program designs
    • Preparation: Ensuring that the learning environment is structured to support the program design
    • Action: Intentional delivery of skill building instruction and activity
    • Feedback: Providing targeted real-time task-level, process-level or self-regulating feedback during the action phase
    • Reflection: Conducting reflection activities that promote the learner to reflect on and articulate what they have learned over a course of time (day, week, month..)
    • Improvement: Using information from a variety of sources to support the improvement of a youth, adult skill-builder and program design

Section 3: Using Evidence

All stakeholders use formative and summative 21st century skill assessment tools to inform skills development…

MORE +

  1. Goal: Further develop MHA Labs assessments to meet the major developmental milestones of youth which include graduating from high school college and career ready; skill-based recommendation and admissions to college and employment; and skill based performance review and promotion on the job.
  2. Targets
    • Teachers
    • Out-of-School Time Instructors
    • Counselors & Mentors
    • College & Career Advisors
    • Workforce Developers
    • Employers
  3. Impact Metric Overview: MHA Labs tracks the use of its assessments across cradle to career contexts. MHA Labs has developed this system of formative and summative assessment tools to benchmark skills performance in context and communicate results. Use of assessment is not limited only to rating the scorecard which is typically where the majority of youth development assessment end their function. MHA Labs prioritizes the formative and summative use of assessment data to support, develop and translate a youth’s skills into asset building and applied productivity.
  4. Impact Metric Rationale: MHA Labs twenty-member volunteer Data and Impact Alliance developed a comprehensive set of data use cases to support the rationale for assessment use.  Alliance members debated each use case through the lenses of research and evaluation science as well as the real-world implications. All use cases had to reach 100% agreement among Alliance members.
  5. Impact Metric Tracking Items
    • Collecting evidence of skills performance
    • Rating Assessments
    • Using Data for skills development
    • Using Data for continuous improvement

 

The use cases below outline the rationale behind tracking assessment usage:

Human Achievement Quotient – Formative Assessment Only>>

  • Provide a formative assessment to benchmark youth skills development over time
  • Provide verbal communication feedback to youth on skills development
  • Provide written communication feedback to youth on skills development
  • Develop communication practices or tools that engage youth in a discussion of their skill strengths and areas for growth
  • Co-create individual development plans where youth document their skill development goals and their plan to reach those goals
  • Provide youth with suggestions and recommendations on programs or training opportunities aligned to their individual skills development goals and plan
  • Provide verbal communication feedback to parents on their youth’s skills development
  • Provide written communication feedback to parents on their youth’s skills development
  • Develop communication practices or tools that make parents aware of their youth’s skill strengths and areas for growth
  • Develop communication practices or tools that engage parents in a discussion about their youth’s skill strengths and areas for growth
  • Co-create individual learning plans where youth and their parents document the youth’s personal development goals and their plan to reach those goals
  • Provide parents with suggestions and recommendations on programs or training opportunities aligned to their youth’s skills development goals and plan
  • Share standardized skill assessment information between persons working with the same youth
  • Develop and modify skill building lesson units or program plans based on youth’s skill strengths and growth areas. Data sourced from program/class level aggregate data for each skill
  • Develop and modify skill building instructional strategies based on youth’s skill strengths and growth areas. Data sourced from program/class level aggregate data for each skill
  • Develop and modify skill building project-based learning activities based on youth’s skill strengths and growth areas. Data sourced from program/class level aggregate data for each skill
  • Identify professional development programs that build instructional knowledge and strategies in areas where youth need more growth. Data sourced from program/class level aggregate data for each skill

MHA Labs Human Achievement for College and Career Recommendations:  Higher Education Uses>>

  • Implement a summative point-in-time” assessment of a youth’s skill level for the purpose of a higher education admissions recommendation
  • Providing written communication feedback to youth about their ratings and implications for entrance into higher education
  • Provide verbal communication feedback to youth about their ratings and implications for entrance into higher education
  • Provide youth with suggestions on college preparatory programs that align to their skills
  • Provide youth with suggestions on college scholarship programs that align to their skills

MHA Labs Human Achievement for College and Career Recommendations: Career Uses>>

  • Provide a summative point-in-time” assessment of a youth’s skill level for the purpose of entrance into a job or career path
  • Provide a summative point-in-time” assessment of a youth’s skill level for the purpose of a work-based learning opportunity or program
  • Provide a summative point-in-time” assessment of a youth’s skill level for the purpose of a job placement recommendation
  • Provide a summative point-in-time” assessment of a youth’s skill level for the purpose placement in a job-training program
  • Provide a summative point-in-time” assessment of a youth’s skill level for the purpose placement in a certification program
  • Provide youth suggestions on workforce development programs or training opportunities that align to their skills

MHA Labs Employee Appraisal>>

  • Provide verbal communication feedback to employees and interns about their ratings and implications for job performance
  • Providing written communication feedback to employees and interns about their ratings and implications for job performance
  • Provide data for recognizing and celebrating the success of an employee or intern’s job performance
  • Provide a summative “point-in-time” assessment of an employee or intern’s skill level for the purpose of an employee performance appraisal
  • Provide a summative point-in-time” assessment of an employee or intern’s skill level for the purpose of a job promotion
  • Provide a summative point-in-time” assessment of an employee or intern’s skill level for the purpose of a salary review
  • Provide data for planning and revising workforce development programming and supports

Section 4: Tracking Output

Youth experience increased access to opportunity based on the use of their 21st century skills performance…

MORE +

  1. Goal: Track how MHA Labs formative assessment data increases skill building supports for youth in their learning setting. Track how MHA Labs recommendation and employer appraisal data is effectively used to increase access to opportunity. Goal is to use MHA Labs recommendation data as a more predictive tool for education and job opportunities. For employers, goal is to use MHA employer appraisal data as a skill-based tool for promotion and advancement.
  2. Targets:
  • Increase in Skills Development Supports
  • Access to Enrichment Programs
  • Access to Selective Schools
  • Access to College
  • Access to Scholarships
  • Access to Internships
  • Access to Employment
  1. Impact Metric Overview: Increasingly, assessment data is used to either summatively assess a youth’s performance or measure growth over time to evaluate the impact of the learning environment. Yet most subject matter experts agree that the measurement of behavioral still is contextual and subjective. As a result, behavioral assessment data is not deemed a reliable source for measuring growth over time or for value-add analysis. Instead, MHA Labs prioritizes the use of assessment data for skills development and access to opportunity.
  1. Impact Metric Tracking Items:
  • Youth Outcomes Tracking System
  • Predictive Validity Studies
  • Longitudinal Research Studies

New University of Chicago – MHA Labs Impact Research Project: Assessing 21st Century Competencies for Urban Youth>>

Read more about MHA Labs outcomes research project with Lindsey Richland, University of Chicago Department of Comparative Human Development funded by the Hymen Milgrom Supporting Organization. 

This new research-practitioner collaboration will examine 21st Century skills, cognitive and non-cognitive competencies necessary for high quality employment in the 21st century, in urban youth.  The project aims are to better understand 1) the relations between cognitive and non-cognitive competencies and employment trajectories, 2) competency ratings and the context of evaluation, and 3) the role of evaluating competencies on a training program. Specifically, this research project will examine the role of assessing and training 21st Century Skills on ethnically diverse, economically disadvantaged youth’s employment and post-secondary schooling trajectories in the context of a novel internship program providing skills training and corporate IT employment for high school seniors (Genesys Works, http://www.genesysworks.org/twincities/).

Approximately 300 rising seniors selected to participate in Genesys Works in Minneapolis will be followed through an eight-week summer training program, after which some of the youth will be hired into one-year paid IT internship positions in major local and national corporations in an afterschool capacity. The research program will examine relations between a) 21st Century Skills as assessed by a range of people across three contexts: employers, teachers, and community members, b) Cognitive processing skills including working memory, executive function, and relational reasoning, and c) employment trajectory at three time-points: hiring decisions, employee review/ raise decisions, and post-program employment or post-secondary education persistence.  These scores also will be used to qualitatively examine how the use of free MHA tools for identifying and assessing key 21st century competencies (both formally and informally) impacted the training provided to individual students as measured by weekly program grades, written feedback comments, and performance on the technical curriculum and assessments.

Section 5: Measure Outcomes

All stakeholders use formative and summative 21st century skill assessment tools to inform skills development…

MORE +

  1. Goal: Track how many youth engaged in 21st century skills development programs meet their college, career and life milestones.
  2. Impact Metric Overview: MHA Labs will focus on tracking progress on established, measurable milestones that are proven to impact long term social and economic stability for youth, such as high school graduation, college completion and living wage employment pathways. Conversely, MHA Labs will also measure its toolkit’s ability to reduce disciplinary action and juvenile justice interactions which work to limit a youth’s achievement of positive milestones. MHA Labs will take into consideration the entire readiness skills portfolio of students when measuring to what extent 21st century skills were responsible for success. MHA Labs has been consistent in its messaging that 21st century skills success is part of a larger readiness equation that drives achievement in school, college, career and life. That 21st century skills are not sufficient in their own right but must be combined with competency in literacy, math, technical skills, and college and career navigation abilities. These four domains have been determined to be the primary components of college and career readiness in the current economy.
  3. Impact Metric Rationale: At this time, MHA Labs in constricting its outcome measures to widely accepted reliable measurement instruments, school data, college persistence data and employment data. MHA Labs needs to do this in order to successfully validate its own assessment tools that are more subjective in nature.  MHA Labs does support the stand-alone role of 21st century skills in the development of outcomes attributed to social emotional and other noncognitive factors. MHA Labs also endorses research findings that demonstrate the long term positive impact that social emotional competencies and attitudes have on future achievement.
  4. Targets
    • Successful transition into Kindergarten
    • On-Track with no achievement gap by 3rd Grade
    • Elementary On-Track
    • Successful transition from Elementary to High School
    • High School On-Track
    • High School Graduation
    • Attendance Rates
    • Disciplinary Actions
    • Juvenile Justice Interactions
    • Post-Secondary Enrollment
    • Post-Secondary Retention
    • Post-Secondary Completion
    • Job Placement
    • Job Retention
    • Job Advancement
    • Satisfaction with School and Program Experiences
    • Satisfaction with Career Choice
    • Satisfaction with Life Standing