Designing and Assessing Learning Outcomes: A Framework for Co-Curricular Sustainability Programs
Dilafruz R. Williams
Vicki L. Wise
Portland State University
Abstract: In recent years, Co-Curricular Sustainability Programs (CoCSPs) have been established at many higher education institutions. However, few such programs have developed learning outcomes or assessment processes to measure the types of learning that are occurring within their programs. This case study creates synthesis between sustainability education and student leadership competencies and shares a place-based framework for designing and assessing learning outcomes. By utilizing learning outcomes and assessment processes, leadership educators can strengthen the effectiveness of their sustainability programs and facilitate transformational learning experiences for students across campus.
Keywords: Learning outcomes, Co-Curricular Sustainability Programs, Assessment
In recent years, Co-Curricular Sustainability Programs (CoCSPs) have grown substantially in terms of size and impact in higher education. CoCSPs may be housed within a variety of different university departments that include facilities, academics, student affairs, and sustainability offices. Intended to complement classroom learning, CoCSPs offer opportunities for students to participate in sustainability-focused activities on campus and in their communities. Some examples of such experiences include Earth Day festivals, residence hall peer-to-peer education teams, volunteer programs, community service projects, garden work parties, committee service, and field trips. Through these Co-Curricular learning environments, students may participate in university activities such as implementing climate-action plan goals, helping to manage campus waste streams, or maintaining campus greenspaces (AASHE Education, 2012). Students who participate in CoCSPs include leaders of student organizations, volunteers, interns, graduate assistants, student employees, and any other students who participate in university-sponsored sustainability activities outside the classroom. Considering the wide range of students who participate in these activities, there is great potential to use intentional learning outcomes (LOs) to deepen the connection between sustainability and the student experience across campus.
In addition to supporting student learning through significant learning experiences, CoCSPs can benefit from designing LOs in many ways. These benefits include:
- ongoing enhancement and refinement of programs and activities;
- recording organizational growth and impact within the campus community over time;
- identifying how departmental initiatives fit into the mission of the university;
- providing opportunities to build partnerships and collaboration;
- gaining student perspectives of departmental initiatives to ensure that sustainability messages are relevant to the student community;
- collecting student success stories to share as part of internal and external sustainability communications;
- advertising of leadership experiences accurately, leading to greater participant satisfaction;
- clarifying departmental and organizational goals, priorities, and areas of expertise; and
- justifying program funding.
Developing LOs and, ultimately, assessment processes require an initial investment of time and energy, but these practices build the foundation for a more robust, resilient departmental system by connecting to the university’s educational mission. This case-study shows a place-based holistic approach to LO framework that allows departments to create outcomes that are based on research and relevant to their specific institutional context.
Co-Curricular Sustainability Program at Portland State University: A Case Study
The Sustainability Leadership Center (SLC) was launched as a CoCSP at Portland State University (PSU) in 2009 with the mission of integrating sustainability into the student life experience. With a densely populated, non-traditional urban campus and more than 28,000 students, PSU’s campus environment offers many diverse perspectives and opportunities for experiential learning. PSU’s motto is Let Knowledge Serve the City, and the institution prioritizes community based learning, civic engagement, diversity, and sustainability. Sustainability is one of PSU President’s initiatives as well as a Campus Wide Learning Outcome for undergraduate students. This organizational context provides a supportive and encouraging atmosphere for exploring sustainability leadership and Co-Curricular learning activities in the SLC and other campus programs.
The SLC is funded through the Institute for Sustainable Solutions, PSU’s hub for sustainability research, planning, and curriculum integration. Organizationally, the SLC reports to the Dean of Student Life within PSU’s Enrollment Management and Student Affairs (EMSA) division. This unique partnership and placement within the organization allows for seamless integration with other student services such as the Multicultural Center, Campus Recreation, Student Activities and Leadership Programs, Advising and Career Services, and Orientation programs (See Appendix A for organizational location of SLC). These relationships increase the diversity of program applicants and tie SLC initiatives to the university’s student life agenda. Because of strong partnerships across campus, the SLC is also able to recruit diverse program participants who represent many perspectives, majors, backgrounds, and skill sets. This cross-pollination creates a more valuable learning experience for all participants and strengthens the ability of the department to position sustainability leaders as integral and valuable members of the campus community.
The SLC is staffed by one full-time coordinator, one graduate assistant, six part-time student employees, and a rotating team of interns. Its departmental goals and vision have been influenced by PSU’s institutional context, feedback from program participants, the Institute for Sustainable Solutions, the campus student life agenda, and relevant academic programs such as the Leadership for Sustainability Education specialization within the Graduate School of Education. The SLC utilizes a holistic perspective of sustainability that integrates systems thinking, social justice, deep ecology, participatory processes, and wellness into its sustainability leadership philosophy. Because many students rotate in and out of the organization each academic year, it is important for SLC programs to be easily managed by students and to include effective assessment processes. These frequent assessments allow student program coordinators to receive feedback term-by-term in order to learn about their leadership styles and provide more intentional experiences for student participants.
The SLC participates in shaping PSU’s larger institutional sustainability agenda. For example, SLC representatives fill roles that include membership on the Sustainability Communications Committee, the Student Affairs Assessment Council, the Diversity Action Council, and various advisory boards. SLC staff also lead initiatives, host workshops, events and trainings that reach throughout the campus community. This case study focuses specifically on the design and assessment of LOs within the SLC’s core experiential programs as addressed in Figure 1.
Figure 1. Sustainability Leadership Center Core Programs
|Sustainability Leadership Center Core Programs||Description of Program|
|Sustainability Volunteer Program||20-25 students commit to 2-4 hours per week for least one term. Each Task Force collaboratively designs and implements a project based on topical areas:
|EcoReps||Up to 15 students in residence halls participate for 2-4 hours per week for one academic year:
|Student Sustainability Leadership Council||Up to 20 student representatives are selected each academic year as delegates from departments and student initiatives from across campus. This group cultivates and builds the student sustainability vision, provides feedback on ways to improve the student experience, and interacts with administration.|
|Field Trips||Offered quarterly, 15 student leaders from across campus visit and serve at community organizations that focus on sustainability. Preference is given to SLC participants.|
Together, these organic and evolving peer-to-peer experiential learning programs offer a “web of opportunities” for students to participate in and co-create collaborative intentional learning communities. For example, a student may initially apply to be an EcoRep because of an interest in making their residence hall building more sustainable or gaining leadership skills. This student may later choose to participate in the Cultural Sustainability Task Force to understand how social justice and equity relate to sustainability. After gaining confidence and forming additional connections in the university, the student may apply for an internship to learn about food systems or sit on the SLC’s Student Sustainability Leadership Council. The student has thus navigated and built a self-directed leadership experience based on their interests and strengths while contributing to the growth of the organization. Assessment is used for tracking students’ learning and individual success stories as they navigate their leadership journey through SLC activities and their academic studies.
Many SLC participants continue their leadership journeys into other positions on campus that include resident assistants, peer mentors, community garden coordinators, research positions, and student group leaders. As a result, other student leadership programs have identified unique connections between sustainability and their own departmental goals. For example, the Women’s Resource Center’s student-led EcoFemme Action Team has incorporated sustainability into the department’s mission and designed an annual series of events called Social Sustainability Month. Social Sustainability Month events tie environmental issues to topics such as gender equality, power and privilege, communication, and social justice. The Women’s Resource Center is just one example of how sustainability LOs relate to and can strengthen a variety of student leadership programs across campus.
Designing Learning Outcomes for Co-Curricular Sustainability Programs
What does it mean to be a well-rounded sustainability leader, and how can CoCSPs cultivate sustainability leadership competencies in students? Because sustainability programs focus on the relationships between a wide range of environmental, social, and economic issues, narrowing the possibilities to a list of core competencies and outcomes can initially be daunting. Students need to simultaneously gain comprehension about a variety of important sustainability issues, develop their own interests and intrinsic motivations, and build the skills necessary to design solutions to complex sustainability problems (Task Force, 2014). The SLC has catalogued our process for creating manageable steps to support leadership educators who may have varying levels of familiarity with LOs, assessment, student leadership, or sustainability competencies.
In 2011, the SLC participated in Enrollment Management and Student Affairs’ (EMSA) division-wide initiative around assessment. SLC’s process for creating LOs was supported step-by-step by the EMSA Assessment Director and connected with other leadership programs that were undergoing similar efforts. The SLC Coordinator attended multiple conferences and workshops on topics such as assessment, service learning, student leadership, LOs and social justice, through the American College Personnel Association (ACPA) and PSU. The SLC timeline for designing learning outcomes and assessment processes is described below.
- In winter 2011, the EMSA division-wide assessment initiative began.
- In spring 2011, the SLC Coordinator attended assessment trainings and reviewed student leadership and sustainability education literature.
- In summer 2011, the LO work group brainstormed possible competencies and researched institutional context.
- In fall 2011, the SLC student staff provided feedback on LOs; pilot-tested assessment to program participants.
- From spring through summer 2012, the Director of Assessment analyzed pilot-test data, and data was used to further refine programs, LOs and assessment instruments.
- In fall 2013, the SLC Coordinator made minor revisions to LOs and assessment instruments.
During the process of designing LOs, the SLC identified several key processes that can support the development of a comprehensive set of sustainability leadership competencies. Whereas each institution and program should consider their unique needs and goals while developing outcomes, the following guidelines can be helpful in navigating the complexities of compiling information from various sources, considering institutional context, and settling on a final set of competencies to implement in a CoCSP.
1. Use the design process as a learning experience. Establishing and measuring LOs is an ongoing process that benefits from revision over time as the organization changes and as additional research is published in the areas of student leadership and sustainability education. Leadership educators can utilize the iterative assessment cycle shown in Figure 2 to create a cycle of continuous inquiry, feedback, reflection, and application of learning to their assessment processes. Include student leaders in the process so they can gain practical skills in areas that include program development, leadership development, research, survey design, writing, and data analysis. Students and staff can also participate in tying activities and events to identified competencies. SLC student staff members gave presentations on LOs and assessment topics at conferences and were able to include these skills in their portfolios and resumes. In the LO generation process, four student staff were asked to participate based on their interest in shaping the future of the program, their knowledge of sustainability and student leadership, and the length of time spent working in the SLC. In later stages of the process, a larger group of students was included.
2. Convene a working group. Identify a small group that will support the process of designing competencies. This team may include the coordinator of the CoCSP, designated students, and an assessment coordinator. A supportive faculty partner who is skilled in curriculum design or a representative from Student Affairs who has a deep understanding of student leadership may also be valuable members of the working group.
3. Clarify purpose in designing learning outcomes. When designing learning outcomes, it is important to begin by specifying why outcomes are needed and how they will be used to strengthen the program. Identify the depth of experience that can realistically be offered to students given the department’s goals, resources, expertise and partnerships. How much time will students spend participating in the experience? What knowledge, experience and skills do the CoCSP’s leadership educators already hold, and how much investment can be made to gain additional skills? Detailed discussion about the institutional value of LOs is available in Learning Reconsidered 2: A Practical Guide to Implementing a Campus-wide Focus on the Student Experience (Keeling, 2006)
4. Begin with what is already known. Brainstorm possible LOs as a group. The goal at this point is to generate a variety of creative ideas that may complement, overlap, or even appear to contradict each other. Think about the attitudes, behaviors, knowledge and skills exhibited effective student leaders. Consider specific sustainability and leadership topics on which the program focuses. Encourage the team to use creative descriptive words and find interesting connections between concepts.
5. Align to institutional goals and vision. If the CoCSP’s supporting institution has articulated undergraduate campus wide learning outcomes, ensure that program LOs tie to each outcome in some way. Program LOs should take into consideration the mission, vision, and values of the institution. It is also valuable to consult with individuals who coordinate the student leadership and campus life agendas. These partnerships will likely be found within Student Affairs departments or community engagement programs. Ensure that the CoCSP’s LOs tie to these goals as well. Individuals working with sustainability curriculum can also ensure that the outcomes are tied to the institution’s sustainability literacy goals for graduates.
6. Find common themes across a variety of sources. In recent years, there has been increased interest in sustainability competencies, as evidenced by multiple published lists of sustainability competencies from a variety of sources and with various focal areas. When comparing compilations of LOs, many competencies will be duplicated or may only vary slightly. The SLC chose to consult competency lists that form a nexus between student leadership theory and sustainability education theory because of the CoCSP’s focus, goals and institutional context. From an extensive literature review, the following sources were chosen as the most relevant for developing LOs for CoCSPs.
a) Student Leadership Standards, The Council for the Advancement of Standards. These standards were created to support student leadership programs. They ensure that LOs are backed by decades of research on student development and leadership. Sustainability and systems-thinking concepts have been embedded throughout the standards. The CAS Standards include four themes or dimensions of leadership that build in complexity: Foundations of leadership, personal development, interpersonal development, and the development of groups, organizations, and systems (CAS, 2003).
b) Change Agent Skills Required for a Sustainable Future, American College Personnel Association (ACPA). ACPA separates the dimensions of student leadership for sustainability into three categories understanding, motivation, and skills. These categories depict how leadership characteristics should be built on a foundation of content knowledge. ACPA’s Change Agent Skills include characteristics such as resilience, optimism, patience, assertiveness, self-awareness, and curiosity. ACPA also contains a set of specific change agent skills that include the ability to:
- collaborate, network, develop alliances and coalitions, and build teams;
- adjust to the diverse and changing needs of both individuals and society as a whole;
- set realistic and clearly defined goals and objectives; and
- listen to others and incorporate their ideas and perspectives (Task Force, 2014).
c) The Five Ecoliterate Practices, Center for Ecoliteracy (2012). With the goal of “integrat[ing] emotional, social, and ecological intelligence,” the Five Ecoliterate Practices focus on understanding natural systems, interdependence, deep ecology, and systems thinking. These practices are:
- developing empathy for all forms of life;
- embracing sustainability as a community practice;
- making the invisible visible;
- anticipating unintended consequences; and
- understanding how nature sustains all life.
d) Relational Leadership Model. This model focuses on relationships as the key to effective leadership. The Relational Leadership Model is used in many student leadership programs because of its simplicity and its emphasis on equitable processes. The model also focuses on the value of “head, hands, and heart learning”, which is particularly relevant to experiential education programs. Relational leadership is ethical, inclusive, and process-oriented (Komives, 1998).
e) CoCSP leaders may choose to seek other sources of competencies that are relevant to their specific program or institutional context. For example, Wiek, Withycombe, and Redman (2010) offer key competencies in sustainability for academic program development. The Association for the Advancement of Sustainability in Higher Education also offers a variety of opportunities to learn more about academically-oriented competencies for sustainability courses (AASHE, 2014). As the fields of sustainability education and leadership continue to grow in higher education, additional compilations and lists of LOs are likely to be developed for specific learning contexts.
7. Compile competencies. Once relevant lists of outcomes have been identified, begin to use these sources to supplement the original brainstorm list that resulted from steps four and five. Analyze and connect overlapping concepts and connect them whenever possible while taking care to maintain the integrity of the sources and preserve the original intent of the competencies. Continue this refinement process until a concise list of competencies results.
8. Rank learning outcomes by stages of leadership identity development. If the CoCSP working group chooses, competencies can be ranked and connected to themes that reflect the stages of leadership development. For more information on the stages of identity students undergo while engaged in student leadership experiences, consult the Leadership Identity Development Model (Komives, Mainella, Longerbeam, Osteen & Owen, 2006).
By following the above steps, the SLC identified program LOs as seen in Figure 3:
Figure 3. Sustainability Leadership Center Learning Outcomes
Knowledge and Awareness
Building Relationships and Systems
Assessing Learning Outcomes
To measure learning, outcomes must be assessed. PSU’s EMSA division has created a cyclical model (see Figure 2) that depicts the iterative nature of the assessment process. These six steps allow CoCSPs to utilize assessment for ongoing reflection and improvement. The steps also share how results can be used to clarify organizational direction, which then revises outcomes and assessment. The value of transparency surrounds the entire assessment cycle. With time, assessment can be integrated into the flow of the academic year and become an integral part of organizational culture.
Schuh and Upcraft (2001) define assessment as “an any effort to gather, analyze, and interpret evidence which describes institutional, departmental, divisional or agency effectiveness” (p. 18). There are many methods of assessing learning outcomes. Some methods are direct measures of student learning where they demonstrate what they know and can do, while other methods are indirect assessment where students report perceived changes in attitudes and knowledge and skills. The most appropriate instrument depends on the learning outcome. Included are descriptions of various assessment methods we have use in our SLC programs, including surveys, mid-term reflection, legacy projects and transition binders, photos and artistic reflections.
Surveys. Surveys can be designed to collect both qualitative and quantitative data about students’ perceived learning experience and their changes in attitudes and beliefs. At the end of each term, the SLC survey measured students’ experiences using both quantitative and qualitative questions. Quantitative questions were measured on a 1-5 scale (1=not at all; 5=very much). A few examples are included.
- As a result of your experience, do you have a deeper understanding of the connections between ecological, social and economic systems?
- Can you more clearly imagine and strive for an inclusive and socially-just world that appreciates multiple perspectives of sustainability?
- Do you feel more connected to (name of institution)?
- Do you feel more confident as a role model for sustainable lifestyle choices?
- Did you have the opportunity to design a program or project?
To understand what changes occurred for students, the SLC survey also contained qualitative questions. Students were asked to report evidence that the program had an impact on their learning. For example, students were asked to share an example of a time when they felt they were a leader, and to share an experience in the term that deepened their understanding of sustainability.
Surveys can also offer an opportunity for students to give feedback on the program or provide general reflections about their experiences. The SLC administers a survey to all program participants at the last group meeting of each term and on all field trips with the use of smart phones, laptops, or paper. The SLC uses this feedback for program improvements. In addition, student quotes from these reflections can be used in program brochures, websites, marketing materials, and when justifying program funding.
Journaling. The SLC student staff creates small journals from recycled paper to use for guided reflection at team meetings. Group sessions often begin with a check in question that gently prompts students to reflect on their learning goals and leadership experiences by written reflection and a brief group discussion. Journal questions prompt students to share their learning and growth processes. The SLC staff asks questions such as: What challenges have you overcome in your role? What new partnerships and networks you have created? How does sustainability relate to your educational and career goals? What is your leadership style?
Mid-term reflections. Student staff and program participants benefit from being reminded of their LOs and goals midway through the term to ensure that program activities are aligned with outcomes. Reflections also allow students to see their own progress and encourage them to speak up if their needs are not being met. This practice also refocuses students on the intended learning outcomes of the program.
Legacy projects and transition binders. Student interns and staff are strongly encouraged to catalog their work, both in terms of their tangible output, as well as the lessons they learned during their time in the position. Most group projects within the SLC are also designed with the intention of leaving either a measurable outcome or finished product to mark the completion of a leadership experience. This practice allows institutional knowledge to be permanently recorded and provides an opportunity for students to identify and reflect on their learning. In the SLC, legacy binders have been created for the Earth Days Festival, the EcoReps program, Farmworker Awareness Week, and Social Sustainability Month. Program participants are also encouraged to write short blog posts about their projects and experiences. These are published online with editorial support from colleagues in the Institute for Sustainable Solutions. Posts have reflected upon field trips, events, campaigns, and specific sustainability topics that include fair trade and composting.
Photos. Photos from events can depict students engaged in activities and participating as a team. These photos can be included in grant applications, annual reports, presentations, printed materials, and websites. Examples of photo opportunities include students engaged in group discussions, planting gardens, leading events, hosting food drives, taking a sustainability pledge, and creating useful items from re-purposed materials.
Artistic reflection. This assessment method allows students to explore their leadership identity and experiences through the use of concepts, colors, symbols and other creative elements. During a spring Art in the Garden event, the SLC Cultural Sustainability Task Force provided students with blocks of re-purposed wood, paint, magazines, and other art supplies and asked the question “Where are you from?” This question was intentionally open-ended and allowed students to answer the question in whatever way personally resonated with them. The collected pieces of art were displayed together in a central location within the sustainability office suite along with a description of the project and its goals.
Case Study Results and Findings
Assessment allowed the SLC to identify the types of learning that were occurring as a result of its activities and leadership development processes. This feedback allowed the SLC Coordinator and each student coordinator to receive data from the students they mentor and set goals for improvement. Data were also aggregated to give feedback on SLC programs as a whole, term-by-term, for the entire academic year, and for multiple years. Staff gained confidence and learned about their own leadership styles since they received direct feedback from program participants.
SLC learned through data review that the most effective learning outcomes–where students reported the greatest changes as a result of the program—were:
- building strong support networks and community;
- becoming role-models for sustainable lifestyle choices;
- gaining practical knowledge of critical global issues that can be put into action at the local and organizational level;
- increasing engagement and connection with place by leaving a legacy;
- feeling supported in balancing priorities and maintaining appropriate workload; and
- increasing their autonomy and self-determination.
The data also showed us areas of needed improvement. For example, students reported lower levels of:
- deep understanding of interdependence between social, ecological, and economic issues;
- exposure to a variety of sustainability initiatives and topics;
- practice using collaborative facilitation and group dynamic skills; and the
- ability to imagine and strive for an inclusive and socially-just world that appreciates multiple sustainability perspectives.
Designing and assessing learning outcomes can strengthen the effectiveness and impact of Co-Curricular Sustainability Programs and help create significant learning experiences for students. The framework presented offers a process for compiling student leadership and sustainability education theories and competencies into a place-based approach for designing useful learning outcomes and assessments. Even though the learning outcomes described here are tailored specifically for co-curricular programs, they may also be relevant for designing experiential classroom activities and other learning experiences. Learning outcomes for Co-Curricular Sustainability Programs have the potential to connect sustainability to the student life experiences on campuses and cultivate well-rounded sustainability leaders who have the necessary competencies to find solutions to complex sustainability challenges.
Assessment Planning and Practice (2014). Portland State Enrollment Management & Student Affairs. http://www.pdx.edu/studentaffairs/assessment-planning-and-practice
AASHE (2014). Education and research resources. Retrieved from Association for the Advancement of Sustainability in Higher Education websitehttp://www.aashe.org/resources/education-research-resources/
Council for the Advancement of Standards in Higher Education – CAS (2003). The role of student leadership programs: CAS standards contextual statement. http://www.cas.edu/getpdf.cfm?PDF=E86F4088-052E-0966-ADCB25F2A9FE7A70
Center for Ecoliteracy (2012). The five ecoliterate practices. Retrieved from http://www.ecoliteracy.org/essays/five-ecoliterate-practices
Keeling, R. P. (Ed.). (2006). Learning reconsidered 2: A practical guide to implementing a campus-wide focus on the student experience. Washington, D.C.: NASPA.
Komives, S, Lucas, N., & McMahon, T. (1998). Exploring leadership: For college students who want to make a difference. San Francisco: Jossey-Bass.
Komives, S. R., Mainella, F.C., Longerbeam, S.D., Osteen, L. & Owen, J.E. (2006). A leadership identity development model: Applications from a grounded theory. Journal of College Student Development, 47(4), 401-402.
Schuh, J. H., and Upcraft, M.L. (2001). Assessment practices in student affairs: An applications manual. San Francisco: Jossey-Bass.
Task Force on Sustainability Change Agent Skills (2014). ACPA-College student educators international. http://www.myacpa.org/files/changeagentskillspdf
Wiek, A., Withycombe, L., & Redman, C. (2011). Key competencies in sustainability: a reference framework for academic program development. Sustainability Science, 6(2), 203-218.
Appendix A: Organizational Location of Student Leadership for Sustainability at Portland State University