New Directions: Analytics and Measuring and Improving Performance
Note that this essay is one of twenty (20) New Directions in Planning essays, which are an online part of and a companion to the SCUP book, A Guide to Planning for Change (Norris and Poulton, 2008). The essays will eventually be available in their own Web home, but we are sharing them here, now, because we want you to have access to them sooner and because this blog environment will let you post comments, if you wish. Please do!
Analytics and Measuring and Improving Performance is written by Donald M. Norris of Strategic Initiatives.
A Guide to Planning for Change can be ordered in SCUP's online bookstore or via the order form you can download here (PDF). Why don't you purchase a copy (for yourself or a colleague) and spend the down time over the holidays refreshing your perspective on higher education planning?
Copyright SCUP, 2008, all rights reserved.
Rigorous measurement and analytics have always been essential to all aspects of successful integrated planning. Recently, growing pressures for performance improvement have elevated the importance of assessment, accreditation, and institutional effectiveness. The current environment of accountability is changing the politics and practices of planning. Colleges and universities are moving not just to “cultures of evidence”, but to “cultures of performance measurement and improvement.” To support these developments, new knowledge sharing technologies, analytical practices, and toolkits have emerged: Key performance indicators, dash boards, balanced scorecards, strategy maps, and performance-based funding. Institutions are deploying a new breed of “action analytics” that drive decision making and active interventions to enhance student success. The cost of analytics will decline and their availability will spread, providing “analytics for the masses.”
In the world of strategic planning and strategy execution, emerging applications for alignment and analytics will be inextricably fused together. Alignment applications will support planners in “setting context, keeping track of progress, and holding people responsible for results.” Analytic applications will blend with alignment, focusing on “setting targets and keeping score (in context), modeling improvements, and enabling interventions and refinements.” Taken together, these tools and applications will empower planners and executers of strategy to align and leverage strategies and actions in ways that have not been possible.
I. What Forces Are Driving Analytics and Performance Measurement
in Higher Education?
Three primary forces are driving higher education analytics: 1) New technologies and solutions that enable new levels of knowledge sharing and analytics, 2) pressures for improved performance in the face of global competition and political visibility, and 3) growing awareness that institutions have substantially suboptimized their data, information, and analysis resources.
First, knowledge-leveraging and analytical practices are advancing in sophistication and proliferation. This has been aided by a host of new software and professional services offerings. These include deploying academic analytics (tools, solutions, and services) to produce actionable intelligence, service-oriented architectures, mash-ups of information/content and services, proven models of course/curriculum reinvention, and changes in faculty practice that improve performance and reduce costs. The open-architecture nature of these solutions enables information planners to extract and share data, information, and knowledge in ways that have previously proven impossible.
Over time, these new offerings will likely support previously unattainable levels of measurement, comparison, and institutional interventions to improve access, affordability, and success for students.
Second, public demand in the United States is escalating for colleges and universities to measure, demonstrate, and improve performance and to provide access to this data. This demand is being driven by a variety of forces and interests. The most compelling is the stark fact that the international standing of the United States is slipping. In spite of the relative dominance of U.S. leading universities and their world-class reputations, the nation is losing ground in terms of the overall educational attainment of its population. The United States is also deficient in the across-the-economy-and-workforce competencies necessary for success in the global economy. Additionally, mid-tier institutions are increasingly at risk of falling behind U.S. and international competitors in their ability to track their performance and identify areas where they need curriculum and process reinvention and innovation. As a consequence, the United States faces projected declines in per capita income and economic competitiveness.
Third, most colleges and universities are awash in data. Unfortunately, they lack the capacity to seamlessly turn their data into meaningful information. Nor can they easily access, combine, and repurpose that information to support analysis, drive decision making, and improve student success. In many cases, the data they need are hiding in plain sight. They are frozen in place by sub-optimized data models (i.e., either ERP databases or ineffective data warehouse designs), by difficult-to-use reporting tools, and/or by the proprietary nature of some systems. Rather than crafting analytical applications that create optimized data, information, and analysis environments, most colleges and universities have focused on individual analytic tools, with disappointing results. It’s time to change perspectives by focusing on analytic applications that optimize data, information, and analytics resources.
The impact of these drivers will increase in the near- and long-term future.
II. What Are the Emerging New Directions in Analytics and Performance Measurement?
SCUP has a number of core resources by Hollowell. Middaugh, and Sibolski and Nedwek (1) that are essential to understanding how the measurement and improvement of performance can be integrated into campus planning processes. The new directions in analytics fit well into these frameworks, but will dramatically extend organizational capacity.
Open-Architecture Analytics and Optimized Data/Information/Analytics. A series of articles in the EDUCAUSE Review and other sources (2) have described the emergence of a new generation of analytics based on open architecture applications and Web 2.0 practices. These new analytics enable planners to escape the limitation of the “stack” of administrative ERP and legacy systems and other proprietary limitations.
Planners can extract data, information, and knowledge (information in context) from a wide range of sources: administrative and academic ERP, assessment, institutional alignment, and external information services. The new analytics are based on optimization of data resources through careful data modeling/mapping and the optimal integration of data warehouses, extract transfer and load (ETL), online analytical processing (OLAP), and business intelligence (BI) tools. These data can be analyzed in a “cloud” of open-architecture applications and viewed through a presentation layer consisting of dashboard, portfolios, and/or portals.
Using these tools enables institutions to leap from static reporting and limited dash boarding to dynamic viewing of data, drawing on a virtually unlimited range of options. Decision makers can use dashboards to view the overall status of indicators in real time, change the variables on the fly, then drill down into examine the detail on groups of students, faculty, projects, or other measures of interest. These tools can be used to create alerts on students at risk, then intervene through intrusive advisement and actions to support students needing assistance.
These combinations of open-architecture tools, data modeling/mapping services, and solutions are styled “action analytics” because they enable a new level of information-based decisions, interventions, and actions to improve performance. These tools will be available not just to power users, but to grassroots staff and faculty needing contextualized information to support decision making and actions. One of the driving forces of analytic applications will be “analytics for the masses.”
To achieve these goals, institutions will need to do a better job of optimizing their data, information, and analytics environments. Too many institutions have assumed that their data and information were available for analysis when in reality they were stuck in suboptimized data structures that limited extraction, combination, and analysis. Such data were “hiding in plain sight.”
Key Performance Indicators and Dashboards. Key performance indicators (KPIs) and executive dashboards have been part of the planner’s toolkit for many years (3). They have been used actively in campus-wide strategic planning and strategic enrollment management (SEM) and academic planning. The new generation of open-architecture action analytics will greatly accelerate the use of KPIS and dash boarding and will make them much more dynamic and many faceted.
Balanced Scorecards and Strategy Maps. Kaplan's and Norton’s books on balanced scorecards and strategy maps (4) have taken the for-profit world by storm. Over time, they have changed the practice of planning and executing strategies that are aligned with the mission, vision, and targeted outcomes of enterprises. These principles and practices are being applied by a number of leading-edge colleges and universities.
As institutions leverage action analytics, they will likely turn to balanced scorecards and strategy maps to align strategies, actions, outcome targets, and measurement. In What Every Campus Leader Needs to Know About Analytics, Norris and Leonard have provided a checklist for campuses to use in assessing their campus data, information, and analytics environments and the “analytics IQ” of their campus leadership team.
Aligning Learning and Work. Among the greatest potentials for open-architecture analytics will the increased sharing and analysis of data, information, and knowledge between PK-12, PSE, and employment/workforce settings (5). These analytics will improve preparation for work and transitions among and between PK-12, PSE, and workforce settings.
III. How Will These New Directions Affect Integrated, Strategic, Aligned Planning?
Open-architecture analytics will enhance the capacity of campus planers of all kinds to integrate and align planning and the execution of strategy. These practices will enable planners and strategist to extract and combine data from internal and external data sets that have previously been isolated “silos”. Data that are now “hiding in plain sight” can be integrated into a strategic analytics capability. Advanced analytics will dramatically accelerate the acceptance of and time frames for evidence-based decision making.
These principles are discussed in greater detail on pages 17-32 of Chapter 2: Integrated, Strategic, Aligned Planning in A Guide to Planning for Change. Three figures in this chapter as especially instructive in understanding the potential of advanced analytics:
• Figure 2.4: Information/Analytics Support for Different Types of Planning and Decision Making;
• Figure 2.7: Aligning Planning, Accreditation, Continuous Improvement, and Performance Measurement; and
• Figure 2.8 Combining Alignment and Analytic Applications.
The following figure summarizes the jump shift that will be achieved by tomorrow’s action analytics, used in support of integrated, strategic, aligned planning and decision.
Perhaps the most potentially transformative element is making intuitive, dynamically changing analytics available to front-line faculty and staff. This is likely to enable colleges and universities to change responsibilities, reshape job descriptions, and change administrative and academic support policies and practices.
IV. Resources
The following resources are critical to understanding analytics, performance measurement, and improvement.
1) Core Resources on Measurement and Performance in Planning:
3) Key Performance Indicators and Dashboards
Links to online examples of Key Performance Indicators and Dashboards. Examples:
• Minnesota State Colleges And Universities (MnSCU)
• University of Miami
• Others to be provided
4) Balanced Scorecard, Strategy Maps
Links to online examples of Balanced Scorecard/Strategy Maps to be provided. Examples:
• University of Washington
• Rochester Community and Technical College
• University of Fort Hare (South Africa)
• Curtin University of Technology (Australia)
• Others to be provided
5) Aligning Work and Learning
6) Case Study Descriptions of Analytics
Contact
Donald M. Norris, Ph.D.
President, Strategic Initiatives, Inc.
dmn@strategicinitiatives.com
(703) 450-5255
Analytics and Measuring and Improving Performance is written by Donald M. Norris of Strategic Initiatives.
A Guide to Planning for Change can be ordered in SCUP's online bookstore or via the order form you can download here (PDF). Why don't you purchase a copy (for yourself or a colleague) and spend the down time over the holidays refreshing your perspective on higher education planning?
Copyright SCUP, 2008, all rights reserved.
Chapter 6: Analytics and Measuring and Improving Performance
Rigorous measurement and analytics have always been essential to all aspects of successful integrated planning. Recently, growing pressures for performance improvement have elevated the importance of assessment, accreditation, and institutional effectiveness. The current environment of accountability is changing the politics and practices of planning. Colleges and universities are moving not just to “cultures of evidence”, but to “cultures of performance measurement and improvement.” To support these developments, new knowledge sharing technologies, analytical practices, and toolkits have emerged: Key performance indicators, dash boards, balanced scorecards, strategy maps, and performance-based funding. Institutions are deploying a new breed of “action analytics” that drive decision making and active interventions to enhance student success. The cost of analytics will decline and their availability will spread, providing “analytics for the masses.”
In the world of strategic planning and strategy execution, emerging applications for alignment and analytics will be inextricably fused together. Alignment applications will support planners in “setting context, keeping track of progress, and holding people responsible for results.” Analytic applications will blend with alignment, focusing on “setting targets and keeping score (in context), modeling improvements, and enabling interventions and refinements.” Taken together, these tools and applications will empower planners and executers of strategy to align and leverage strategies and actions in ways that have not been possible.
I. What Forces Are Driving Analytics and Performance Measurement
in Higher Education?
Three primary forces are driving higher education analytics: 1) New technologies and solutions that enable new levels of knowledge sharing and analytics, 2) pressures for improved performance in the face of global competition and political visibility, and 3) growing awareness that institutions have substantially suboptimized their data, information, and analysis resources.
First, knowledge-leveraging and analytical practices are advancing in sophistication and proliferation. This has been aided by a host of new software and professional services offerings. These include deploying academic analytics (tools, solutions, and services) to produce actionable intelligence, service-oriented architectures, mash-ups of information/content and services, proven models of course/curriculum reinvention, and changes in faculty practice that improve performance and reduce costs. The open-architecture nature of these solutions enables information planners to extract and share data, information, and knowledge in ways that have previously proven impossible.
Over time, these new offerings will likely support previously unattainable levels of measurement, comparison, and institutional interventions to improve access, affordability, and success for students.
Second, public demand in the United States is escalating for colleges and universities to measure, demonstrate, and improve performance and to provide access to this data. This demand is being driven by a variety of forces and interests. The most compelling is the stark fact that the international standing of the United States is slipping. In spite of the relative dominance of U.S. leading universities and their world-class reputations, the nation is losing ground in terms of the overall educational attainment of its population. The United States is also deficient in the across-the-economy-and-workforce competencies necessary for success in the global economy. Additionally, mid-tier institutions are increasingly at risk of falling behind U.S. and international competitors in their ability to track their performance and identify areas where they need curriculum and process reinvention and innovation. As a consequence, the United States faces projected declines in per capita income and economic competitiveness.
Third, most colleges and universities are awash in data. Unfortunately, they lack the capacity to seamlessly turn their data into meaningful information. Nor can they easily access, combine, and repurpose that information to support analysis, drive decision making, and improve student success. In many cases, the data they need are hiding in plain sight. They are frozen in place by sub-optimized data models (i.e., either ERP databases or ineffective data warehouse designs), by difficult-to-use reporting tools, and/or by the proprietary nature of some systems. Rather than crafting analytical applications that create optimized data, information, and analysis environments, most colleges and universities have focused on individual analytic tools, with disappointing results. It’s time to change perspectives by focusing on analytic applications that optimize data, information, and analytics resources.
The impact of these drivers will increase in the near- and long-term future.
II. What Are the Emerging New Directions in Analytics and Performance Measurement?
SCUP has a number of core resources by Hollowell. Middaugh, and Sibolski and Nedwek (1) that are essential to understanding how the measurement and improvement of performance can be integrated into campus planning processes. The new directions in analytics fit well into these frameworks, but will dramatically extend organizational capacity.
Open-Architecture Analytics and Optimized Data/Information/Analytics. A series of articles in the EDUCAUSE Review and other sources (2) have described the emergence of a new generation of analytics based on open architecture applications and Web 2.0 practices. These new analytics enable planners to escape the limitation of the “stack” of administrative ERP and legacy systems and other proprietary limitations.
Planners can extract data, information, and knowledge (information in context) from a wide range of sources: administrative and academic ERP, assessment, institutional alignment, and external information services. The new analytics are based on optimization of data resources through careful data modeling/mapping and the optimal integration of data warehouses, extract transfer and load (ETL), online analytical processing (OLAP), and business intelligence (BI) tools. These data can be analyzed in a “cloud” of open-architecture applications and viewed through a presentation layer consisting of dashboard, portfolios, and/or portals.
Using these tools enables institutions to leap from static reporting and limited dash boarding to dynamic viewing of data, drawing on a virtually unlimited range of options. Decision makers can use dashboards to view the overall status of indicators in real time, change the variables on the fly, then drill down into examine the detail on groups of students, faculty, projects, or other measures of interest. These tools can be used to create alerts on students at risk, then intervene through intrusive advisement and actions to support students needing assistance.
These combinations of open-architecture tools, data modeling/mapping services, and solutions are styled “action analytics” because they enable a new level of information-based decisions, interventions, and actions to improve performance. These tools will be available not just to power users, but to grassroots staff and faculty needing contextualized information to support decision making and actions. One of the driving forces of analytic applications will be “analytics for the masses.”
To achieve these goals, institutions will need to do a better job of optimizing their data, information, and analytics environments. Too many institutions have assumed that their data and information were available for analysis when in reality they were stuck in suboptimized data structures that limited extraction, combination, and analysis. Such data were “hiding in plain sight.”
Key Performance Indicators and Dashboards. Key performance indicators (KPIs) and executive dashboards have been part of the planner’s toolkit for many years (3). They have been used actively in campus-wide strategic planning and strategic enrollment management (SEM) and academic planning. The new generation of open-architecture action analytics will greatly accelerate the use of KPIS and dash boarding and will make them much more dynamic and many faceted.
Balanced Scorecards and Strategy Maps. Kaplan's and Norton’s books on balanced scorecards and strategy maps (4) have taken the for-profit world by storm. Over time, they have changed the practice of planning and executing strategies that are aligned with the mission, vision, and targeted outcomes of enterprises. These principles and practices are being applied by a number of leading-edge colleges and universities.
As institutions leverage action analytics, they will likely turn to balanced scorecards and strategy maps to align strategies, actions, outcome targets, and measurement. In What Every Campus Leader Needs to Know About Analytics, Norris and Leonard have provided a checklist for campuses to use in assessing their campus data, information, and analytics environments and the “analytics IQ” of their campus leadership team.
Aligning Learning and Work. Among the greatest potentials for open-architecture analytics will the increased sharing and analysis of data, information, and knowledge between PK-12, PSE, and employment/workforce settings (5). These analytics will improve preparation for work and transitions among and between PK-12, PSE, and workforce settings.
III. How Will These New Directions Affect Integrated, Strategic, Aligned Planning?
Open-architecture analytics will enhance the capacity of campus planers of all kinds to integrate and align planning and the execution of strategy. These practices will enable planners and strategist to extract and combine data from internal and external data sets that have previously been isolated “silos”. Data that are now “hiding in plain sight” can be integrated into a strategic analytics capability. Advanced analytics will dramatically accelerate the acceptance of and time frames for evidence-based decision making.
These principles are discussed in greater detail on pages 17-32 of Chapter 2: Integrated, Strategic, Aligned Planning in A Guide to Planning for Change. Three figures in this chapter as especially instructive in understanding the potential of advanced analytics:
• Figure 2.4: Information/Analytics Support for Different Types of Planning and Decision Making;
• Figure 2.7: Aligning Planning, Accreditation, Continuous Improvement, and Performance Measurement; and
• Figure 2.8 Combining Alignment and Analytic Applications.
The following figure summarizes the jump shift that will be achieved by tomorrow’s action analytics, used in support of integrated, strategic, aligned planning and decision.
Perhaps the most potentially transformative element is making intuitive, dynamically changing analytics available to front-line faculty and staff. This is likely to enable colleges and universities to change responsibilities, reshape job descriptions, and change administrative and academic support policies and practices.
IV. Resources
The following resources are critical to understanding analytics, performance measurement, and improvement.
1) Core Resources on Measurement and Performance in Planning:
- Hollowell, D., M.F. Middaugh, and E. Sibolski, Integrating Higher Education Planning and Assessment: A Practical Guide, SCUP, 2006.
- Nedwek, B.“Linking Quality and Accountability,” in Doing Academic Planning: Effective Tools for Decision Making, Brian P. Nedwek (ed), Ann Arbor, Society for College and University Planning, 1997, pp. 137-145.
- Norris, D.M. and L. Baer, “Action Analytics, Alignment, and Strategic Planning,” Concurrent Session, SCUP Annual Conference, Montreal, CA, July 23, 2008.
- Norris, D.M. and J. Leonard, "What Every Campus Leader Needs to Know About Analytics," White Paper, March 7, 2008.
- Norris, D.M., L. Baer, J. Leonard, L. Pugliese, and P. Lefere, “Action Analytics: Measuring and Improving Performance That Matter in Higher Education,” EDUCAUSE Review, Jan/Feb 2008. http://www.educause.edu/ir/library/pdf/ERM0813.pdf and "Framing Action Analytics and Putting Them to Work," EDUCAUSE Review, Jan/Feb 2008 (electronic only).http://connect.educause.edu/lib/EDUCAUSE+Review/FramingActionAnalyticsand/45826, EDUC.
- Davenport, T.H. and J. C. Harris, "Competing on Analytics: The New Science of Winning." Boston: Harvard Business School Press, 2007.
- Campbell, J.P, P. B. DeBlois, and D. G. Oblinger, “Academic Analytics: A New Tool for a New Era”; July/August 2007 issue of EDUCAUSE Review (vol. 42, no. 4).
Graves, W.H. “Voluntary Counter-Reformation: Stepping Up to the Challenge,” July/August 2007 issue of EDUCAUSE Review (vol. 42, no. 5). Norris, D.M., P. Lefere, and J. Mason, “Making Knowledge Services Work in Higher Education,” EDUCAUSE Review, Sept/Oct 2006 (vol. 41, no 4). Graves, W.H. “Improving Institutional Performance through IT-Enabled Innovation,” EDUCAUSE Review, vol. 40, no. 6 (November/December 2005), pp. 78–99. Eckerson, W.W, "The Rise of Analytics Applications: Build or Buy," White Paper, The Data Warehouse Institute, 2002.
Sapp, M. article from SCUP 42 (To be provided) Howard, R.D. and K. W. Borland, Jr. (ed), “Balancing Qualitative and Quantitative Information for Effective Decision Support, “ New Directions for Institutional Research, No 112, San Francisco, Jossey-Bass, 2001. Dolence, M.G, D.J. Rowley, and H.D. Lujan, "Working Toward Strategic Change: A Step-By-Step Guide to the Planning Process," San Francisco: Jossey-Bass, 1997. Dolence, M.G. and D.M. Norris, “Using Key Performance Indicators to Drive Strategic Decision Making,” in New Directions for Institutional Research, Using Performance Indicators to Drive Strategic Decision Making, V.M.H. Borden and T.W., Banta, Number 82, Summer 1994.
• Minnesota State Colleges And Universities (MnSCU)
• University of Miami
• Others to be provided
4) Balanced Scorecard, Strategy Maps
Bally, R, “Balanced Scorecard Deployment for Integrated Planning in Higher Education,” Concurrent Session Presentation at SCUP Annual Conference, July 22, 2008. Kaplan, R.S. and Norton, D.P, “How to Implement a New Strategy Without Disrupting Your Organization,” Harvard Business Review, December 2006. Scholey, C. and H. Armitage, “Hands-on Scorecarding in the Higher Education Section,” Planning for Higher Education, Vol. 35, No. 1, October-December 2006. Kaplan, R.S. and D. P. Norton, "Strategy Maps: Turning Intangible Assets into Tangible Outcomes," Boston: Harvard Business School Press, 2004. O’Neil, H.F. Jr., E. M. Bensimon, M. A. Diamind, and M.R. Moore, “Designing and Implementing an Academic Scorecard,” Change, November/December 1999, pp. 33-40. Kaplan, R.S. and D. P. Norton, "The Balanced Scorecard: Turning Strategy into Action," Boston: Harvard Business School Press, 1996.
• University of Washington
• Rochester Community and Technical College
• University of Fort Hare (South Africa)
• Curtin University of Technology (Australia)
• Others to be provided
5) Aligning Work and Learning
Norris, D.M. L. Baer, J. Leonard, L. Pugliese, and P. Lefere, “Action Analytics: Measuring and Improving Performance That Matter in Higher Education,” EDUCAUSE Review, Jan/Feb 2008, p. 5-12.
Norris, D.M. L. Baer, J. Leonard, L. Pugliese, and P. Lefere, “Action Analytics: Measuring and Improving Performance That Matter in Higher Education,” EDUCAUSE Review, Jan/Feb 2008, p. 5-12. Examples from Minnesota State Colleges and Universities, Capella University, University of Toledo, Indiana University. Campbell, J.P, P. B. DeBlois, and D. G. Oblinger, “Academic Analytics: A New Tool for a New Era”; July/August 2007 issue of EDUCAUSE Review (vol. 42, no. 4) . Examples from Purdue University, Baylor University, Sinclair Community College, Northern Arizona University. Coppin State University Case Study Cuyahoga Community College Case Study Colorado State University Case Study University of Maryland Eastern Shore Case Study
Donald M. Norris, Ph.D.
President, Strategic Initiatives, Inc.
dmn@strategicinitiatives.com
(703) 450-5255
posted by Terry Calhoun at
10:31 AM
0 Comments:
Post a Comment
Subscribe to Post Comments [Atom]
<< Home