WebFOCUS Streamlines the Curriculum at the University of North Texas Health Science Center
Snapshot
| Organization The University of North Texas Health Science Center, which trains several types of health professionals, injecting more than $244 million into the Texas economy each year. A 110-member physician group manages 190,000 patients each year. The center also conducts research through six Institutes for Discovery. |
| The Challenge To streamline the process of collecting and distributing course evaluation results to various constituencies. |
| The Strategy Web-based collection, analysis, and distribution of information that can be entered and retrieved from virtually any location. |
| The Results Dramatic reduction in the time required to process and deliver evaluation results – from days to seconds. |
| Information Builders Solution WebFOCUS, Information Builders Consulting, iWay Software. |
Attending medical school is an huge commitment, causing medical students to plan their time very carefully. Thanks to Information Builders, the task of evaluating courses has become much easier at the University of North Texas Health Science Center (HSC). IT pros at HSC used Information Builders' WebFOCUS and iWay Software to create an Internet-based course assessment program that delivers customized reports tailored to individual needs.
"Many people depend on our department for information, including students, instructors, course directors, department chairs, deans – all the way up to the president of the institution," explains Jerry H. Alexander, Ph.D., Associate Professor of Medical Education and Director of Academic Information Services at HSC. "By offering candid assessments and detailed feedback on the curriculum, students can quickly share insights to help us maintain and enhance curriculum quality."
The University of North Texas Health Science Center is dedicated to the advancement of all three disciplines of medical science: education, research, and patient care. The 15-acre, $71 million complex is located in the Cultural District of Fort Worth, Texas. The institution has a combined faculty of more than 200, a staff of 900, and a cadre of 300 volunteer community physicians.
Alexander works in Academic Information Services in the Department of Medical Education, the department responsible for academic computing. Its mission is to support the HSC's commitment to excellence in education by providing a wide range of academic information support services to a broad constituency of the center's administrators, faculty, staff, and students. In practice, that takes the form of extensive testing services, external examinations, information services, and statistical services. "We generate a lot of data daily," says Alexander.
Diagnosing the Requirements
The HSC has always given students the opportunity to evaluate instructors and courses. In the past, however, course evaluations were tallied and distributed using paper reports that had to be printed, sorted, packaged, labeled, and mailed. "It was slow and cumbersome," admits Alexander. "We needed to streamline the data collection process so we could get evaluation results to our various constituencies in a more timely manner."
Academic Information Services considered a number of turnkey reporting solutions and several report development tools before settling on WebFOCUS. "We are a creative team that designs and develops much of our own software," continues Alexander. "We have lots of experience with Information Builders' technology. When we learned about WebFOCUS, we were excited at the prospect of migrating and enhancing our existing software and expertise. We recognize the leadership Information Builders has exhibited in the 1980s and 1990s, and we trust that the company will remain state-of-the-art."
"Information Builders' offerings ranked above other business intelligence tools in several categories," Alexander recalls. "We liked the robust reporting capabilities, such as the ability to create custom reports and to dynamically color-code output variables based upon predetermined values. Much of our output is graphical, which is why we also liked the embedded graphic capabilities so we don't have to export the data to another vendor's product. The ability to drilldown on values for purposes of elaboration was another attractive feature."
HSC turned to iWay Software, an Information Builders company, to access data from various sources. In addition, HSC enlisted help from Information Builders Consulting for the initial migration from DOS to Microsoft Windows NT. The organization also purchased a training package of advanced reporting techniques. "We had limited experience with HTML and we learned as we went along," says Alexander. "WebFOCUS was relatively straightforward to learn."
Building the System
The development team quickly gained proficiency with the WebFOCUS software. Using this new expertise, they set out to design and implement Quality, a system to collect and distribute course and instructor evaluation data in real time via the Internet. There were two phases to the project: collection of the data from students and distribution of the data to the various constituencies.
The collection process was tackled first. According to Alexander, this was no small undertaking. Unlike traditional undergraduate education, where there is one instructor per course, most medical school courses have 10 to 20 instructors. The collection process is further complicated by medical students in their third and fourth years of study performing medical clerkships at hundreds of locations throughout the country. "The paperwork can be overwhelming," Alexander says.
Developers designed Web forms for inputting course evaluation data. This way, instead of filling out a paper form that had to be manually tallied, students could evaluate courses and clerkships by responding to a series of online questions. The developers also created a photo gallery of instructors, to make it easy for students to identify all the instructors who taught a particular course.
"Quality was designed to bridge the gap between evaluation and curriculum planning through computerized course and instructor evaluations," says Alexander. "We analyze the data with WebFOCUS, using a five-point Likert-type scale to create a satisfaction index for every course and every instructor. Having a fourth-generation language underlying WebFOCUS is very powerful."
Crunching the Data
Today, evaluation reports are online for all courses; HSC allows students to enter and retrieve information from any computer with Internet access. Quality also lets students supply daily curriculum comments to the course directors and anonymous e-mail to course faculty. Course comments are accumulated and categorized by organization, presentation, instructor/ student interaction, pace, workload, tests, support materials, and services. These daily curriculum comments allow the course director to make adjustments while the course is in progress.
Following analysis, databases are updated and reports are created to summarize evaluations for a course or clerkship as well as for each individual instructor. Composite reports, which average instructor results for a given course, are also created. These reports serve as an important benchmark for instructors to compare individual evaluations. Report elements are color-coded, with statements rated most highly shown in green and those rated lowest shown in red. Additionally, a "satisfaction index" is calculated for each course or clerkship and each instructor.
Alexander says the new system saves hundreds of hours in staff time and thousands of sheets of paper each year. Clerks no longer have to collect and enter about 65,000 data points from evaluation forms. Additional savings stem from the instantaneous delivery capabilities of e-mail and the Web, which makes the processes of hand-sorting reports, packaging them in envelopes, labeling the envelopes, and transporting them to the mail room obsolete.
"The time required to process and deliver results dropped from days to seconds," says Alexander. "With electronic reporting, what used to take 30 hours now takes just a few minutes. We formerly had to distribute course evaluations manually. Now WebFOCUS automatically creates and archives HTML reports, which are immediately accessible to authorized users. There are no longer any distribution chores."
WebFOCUS is deployed on a Microsoft Windows NT server. Numerical data is analyzed and stored in a FOCUS database while student registration data resides in an Oracle database on a UNIX platform, accessed via iWay Software. Long comments are stored in a PostgreSQL database on a Linux system and are retrieved using Perl scripts. WebFOCUS combines these various types of data dynamically each time reports are generated.
Looking Ahead
At present, the Quality system supports the evaluations of 741 students for about 200 required and elective courses within the HSC and several hundred evaluations of faculty, who teach multiple courses. When fully developed, the Quality system will include evaluations for courses and instructors, clinical clerkships and preceptors, and postgraduate residents and faculty. It will also feature an extensive testing module for maintaining item statistics and grade rosters, along with a Student Progress area for tracking current and historical grades, student progress, and class rank. "We use conditional styling on the matrix," says Alexander. "It is color-coded so problem areas can be quickly spotted."
Alexander believes that the extensive business intelligence capabilities of WebFOCUS will become increasingly more important over time. "We plan to use WebFOCUS to compare the satisfaction index and do trend analysis across years," he adds. "For example, we can use it to learn how a particular course compares with others, how certain clinical training sites compare with others, and the relative strengths and weaknesses of evaluation elements within sites. We have been impressed with Information Builders' commitment to enhance the product and incorporate new Web developments as they become known."
