At the 2011 ASTC Annual Conference in Baltimore, a session entitled “Exhibit Evaluation: Useless Bureaucratic Hurdle or Valuable Tool?” sparked a particularly spirited discussion. The session had its origins in a provocative post  on ASTC’s listserv (ISEN-ASTC-L) in January.

Held Monday, October 17, the session was moderated by Sam Taylor of Pittsburgh’s Carnegie Museum of Natural History. It began with a series of five-minute presentations addressing the question of evaluation’s value.

First, Dave Ucko, formerly of the U.S. National Science Foundation, highlighted evaluation’s usefulness in providing accountability to the federal government, adding to our knowledge base, continuing to professionalize the field, and strengthening projects.

Next, Charlie Carlson of San Francisco’s Exploratorium (who emphasized that his statements do not reflect the position of his institution) pointed out that there are lots of exhibits that have succeeded without formal evaluation. “[Evaluation] does not directly result in a memorable, positive visitor experience,” he said.

Martin Weiss of the New York Hall of Science in Queens disagreed, calling evaluation “extremely important,” but asserting that “our profession has to strive to make evaluation better and more usable.”

As the sole independent evaluator on the panel, Ellen Giusti remarked, “Charlie says we know when an exhibit is popular. Popularity is not always the key to success.” She stressed the value of evaluation in determining whether an exhibition’s goals have been met, and also reminded the audience that lessons learned from one project can be applied to the next.

Finally, Paul Orselli of Paul Orselli Workshop (POW!) in Baldwin, New York, discussed the importance of having internal capacity for both exhibition development and evaluation, as well as the need to diversify evaluation. “We’ve sort of built up what I would characterize as evaluation monoculture,” he said. “I wonder if we could widen the view of notions of evaluation, so real physical prototyping becomes a more valued part of this, and exhibit people become more truly part of a partnership [with evaluators].”

Next the question was put to the crowd, which included exhibit developers, evaluators, and other professionals from around the world. An impassioned debate ensued. Here are some comments from this discussion:

• “Peer evaluation in some safe setting—a discourse about what’s worked and what hasn’t—would be more useful than professional evaluators’ feedback.”

• “Evaluation has tilted toward how people are changed after seeing an exhibition. There’s not enough emphasis on what people do and see in the exhibition…That’s why people go to museums, not because they [ask themselves], ‘What are the cognitive outcomes our kids will have?’”

• “Evaluation is one way to learn about what we do and the effect of what we do on our visitors—one way to learn about our own practice. I view it as a learning tool, and that keeps me going because I don’t know everything and I never will.”

• “I think evaluation should be like a visit from the health department to a café. They can show up at any time and gather information. It ought to be that the Spanish Inquisition can descend on your exhibition and really give you a bad time. That would be much more exciting.”

• “In evaluations, it’s easy to learn about all the things the project achieved, but you really have to squint to see what failed. We should put a book together of failed projects—that’s how the field advances.”

• “If you want future funding, the only way to get it is to have positive report. I find that very problematic. [It makes it] difficult to actually get honest evaluations.”

• “It should be a requirement that 10% of exhibit project money is left after the exhibit opens so you can actually go back and do remedial work to make it a truly great exhibit.”

• “You can’t measure everything that matters and everything that matters can’t necessarily be measured.”

• “If you just toss out the first idea you have and it works, congratulations…but it helps to have an outside person to say, ‘Let’s walk through the data. What do we have to do to make that work?’”

Owen Gaffney, director of communications at the International Geosphere-Biosphere Programme (IGBP), Stockholm, Sweden, delivered a riveting featured session at the 2011 ASTC Annual Conference in Baltimore on Sunday, October 16.  Using striking visualizations, Gaffney showed how human impact on the planet has increased exponentially since 1950—a phenomenon known as the Great Acceleration. Human impact, he explained, is determined by the intersection of population, affluence, and technology.

“Humanity has pushed the planet so much that we’ve reduced its resilience,” Gaffney said. “We’re getting close to the edge.” He said that we need to stay within nine boundaries, and we’ve already crossed three of them: climate change, the nitrogen cycle, and biodiversity loss.

Although he cited some grim statistics, Gaffney also sounded an upbeat and hopeful note. He held up the improvements in the ozone hole over Antarctica as a positive example. “This shows we can act globally on these issues, and this action can be effective.”

In that vein, Gaffney invited science centers to host their own events aligned with Planet Under Pressure: New Knowledge Toward Solutions. Gaffney is directing communications for this major scientific conference, which will be held in London in March in advance of the UN summit on sustainable development, Rio+20. ASTC is working with Planet Under Pressure to provide support and resources for participating science centers, as well as to develop 12 debates to be held on four continents.

Gaffney concluded, “We risk crossing a planetary threshold, but solutions exist. We need to act.”

The Live Demonstration Hour has long been a highlight of the ASTC Annual Conference, and this year’s wowed the audience once again. If you missed it (or want to watch again!), videos of each demonstration are available through the links below.

Niki Hord, Maryland Science Center, Baltimore

Adiel Fernandez, New York Hall of Science, Queens

Jonah Cohen, The Children’s Museum, West Hartford, Connecticut

Eddie Goldstein and Jodi Schoemer, Denver Museum of Nature and Science, Colorado

Steve Spangler, Steve Spangler Science, Denver, Colorado

About the image: Jonah Cohen launches an air pressure rocket. Photo by Christine Ruffo

ASTC’s Youth Inspired Challenge is one year old, so what better way to celebrate than by having a birthday party in the 2011 ASTC Annual Conference Exhibit Hall? Amid party hats and cake, conference attendees met science center youth program participants from the New Jersey Academy for Aquatic Sciences, Camden; Pacific Science Center, Seattle; the Philadelphia Zoo; and the National Aquarium, Baltimore. Libby Redda from Pacific Science Center and Jeremy Martinez from the National Aquarium addressed the crowd, sharing their experiences and describing how they personally have benefitted from the institutions’ youth programs.

ASTC’s Youth Inspired Challenge is designed to expand the impact of science centers and museums to assist our youth to become the innovative and creative thinkers needed for the 21st-century workforce. ASTC-member institutions will offer valuable science education and youth employment programs outside the classroom to engage youth in a minimum of 2 million hours of science enrichment through STEM-centered youth development programs. In the program’s first year, nearly 14,000 youth across the globe were reached during more than 702,000 out-of-school hours. Click here to learn how your institution can join the program.

ASTC would like to thank the chaperones and youth participants for joining our celebration!

About the image: Libby Redda, Pacific Science Center, and Jeremy Martinez, National Aquarium. Photo by Christine Ruffo

Approximately 84% of ASTC-member institutions have some type of teacher training program. While science centers and museums are doing a great deal to help strengthen the teaching of science, and the skills that develop in out-of-school environments are being implemented quickly into the classroom, more has to be done. And Ellen Futter, president of the American Museum of Natural History (AMNH), underlined the imperative: “Failing to improve science teaching in schools will have dire consequences.”

The “Museums and Teachers: Partnerships with a Purpose” featured session opened with a presentation from Dr. Patricia Simmons, the new president of the National Science Teachers Association (NSTA), and head of the department of Math, Science, and Technology Education at North Carolina State University. In 2010, ASTC joined NSTA’s Alliance of Affiliates as the professional organization representing informal science education, and as Simmons mentioned, both organizations share similar advocacy goals, which include ensuring that the Elementary and Secondary Education Act (ESEA) is reauthorized.

Over 80% of the fastest growing professions (e.g., health care, IT) require science skills, and despite high unemployment rates, 1/3 of U.S. manufacturers report significant skills shortage in STEM areas. Futter called for science centers, natural history museums, zoos, aquaria, etc., to assume a more active role in improving science education.

“Science centers have always been places of inspiration,” she said. “But as valuable as that role has been, science centers are and need to be as essential a part of the formal learning landscape as well as the informal one.”

AMNH is participating in Urban Advantage, a collaborative program focused on supporting and improving the teaching and learning of public school science education. Urban Advantage reports that participating students perform better than those who don’t. The museum is also about to launch the nation’s first masters degree-granting program for teachers in earth science, co-taught by museum scientists and educators. (AMNH is already the only PhD-granting museum in the U.S.)

“We dare not smother the fire of intellectual curiosity,” Futter said. “Science centers and museums must make our resources broadly available to improve science education.”

Click here to watch a video from this session.