PROGRAM ASSESSMENT - 2023
Chautauqua Dialogues
Prepared for: The Department of Religion and the IDEA Office
Prepared by: Roger Doebke, Lynn Stahl and Robin Harbage
November 29, 2023
A YEAR OF EXPERIMENTATION, GROWTH, AND SATISFACTION
TABLE OF CONTENTS
Page 4 - DATA: Raw data for participants, facilitators, and venues
Page 7 - Assessments from Participants - Extracting the data and comments from 1,046 assessment cards.
Page 12-Assessments from Facilitators
Page 15 - Assessment of Venues
Page 16 - Assessment of Focus Dialogue & Venues
Page 17 - Assessment of Institutional Support
Page 19 - Assessment of Facilitator Training
Page 21 - Promotion Recommendations for 2024
Page 22 - The Red Bench Project
Page 22 - The Listening Tool
Page 23 - Agenda for 2024
Page 25 - Leadership Team for 2024
DATA
The 2023 Season provided CHQ Dialogues to 1143 participants. This was a 25% increase in participation over 2022. The average of 127 participants per week varied from a low of 80 in Week 2, to a high of 203 in Week 8. The participation seems driven by the subject matter of the weekly theme and the number of full access versus gate access passes. Note, Week 3 (Will the Center Hold), Week 4 (State of Believing) and Week 8 (Resilience of Democracy) were interesting themes for dialogue given potential political disagreement on these topics, compared to themes for Weeks 5 through 7. Weeks 1 and 9 simply had lower overall attendance, and Week 2 was dominated by families with gate access passes.
Over the nine weeks, 65 separate facilitators participated in CHQ Dialogues to provide 168 unique sessions, though in a few cases participants of two venues were combined when attendance was low at a location near a concurrent Dialogue. When adding up all facilitator participation, including co-facilitators at a single session, facilitators participated in 277 Dialogues. Of the 65 participating facilitators, 20 individuals facilitated 60% of the total Dialogues. As in seasons past, a core of our facilitators are the backbone of our facilitation team, but many come for only a week or two, and take the time to participate while at Chautauqua.
Participation by venue varied significantly from an average high of 15.8 in the AAHH to a low of 3.0 for the Women’s Club sponsored lecture. Of the 19 venues, 4 of the top 5 attended venues were on Thursday or Friday, with Friday representing 3 of the top 5 attended venues. This may represent increased interest in joining discussion as the week progressed, or the value of repeated CHQ Dialogues impressions during the week. Of the 8 lowest attended venues, 6 were at 12:30, reflecting the redundancy of multiple venues for this time slot, when 2 or 3 venues were available. The Graybiel House had particularly low attendance each day that it was used.
Assessments from Participants
We received 1046 completed assessment cards during the 2023 Season. The following information was gleaned from reviewing those responses. We asked attendees to respond to six questions and provided space for comments. Sample responses were chosen based on their being representative of numerous similar comments.
Responding to the question: Is this the first time you have attended a session?
55% responded that this was the first time they had attended a session. Increasing from 30% in 2022 to 45% in 2023 saying they had attended previously.
Responding to the question: Did you consider this a good experience?
Well over 99% responded “Yes” to this question. In fact, only 7 respondents said they did not consider it a good experience.
Sample responses:
“The rules for engagement for talking were good.”
“I am attending Chautauqua for the first time. A very good open discussion.”
“I am energized by the Dialogues.”
“Keep it going. It's terrific..”
“Thanks! Great addition-new tradition.”
“Excellent opportunity to practice mindful listening.”
“A humanizing experience.”
“My first time - I enjoyed it a great deal!”
“Thank you, and thank you for the Lutheran House, it's a wonderful place.”
“This is a great program to further enhance the Chautauqua experience.”
“This was great. I am so glad I came. WOW!”
“Yet another reenforcement of a sense of community, shared values and trust at Chautauqua.”
Responding to the question: On a scale of 1 to 5, with 5 being best, how would you rate your facilitator?
In 2022 54% of participants gave facilitators a rating of 5; in 2023 69% gave the facilitator a 5 rating.
24% of participants gave facilitators a rating of 4.
6% of participants gave facilitators a rating of 3.
1% of participants gave facilitators a rating of 1 or 2.
With 93% of the participants giving facilitators either a 4 or 5 rating, a significant increase in the quality of the facilitator’s performance was evident in 2023 over 2022.
Sample responses:
“Good facilitators brought all people into the conversation.”
“Effective facilitators, interesting discussion focused on action.”
“Lynn and Shahid make a great team.”
“I would enjoy learning to be a Dialogue leader.”
“The calm atmosphere led by Catie and Steve was wonderful.”
“Great job, MJ & Chick.”
“Barbara is the best facilitator I’ve had. She controlled the dominators and made clear the rules of the road and engagement..”
“Kriss’ calming energy makes her a fabulous facilitator.”
“Our facilitator was excellent.”
“John and Kriss are gifts to me. What a privilege it has been to be with them today.”
Editors Note: Again this year a participant wrote, “Anita is Awesome!!” Another participant wrote “Our facilitator (aged 90) was excellent. Age is not a disqualifier for remarkable leadership and thought.”
Responding to the question: Do you feel any different about any topic discussed after hearing what others in your group had at to say?
57% responded “Yes”
Sample responses:
“I will be taking a 2nd look at a speaker.”
“I walked away with ideas of what to do going forward, even though it’s on a small scale.”
“Solidified my ideas from the lecture.”
“It didn’t make me feel different , but did make me see things in broader ways.”
“It’s good to listen to yourself as well as to others.”
“I don’t feel any different, but I sure came away with new thoughts.”
“Topics that resonated with others reminded me of things I forgot.”
Responding to the question: Did talking about ideas you heard in the lectures help you process the substance of the lectures?
90% of respondents said the Dialogues helped them process the substance of the lectures.
Sample responses:
“Great idea to synthesize the week with different perspectives.”
“Best process part of anything at the Institution.”
“It helped me to better understand the concerns of others.”
“Listening to others was very helpful.”
“My assumptions were confirmed that face to face dialogue works.”
“Hearing my dialogue partners share their thoughts that differed from mine allowed me to recognize my own limiting judgements and close mindedness.”
“Discussion by other people helped pull ideas together for me.”
“These Dialogues and the booklet for notes are a great idea! Thank you!”
“It supported my view that if we can talk respectfully face to face, we can help resolve conflict and settle problems.”
“I learn something new every moment I am at Chautauqua and especially here!”
Responding to the question: Would you attend a Dialogue in the future?
Well over 99% of respondents said they would attend a Dialogue in the future. Out of the total responses only 8 participants said they would not return in the future.
Sample responses:
“Keep the Dialogues going. More red benches.”
“Please keep the Dialogues going. I love the new no sign-up format.”
“Best part of Chautauqua and that’s saying a lot.”
Editor’s note: We received an overwhelming number of comments saying thank you, I loved it, don’t stop doing this and “great job” to the facilitator.
Responding to the request for additional comments:
“Excellent discussion”
“Chance to share.”
“So interesting.”
“Great experience. Singular opportunity to exchange thoughts.”
“I really loved this experience.”
“Love this Dialogue! Great work!.”
“Great group! Good conversation and ideas.”
“Keep these going.”
“I was encouraged with the groups overall desire to see and participate in civil dialogue.”
General Negative Comments:
Some complained about the sessions being too short. A couple of comments were made about the dificulty of finding the location of a venue. A few complaints about facilitators who engaged too much in the discussion. We have retained the assessment cards and they can be made available for viewing.
Assessments from Facilitators:
This summarizes the feedback from our post season facilitator survey for 2023. We had a remarkable response with more than half of the participating facilitators responding.
Responding to the question: What made CHQ Dialogues a good experience for you?
Overwhelmingly. responses listing benefits of CHQ Dialogues for facilitators described (1) the opportunities to meet and engage with other facilitators, (2) listening to the fresh and differing perspectives of the many participants, and (3) meeting new people at Chautauqua.
Many facilitators appreciated the training and opportunity to learn and practice the skills of being a facilitator. Several participants also expressed gratitude for the free lunches.
Responding to the question: What changes would you make to CHQ Dialogues in the following areas:
- Scheduling
Facilitators indicated the Saturday Women’s Club Dialogues were poorly attended. Several also suggested the 12:30 Dialogues may have been over scheduled given poor attendance. These are both addressed in our recommended scheduling for 2024.
- Organizational meeting
There was almost unanimous agreement that the Monday organizational meetings were valuable. One facilitator mentioned they may have been too long, particularly when facilitator feedback became redundant.
Most participants agreed they found value in hearing from other facilitators, especially regarding how to handle dificult participants. A few specific suggestions included:
• Begin each meeting by going around the circle introducing each participant.
• Send out notes of suggestions from each meeting.
• Frequently remind facilitators to be facilitators, not participants.
• Allow a couple of experienced facilitators to describe best practices they employed.
Responding to the question: How would you rate the Venues?
The two most common venue complaints beyond poor attendance at the Hall of Missions on Saturday, were the Graybiel House, which is not a good space and hard to find even with additional signage, and the Methodist House at 3:30 due to noise from the Amphitheater rehearsals.
Several facilitators mentioned we should emphasize in marketing the Dialogues, that the venues are merely convenient locations and the Dialogues are not related to the denominations, so everyone is welcome at all venues.
Responding to the question: What do you think about the Dialogue structure?
There was unanimous praise for the structure of the Dialogues. Specific comments mentioned the simplicity of following the process, the value of sticking to the weekly theme for participant inclusion and having opportunities to be paired with other facilitators.
A few participants clearly want to be solo facilitators. We were fortunate to have enough facilitators to assign two to most venues, and we should make it clear that facilitators should agree in advance as to the role each co-facilitator will play. We will encourage experienced facilitators to release as much responsibility as possible so that the less experienced facilitator will feel comfortable as the lead. We had one unfortunate experience where a male facilitator asked a female facilitator to “be the scribe”, clearly displaying gender bias.
Responding to the question: Do you have other suggestions for improvement in marketing, recruiting, or training?
Many suggestions centered around marketing CHQ Dialogues. Several of the facilitators showed strong interest and so this could be a good activity to recruit additional help.
• Provide more articles in CHQ Daily explaining the Dialogues.
• Place large ads in the Monday or Tuesday CHQ Daily.
• Change the announcement at the Amphitheater. Several stated the announcement needs to be more specific, clearer, varied and more welcoming. It does not currently stand out and misses the mark for anyone who hasn’t previously participated. A suggestion was made to place the announcement before the Q&A is announced, which would differentiate this announcement, when most people are in the amphitheater and when most participants are already thinking about the lecture . Even if they are about to exit, they may be more receptive to a hearing about a venue where they could engage in Dialogue.
• Request that Roads Scholars and other organizations to announce and participate in Dialogues.
• At the end of each CHQ Dialogue, encourage the participants to urge others to attend upcoming Dialogues by “word of mouth”. Perhaps even pass out materials to share or ask participants to share the brochure they’ve already been given with another guest, encouraging them to attend.
• Tell participants how they can become facilitators.
Suggestions regarding training included providing and encouraging annual online refresher training for all facilitators, and emphasizing the potential roles of cofacilitators. Facilitators specifically requested a checklist for use with facilitation. Cofacilitators could use this to divide responsibilities. This fits well with the checklist we’re developing which can be used before, during and after each Dialogue.
Assessment of Venues
We have so appreciated the support and participation of the denominational houses. Ten of these houses contributed their space and support each week. One hosted four per week; four hosted two each week; and five hosted one per week. In addition the AAHH and the Hall of Missions were used once a week during the Season.
It was determined that there were too many Dialogue groups at 12:30 each day so that the number will be reassessed for 2024. The confusing location of one of them posed a challenge so the attendance was greatly diminished. During the 2024 season, the facilitators of the simultaneous Dialogue groups will be given each other's cell numbers, so if one group only has a few participants, the facilitators will be able to connect with one another and perhaps combine the groups.
Our leadership team will continue to do more direct work with the denomination house hosts, especially since there seems to be a turnover in hosts each season. Many do not fully grasp their role and importance of encouraging their house guests to participate in any of the weekly Dialogues anywhere on the grounds. Again, we will provide material for the hosts to distribute at their weekly orientation sessions. We also will continue working with the hosts to arrange appropriate seating as well as establishing a quiet area. Some of the houses provide water and even a little treat like raisins, but that is totally optional on the part of the hosts!
Assessment of Focused Dialogues:
Again this summer the AAHH Dialogues focused solely on the Wednesday afternoon AAHH speaker. They were held on Thursday afternoon, a half hour later, at 4:00 pm, so the participants could walk the extra distance following the 2:00 lecture. Several of these Dialogues involved huge groups which had to be split in half. Fortunately, there were at least two facilitators assigned to the 4:00 Thursday Dialogues each week, in case this actually happened. In looking to 2024, we have located a second location in the immediate neighborhood of the AAHH so the division can be done with the least amount of disruption. Ideally, the AAHH will have either an expanded facility or a tent erected outside, that could accommodate the overflow.
The second focused Dialogue was held at the Hall of Missions on Saturday afternoons, immediately following the Women’s Club Saturday lectures. After two years of having minimum attendees, we will not continue these Dialogues in 2024. Either those at the Women’s Club lectures wished to attend other conflicting opportunities, or meet directly with the speaker at 5:00, which seemed to conflict with the Dialogue.
Assessment of Institutional Support
All of our Leadership Team agrees that support from Institutional Staff was excellent in every way. Support was provided by the Institution in both tangible and intangible ways. Across the board, Staff did everything they could to make the Dialogues a success.
Also tangible was the senior staff support. Primary support was provided by Amit Taneja and Melissa Spas. One or more senior staff attended our Monday Facilitators
Lunch at Hurlbut Church each week. They always welcomed the facilitators and told them about the Institution’s support for and commitment to the Dialogues. Staff also provided appropriate assistance in answering questions that dealt with larger issues and concerns.
• Staff was instrumental in promotion of the Dialogues. Signs for every venue were provided and proved very beneficial in promoting the Dialogues and identifying the host venues.
• Staff agreed to several important initiatives during the season, including distribution of the Lecture Journals before the10:45 at the AMP.
• Staff enhanced the presence of the Dialogues on the Institution website.
• High quality facilitator “badges” were provided by the Institution and facilitators were encouraged to wear them throughout the week. This had the effect of acknowledging the facilitators and invited the public to engage them with questions about the program.
• Staff announced the Dialogues at all 10:45 and 2:00 Lectures and had a brief video slide announcement about the Dialogues running before the 10:45.
• Staff arranged for a feature article in the Daily.
• CHQ Dialogues was invited to participate in the Sunday Community Group Fairs on Bestor Plaza. Numerous facilitators engaged with visitors who sought more information about the program.
The intangible was the afirmation and encouragement provided by Staff.
• Preseason, Mellisa and Amit worked diligently with the Dialogues leadership to define what could be done to make the Dialogues more relevant and more visible. The results during the season were evident. Many Chautauquans expressed renewed interest in the program and lauded the commitment of the Institution.
• Staff members who were not directly involved with the Dialogues made a specific point of telling us how much they supported the program.
Assessment of Facilitator Training
The off-season training completed in preparation for the 2023 season followed the format of those presented for the 2022 season. We had a total of 32 people who participated in the training. We edited the Manual and prepared new training materials for 2023. Facilitators reported that the materials used in training had substantial influence on their growth and skill level in leading the Dialogues. It is apparent that there is a need to continue to train new facilitators by offering an introductory online class. Therefore, we will repeat this level of training for a limited number of people who want to join for the first time or get a refresher. This class will purposely be limited because we have enough previously trained people who want to facilitate on their own rather than co-facilitate, and we want to give them that opportunity.
There is also an opportunity to create a Level 2 online class for those facilitators who have completed the basic class and have had the first hand experience of facilitating Dialogues at Chautauqua. Envisioned as a more inter-active experience the Level 2 class will address honing skills and discussing how to engage in the “real life” challenges of active facilitation.
Hal Simmons of our leadership team has designed an application form for all new facilitators. We learned in 2022 that we just didn’t know enough about the new people coming in and didn’t have the time to get to know them before making assignments.
Because of the remote nature of our engagement with new facilitators we have learned that we cannot rely on any assumptions about their skill level. We therefore plan to assign all new facilitators to the Observer status to start with, followed by cofacilitating for the number of sessions necessary based on personal assessments. Thereafter, they will be designated as a facilitator or released from the program. From our pool of experienced facilitators we plan to select people we will designate as Senior Facilitators and use this group more for training and mentoring.
We recognize the importance of presenting a consistent product, one that follows the Facilitators Manual. The only way to ensure consistency and quality is to audit the performance of our facilitators. To this end, we intend to make this part of our program
more robust in 2023. Senior Facilitators will devote more time to observing Dialogues and will also meet with facilitators to conduct evaluations.
We will also provide better tools for our facilitators. In reviewing the note taking from 2022 it is evident that our attendance sheet is not the useful tool it could be for facilitators to take the kind of notes they need to facilitate the conversations. Notes are an important part of not only starting the conversation but knowing where to take it, and we will aim will to provide a more sensible road map to accomplish great dialogues.
Our weekly lunch kickoff meeting for facilitators proved invaluable and we would like to continue them. In fact we believe them to be so important that we think they should be mandatory. They became an important part of sharing, training and camaraderie.
Mentoring (Audit) Program for 2024
We have planned for several years to implement an auditing program as part of our ongoing training. Now renamed, the “Mentoring” Program will finally get started in 2024 with Hal Simmons taking on responsibility for its implementation. We envision mentors attending sessions as observers and then engaging with the facilitator of the session to talk about what went well and what might need some adjustment or improvement. Hal has previously helped us in this regard with particular individuals and he is well suited for this work. We believe the development of this program is essential to the sustainability of the CHQ Dialogues.
A post dialogue checklist has been developed for our facilitators. This checklist will both provide a list of prescribed actions to take during the Dialogue and a means for facilitators and mentors to review whether the expected steps were taken and how effective these action were during the Dialogue.
AGENDA FOR 2024
Additional Training
One of our objectives for 2020 (which became 2023) was to find a way to incorporate race competency training for all facilitators. In 2022 many of our facilitators attended the training offered by Amit Taneja.
Perhaps it was the content of the programming and themes of the Institution, perhaps the emergence of the African American Heritage House, or both, that made race a common issue participants wanted to discuss in the Dialogues during the 2019 season and again in 2022. Our awareness may also be tied to the fact that the dialogues at the AAHH were coupled to the sponsored lecture which specifically addressed race each week. In any event, it cannot be denied that race was and most likely will be an issue to be discussed in the Dialogues which mandates that the facilitators gain a level of race competency. To this end, we expect to work with Amit Taneja to incorporate this training into our spring schedule. We plan to offer diversity training for our facilitators regarding unconscious bias. "The End of Bias - A Beginning" by Jessica Nordell and "See No Stranger" by Valarie Kaur will serve as primary sources, with videos and group discussion. The training will focus on informing facilitators of the many forms of unconscious bias, how they personally both experience and perpetuate bias, and processes for reducing unconscious bias.
Signage at each venue that will announce full sessions.
“PLEASE
DO NOT DISTURB”
“Sorry
we couldn’t accommodate you, this session is full”
The Red Bench Project
In 2022 we proposed forming a partnership with iAct, Interfaith Action of Central Texas (interfaithtexas.org), whose mission statement alines with that of Chautauqua. “The Red Bench” program was initiated with the placement of 6 red benches on the grounds for the summer of 2023. We believe this has proven to be a good tool to bring civil discourse to the public square. It has proven to invite interest by saying, “if you sit here, you are inviting conversation.” We heard nothing but positive comments from the public and believe that an expansion of the number of benches is justified as a symbol of our interest in creating good conversations.
The Lecture Journals
In 2023 we started to create a culture of “listening” and then engaging with each other in Dialogues. We distributed 1000 of the new Lecture Journals at the AMP. Journals were given out only to those who expressed a sincere interest in the concept of journaling. Each person who received a journal also received a short verbal introduction to their purpose and agreed to participate in the process. Those who listened and received a journal were excited and grateful to the Institution for providing the journals. The journals were distributed only on Monday and Tuesday before the 10:45 Lecture and only to those who entered the AMP through the North gate. For 2024 we believe the quantity printed should be increased to 3,000 pieces, and we would distribute them at both the North and South gates to the AMP and perhaps, before the 2:00 lecture.
PROMOTION FOR 2024
We can’t say it often enough. People still think that selection of a venue has something to do with religion.
Our weekly population is now composed of so many folks who are at Chautauqua for a week or two, our promotion efforts need to be oriented to repeat week after week. We believe that a daily column in The Chautauquan explaining the opportunities as well as listing them is appropriate. A weekend edition spread would do wonders for awareness.
We also want to continue announcements at the 10:45 and 2:00 lectures. We believe an invitation between lectures and Q&A inviting the audience to continue their learning after the lecture by actively participating in Dialogues specifically pertaining to the lecture, will both attract more CHQ Dialogues attendees and demonstrate the richness of experiences at Chautauqua beyond the lecture hall.
For 2024 we have considered the question of why more people don’t attend a Dialogue. Since our feedback from those who do attend one is so positive we looked further into the comments made on the assessment cards and have arrived at what we think might be a reason for not attending. FEAR. It seems clear that most people attending have never attended anything like this before. For them it is a new experience. We also know from assessment comments that attendees are surprised that people are interested in what they have to say. We believe our challenge is a cultural one, particularly amongst women in the age group we serve. The culture in which they grew up discouraged women from speaking and/or rendering opinions etc. Even today our culture includes a marginalization of women’s voices. Our experience also shows that some who do attend the Dialogues are reluctant to speak and we believe this is for the reasons cited above.
We believe the action item here is to include in our promotion campaign a designed display ad similar to that below:
Your Voice Matters
Welcome to Our CHQ Dialogues We will listen to you, we will learn from you and we will honor your opinions.
Leadership Team for 2024
Roger Doebke
Lynn Stahl
Robin Harbage
Data Manager
Includes Scheduling Weekly Facilitators
Collecting Attendance Data
Hal Simmons
Mentoring
Community Fairs
Cathy Digel
Mentoring
Lecture Journal Distribution
Robert Cahn
Publicatiion Productiion