2022 Member Survey Results

[Evan Reiter]

More than 250 members (15%) took part in our 2022 Member Survey, which was available between between September 15 - November 15, 2022. The Administration Team is grateful to those who took the time to provide feedback on the community and ideas for how we can continue to offer great service to our members.

We’re pleased to provide a snapshot of the survey results below. If anyone is interested in seeing additional data, feel free to reach out to me. We're happy to make additional data available for members to complete their own analysis! 

BVA members are represented by all age categories, with the largest demographics being 56+ (28%) and 26-35 (17%): 

 

 

Consistent with previous years, controllers tend to be younger while members of our community tend to be older. The trends have been relatively constant year-to-year, except that the age of our controllers has become more evenly distributed since 2018: 

 

 

76% of our members connect from Eastern or Central time, although the community has members from around the world. This is part of the reason we consistently schedule events to benefit this time zone (i.e., weekday evenings or weekend afternoon/evenings). 

 

 

A little less than half of our members have some form of real-world certificate; most of those are pilots. 47% of our members do not have any real-world flight experience.

 

 

Generally, members were pleased with ATC realism and courtesy, our overall community feel, and the timing and experience with our events. The summary below represents average scores out of 5. In all cases, perceptions are consistent between controller and pilot members:

 

 

Consistent with previous years, members are pleased with the community experience, with 90% of members rating their experience as “Very Good” or “Excellent”. 64% of you rated the overall experience as "Excellent", consistent with 2020:

 

 

This year, we asked members what the primary or most important single source was for finding information about our events. We found that (overall) our website is the most common location members look to. However, digging further into the data, we can see that there is a difference between controller and pilot members. While controller members look to the website most (62%), pilots rely more on the email reminders (40%) to find out about upcoming events:

 

 

In addition to reviewing the demographics and data, the Administration Team also read through thoughtful, insightful suggestions from those who completed the survey. We read each piece of feedback individually and have already followed up with approximately 10 members about their questions. 

 

If you'd like to get access to additional data or the full dataset to conduct your own analysis, feel free to reach out! 

If you have questions, comments, or feedback about the community, please don’t feel the need to wait for our 2024 survey! Reach out to the Administration Team, submit ATC feedback, or share your thoughts on our forums or in Discord and TeamSpeak.

 

 

[Evan Reiter]

A few thoughts from the member survey, specifically for controllers. I'm posting here in the community-wide section of the forums because I think all members may be interested in seeing these points too. 

 

We're younger than average.

 

75% of controllers responding to the survey are under 35. By comparison, 63% of pilots are over 35. Remember that if you were looking around the room at Tea Party, that room is not representative of our broader community. 

 

We have members from around the world.

 

I recently received a great message from one of our members that nicely summarizes the importance of expectation-setting around ATC. I'll paraphrase here:

 

"I have a hard time making it to my computer before 9:30/9:45pm in the evenings due to getting my kids down to bed. Thus, I miss a majority of the nights on BVARTCC due to signing on too late. That's not their fault - and I am going to try to adjust our nightly schedule here too - but I wondered about if there might be specific individuals who I can assume will be online later than others."

 

Members with families or who are in later (and earlier) time zones may not have the same opportunities to enjoy our coverage. We can't be online at all hours of the day, but we can help set expectations for our members by using the ATC Timetable and Controller Info - especially if we're signing on at times that are outside of our typical peak hours (weeknight evenings, weekends).

 

Pilots love the ATC Timetable and Controller Info. Or, at least, they say they do...

 

We received several comments in the survey about pilots who login to start a flight, only to find ATC is about to close. This has been a point of discussion across VATSIM since...forever? Being a volunteer community, there is never an expectation or guarantee of coverage. However, when you know the amount of time you intend to spend online, there are two things you can do to help set expectations: the ATC Timetable and Controller Info. Adding yourself to the ATC Timetable even after you've signed on can still be helpful. Similarly, please consider adding an expected logoff time (when you know it) to your Controller Info. 

Although it came up quite frequently in the long answer section of the Member Survey, it was interesting to me that the quantitative results don't necessarily reflect this, with the question "I use the ATC Timetable at bvartcc.com to plan my flights around expected ATC coverage" getting the lowest average score in the section on community perception:

 

 

ATC training process.

 

Out of 265 responses, we received 6 that rated the overall membership experience as a 1 or 2 out of 5. Of these 6 responses, 4 specifically referred to our ATC training program or exams as areas why they felt the community needed to improve. As those of us who remember K90 can attest to, the training program has changed dramatically in the past 2 years. While we have improved our self-study material, added new videos, and made the general process easier, it's also taken longer to get assigned a Mentor or Instructor than it might have at the beginning of 2020. 

 

Finding the right balance of realism while remembering this is a hobby is the supreme challenge for each VATSIM facility. We take survey responses very seriously and have committed as an A-Team to review the training program this year. This will start with a data gathering effort on our part as well as seeking input from the controller community. This work may well result in changing nothing, because a lot of thought has gone into the program as it's currently designed and the quantitative survey responses indicate things are going quite well. However, we're always open to making improvements and adjustments. Expect to hear more on this in the coming weeks. 

 

Let me be clear though: these are 4 responses out of 265. On average, controllers rated our training program 4.5/5 (consistent with 2020) including the exam process (4.4/5) and the ease of scheduling training (4.2/5). So we're not looking to make dramatic changes or solve a problem that isn't broken; and we're very happy to see that all members continue to feel BVA is providing a great membership experience (4.5/5, across both pilot and controller members).

[William Walker]

Data makes my scholarly heart warm back up. Thank you for sharing this!