Stephen Yasko

Public Radio Listener Survey Question Construction

Question construction needs to be thoughtful and tested.  Even if you think you have a perfect question, designed to get the information you want, ask staff and volunteers for their response.  It’s measure twice; cut once.  I can’t tell you how many times I thought I had a great question that turned out to be a dud because I had one unclear word in it.

So what is it you want to know?

Do you really know what’s important to your audience?  Answering that question is the over-all goal of any survey.  Listeners don’t always make the distinction between your station’s programming and national programming.  You should always construct your questions to focus your listeners minds on specific programming elements if that is important you.

Let’s go through a series of questions to get at what community issues are important to your respondents.  This is a basic question sequence that can be used to ask about any part of your station including fundraising and news.  We’ll cover the sequence using news focused questions.  Remember too that this is a general sequence and it might take one, two, three or more questions on a topic to get the information you’re looking to acquire.

Question 1:  Unaided Recall:   What Three Issues facing our local community do you spend time thinking about? The order is not important, just that you spend time thinking about them.

The question should have three text boxes to capture the information.  Notice the words “local community” are used.  Local differentiates from national or state-wide issues and “community” so they focus on their definition of local, not yours.  Asking someone about issues affecting Baltimore is going to get you different data from respondents who live 30 miles outside the city.  Once we get their home zip code later on in the survey then we can compare and contrast what different parts  of our listening area.

Question 2:  Prioritization:  On the left are list of issues many would say are facing our local community. Drag each one to the box on the right and order them by dragging them up and down to put them in order of you think are the most important to you to the least important.

You can take a few approaches to determine the list of issues for the left side.  I advise that focus on topics from the  beats that your station currently covers. This, combined with question 1, will give you an idea just how well your editorial is matching up with what your respondents want to know.

The topics on the right should be two or three words that can be “semi-specific.”  “Transportation” has too wide a meaning.  “Transportation Infrastructure” narrows it down some and might more in aligned with your content. “Public Transportation Infrastructure” makes it even more specific. I would choose “Transportation Infrastructure” as it would have similar meanings to drivers and public transportation riders.  You may need to add a “how do you get to work question” later on the in the survey to determine if your listeners are more interested in road construction issues versus subway delays.

This question type will return a score and ranking from all of your respondents giving you a nice list of their priorities that you might use to guide the amount coverage you give each topic.  You should also include a free form text box for comments.  “Is there anything you’d like us to know about these topics?  It could be something going on in your community or something about how we covered one of these topics.”

Question 3: Performance:  This is the question that gets to the heart of the matter.  Here we ask respondents for their opinion about how well your station fills their needs.  I like to ask this one in a table of statements and use a 1-5 strongly disagree to strongly agree scale.  Generally your listeners are going to give you high marks for your coverage but we’ll talk about that in a future post about analyzing your data but generally, any statement that averages less than 3.75 should be a topic of conversation in the station.

“Below are a series of statements about WXXX. Please tell us how much you agree with each.”
“WXXX covers the kind of stories I need to know about”
“WXXX reporters work to tell stories without injecting their own opinions”
“WXXX covers stories that no other TV or radio station covers”
“WXXX  stories have facts and figures that can’t be disputed”
“WXXX stories are clear and easy to understand”

You can see that you can offer soft or hard statements about your coverage.  I would also include a comments box here for feedback. “Do  you have any insightful comments about our coverage that you think would be helpful in the future?”

I’ll demonstrate how this sequence can be constructed to shed light on your fundraising program.