GuideStar Research
Contact Us FREE Research Tools Quotes or Proposals
Technology & Demos
Overview
EMMS
Online Report Center
Multi-Language Surveys
Security
Request a
Survey Quote
 
Online Surveys FAQ
Our online survey response rates are not consistent. They are excellent on one survey and poor on another. What can we do to assure adequate response rates, especially from specific segments?

How do you determine the survey sample size and segment sample sizes needed for a valid scientific sample of the population?

How do I conduct a SMART survey when only some of our target population have provided e-mail addresses and others may not have access to e-mail or the Internet?

Isn't there a problem with online surveys of a lack of control in maintaining a legitimate random sample when people can self-select to take the survey?

I have a concern about people taking our surveys on the Web site who are not authorized. How do I know which survey data is legitimate and which is not?

What about the confidentiality of people's identity and responses in e-mail surveys? When we use e-mail, our company can read it if they want to. Can they read our survey responses as well?

We do not have the internal expertise and resources to conduct surveys of any kind let alone SMART surveys. Do you have any suggestions?

I have heard that SMART surveys cost less and are faster than print surveys. Is this really true, and if so, how much faster and cheaper are they?"

Our online survey response rates are not consistent. They are excellent on one survey and poor on another. What can we do to assure adequate response rates, especially from specific segments?

Good consistent response rates result from doing all the basics correctly (and aggressively). It starts with a survey design that offers a true value proposition to the participants. To develop a true value proposition, you need to talk with, or meet with, a few groups of prospective participants and explore their concerns and issues and the various types of value options (survey content, incentives, confidentiality, etc.) which would appeal to them. Then test the value proposition with one or two more groups.

With a tested value proposition, you next have to communicate the proposition ahead of time via e-mail (and print if appropriate) to create awareness and acquire buy-in.

The cover message which accompanies the questionnaire must resell the value proposition and lead quickly to the act of clicking open the questionnaire. The opening view of the questionnaire page must be simple, inviting and friendly with the first couple of responses right at hand to begin their engagement with the questionnaire quickly.

If you are going to analyze your findings by segments, your communications and value proposition should be tailored if possible to each segment's interests. Also, if possible, use custom e-mail cover messages for each segment with their value proposition clearly explained in the invitation. If your address list is large enough, it is a good idea to have a secondary list of additional e-mail addresses of people in your segments in reserve which can be used on an "as needed" basis.

Give people adequate notice and deadlines. Although many responses, as many as 50% of what you need may come in within 24-48 hours, many SMART survey respondents tell us they would like a week to ten days to complete a survey. Responses should be tracked on a daily or every other day basis (both overall and by segment). At the halfway mark, targeted e-mail message reminders should be sent to non-responders. This practice should be completed every couple of days through the end of the survey.

Deadline extensions of 24-48 hours should be planned into the schedule in advance as a precaution. If needed, a deadline extension announcement should be sent in the afternoon of the last day to non-responders. Also, if needed, send invitations to some or all of the people in your reserve database.

Incentives can also help increase response rates. It is best to make them substantial and to introduce them right at the beginning so that the motivational value can be incorporated into the value proposition and applied as part of the overall frequency of communications during the survey process.

As a last resort, if you still have not achieved a required sample size, you can employ a telephone survey facility to conduct a limited number of outbound interviews to complete your sample. There are also two things you can do with an undersize sample at the statistical analysis level. One is to carefully weight the sample overall and by segments to adjust the database to reflect each population's size in the sample - which you should do with every survey sample - before beginning the data analysis. The other is to use analytic software such as SPSS Exact Tests, which analyzes and adjusts statistical reports to account for small sample sizes. Both of these techniques should be done by a research professional.

Return to Top
How do you determine the survey sample size and segment sample sizes needed for a valid scientific sample of the population?

There are pre-calculated tables available for this as well as software such as SPSS's Sample Power. We have created two free, easy-to-use tools for this that are available here on the Web site at free research tools.

One is a table in Microsoft Word97, which tells you the sample sizes required for populations up to 10,000 at various confidence levels and margins for error. The other is a software applet for SPSS users, which will calculate the margin for error of a given sample for any size population and will also calculate the exact sample size required for any population.

Return to Top
How do I conduct a SMART survey when only some of our target population have provided e-mail addresses and others may not have access to e-mail or the Internet?

The answer may be for you to consider a dual-media survey, which uses both an electronic survey to quickly, and inexpensively reach those who are online and another survey method (print, telephone, etc.) to collect data from those who are not online.

We do this routinely for many of our clients. The world is in transition with many people already working and relying on the online world and many who are not yet. The dual-media survey reflects the reality of the world we live in. It may sound more complicated than it is, but most professional survey firms working with online surveys should be able to do it for you. There will be some extra costs involved because two questionnaires will have to be created, one for each media, and there may also be some additional costs for database merging. Outside of these two functions, costs should not be any different than conducting a survey in a single media.

Return to Top
Isn't there a problem with online surveys of a lack of control in maintaining a legitimate random sample when people can self-select to take the survey?

This is no more of a problem than with traditional paper surveys. With traditional print surveys, the development of a random sample list of every "nth" name is done before the invitations and questionnaires are sent out. The same is true of e-mail address lists for online surveys. Keep in mind that no matter how you randomize your lists, survey participants always self-select as to whether to participate after receiving the questionnaire.

The same is true of people visiting a Web site and encountering a survey offering. It is the same as if they encountered a survey offering in the mail or in a publication.

If you're fortunate enough to have a larger sample size than you need, you can always randomize the completed data records you have if you wish, however, most researchers are delighted to have a larger sample than they need because it reduces the margin for error on the survey.

For those who are very concerned about this, you can conduct outbound telephone interviews with non-responders to determine whether their responses are statistically different from those who responded and factor that information into the analysis. Pilot surveys can also help in this regard.

Return to Top
I have a concern about people taking our surveys on the Web site who are not authorized. How do I know which survey data is legitimate and which is not?

If you are conducting a public survey which is available to any visitor on your Web site, you must design your survey with qualifying filter questions built-in so that inappropriate records can be identified and discarded during the database cleaning process.

If you are working with a defined list of prospective survey participants who are being invited to participate in your survey, you can send them qualifying passwords and individual survey ID codes to use when taking the SMART survey. These authorize their access to the questionnaire and validate their responses. This assures there will be no unauthorized participants or multiple responses from the same person.

If you don't want participants to deal with passwords and survey ID codes, we can program our e-mail invitation lists with special individualized codes which will be automatically checked against an authorized database when a participant begins an SMART survey questionnaire on our Web site. This is transparent to the participant and eliminates the need for participants to deal with entering passwords, etc.

Return to Top
What about the confidentiality of people's identity and responses in e-mail surveys? When we use e-mail, our company can read it if they want to. Can they read our survey responses as well?

The issue is really more about people's concern about confidentiality than the reality of the confidentiality because the reality is that SMART surveys can be made quite confidential.

It is true that the law says that e-mail on company systems is company property and can be reviewed; however, we use a number of safeguards to protect people's confidentiality on SMART surveys. First, response data entered in our questionnaire software cannot be saved on a PC's hard drive. Only the blank questionnaire itself can be saved. The responses only exist during the time they are being entered. Once sent, there is no record on the respondent's hard drive.

As far as the e-mail file being captured in transit, only the data is transmitted so it is relatively meaningless to a reader who does not have the translation software program plus the individual custom program created to read the data from that specific survey.

It is also highly unlikely that a corporation's IT resources would be directed away from mission critical tasks to try to capture mostly undecipherable raw data from a survey.

In addition, if a client desires, we can apply SSL (Socket Security Layer) which encrypts each file from the browser to the destination server in the same way credit card transaction information is protected on the Internet.

Plus, we use a third party's neutral server on the Internet to strip out any e-mail address identification, which might be present in the file, before the response file is forwarded to us. When we receive the file, it really is anonymous except for data entered voluntarily by the respondent.

What is important is responding to people's real concerns about confidentiality in advance to allay their concerns as much as possible. This should be done by reassuring them in every communication that the survey is confidential along with a brief explanation of the safeguards in place to protect the confidentiality of their responses.

Additionally, we find that most of our clients prefer to have the Web site survey hosted on our password protected Web site because people are often more comfortable interacting electronically with an independent third party firm that guarantees to protect the confidentiality of their responses.

It is generally considered true that higher response rates can be achieved by using a third party research firm's Web site to collect confidential data.

Return to Top
We do not have the internal expertise and resources to conduct surveys of any kind let alone SMART surveys. Do you have any suggestions?

Like any business task, you have to assign resources to get it done. If the information a survey can provide is important to your success, put some resources against it. There are some basic survey software packages which can be self-taught, and Microsoft FrontPage 2000 has form functionality to create questions and collect information from Web site visitors. These can be useful for short polls and simple surveys if someone in your organization has the time to master the learning curve.

However, if your needs for sample design, questionnaire construction, question writing, data collection, data management, data analysis and report preparation are anything more than rudimentary, I would strongly suggest calling in a professional resource, either an individual consultant or a research firm, to help you successfully complete your survey project.

Return to Top
I have heard that SMART surveys cost less and are faster than print surveys. Is this really true, and if so, how much faster and cheaper are they?

It is true that SMART surveys can cost less and deliver findings faster than print surveys. They also generally deliver higher response rates and better quality comments on open-end questions.

How much "faster and cheaper" depends on the questionnaire and the participants. Shorter SMART survey questionnaires deliver quicker responses than lengthier, complex questionnaires. They also take less time to produce and deploy. Some populations are more high-energy cultures than others and respond more quickly.

In general, it takes anywhere from a day to a week to program and deploy an SMART survey once the questions are in hand. There is no mailing time involved. Most responses will come back within a few days with the rest coming in over a week to 10 days.

Reports can be ready within hours or days after the responses are collected depending upon the complexity of the reports because the data is already in electronic form. And the open-end comments are already typed, so reporting of these findings is eminently quicker and easier. Because people are typing their comments, they tend to write more complete ideas and expand on them. The whole process can be as short as 48-72 hours or as long as several weeks depending on the nature of the survey.

Regarding cost savings, there are no printing or postage costs, and the communications and reminders can be sent within minutes and include another copy of the questionnaire so you are assured everyone has a copy during the entire period of the survey. And, of course, with a Web-based survey, the questionnaire is always available. There are also no data entry costs and no transcription costs for open-end comments.

A general rule of thumb is that SMART survey data collection can cost a third to half of what print survey data collection costs. There is no way to estimate the benefit and opportunity costs saved by having information weeks or months sooner than from a print survey, but these can be considerable in a competitive, demanding environment.

Return to Top
 
©2012 GuideStar Research 212 426-2333 | Contact Us Privacy Policy