The good news is that lots of people have been there before you, so there’s no need to reinvent the wheel!  Various online survey tools have captured this experience, making it very easy for a beginner to design a questionnaire, and also saving time for those experienced in designing surveys.

Whether your survey is to be online, a phone survey, or a postal survey, it’s well worth using a survey design tool to develop your questions. Then, if need be, you can transfer them to a printed version.

The tool I use, and thoroughly recommend to my clients, is Survey Monkey, which you can find at www.SurveyMonkey.com. It’s even free for up to 10 questions and up to 100 responses. Beyond that there’s a very affordable monthly fee, which you can end at any time.

Five ways Survey Monkey slices chunks out of questionnaire design time

1. Sample question types

When you Add a question, there’s a drop down list of questions types to choose from. For example: multiple choice,  rating scale, textbox etc.

If you’re not sure which type of question you need, simply click on the ‘Examples’ to see what each question type looks like.

2. Automatic layout

Suppose you’ve chosen to use a multiple choice question. You simply enter the question, and the possible responses. Hey Presto! Survey Monkey automatically lays it out for you.

3. Easy to change the question type

Once you see your question you may decide multiple choice isn’t right; a rating question would be better. Simply change the question type, and you’ll be prompted to add your rating categories eg low, medium, high. The layout is changed instantly.

4. Easy to reorder questions

As you develop your questionnaire, the chances are you’ll realise, for example, that question 5 about the type of training undertaken, needs to come before question 8 which asks their opinion of the training. No problem - just drag and drop, and the questions are automatically renumbered for you.

5. Easy to circulate for feedback

You can quickly send your questionnaire to people for comment before you finalise it. Either send them the link to the online survey, or send them a PDF version which is easily printed.

When you have a questionnaire to develop, use an online survey design tool such as Survey Monkey. You’ll get ideas for question types to use, and progress much more quickly and with more flexibility than you ever could using word processing.

Imagine you’ve been asked to book a restaurant in a strange town for your group’s dinner. And they agreed they would prefer Greek. You trawl through Yellow Pages, but can’t find a Greek restaurant, so you go for Thai instead. How embarrassing when you find a Greek restaurant right next to the Thai one! It turns out you were using an old edition of Yellow Pages.

Similarly for your list, it needs to be robust to give you a representative sample of feedback from your survey.

Three ways to ensure your list is robust

1. Ensure it’s up to date

It’s easy to check the date of Yellow Pages. But what about a list of businesses on a website, or from your organisation’s database?

• It’s surprising how many lists have no dates. So you may need to do some investigation to check the source of the list.
• If it’s a year or more old, you will probably need to update it. It’s a good idea to check how the list was collated in the first place, so you can follow the same process.

2. Ensure it’s comprehensive

• Check your list includes the full range of people you want to consult.
  Suppose you need to contact a sample of people from the construction sector. Your list will need to include builders, joiners, electricians, plumbers etc. If for some reason your list has no plumbers, you will need to add them to the list, or your survey will not represent all the trades.
• Check your list represents the time period you want feedback from.
  If you seek feedback from your clients during the past two years, you need to be sure your list is complete for two years, so your sample will be a true representation of all your clients. If the first six months of your project are missing, your sample will not represent the past two years.

3. Eliminate any duplicates

When lists are updated, extended, or merged, there’s a real danger of introducing duplicates. If someone on the list is contacted twice they won’t have much faith in your project, and it’s a waste of your resources. Also, if you are drawing a sample from your list, your sample will be distorted by duplicates.

Far better to eliminate any duplicate entries first! Here’s how:

• Use the ‘Sort’ facility in Word, Excel or other software to put your list into alphabetical order.
• Scan your list for duplicate entries.

To ensure your list forms a robust basis for your survey, you need to check that it’s up to date, comprehensive and free from duplicates. Then youcan be confident your survey has a firm foundation.

Remember the last time you received an email asking you to ‘complete a quick survey’, or you opened an envelope to find a questionnaire inside, or you answered the phone to hear the dreaded “Please could you spare a few minutes…”.

How did you feel?

The trick to getting a good survey response is to put yourself in the shoes of the recipient, and identify what would make you more likely to be motivated to participate.

Three tried and tested strategies to increase your response rate

1. Provide an incentive

People are busy, they need a good reason to give up time to answer your questions. Providing an incentive is a good ‘carrot’ to attract people who may be swithering about whether to take part in your survey.

Incentives can take many forms: from money to vouchers to goods to a free service or subscription

• Check your organisation/funders policy on incentives – if it excludes spending money on them, either find something you can offer ‘in kind’, or get an organisation or business to donate the incentive.

• Consider the target group for your survey, and brainstorm a list of things they would value. I have used gift vouchers, money, books, and a free session.

• Describe your incentive in an attractive way, and highlight it early on in the Introduction to your survey.

2. Make it easy

The easier your questionnaire is to complete, the more people will fill it in. Any barriers provide an excuse to ‘put it on one side’. We recently received a questionnaire 6 years after the close of the survey!

• Start with a question which is very easy to respond to.

• Use ‘tick box’ responses for the majority of questions.

• For postal surveys, provide a ‘post paid’ return envelope.

3. Keep it short

The shorter your survey, the more responses you are likely to get. This is particularly the case with a printed questionnaire, where the respondent can immediately see how ‘long’ it is.

• Test your questionnaire with a few ‘real’ respondents to find out how long it takes to complete. If it’s more than 10 minutes consider pruning your questions (though an acceptable time will vary for different audiences).

• Include the time the questionnaire will take to complete in the Introduction to the survey.

• Be creative with layout to make the questionnaire look ‘short’.

By providing an incentive, and making your questionnaire quick and easy to complete, you will increase the number of people who respond. This in turn makes the feedback from your survey more robust.

1. Get people on board

Getting people on board at the outset saves hassles later on. You don’t want to work all the way through the evaluation, come up with a good set of recommendations and then find senior management have other priorities, or project staff are resistant to the proposed changes.

Keys

• Prepare your reasons why the evaluation will benefit: your organisation / the project / partners / stakeholders / other parties, taking into account the wider picture of these parties, and how your project fits in.
• Be prepared to genuinely listen to difficulties the various parties may raise, and then seek ‘win-win’ solutions to these difficulties. This may involve modifying your plans.
• Get people on board at the outset, that way you can create a positive environment from the start.

2. Foster a positive environment

A positive environment around the evaluation will help diminish the threat that some people feel when they hear the ‘evaluation’ word. Those closely involved with the inception of the project, and those running it on a day to day basis may think of it as ‘their baby’, and feel defensive at the thought of it being ‘assessed’, particularly if an outsider is involved. This might discourage them from engaging fully in the evaluation.

Keys:

• Tell people about the evaluation as early as possible, and take every opportunity to remind them about it.
• Keep them fully informed as the evaluation progresses, particularly of changes.
• Show your own positive attitude in practical ways, and make it clear you’re personally learning from the evaluation.

3. Treat the evaluation as a learning opportunity

A good way to take the threat out of the evaluation is to treat it as a learning opportunity. In this way ‘criticisms’ can be turned into ‘suggestions’, and ‘mistakes’ into ‘lessons for the future’.

Keys:

• Proactively adopt a no-blame culture yourself; where mistakes are OK so long as lessons are learned.
• Encourage this learning culture within your team and wider sphere of influence.
• Build a requirement to learn from the evaluation process into the evaluation specification.

4. Allocate time to the evaluation

If you’re short of time you may plan to contract some or most of the evaluation out to a consultant. If this is the case, you’ll still need to spend time helping the evaluator understand what your project aims to achieve, what your stakeholders are looking for etc. Otherwise you may end up with an evaluation report which isn’t particularly helpful to you.

Keys:

• Schedule enough time to ensure any consultant involved really understands your project – how it originated, its aspirations, and how it works on a day to day basis, They will also need to know how your organisations works.
• Schedule admin time to collate lists of clients, stakeholders, project achievements, and any monitoring data. It’s very unlikely an outsider can do this for you.
• Schedule regular times into your diary to be in touch with the evaluation. If everything runs smoothly they may not all be needed; if there are hitches you’ll have the time to deal with them.

5. Develop a specification for your evaluation

Without a spec, like a meeting without an agenda, evaluation feedback will drift off in all directions, and you’ll have a hard job drawing any conclusions.Even if you plan to carry out the evaluation yourselves, you’ll need a specification of what it aims to achieve, as well as any constraints such as timescale and budget.

You can use the evaluation specification as the basis of a contract with any consultant you decide to appoint to help with the evaluation.

Keys:

• Prepare the evaluation aims and objectives in consultation with colleagues, project staff, funders and any partners. This will help get them on board, and the whole process will focus attention on your project.
• Focus the evaluation on key project objectives; usually it’s impractical to check out achievement of all of a project’s aims and objectives.
• Include timescale and budget, and any suggestions you have for the evaluation methods. Be open to revising the specification if needed as the project progresses.

You’ve probably noticed that three of the five steps to a hassle-free evaluation are ‘people’ factors: getting them on board, fostering a positive environment and treating the process as a learning opportunity.

Taking steps to address these ‘people’ aspects greatly increases your chances of setting up the evaluation, running it and achieving positive follow-up. Someone inside the organisation needs to be involved in addressing these steps; they can’t readily be achieved by an outsider, though an outside perspective can certainly help.

Of course there can be all sorts of technical hitches too, and an experienced evaluator will help you take care of these. Allocating time to the evaluation and having a clear evaluation specification will minimise these hitches.

Address these five steps and you’ll be well on the way to a hassle-free evaluation!

May Johnstone, 2007, www.mayjohnstone.co.uk. Please feel free to circulate this article provided it is used in its entirety, including this acknowledgement.