• twitter
  • facebook
  • google+
  • youtube
  • Follow Me on Pinterest
Frequently Asked Questions

 

Q: Can you test the reliability of your survey before you give the survey?

No. Survey Reliability is determined by the answers on the survey

Q: What is the difference between Statistical Power and Survey Reliability

Statistical power is often used to determine the number of people (sample size) needed for your survey based on the overall population. So if you known the population of a city or a company or a school, you could determine the appropriate number of people you need to survey to have a representative sample of a population. However, statistical power does not determine whether your survey is reliability. The analysis only assumes your survey is reliability. On the other hand, Survey Reliability measure whether the answers to questions that make up your survey is reliability.

Q: Can negatively worded/coded questions affect survey relaiblity scores?
Yes, Before using Relicheck, you may want to reverse code negatively worded  questions or questions that are coded the opposite of the other questions in your survey. For instance if in your survey 1 = strongly disagree and 5 = strongly agree, you have a questions ask, “Do like your boss” and then ask the question “Do you dislike your boss?” If the once answer is 5 = strongly agree and the answer to the second question is 1 = strongly disagree, the questions are product the same answers but are worded opposite of each other.

When testing Survey Reliability, Relicheck will treat all questions as positively worded. It means that questions can have internal consistent, but the analysis will show the negative question as having a large negative ITC (greater than -3.0). So, if you did not reverse cod questions before uploading the survey to Relicheck, keep in mind that questions with high negative ITC could be negatively worded/coded.
Q: Can the number of participants affect survey relaiblity scores?.
Yes, Common practice is that before a survey can be questioned for reliability, if survey should have a 1: 10 ratio of participants to questions. This means that for each question on your survey, you should other ten participants answer the question. So if your survey consists of 10, to get an accurate reliability score, you must have at less 100 people. Few researchers suggest that a 5:1 ratio is appropriate.
10:1 Ratio 5:1 Ratio
# of Questions on survey

# of participants needed

 

# of Questions on survey

# of participants needed

1

10

 

1

5

2

20

 

2

10

3

30

 

3

15

4

40

 

4

20

5

50

 

5

25

6

60

 

6

30

7

70

 

7

35

8

80

 

8

40

9

90

 

9

45

10

100

 

10

50

11

110

 

11

55

12

120

 

12

60

13

130

 

13

65

14

140

 

14

70

15

150

 

15

75

16

160

 

16

80

17

170

 

17

85

18

180

 

18

90

19

190

 

19

95

20

200

 

20

100

Q: What are symbols for reliability?
r = reliability
α = alpha (Cronbach Alpha)
Q: What is a good reliability score?

Reliability scores range from 0 to 1.0, with 1.0 being the highest. A commonly-accepted rule of thumb is that a reliability score between .60 and.70 indicates acceptable reliability and .80 or higher indicates good reliability. High reliabilities (0.95 or higher) are not necessarily desirable, as this indicates that the questions may be entirely redundant. The goal a survey is that it is a reliable instrument for which scores on similar questions relate (internally consistent), but also for each questions to contribute some unique information as well.

Q: What is survey reliability and why is it important?
Survey Reliability is a measure that determines the reproducibility of your survey. In other words, survey reliability determines if the answers produced by your survey can be used over and over again to produce reliability answers. <br />
Why is survey reliability important? Surveys provide critical information needed to make valuable decisions. However, not all surveys produce reliable answers. The questions on your survey may seem reasonable to answer. However, the questions may not make sense to the person taking the survey. Therefore the survey can produce non-reliable answers. It is important that the answers on your survey are reliable and provide relevant information you can use.
Q: What type of reliability test does Relicheck use?
There are several types of reliability: test-retest reliability, alternate-form reliability, internal consistency reliability, internal consistency reliability, intraobserver reliability, and interobserver reliability. The commonly used reliability test is internal consistent reliability. Relicheck uses this reliability test to determine survey ability. This test of reliability examines the relationship between the relationship and whether the questions are the survey product consistent responses.
Q: What is Item Total Correlation?
Item Total Correlation (ITC) (aka Inter-Item Correlation) is the relationship (correlation) between a question (item) and the sum of the remaining questions on the survey. For instance, if you had a survey with 10 questions, the ITC will be between question 1 and the sum of questions 2 – 10, and question 2 and the sum of questions 1 and 3 through 10, etc. The ITC ranges from 0 to 1.0 with 1.0 being the highest.