A laboratory analog of naturalistic observation was used to examine the effect of feedback on the reliability of observers working in pairs. Observer reliability was assessed using 2 methods of measurement: percent agreement of observer protocols to criterion protocols,and percent intro-pair agreement. Three levels of feedback were employed: total feedback in the fomr of intra-pair agreement and agreement with criterion protocols, intra-pair agreement feedback only, and no feedback. Results indicated a significant effect due to method of assessment: intra-pair agreement scores were significantly higher than observer to criterion agreement scores. Significant differences were also found for the feedback effect. The reliability for the total feedback group continued to improve under both methods of assessment; the intra-pair agreement-only group showed improvement in the intra-pair scores, but indicated decreases in reliability when assessed in terms of the criterion protocols; and the reliability for the no-feedback group decreased under both methods of assessment. Further results revealed that within each treatment condition intra-pair agreement scores were significantly higher than between pair agreement scores. The possible implications of these results for observational research, and suggestions for maintining observer reliability, were discussed.