First Cycle

First user-centered-design cycle: proof of concept

The goal is to design a usable CAVL (computer-assisted vocabulary learning) software for the visually impaired. This means that the primary output mode should be auditory. In terms of content, there should be a good balance between addressing vocabulary items with regard to their meaning and their orthographic form. 

The key objective of the first cycle in the development of AVoS was to evaluate the basic concept of an auditory CAVL application for the visually impaired. Details of this study can be found in (Stein, 2010). The evaluation of an AVoS prototype took place under controlled testing conditions at the Carl-Strehl Gymnasium in Marburg, Germany. 15 visually impaired pupils from levels 6 to 9 participated in the study, aged 12 to 16. The evaluation was divided into three parts. (1) The subjects were asked to complete a questionnaire to enquire about their vocabulary study and computer usage habits. (2) Test persons worked with the AVoS prototype and completed a pre-test and a post-test to measure their progress. This served to measure both spelling and vocabulary performance objectively. To this end, a randomised and controlled [2 x 2] factorial between-groups design was adopted. The subjects were assigned randomly to one of two orthographic feedback conditions: simple auditory feedback (S-AF) and prosodically enhanced auditory feedback (PE-AF). In the S-AF condition, words were spelled aloud in the case of a spelling error. In the PE-AF condition, words were also spelled aloud in the case of a spelling error but the errors were located by prosodically enhancing their position. (3) The students filled in a questionnaire to report their user experience. The following gives an overview of our findings from the evaluation upon the completion of the first cycle in the AVoS development.

(1) What are typical vocabulary study and computer usage habits among visually impaired language learners?

The test subjects of this study relied extensively on braille reading and list learning when studying foreign vocabulary and orthography. They did not benefit from index cards in braille script or from the auditory output of the PC. Further, subjects singled out spelling as their biggest handicap in the L2 classroom. In terms of mental orthographic representation, they indicated that they picture spelling both haptic and auditorily. When using the computer, the test subjects relied on both the screen reader and the braille display but prefered the screen reader. Spelling errors in computer documents were commonly corrected using the spelling option of the screen reader or the brailler display, again with a preference for the screen reader. 

(2) Do test subjects improve their vocabulary and spelling performance when they use AVoS? Does the prosodically enhanced feedback have any effect on perfomance?

Working with AVoS improved both vocabulary performance (Pillai’s trace: F(1;9)=129.452, p<0.001 (two-sided)) and spelling performance (Pillai’s trace: F(1;9)=129.898, p<0.001 (two-sided)) significantly between the pre-test and the post-test. The prosodically enhanced auditory feedback (PE-AF) contributed to a significantly larger learning gain for spelling performance than the simple auditory feedback (S-AF) (Pillai’s trace: F(1;9)=5.030, p=0.026 (one-sided)). For vocabulary performance no difference between the two conditions was found. Subjects who received the prosodically enhanced feedback were able to correct their mistakes faster. However, this result was not significant.

(3) Which features of AVoS work well and which could be improved?

The test subjects classified the design of the feedback generally as usable, irrelevant of which testing condition they were assigned to. 14 out 15 test subjects indicated that they would like to work with such a tool to study vocabulary and spelling. They pointed out that they found it helpful that the entire word was spelled after a mistake, that words were repeated until they were known, that they had access to several control options, and that they could only move on to the next word once the correct word had been entered. In terms of the general system design, most subjects commented positively on the diversity of the feedback utterances, while some found this feature somewhat annoying. The missing braille line was not a problem, it was rather seen as a benefit by some subjects. The speech synthesis that was adopted received largely negative comments and should ideally be replaced by better voices.