Colombian Sign Language Dataset (CoL-SLTD)
Description
Overview
Sign language, in general, preserves the structural communication Subject-Verb-Object, expressed as a visual combination of hand shapes, articulator locations, and movements (Stokoe, W. C. (1980). Sign language structure). The motion information is considered the core of the SL, allowing, among others, to differentiate signs related to the pose and also to define the verbal agreement in the sentences (Sandler, W. (2012). The phonological organization of sign languages). For instance, in American SL, the expression of "I give You" has a similar geometrical description that "You give her", the biggest difference is given by motion direction. Also, while the handshapes represent noun classes, the combination with motion patterns could represent associated verbs and complete utterances (Supalla, T. (1986). The classifier system in American sign language).
Complexity
The proposed dataset incorporates interrogative, affirmative, and negative sentences from Colombian Sign Language. Furthermore, this dataset includes different sentence complexities such as verbal and time signs that define subject and object relationships, such as the phrase: "Mary tells John that she will buy a house in the future". This dataset also includes signers of different ages to avoid bias in the analysis and to capture a large variability for the same language.
In CoL-SLTD, so far, there are 39 sentences, divided into 24 affirmatives, 4 negatives, and 11 interrogative sentences. Each of the sentences has 3 different repetitions, for a total of 1020 sentences, which allows capturing sign motion variability related to specific expressions. Also, the phrases were performed by 13 participants (between 21 to 80 years old), with sentence length between 2 to 9 signs and the videos have an average length of 3.8 seconds and an average number of frames of 233.
Evaluation Scheme
Two different evaluations are proposed over Col-SLTD. In a first evaluation, a signer independence split aims to evaluate the capability to translate sequences of signers no seen during training. In this split, a total of 10 signers were selected for training and 3 signers with different ages for testing. In a second evaluation, the task should report the capability to generate sentences not seen during training. In this task, a total of 35 sentences were selected in training and 4 sentences in testing. The words in test sentences have the highest occurrence in training and the sentences involve affirmations, negations, and interrogations. The following table summarizes the statistics per split.
