Last week I started the task of evaluating the language models that would be trained on the data without outliers in sentence length. This involved running a couple programs with each of the languages and training the parsing program which can take several hours per language depending on the amount of data. After the models were trained I had to use them to parse a set of testing data and compare it to models that were trained on the entire set. For most of the languages there was a 5-15% decrease in performance but this can mainly be attributed to the significant loss in training data, which means the program may not get to see a certain word in the training data that it sees on the testing data. In this type of scenario most of these types of programs will assume it to be a proper noun since it is more likely to not have seen a proper noun than an uncommon verb. This can lead to problems especially when the language you are training starts with a low amount of sentences to train with for example the set of Ukrainian sentences I have available only has 37 sentences and after trimming the data its accuracy dropped to about 3%.
0 Comments
|
|