Creating parallel and comparable corpora for work in domain specific areas of language

Creating parallel and comparable corpora for work in domain specific areas of language

Belinda Maia

FLUP

Parallel corpora - definition

• “A parallel corpus is a collection of texts, each of which is translated into one or more other languages than the original. The simplest case is where two languages only are involved: one of the corpora is an exact translation of the other. ....... The direction of the translation may not even be known”.

Parallel corpora - uses

• “Parallel corpora are objects of interest at present because of the opportunity offered to align original and translation and gain insights into the nature of translation. From this work it is hoped that tools to aid translation will be devised. Probabilistic machine translation systems can moreover be trained on such corpora”.

Comparable corpora - definition

• “A comparable corpus is one which selects similar texts in more than one language or variety. There is as yet no agreement on the nature of the similarity, because there are very few examples of comparable corpora”.

Comparable corpora - uses

• “The possibilities of a comparable corpus are to compare different languages or varieties in similar circumstances of communication, but avoiding the inevitable distortion introduced by the translations of a parallel corpus”.

Quotations from:

• EAGLES - Expert Advisory Group on Language Engineering Standards
• Guidelines – 1996 – at:
• http://www.ilc.pi.cnr.it/EAGLES96/browse.html

Parallel corpora - alignment & annotation

• Most common form of alignment

= at sentence level

• E.g. Text aligners:
• WORDSMITH – recognizes full stops only
• WinAlign – TRADOS – recognizes a certain amount of formatting, paragraphs, numbers, tagging
• Ongoing research to align at:
• term/word level
• tag level