preview

What Is The Process Of Front End Text Processing

Decent Essays

\subsection{Front-end text processor} \label{subsec:front-end}
Front-end text processor produces Linguistic representation from the input text. This component does the majority of the text pre-processing and then gives the Linguistic Representation of the text to the Statistical Model. The task of pre-processing varies based on what kind of Front-end it is. There can be two types of front-end.
\begin{enumerate}
\item Trained front-end. \item Minimal front-end.
\end{enumerate}
\subsubsection{Trained Front End}
By trained front end, we mean a front end text processor which is trained on a specific language using language specific rules, grammar etc, e.g: Front end text processors of Festival \cite{festival}, Mary Text to …show more content…

It depends on what type of text we are trying to normalize. There is no one consistent algorithm to normalize texts. It varies from language to language. One example of text normalization can be processing of NSW or Non-Standard Words, such as- year, cardinal number, ordinal number, acronyms etc. \par For example: For example: Consider the Bangla sentence, \textbengali{"আমি পরীক্ষায় ৭ম হয়েছি।"} Here, \textbengali{"৭ম"} is an ordinal number which is supposed to be pronounced as \textbengali{"সপ্তম" }। During text normalization we can replace \textbengali{"৭ম"} with \textbengali{"সপ্তম" }. \item \textbf{POS Taggging}\\ POS tagging or Parts of Speech tagging is labeling the tokens as their Corresponding Parts of Speech. POS tagging is done keeping in consideration what parts of speech the token is in its sentence. POS tagging helps in deciding prosodic information as different parts of speech tend to be pronounced differently sometimes.
.
\item \textbf{Phoneme Detection}\\ Phoneme detection is creating the pronunciation model, which contains phoneme set, that the text must follow. The tokens are basically broken down into phonemes by looking up a dictionary or some rules. A vast knowledge of the language is required to create the phoneme set and the dictionary. \item \textbf{Phrase Break}\\ Finding the proper position to put the prosodic break is another important step in

Get Access