Previous Page Table of Contents Next Page

8. Enhancing Thai AGRIS management with NLP techniques

To enhance the performance of the Thai AGRIS management, both in knowledge repositories and service provision, the centre has collaborated with the Specialty Research Unit of Natural Language Processing and Intelligent System Development (NAiST), within the computer engineering department at Kasetsart University. Some of the cooperative work has been mentioned in the previous chapters but the following specifically looks at the efforts to enhance the performance of the Thai AGRIS system by applying natural language processing (NLP) techniques.

The Thai AGRIS system currently includes two important functions: storage and retrieval. In the storage subsystem, Kasetsart researchers have developed an annotation tool for attaching metadata to a document. Additionally, a converter tool is being used to generate text representation from major electronic file formats. An automatic indexing tool is used to extract important keywords from the document. An image-enhancement tool is used to remove noise from the document image. In the retrieval subsystem, the researchers have developed an intranet search engine that provides a standard Boolean-query mechanism. Furthermore, it provides query expansion using ontology knowledge for improving the retrieval process efficiency.

Background and motivation

There are three main problems in Thai AGRIS management that affect the performance of both the information retrieval system and the machine translation system:

To enhance the Thai AGRIS management performance and address these three primary problems, the Thai AGRIS Centre has collaborated with Kasetsart’s NAiST laboratory in a project called Development of Agricultural Information System for Knowledge Acquisition and Dissemination. It is a five-year project, which began in 2001, to integrate Thai language processing techniques to Thai agricultural information processing. It consists of:

From STREDEO to Thai AGRIS management

STREDEO3 multimedia, multilingual document storage, retrieval and delivery system for e-organization is a prototype document management system being developed by the NAiST laboratory (research started in 2001). Because the Thai AGRIS Centre was using the same concept model, the STREDEO was applied to the Thai AGRIS system management, consisting of automatic document indexing for both image and text, automatic document classification and dissemination, and intelligent search engine with query expansion:

3 Funded by KURDI (Kasetsart University Research and Development Institute).

Previous problems in document management systems of the Thai AGRIS Centre

In general, document management systems still have a few shortcomings that need to be resolved:


Electronic documents usually store in a variety formats. Some of them are not open formats and the mechanisms for converting their content are not available.

Document representation is generally not enough for effective retrieval. A good designed tool must be developed to ease the task of annotation, such as including automatic indexing with the metadata annotating tool.

Maintenance of document images. Scanned documents may contain noise, which should be removed before further processes such as OCR.


Use of controlled vocabularies in natural language processing on search queries causes low precision.

In light of these problems, the STREDEO model has been applied to Thai AGRIS management as follows:

Annotation tool for intelligent management of knowledge repositories

Figure 8 shows the overview of a developed system, consisting of an annotation tool, an image enhancement tool and a search engine module.

Figure 8: Electronic document management system architecture

Figure 8

The annotation tool is responsible for generating document representation. It consists of three main components: converter (converts the electronic document to standard text file format), automatic indexing (used to generate keywords of document automatically) and annotation interface (allows user to attach metadata to the document).

To remove the noise from the document image, NAiST researchers have developed an image enhancement tool that is essentially a pipeline of three image enhancement processes consisting of deskew, auto crop and auto align.

Smart search for smart service

The search engine module is derived from the Kasetsart University search engine, which uses a Thai text retrieval system for intranet document searching. It has two functions: basic search and advanced search. The basic search, like several search engines, provides a standard Boolean-query search mechanism.

The advanced search is used to increase the efficiency of a search engine. To improve the effectiveness of information retrieval, the query expansion has been developed. It uses knowledge from metadata information and an ontology in order to uncover indirect connections between query terms and document content. Both of them can improve searching performance.

Automatic Thai ontology construction tool

Ontology plays an important role in many applications, such as information integration and document classification in taxonomies, including the information retrieval system. To enhance a search engine performance, ontology is used to boost the correctness and coverage of the system. However, there remains a problem of building large and adequate ontology within a short time frame and at low cost. The following present both a recycling approach to engineering ontology and general maintenance architecture for discovering new instances from text.

An overview of the ontology construction and maintenance system

Figure 9 illustrates on overview of the ontology system, which contains four steps: lexical pre-processing (for identifying word strings), ontology term acquisition and relation extraction (to extract all single nouns and compound nouns from the tagged corpus and their relations), and term/relation organization (to organize the terms according to the relations).

Figure 9: Thai ontology construction and maintenance system architecture

Figure 9

From Thai AGROVOC to ontology relationship

The AGROVOC database was converted into the text file to recycle the original relation to ontology relationship (e.g. from BT/NT to IS-A). However, all BT/NT relations in AGROVOC could not be defined to the IS-A relation. Their semantic relations could be defined as “ingredient of” and “property of”.

Heuristic method has been used to resolve this problem. A compound noun in NT will be processed to find a head noun; if head noun is consistent to BT, their relation will be defined to IS-A relation. If they are not compatible, the system will alert an expert to confirm this relation. This method will create many sub-trees where the root is a top term in AGROVOC. In the future those top terms will be extended by using the corpus-based ontology extraction algorithm for maintaining and constructing large ontology.

Previous Page Top of Page Next Page