Evaluation of Natural Language Processing Techniques for Information Retrieval

Abstract

Purpose: The main objective of the study was to investigate the evaluation of Natural Language Processing techniques for information retrieval.

Methodology: The study adopted a desktop research methodology. Desk research refers to secondary data or that which can be collected without fieldwork. Desk research is basically involved in collecting data from existing resources hence it is often considered a low cost technique as compared to field research, as the main cost is involved in executive’s time, telephone charges and directories. Thus, the study relied on already published studies, reports and statistics. This secondary data was easily accessed through the online journals and library.

Findings: The findings reveal that there exists a contextual and methodological gap relating to Natural Language Processing techniques for information retrieval. Preliminary empirical review revealed that NLP methods significantly improved the accuracy and efficiency of information retrieval systems. Through systematic evaluation, various NLP techniques, including tokenization, named entity recognition, semantic parsing, and word embeddings, were found to enhance retrieval performance across diverse datasets and domains. By considering context and user intent, researchers aimed to develop more contextually aware and personalized information retrieval systems. The study emphasized the need for further research to explore hybrid approaches and domain-specific adaptations, ultimately highlighting the transformative potential of NLP in revolutionizing information access and utilization.

Unique Contribution to Theory, Practice and Policy: Information Foraging theory, Relevance theory and Cognitive Load theory may be used to anchor future studies on Natural Language Processing techniques. The study provided recommendations to enhance information retrieval systems. It suggested integrating advanced NLP techniques such as named entity recognition and sentiment analysis to improve query understanding and document relevance. Additionally, the study recommended adopting word embeddings and semantic parsing techniques for better semantic understanding of user queries and documents. It emphasized the importance of domain-specific adaptations and continuous evaluation of NLP techniques to tailor them to specific application domains and keep pace with evolving user needs and technological advancements.