*Sri Ramakrishna Engineering College, Coimbatore, India
**Department of Electronics and Communication Engineering, M. Kumarasamy College of Engineering, Karur, India
Online published on 14 October, 2016.
Most of the deduplication process requires similarity function which address whether the two entries are duplicate or not by setting the threshold. Setting the threshold is an important issue and it relies more on human intervention. The proposed work addressed two problems: first to find the optimal equation using Genetic Algorithm(GA) and next it adopts an intelligence algorithm, Artificial Bee Colony (ABC) to get the optimal threshold to detect the duplicate records more accurately and also it reduces human intervention. Two dataset Restaurant and CORA are considered to analyze the proposed algorithm.
GA, ABC, Similarity metrics, Cosine Similarity, Levenshtein Distance
