Database Record Duplicate Detection System using Simil Algorithm

Abstract

As more data is populated into the database table, there is the tendency for the table to store duplicate or redundant record which results in the consumption of data spaces in the database and also in the storage device where the database resides. Despite the ever-increasing memory capacities of devices, significant benefits can still be realized by reducing the bytes size needed to represent an object when it is stored or retrieved from the database. This is quite beneficial to mobile devices with limited storage, reference data, e-mail, where sequences of large bytes are repeated and data transmitted over lowbandwidth or congested links. Reducing bytes equates to eliminating unneeded data, and there are numerous techniques for reducing redundancy when objects are stored or sent. This study implemented a database record duplicate detection system using simil algorithm as the reduction technique to achieve efficiency in detecting and reducing the presence of duplicate records in a database and hence provides an automated means of executing database record optimization.