Plagio is a simple implementation of Shingles Algorithm for searching near duplicate documents Apache Spark engine.
Plagio was developed for my thesis Parallel and Distributed Algorithms for Large Text Datasets Analysis as a demonstration of using distributed processing solutions, such as Apache Spark, for distributed textual data processing.
Program available on "as is" basis and created only for education purposes.
- Java Development Kit 1.8
- Apache Spark 1.5.2
MIT