Skip to content
wingsofovniaPublic

About

Implementation of Shingles Algorithm for searching near duplicate documents using Apache Spark

Resources

Stars

2 stars

Watchers

2 watching

Forks

Latest commit

 

History

36 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Plagio

Plagio is a simple implementation of Shingles Algorithm for searching near duplicate documents Apache Spark engine.

Plagio was developed for my thesis Parallel and Distributed Algorithms for Large Text Datasets Analysis as a demonstration of using distributed processing solutions, such as Apache Spark, for distributed textual data processing.

Program available on "as is" basis and created only for education purposes.

Requirements

  • Java Development Kit 1.8
  • Apache Spark 1.5.2

License

MIT

About

Implementation of Shingles Algorithm for searching near duplicate documents using Apache Spark

Resources

Stars

2 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors

Languages