CI/CD: Duplicate issue detection and marking

[iamdwaseem]

🚀 CI/CD Enhancement: Automated Duplicate Issue Detection

📌 Problem

Currently, contributors must manually search through existing issues before opening a new one.

This process:

• Is time-consuming
• Is often skipped
• Leads to duplicate issues
• Increases manual triaging effort for maintainers

As the repository grows, identifying duplicate issues becomes more difficult and inefficient.

---

💡 Proposed Solution

Introduce an automated duplicate issue detection system using a GitHub Action and bot-based workflow.

🔄 Workflow Overview

1. A new issue is opened
2. GitHub Action is triggered
3. The system compares the issue title and description with existing issues
4. Semantic similarity is calculated
5. If similar issues are found:
   • A comment is posted listing related issues
   • A label such as possible-duplicate is added

⚠️ The system will only suggest duplicates and will NOT auto-close any issue.

---

⚙️ Technical Approach (High-Level)

• Trigger: on: issues: [opened]
• Use semantic similarity (e.g., sentence-transformers)
• Compute cosine similarity scores
• Apply configurable similarity threshold
• Comment with top similar issues
• Add label for maintainer review

Threshold and behavior can be configurable via repository variables.

---

🎯 Expected Impact

• Reduced duplicate issues
• Faster issue triaging
• Cleaner issue tracking
• Improved contributor experience
• Reduced maintainer workload

---

🛡️ Safety Considerations

• No issue will be automatically closed
• Maintainers retain full control
• Only non-destructive suggestions will be made

---

🖼️ Screenshots

N/A (CI/CD enhancement proposal)

---

📝 Record

• I agree to follow this project's Code of Conduct
• I want to work on this issue

[iamdwaseem]

Hi @Saahi30 👋

I would like to work on implementing automated duplicate issue detection for this repository.

🎯 Goal

When someone opens a new issue:

• Workflow triggers automatically
• Fetch existing issues
• Convert issue text → embeddings
• Compute semantic similarity
• If similar → comment + add label
• No auto-closing (maintainer-controlled)

---

🧱 Proposed Architecture

New Issue Opened
↓
GitHub Action Triggered
↓
Python Script Runs
↓
Fetch Issues via GitHub API
↓
Generate Embeddings (sentence-transformers)
↓
Cosine Similarity
↓
If similarity > threshold:
→ Comment
→ Add possible-duplicate label

---

⚙️ Implementation Plan

1️⃣ Workflow Setup

• Create .github/workflows/duplicate-check.yml
• Trigger on:

  on:
    issues:
      types: [opened]

• Use Ubuntu runner + Python 3.10
• Install required dependencies

---

2️⃣ Issue Fetching

• Fetch open issues via GitHub REST API
• Exclude current issue
• Limit to recent 100 issues (performance optimization)

---

3️⃣ Semantic Similarity

• Use all-MiniLM-L6-v2 (pre-trained sentence transformer)
• Generate embeddings for:
  • New issue
  • Existing issues
• Compute cosine similarity
• Apply configurable threshold (e.g., 0.6)

---

4️⃣ Automated Response

If similarity exceeds threshold:

• Post comment listing top similar issues
• Add label: possible-duplicate
• No automatic closing

---

🛡️ Safety Measures

• No destructive actions
• Maintainers retain full control
• Threshold configurable via repo variable
• Only suggestion-based detection

---

I would be happy to implement this following the repository’s contribution guidelines.

Please assign this issue to me if approved. 🙌