🧪 DECIMER Image Transformer 🖼️

Deep Learning for Chemical Image Recognition using Efficient-Net V2 + Transformer

---

📚 Table of Contents

• 📝 Abstract
• 💡 Method and Model Changes
• ⚙️ Installation
• 🚀 Usage
• ✍️ Hand-drawn Model
• 📄 Citation
• 🙏 Acknowledgements
• 👨‍🔬 Author
• 🌐 Project Website
• 🏛️ Research Group

---

📝 Abstract

The DECIMER 2.2 project tackles the OCSR (Optical Chemical Structure Recognition) challenge using cutting-edge computational intelligence methods. Our goal? To provide an automated, open-source software solution for chemical image recognition.

We've supercharged DECIMER with Google's TPU (Tensor Processing Unit) to handle datasets of over 1 million images with lightning speed!

---

💡 Method and Model Changes

🖼️ Image Feature Extraction Now utilizing EfficientNet-V2 for superior image analysis

🔮 SMILES Prediction Employing a state-of-the-art transformer model

🚀 Training Enhancements

1. 📦 TFRecord Files - Lightning-fast data reading
2. ☁️ Google Cloud Buckets - Efficient cloud storage solution
3. 🔄 TensorFlow Data Pipeline - Optimized data loading
4. ⚡ TPU Strategy - Harnessing the power of Google's TPUs

---

⚙️ Installation

# Create a conda wonderland
conda create --name DECIMER python=3.10.0 -y
conda activate DECIMER

# Equip yourself with DECIMER
pip install decimer

---

🚀 Usage

from DECIMER import predict_SMILES

# Unleash the power of DECIMER
image_path = "path/to/your/chemical/masterpiece.jpg"
SMILES = predict_SMILES(image_path)
print(f"🎉 Decoded SMILES: {SMILES}")

📥 Models: downloaded on first use

import DECIMER does not download or load anything. The model weights are not part of the pip package; they are downloaded from Zenodo the first time they are needed and cached afterwards:

• The first predict_SMILES(...) call downloads the printed-structure model (about 300 MB) if it is missing, loads it, and keeps it in memory. Later calls reuse it.
• The hand-drawn model is only downloaded and loaded by the first predict_SMILES(..., hand_drawn=True) call. (Up to version 2.8.0 both models were downloaded and loaded when DECIMER was imported.)
• To pay for the download and the loading at a time of your choosing (at the start of a server, or when building an image), call load_models() once. load_models(hand_drawn=True) loads both models, as import DECIMER did before.

import DECIMER

DECIMER.load_models()  # optional: download if missing and load now
SMILES = DECIMER.predict_SMILES("path/to/your/chemical/masterpiece.jpg")

The models are stored in $PYSTOW_HOME/DECIMER-V2/ (~/.data/DECIMER-V2/ by default), where PYSTOW_HOME is read when the models are first needed. A directory that already holds DECIMER_model/ (and, for hand-drawn structures, DECIMER_HandDrawn_model/) is used as it is, without any network access, so an image with pre-installed models works offline:

export PYSTOW_HOME=/opt/decimer-models   # contains DECIMER-V2/DECIMER_model/saved_model.pb, ...

---

✍️ DECIMER - Hand-drawn Model

🌟 New Feature Alert! 🌟

Our latest model brings the magic of AI to hand-drawn chemical structures!

---

📄 Citation

If DECIMER helps your research, please cite:

1. Rajan K, et al. "DECIMER.ai - An open platform for automated optical chemical structure identification, segmentation and recognition in scientific publications." Nat. Commun. 14, 5045 (2023).
2. Rajan, K., et al. "DECIMER 1.0: deep learning for chemical image recognition using transformers." J Cheminform 13, 61 (2021).
3. Rajan, K., et al. "Advancements in hand-drawn chemical structure recognition through an enhanced DECIMER architecture," J Cheminform 16, 78 (2024).

---

🙏 Acknowledgements

• A big thank you to Charles Tapley Hoyt for his invaluable contributions!
• Powered by Google's TPU Research Cloud (TRC)

---

👨‍🔬 Author: Kohulan

---

🌐 Project Website

Experience DECIMER in action at decimer.ai, brilliantly implemented by Otto Brinkhaus!

---

🎓 Maintained by the Kohulan @ Steinbeck Group

Natural Products Cheminformatics Research Group
Institute for Inorganic and Analytical Chemistry
Friedrich Schiller University Jena, Germany

---

⭐ Star History

---

📊 Project Analytics

Made with ❤️ and ☕ for the global chemistry community

© 2025 Kohulan @ Steinbeck Lab, Friedrich Schiller University Jena