WeedDiffusion: A Dual-Branch Synthetic Augmentation Framework for Weed Mapping

Precision agriculture depends on accurate crop-weed discrimination to enable efficient weed control and sustainable crop production. However, building large-scale annotated datasets is time-consuming and costly. In this work, we introduce WeedDiffusion, a novel data augmentation framework that leverages DreamBooth-tuned diffusion models to generate training data for crop and weed segmentation. Our approach consists of two complementary strategies: (i) inpainting crop regions using a DreamBooth model trained on in-distribution crop imagery, and (ii) cut-and-paste generation of weed instances, created by a different fine-tuned DreamBooth model and segmented with Segment Anything. The augmented images are then paired with automatically reconstructed masks for training semantic segmentation models. Experiments demonstrate that WeedDiffusion produces morphologically realistic samples and consistently improves performance across different semantic segmentation architectures. Our augmented dataset resulted in consistent improvements in the mean Intersection over Union. Ablation studies demonstrate the impact of varying the sizes of both the generated synthetic datasets and the real training data used to train the generator, confirming the approach’s data efficiency and scalability. Our framework offers a scalable, model-agnostic solution for enhancing agricultural datasets through targeted, high-fidelity synthetic augmentation. The code is available at https://github.com/pasqualedem/WeedDiffusion.