This peach tree disease detection dataset is a multimodal, multi-angle dataset which was constructed for monitoring the growth of peach trees, including stress analysis and prediction. An orchard of peach trees is considered in the area of Thessaly, where 889 peach trees were recorded in a full crop season starting from Jul. 2021 to Sep. 2022. The dataset includes a) aerial / Unmanned Aerial Vehicle (UAV) images, b) ground RGB images/photos, and c) ground multispectral images/photos. Two experts (agronomists) annotated the dataset by identifying whether the trees are healthy or stressed, where the identified stress is either Anarsia lineatella or Grapholita molesta.