UAVs play an important role in different application fields, especially in rescue. To achieve good communication between the onboard UAV and humans, an approach to accurately recognize various body gestures in the wild environment by using deep learning algorithms is presented in this work. The system can not only recognize human rescue gestures but also detect people, track people, and count the number of humans. A dataset of ten basic rescue gestures (i.e. Kick, Punch, Squat, Stand, Attention, Cancel, Walk, Sit, Direction, and PhoneCall) has been created by a UAV’s camera. From the perspective of UAV rescue, the feedback from the user is very important. The two most important dynamic rescue gestures are the novel dynamic Attention and Cancel which represent the set and reset functions respectively. The system shows a warning help message when the user is waving to the UAV. The user can also cancel the communication at any time by showing the drone the body rescue gesture that indic ates the cancellation according to their needs. This work has laid the groundwork for the next rescue routes that the UAV will design based on user feedback. The system achieves 99.47% accuracy on training data and 99.09% accuracy on testing data by using the deep learning method.