Drones or unmanned aerial vehicles (UAVs) communication technology, which has recently been thoroughly studied and adopted by 3GPP standard (Release 15) due to its dynamic, flexible, and flying nature, is expected to be an integral part of future wireless communications and Internet of drones (IoD) applications. However, due to the unique transmission characteristics and nature of UAV systems including the broadcasting, dominant line of site and poor scattering, providing confidentiality for legitimate receivers against unintended ones (eavesdroppers) appears to be a challenging goal to achieve in such scenarios. Besides, the special features of UAVs represented by having limited power (battery-operated) and processing (light RAM and CPU capabilities), makes applying complex cryptography approaches very challenging and inefficient for such systems. This motives the utilization of alternative approaches enabled by physical layer security (PLS) concept for securing UAV-based systems. Techniques based on PLS are deemed to be promising due to their ability to provide inherent secrecy that is complexity independent, where no matter what computational processing power the eavesdropper may have, there is no way to decrypt the PLS algorithms. This work is dedicated to highlight and overview the latest advances and state of art researches on the field of applying PLS to UAV systems in a unified and structured manner. Particularity, it discusses and explains the different, possible PLS scenarios and use cases of UAVs, which are categorized based on how the drone is utilized and employed in the communication system setup. The main classified categories include the deployment of the flying, mobile UAV as a 1) base station (BS), 2) user equipment (UE), 2) relay, or 4) jammer. Then, recommendations and future open research issues are stated and discussed.