Improving Accessibility of Remote Drone Control with a Streamlined Computer Vision Approach

Evan Lowhorn, Rocio Alba-Flores

Research output: Contribution to book or proceedingConference articlepeer-review

Abstract

The purpose of this work is to develop a method for classifying hand signals and using the pre-diction output in a drone control algorithm. To achieve this, methods based on Convolutional Neural Networks (CNNs) were applied. The hand signals chosen were the numerical hand signs for one through five for two-dimensional movement with a separate idle signal, and a fist for land. A script was created to automate one minute of training image capture for each class. Transfer learning with PyTorch (Python) was performed using a pre-Trained 18-layer residual learning network (ResNet-18). The training process completed in three minutes and 43 seconds with five epochs and a final overall validation accuracy of over 99%. Implemented with the drone control, the classification performed as desired at approximately 60 predictions per second on desktop and 20 predictions per second on a Nvidia Jetson Nano.

Original languageEnglish
Title of host publication2022 IEEE International IOT, Electronics and Mechatronics Conference, IEMTRONICS 2022
EditorsSatyajit Chakrabarti, Rajashree Paul, Bob Gill, Malay Gangopadhyay, Sanghamitra Poddar
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665486842
DOIs
StatePublished - 2022
Event2022 IEEE International IOT, Electronics and Mechatronics Conference, IEMTRONICS 2022 - Toronto, Canada
Duration: Jun 1 2022Jun 4 2022

Publication series

Name2022 IEEE International IOT, Electronics and Mechatronics Conference, IEMTRONICS 2022

Conference

Conference2022 IEEE International IOT, Electronics and Mechatronics Conference, IEMTRONICS 2022
Country/TerritoryCanada
CityToronto
Period06/1/2206/4/22

Scopus Subject Areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Software
  • Information Systems and Management
  • Electrical and Electronic Engineering
  • Mechanical Engineering
  • Modeling and Simulation

Keywords

  • Convolutional Neural Network
  • Drones
  • Human-Machine Interaction

Fingerprint

Dive into the research topics of 'Improving Accessibility of Remote Drone Control with a Streamlined Computer Vision Approach'. Together they form a unique fingerprint.

Cite this