Development of an AI-driven robotic manipulation framework

This paper introduces a manipulator control system that leverages state-of-the-art artificial intelligence (AI), advanced robotics, and the latest computer vision techniques to enable intuitive and universal robotic control across different robotic manipulators. The intuitive user command interpretation is deployed in the system using ChatGPT API. The vision capabilities of the system rely on YOLOv7 for object detection, Semi-Global Block Matching (SGBM) for object localization, and Principal Component Analysis (PCA) for object orientation. Moreover, MoveIt is utilized for motion planning, and the simulation environment is powered by Gazebo. These components are seamlessly coordinated by a custom state machine integrating into a ROS-based framework. This system allows users to control manipulators completing complex tasks by commands without expert knowledge of robotics. Additionally, a cross-platform kinematics solver architecture is introduced to assess inverse kinematics algorithms for manipulators. Compared to the traditional kinematics solver implemented as ROS plugins, this architecture integrates the sophisticated mathematical capabilities of Matlab with the feasibility of ROS.

Name thesis:degree_name

Master of Applied Science (MASc)

Discipline thesis:degree_discipline

Electrical and Computer Engineering

Grantor

University of Ontario Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Liu, Xiaolong

Advisors dc:contributor.advisor

• Ren, Jing
• Lang, Haoxiang

Handle dc:identifier.uri

https://hdl.handle.net/10155/1859

OAI identifier oai:identifier

oai:ontariotechu.scholaris.ca:10155/1859