Bui Huy Kien, Dang Minh Hieu, Nguyen Duc Hieu Vietnam National University - Vietnam Japan University

The robot frame is crafted from profiled aluminum, measuring 440x500x500 mm as specified. Stepper motors are utilized with a belt transmission system, enabling smooth movement along both the X and Y axes for the laser head. The unidirectional motor is adjustable in speed and rotation direction to dictate the motion. Mounting components, including brackets and flanges, are intricately designed and precision-machined using CNC technology to ensure accurate positioning of the robot's elements.

We chose Jetson Nano for its benefits in image processing and machine learning tasks. The Jetson Nano offers sufficient power to handle complex tasks such as real-time object detection, and it comes with the capability to integrate seamlessly with TensorRT for optimizing performance on the GPU.

Jetson GPIO is a GPIO (General Purpose Input/Output) interface integrated into the NVIDIA Jetson product line, primarily used for interacting with peripheral devices or sensors through GPIO pins on the board. Jetson GPIO provides the capability to control and read signals from these GPIO pins, allowing applications and projects utilizing Jetson to leverage a variety of hardware features and connections. We use the Jetson.GPIO for controlling input and output signals.

Input/Output Ports with the GPIO Library for Jetson Nano

In this project, the GPIO library is used to control 4 DC motors for the forward and backward movement of the robot, 3 stepper motors for controlling the position of the laser , and toggling the laser using the Pulse Width Modulation (PWM) principle for pulse width control.

Make sure that you have cloned the weedding_robot_VJU repository using the git clone command.

git clone https://github.com/hieucoolngau/weeding_robot_VJU.git

Create a new Python file in the weeding_robot_VJU folder and then import the following classes.Tạo 1 file python mới trong folder weeding_robot_VJU sai đó import các class sau

import Jetson.GPIO as GPIO
import time
from step_motor import  StepMotor13, StepMotor2
from dc_motor import Dc_Motor
from laser import Laser

Create an object StepMotor13():

stepMotor13 =  StepMotor13() 
stepMotor2 = StepMotor2() 
laser = laser() #create laser object

After initializing the object, you can use the methods as follows.

stepMotor13.move(GPIO.LOW,GPIO.HIGH,200) #The first attribute defaults to LOW, the second attribute to HIGH represents the direction. You can change the rotation direction by switching from HIGH                                          # to LOW.
                                          #The last attribute is the number of loops; the larger the loop, the longer the stepper motor will rotate.
                                   
#step_Motor2
stepMotor2.move(GPIO.LOW,GPIO.HIGH,200)
#với laser
laser.ON() #Turn on the laser
laser.OFF()#Turn off the laser
laser.CHECK()#When you call this function, the laser will continuously flash off; you can adjust the laser's off time

TensorRT is a library developed by NVIDIA to enhance the inference speed of deep learning models, reducing latency on NVIDIA graphics processing units (GPUs). It can improve inference speed by up to 2-4 times compared to real-time services and is over 30 times faster than CPU performance. In principle, TensorRT is used to deploy libraries that serve machine learning and deep learning, requiring graphics processing for embedded hardware, as illustrated in the diagram below.

Converting libraries to an inference engine with TensorRT

To use YOLO on the Jetson Nano GPU, we perform the synthesis process of the "YOLO engine" for the pre-trained model [15]. This process involves the following steps: creating a Weight Tensor Serialization (WTS) file, installing CMake and Make, building the engine, and testing. The result of this process is the ability to use the trained model on the Jetson Nano GPU, allowing the model to detect objects such as weeds or trees. You can refer to this channel for setup YOLOv7 on JETSON NANO

The conceptual diagram for the integration of computer vision and motion control in a weed-killing robot is depicted as follows. In essence, computer vision is executed through a camera affixed to the robot and YOLOv8n on Jetson Nano. The outcomes include object categorization (weed or plant), confidence level, and the coordinates of bounding boxes around the detected objects. These coordinates consist of 4 parameters: x, y, w, h, representing the central coordinates (x, y) of the box, where w is the width, and h is the height.

These coordinates, along with the object type, are utilized as inputs for managing the robot's actuators. This includes 4 DC motors for locomotion, 3 stepper motors for positioning the laser head in both the (X, Y) directions, and toggling the laser head on/off. The stepper motors will traverse from the initial position to each weed location, activate the laser head to eradicate the weed, and then move on to the subsequent position. This process repeats until no more weeds are detected by the camera, signaling the end of the operation, after which the robot proceeds to the next designated location.

Basically, when you clone our project, YOLOv7 is pre-installed. If you want to learn more about YOLOv7, you can review the source code Yolov7