Ranulfo Bezerra
Projects

Earlier research·2019 — 2021

Construction Vehicle Interaction

Predicting what a human operator is about to do, so an autonomous machine can wait in the right place.

A yellow articulated dump truck being loaded by an excavator on a gravel construction site.
Image: Ranulfo Bezerra

In earthmoving work, autonomous dump trucks share a site with human-operated backhoes, and the cost of misjudging the other party is idle time or a collision. This project used the Beta-Process Hidden Markov Model over 6-axis IMU data to identify the transitions between primitive backhoe motions and predict loading times, with the prediction generalising across operator-and-machine pairs and sensor layouts. It is early work, and it is where the interest in modelling other agents rather than merely sensing them began.

01/Representative publications

3

2020

Prediction of Backhoe Loading Motion via the Beta-Process Hidden Markov Model

Kento Yamada, Kazunori Ohno, Ryunosuke Hamada, Thomas Westfechtel, Ranulfo Bezerra, Naoto Miyamoto, Taro Suzuki, Takahiro Suzuki, Keiji Nagatani, Yukinori Shibata, Kimitaka Asano, Tomohiro Komatsu, Satoshi Tadokoro

IEEE/RSJ IROS 2020·Conference

Abstract +

Backhoe loads sediment onto the bed of dump trucks during earthmoving work. The prediction of backhoe loading time is essential for ensuring safe cooperation between the backhoe and dump trucks. However, it is difficult to predict the instant at which the backhoe is ready to load sediment, because of the similarity in motions observed during gathering sediment. Moreover, since operators have different skill levels, the prediction requires a unique model for each operator. In this study, we attempt to predict the instant at which the backhoe is ready to load sediment into the dump truck. For this purpose, the beta-process hidden Markov model (BP-HMM) is employed to build a backhoe motion model for a specific operator. Time series data of backhoe loading motions for crushed rocks and wood chips, which were measured using 6-axis inertial measurement unit (IMU) sensors equipped at the cab, boom, and arm of the backhoe, were used for modeling with the BP-HMM. Several primitive motions of the backhoe, which occur at the completion of preparation before the loading process begins, were discovered as a result of the motion modeling based on the BP-HMM. We developed the prediction of the instant using three primitive motions. At best, the proposed method could predict the instant with a probability of 67% and 100%, at 6.0 s and 0.7 s before the loading motions began, respectively. This phased prediction can be used to reduce the idle time and risk for dump trucks during earthmoving work with the backhoe.

DOI

2020

Attachable Sensor Boxes to Visualize Backhoe Motion

Kento Yamada, Kazunori Ohno, Naoto Miyamoto, Taro Suzuki, Shotaro Kojima, Ranulfo Bezerra, Takahiro Suzuki, Keiji Nagatani, Yukinori Shibata, Kimitaka Asano, Tomohiro Komatsu, Satoshi Tadokoro

IEEE/SICE SII 2020·Conference

Abstract +

Sensing position and orientation of construction vehicle is an important issue for automation of construction process. We aim to develop sensing and visualization technologies for construction vehicles. Our target construction vehicle is backhoes. Construction vehicles are usually rented in construction fields. However, construction vehicles that can be rented do not have the sensing function. It is too hard to obtain backhoe position and manipulator pose without sensing information. This paper proposes an attachable sensor box that can measure backhoe position and orientation and the manipulator pose. The sensor boxes can be attached on backhoe metal surface by magnetic force without any additional manufacturing on the construction vehicle surface. By using WiFi communication and mobile battery the sensor box can be easily attached on large-size backhoe without wiring. After the work is done, it is easily detached by changing magnetic force power. At loading and scooping motion, backhoe arm and boom have large force exceeding 16 G. To attach the sensor boxes to the arm and boom, we designed an additional magnetic frame that can generate force of 1960 N. The sensor boxes were firmly attached to each joint and prevented to drop by any force from backhoes movement. We also measured the behavior of the backhoe in loading works in hot conditions over 30 °C and visualized backhoe pose and tip manipulator position.

DOI