Integrated Circuits and Systems >

2024 , Vol. 1 >Issue 2: 66 - 79

DOI: https://doi.org/10.23919/ICS.2024.3449791

Special Issue on Selected Papers from ICTA2023

SFTT: A 325 FPS Computational and Hardware Efficient Corner-Detection Accelerator Design for SLAM Applications

  • WEIYI ZHANG ,
  • CHAOYANG DING ,
  • XIAORUI MO ,
  • FEI SHAO ,
  • YIYANG WANG ,
  • YUSHI GUO ,
  • LITING NIU ,
  • CHENG NIAN ,
  • FASIH UD DIN FARRUKH ,
  • CHUN ZHANG
Expand
  • School of Integrated Circuits, Tsinghua University, Beijing 100084, China
CHUN ZHANG (e-mail: ).

CHENG NIAN, (Member, IEEE)

CHUN ZHANG, (Senior Member, IEEE)

Received date: 2024-03-02

  Revised date: 2024-05-29

  Accepted date: 2024-07-30

  Online published: 2024-11-27

Supported by

National Natural Science Foundation of China under Grant(U20A20220)

Fold

Abstract

Simultaneous Localization and Mapping (SLAM) is the process by which a mobile robot can build a map of the surrounding environment and compute its own location. Feature point extraction is one of the key components of a SLAM system. The extraction accuracy and efficiency of corner detection directly affect the overall accuracy and throughput of the system. However, the complexity of corner detection algorithms makes it challenging to achieve real-time implementation and efficient, low-cost hardware design, especially for mobile robots. Harris corner detection class algorithms including Harris and GFTT (Good Feature to Track) have improved accuracy. However, those algorithms require high resource consumption and latency when implemented on hardware platforms. The GFTT achieves higher accuracy than Harris while requiring higher computational complexity. To address the throughput problem, SFTT (Simple Feature to Track), a new Harris class detection algorithm is proposed, and the corresponding hardware accelerator is designed. The proposed SFTT significantly reduced the computational complexity compared with the Harris algorithm and GFTT. Experiments have shown SFTT also achieved slightly higher accuracy compared with the two algorithms. Furthermore, the GFTT accelerator is designed which reaches up to 325 fps at the frequency of 100 MHz. The proposed design has achieved an improvement in throughput by 1.3× times and power efficiency by 1.7× times as compared to state-of-the-art design.

Key words： SLAM; GFTT; corner detection; FPGA; ASIC

Cite this article

WEIYI ZHANG , CHAOYANG DING , XIAORUI MO , FEI SHAO , YIYANG WANG , YUSHI GUO , LITING NIU , CHENG NIAN , FASIH UD DIN FARRUKH , CHUN ZHANG . SFTT: A 325 FPS Computational and Hardware Efficient Corner-Detection Accelerator Design for SLAM Applications[J]. Integrated Circuits and Systems, 2024 , 1(2) : 66 -79 . DOI: 10.23919/ICS.2024.3449791

References

Publishing order | Descend order by publishing year | Descend order by cited within
[1]
R. Mur-Artal and J. D. Tardós, “ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras,” IEEE Trans. Robot., vol. 33, no. 5, pp. 1255- 1262, Oct. 2017.

[2]
A. Suleiman, Z. Zhang, L. Carlone, S. Karaman, and V. Sze, “Navion: A 2-mW fully integrated real-time visual-inertial odometry accelerator for autonomous navigation of nano drones,” IEEE J. Solid-State Circuits, vol. 54, no. 4, pp. 1106- 1119, Apr. 2019.

[3]
E. Rosten and T. Drummond, “Machine learning for high-speed corner detection,” in Proc. Comput. Vis.-ECCV 2006:9th Eur. Conf. Comput. Vis. Proc., Part I. 9, 2006, pp. 430- 443.

[4]
Í. O. d. Oliveira, K. V. Ono, and E. Todt, “IGFTT: Towards an efficient alternative to SIFT and SURF,” in Proc. Int. Conf. Central Europe Comput. Graph., Visualization and Comput. Vis. , 2015, pp. 73- 80.

[5]
J. Weberruss, L. Kleeman, D. Boland, and T. Drummond, “FPGA acceleration of multilevel ORB feature extraction for computer vision,” in 2017 27th Int. Conf. Field Program. Log. Appl. 2017, pp. 1- 8.

[6]
S.-K. Lam, R. K. Bijarniya, and M. Wu, “Lowering dynamic power in stream-based harris corner detection architecture,” in 2017 Int. Conf. Field Program. Technol. 2017, pp. 176- 182.

[7]
C. Harris and M. Stephens, “A combined corner and edge detector,” in Proc. Alvey Vis. Conf. , 1988, vol. 15, pp. 10- 5244.

[8]
S. M. Smith and J. M. Brady, “Susan—A new approach to low level image processing,” Int. J. Comput. Vis., vol. 23, no. 1, pp. 45- 78, 1997.

[9]
D. G. Lowe, “Object recognition from local scale-invariant features,” in Proc. 7th IEEE Int. Conf. Comput. Vis., 1999, vol. 2. pp. 1150- 1157.

[10]
H. Bay, T. Tuytelaars, and L. V. Gool, “SURF: Speeded up robust features,” in Proc. Comput. Vis.-ECCV 2006:9th Eur. Conf. Comput. Vis. Proc., Part I. 9, Graz, Austria, 2006, pp. 404- 417.

[11]
D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperPoint: Self- supervised interest point detection and description,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops , 2018, pp. 337- 33712.

[12]
J. Shi, “Good features to track,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , 1994, pp. 593- 600.

[13]
R. Sun et al. , “A flexible and efficient real-time ORB-based full-HD image feature extraction accelerator,” IEEE Trans. Very Large Scale Integration Syst., vol. 28, no. 2, pp. 565- 575, Feb. 2020.

[14]
C. Forster, Z. Zhang, M. Gassner, M. Werlberger, and D. Scaramuzza, “SVO: Semidirect visual odometry for monocular and multicamera systems,” IEEE Trans. Robot., vol. 33, no. 2, pp. 249- 265, Apr. 2017.

[15]
M. Arjomandi, S. Agostino, and M. Mammone, “Classification of unmanned aerial vehicles,” Report for Mechanical Engineering class, University of Adelaide, Adelaide, Australia, 2006, pp. 1- 48

[16]
K. Dohi, Y. Yorita, Y. Shibata, and K. Oguri, “Pattern compression of FAST corner detection for efficient hardware implementation,” in 2011 21st Int. Conf. Field Program. Log. Appl. 2011, pp. 478- 481.

[17]
T. Imsaengsuk and S. Pumrin, “Feature detection and description based on ORB algorithm for FPGA-based image processing,” in 2021 9th Int. Elect. Eng. Congr. 2021, pp. 420- 423.

[18]
S.-K. Lam, T. C. Lim, M. Wu, B. Cao, and B. A. Jasani, “Area-time efficient FAST corner detector using data-path transposition,” IEEE Trans. Circuits Syst. II: Exp. Briefs, vol. 65, no. 9, pp. 1224- 1228, Sep. 2018.

[19]
S.-K. Lam, T. C. Lim, M. Wu, B. Cao, and B. A. Jasani, “Data-path unrolling with logic folding for area-time-efficient FPGA-based fast corner detector,” J. Real-Time Image Process., vol. 16, pp. 2147- 2158, 2019.

[20]
R. Sun, P. Liu, J. Wang, and Z. Zhou, “A low latency feature extraction accelerator with reduced internal memory,” in 2017 IEEE Int. Symp. Circuits Syst. 2017, pp. 1- 4.

[21]
Q. Zhang, H. Sun, Q. Deng, H. Yu, and Y. Ha, “NORB: A stream-based and non-blocking FPGA accelerator for ORB feature extraction,” in 2023 30th IEEE Int. Conf. Electron., Circuits Syst. 2023, pp. 1- 4.

[22]
Y.-H. Lee, T.-C. Chen, H.-C. Liang, and J.-X. Liao, “Algorithm and architecture design of FAST-C image corner detection engine,” IEEE Trans. Very Large Scale Integration Syst., vol. 29, no. 4, pp. 788- 799, Apr. 2021.

[23]
R. Li, J. Wu, M. Liu, Z. Chen, S. Zhou, and S. Feng, “HcveAcc: A high-performance and energy-efficient accelerator for tracking task in VSLAM system,” in 2020 Des., Automat. Test Europe Conf. Exhib. 2020, pp. 198- 203.

[24]
D. Bojanic´, K. Bartol, T. Pribanic´, T. Petkovic´, Y. D. Donoso, and J. S. Mas, “On the comparison of classic and deep keypoint detector and descriptor methods,” in 2019 11th Int. Symp. Image Signal Process. Anal. 2019, pp. 64- 69.

[25]
A. Neubeck and L. V. Gool, “Efficient non-maximum suppression,” in 18th Int. Conf. Pattern Recognit., 2006, vol. 3, pp. 850- 855.

[26]
D. G. Lowe, “Distinctive image features from scale-invariant key- points,” Int. J. Comput. Vis., vol. 60, pp. 91- 110, 2004.

[27]
K. Mikolajczyk and C. Schmid, “Scale & affine invariant interest point detectors,” Int. J. Comput. Vis., vol. 60, pp. 63- 86, 2004.

[28]
T. Tuytelaars and L. V. Gool, “Matching widely separated views based on affine invariant regions,” Int. J. Comput. Vis., vol. 59, pp. 61- 85, 2004.

[29]
K. Mikolajczyk and C. Schmid, “A performance evaluation of local descriptors,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 10, pp. 1615- 1630, Oct. 2005.

[30]
R. Hess, “An open-source SIFTLibrary,” in Proc. 18th ACM Int. Conf. Multimedia, 2010, pp. 1493- 1496.

[31]
J. Jiang, X. Li, and G. Zhang, “SIFT hardware implementation for real-time image feature extraction,” IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 7, pp. 1209- 1220, Jul. 2014.

[32]
J. Yum, C.-H. Lee, J.-S. Kim, and H.-J. Lee, “A novel hardware architecture with reduced internal memory for real-time extraction of SIFT in an HD video,” IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 10, pp. 1943- 1954, Oct. 2016.

[33]
B. Liu et al. , “An energy-efficient SIFT based feature extraction acceler- ator for high frame-rate video applications,” IEEE Trans. Circuits Syst. I: Reg. Papers, vol. 69, no. 12, pp. 4930- 4943, Dec. 2022.

[34]
M. Lepecq and M. Darouich, “A stream hardware architecture for key- point matching based on a speculative approach,” in 2020 IEEE Int. Symp. Circuits Syst. 2020, pp. 1- 5.

[35]
S. Liu et al. , “Real-time implementation of harris corner detection system based on FPGA,” in 2017 IEEE Int. Conf. Real-Time Comput. Robot. 2017, pp. 339- 343.

[36]
P. Gu, Z. Meng, and P. Zhou, “Real-time visual inertial odometry with a resource-efficient harris corner detection accelerator on FPGA platform,” in 2022 IEEE/RSJ Int. Conf. Intell. Robots Syst. 2022, pp. 10542- 10548.

Options
Outlines

/

〈 〉