A Q-learning Based Continuous Tuning of Fuzzy Wall Tracking

Abstract

A simple easy to implement algorithm is proposed to address wall tracking task of an autonomous robot. The robot should navigate in unknown environments, find the nearest wall, and track it solely based on locally sensed data. The proposed method benefits from coupling fuzzy logic and Q-learning to meet requirements of autonomous navigations. Fuzzy if-then rules provide a reliable decision making framework to handle uncertainties, and also allow incorporation of heuristic knowledge. Dynamic structure of Q-learning makes it a promising tool to tune fuzzy inference systems when little or no prior knowledge is available about the world. To robot, the world is modeled into a set of state-action pairs. For each fuzzified state, there are some suggested actions. States are related to their corresponding actions via fuzzy if-then rules based on human reasoning. The robot selects the most encouraged action for each state through online experiences. Experiments on simulated Khepera robot validate efficiency of the proposed method. Simulation results demonstrate a successful implementation of wall tracking task where the robot keeps itself within predefined margins from walls even with complex concave, convex, or polygon shapes.