Partial end-to-end reinforcement learning for robustness towards model-mismatch in autonomous racing’

Simple item page
dc.contributor.advisorSchoeman, J-Cen_ZA
dc.contributor.advisorJordaan, Willemen_ZA
dc.contributor.authorMurdoch, Andrewen_ZA
dc.contributor.otherStellenbosch University. Faculty of Engineering. Dept. of Electrical and Electronic Engineering.en_ZA
dc.date.accessioned2023-11-20T08:58:04Zen_ZA
dc.date.accessioned2024-01-08T16:08:41Zen_ZA
dc.date.available2023-11-20T08:58:04Zen_ZA
dc.date.available2024-01-08T16:08:41Zen_ZA
dc.date.issued2023-12en_ZA
dc.descriptionThesis (MEng)--Stellenbosch University, 2023.en_ZA
dc.description.abstractENGLISH ABSTRACT: The increasing popularity of self-driving cars has given rise to the emerging field of autonomous racing. In this domain, algorithms are tasked with processing sensor data to generate control commands (e.g., steering and throttle) that move a vehicle around a track safely and in the shortest possible time. This study addresses the significant issue of practical model-mismatch in learning-based solutions, particularly in reinforcement learning (RL), for autonomous racing. Model mismatch occurs when the vehicle dynamics model used for simulation does not accurately represent the real dynamics of the vehicle, leading to a decrease in algorithm performance. This is a common issue encountered when considering real-world deployments. To address this challenge, we propose a partial end-to-end algorithm which decouples the planning and control tasks. Within this framework, a reinforcement learning (RL) agent generates a trajectory comprising a path and velocity, which is subsequently tracked using a pure pursuit steering controller and a proportional velocity controller, respectively. In contrast, many learning-based algorithms utilise an end-to-end approach, whereby a deep neural network directly maps from sensor data to control commands. We extensively evaluate the partial end-to-end algorithm in a custom F1tenth simulation, under conditions where model-mismatches in vehicle mass, cornering stiffness coefficient, and road surface friction coefficient are present. In each of these scenarios, the performance of the partial end-to-end agents remained similar under both nominal and model-mismatch conditions, demonstrating an ability to reliably navigate complex tracks without crashing. Thus, by leveraging the robustness of a classical controller, our partial end-to-end driving algorithm exhibits better robustness towards model-mismatches than an end-to-end baseline algorithm.en_ZA
dc.description.abstractAFRIKAANSE OPSOMMING: Die toenemende gewildheid van selfbesturende motors het aanleiding gegee tot die opkomende veld van outonome wedrenne. In hierdie domein, het algoritmes die taak om sensordata te verwerk om beheeropdragte (bv., stuur en versneller) te genereer wat ’n voertuig veilig en in die kortste moontlike tyd om ’n baan beweeg. Hierdie studie spreek die beduidende kwessie van praktiese model-wanverhouding in leergebaseerde oplossings aan, veral in versterkingsleer (RL), vir outonome wedrenne. Model-wanpassing vind plaas wanneer die voertuigdinamika-model wat vir simulasie gebruik word nie die werklike dinamika van die voertuig akkuraat voorstel nie, wat lei tot ’n afname in algoritme-werkverrigting. Dit is ’n algemene probleem wat teegekom word wanneer werklike implementerings oorweeg word. Om hierdie uitdaging aan te spreek, stel ons ’n gedeeltelike- ‘end-to-end’-algoritme voor wat die beplanning- en beheertake ontkoppel. Binne hierdie raamwerk genereer ’n versterkingsleer (RL) agent ’n trajek wat ’n pad en snelheid bevat, wat vervolgens nagespoor word deur gebruik te maak van ’n suiwer agtervolgstuurbeheerder en ’n proporsionele snelheidsbeheerder, onderskeidelik. Daarteenoor gebruik baie leergebaseerde algoritmes ’n ‘end-to-end’-benadering, waardeur ’n diep neurale netwerk direk (DNN) vanaf sensordata karteer om opdragte te beheer. Ons evalueer die gedeeltelike- ‘end-to-end’-algoritme breedvoerig in ’n pasgemaakte ‘F1tenth’-simulasie, onder toestande waar model-wanverhoudings in voertuigmassa, draai styfheidskoeffisient en padoppervlakwrywingskoeffisient teenwoordig is. In elk van hierdie scenario’s het die werkverrigting van die gedeeltelike- ‘end-to-end’-agente dieselfde gebly onder beide nominale en model-wanpastoestande, wat ’n vermoe demonstreer om komplekse spore betroubaar te navigeer sonder om te verongeluk. Deur dus die robuustheid van ’n klassieke kontroleerder te benut, toon ons gedeeltelike- ‘end-to-end’- bestuursalgoritme beter robuustheid teenoor model-wanpassings as ’n ‘end-to-end’- basislynalgoritme.af_ZA
dc.description.versionMastersen_ZA
dc.format.extentxii, 95 pages : illustrations en_ZA
dc.identifier.urihttps://scholar.sun.ac.za/handle/10019.1/128928en_ZA
dc.language.isoen_ZAen_ZA
dc.language.isoen_ZAen_ZA
dc.publisherStellenbosch : Stellenbosch Universityen_ZA
dc.rights.holderStellenbosch Universityen_ZA
dc.subjectPartial end-to-end; reinforcement learning; robustness towards model-mismatch; autonomous racingen_ZA
dc.subject.lcshReinforcement learningen_ZA
dc.subject.lcshAutomated vehiclesen_ZA
dc.subject.lcshAutomobiles, Racingen_ZA
dc.subject.lcshSimulated annealing (Mathematics)en_ZA
dc.titlePartial end-to-end reinforcement learning for robustness towards model-mismatch in autonomous racing’en_ZA
dc.typeThesisen_ZA
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
murdoch_partial_2023.pdf
Size:
5.08 MB
Format:
Adobe Portable Document Format
Description:
Download
Collections
Masters Degrees (Electrical and Electronic Engineering)