Difference between revisions of "Simulator"

From Elcano Project Wiki
Jump to navigation Jump to search
Line 1: Line 1:
 
= Elcano Carla Simulation External Specification =
 
= Elcano Carla Simulation External Specification =
  
CARLA <http://carla.org/> is an open-source driving simulator. During Q4 2019 we will be developing a  go-between circuit board to transfer data between CARLA simulation and the sensor slots of the Elcano hardware. The CARLA software allows users to emulate trike parameters under controlled conditions within a virtual world, providing an efficient system for testing sensor behavior, design implementation, and benchmarking overall vehicle progression. The CARLA software can simulate a variety of road conditions such as traffic patterns, inclines, and weather as well as provide users control over vehicle parameters including weight, engine power, and aerodynamic properties.
+
CARLA <http://carla.org/> is an open-source driving simulator. During 2019-20 students develop3e a  go-between circuit board to transfer data between CARLA simulation and the sensor slots of the Elcano hardware. The CARLA software allows users to emulate trike parameters under controlled conditions within a virtual world, providing an efficient system for testing sensor behavior, design implementation, and benchmarking overall vehicle progress. The CARLA software can simulate a variety of road conditions such as traffic patterns, inclines, and weather as well as provide users control over vehicle parameters including weight, engine power, and aerodynamic properties.
  
  
A go-between circuit board will transfer data from vehicle systems to CARLA under a variety of test conditions, enabling the software to replicate responses and return sensor data to be reevaluated for accuracy. Connecting the simulation software to components on both the high and low level boards of the Elcano trike including lidar, sonar, throttle, and brake systems, will allow users to immediately see the results of adjustments to vehicle parameters and highlight physical aspects of the trike within the virtual simulation. This project will enable users to efficiently implement improvements and track the immediate effects of design changes without spending the time and resources it would take to transport or operate the trike in real world test scenarios. It enables developers to improve the software without needing to invest in a vehicle.
+
An Arduino Due serves as a router to transfer data from vehicle systems to CARLA under a variety of test conditions, enabling the software to replicate responses and return sensor data to be reevaluated for accuracy. The CARLA Micro-AV Bridge includes the Router, Sensor Hub and Drive-by-wire Arduinos. It permits testing the Elcano trike software, including camera, sonar, throttle, steering and brake systems. Simulation allows users to immediately see the results of adjustments to vehicle parameters and highlight physical aspects of the trike within the virtual environment. This project enables users to efficiently implement improvements and track the immediate effects of design changes without spending the time and resources it would take to transport or operate the trike in real world test scenarios. It enables developers to improve the software without needing to invest in a vehicle.
  
 
[[File:Architecture_CAN_Sim_7_7_20.png|1000px]]
 
[[File:Architecture_CAN_Sim_7_7_20.png|1000px]]
Line 22: Line 22:
 
Code to be executed by a computer that has access to a Carla server, either locally, or through a network.  Controls all objects in Carla simulation.  Retrieves sensor data from Carla, and sends router board.  Also retrieves actuation instructions from the router board and sends to Carla.
 
Code to be executed by a computer that has access to a Carla server, either locally, or through a network.  Controls all objects in Carla simulation.  Retrieves sensor data from Carla, and sends router board.  Also retrieves actuation instructions from the router board and sends to Carla.
  
'''High Level CAN Demo'''  
+
'''Sensor Hub [aka High Level] Processor'''  
Simple demo code for high-level board that sends drive CAN messages repeatedly to the CAN Bus.
+
The same code is used for the virtual or physical vehicle, sending and receiving messages over the CAN Bus. The CARLA Bridge replaces sensor data that would normally come from instruments.
  
'''Elcano C2 LowLevel'''  
+
'''Drive-by-wire [aks C2 LowLevel] Processor'''  
Current Elcano low-level code with minor adjustments. Adjustments discussed in detail later in doc.
+
Current Elcano Drive-by-wirel code. Actuator data is routed to CARLA.
  
 
'''Router Board:'''  
 
'''Router Board:'''  
Code to be executed on the Arduino Due that functions as the router board.  Routes drive commands from low-level to Carla.  Also routes sensor data from Carla to high/low-level boards.
+
Code is executed on the Arduino Due that functions as the router board.  Routes commands from Drive-by-wire to Carla via Python scripts.  Also routes sensor data from Carla to Sensor Hub processor.
  
 
[[File:RouterBoard2020A.jpg|800px|]]
 
[[File:RouterBoard2020A.jpg|800px|]]

Revision as of 04:35, 14 July 2020

Elcano Carla Simulation External Specification

CARLA <http://carla.org/> is an open-source driving simulator. During 2019-20 students develop3e a go-between circuit board to transfer data between CARLA simulation and the sensor slots of the Elcano hardware. The CARLA software allows users to emulate trike parameters under controlled conditions within a virtual world, providing an efficient system for testing sensor behavior, design implementation, and benchmarking overall vehicle progress. The CARLA software can simulate a variety of road conditions such as traffic patterns, inclines, and weather as well as provide users control over vehicle parameters including weight, engine power, and aerodynamic properties.


An Arduino Due serves as a router to transfer data from vehicle systems to CARLA under a variety of test conditions, enabling the software to replicate responses and return sensor data to be reevaluated for accuracy. The CARLA Micro-AV Bridge includes the Router, Sensor Hub and Drive-by-wire Arduinos. It permits testing the Elcano trike software, including camera, sonar, throttle, steering and brake systems. Simulation allows users to immediately see the results of adjustments to vehicle parameters and highlight physical aspects of the trike within the virtual environment. This project enables users to efficiently implement improvements and track the immediate effects of design changes without spending the time and resources it would take to transport or operate the trike in real world test scenarios. It enables developers to improve the software without needing to invest in a vehicle.

Architecture CAN Sim 7 7 20.png


Purpose:

The following is a generalized list of the desired functionality of the simulator.

  • Simulate data for all sensors associated with Elcano Trike.
  • Simulate Elcano behavior to throttle, brake, and steering.
  • Have low and high-level boards function normally with the simulated sensor data without knowing data is being simulated.
  • Allow low and high-level board interaction to be tested.
  • Allow autonomous driving to be tested.

Components:

Simulation: Code to be executed by a computer that has access to a Carla server, either locally, or through a network. Controls all objects in Carla simulation. Retrieves sensor data from Carla, and sends router board. Also retrieves actuation instructions from the router board and sends to Carla.

Sensor Hub [aka High Level] Processor The same code is used for the virtual or physical vehicle, sending and receiving messages over the CAN Bus. The CARLA Bridge replaces sensor data that would normally come from instruments.

Drive-by-wire [aks C2 LowLevel] Processor Current Elcano Drive-by-wirel code. Actuator data is routed to CARLA.

Router Board: Code is executed on the Arduino Due that functions as the router board. Routes commands from Drive-by-wire to Carla via Python scripts. Also routes sensor data from Carla to Sensor Hub processor.

RouterBoard2020A.jpg

RouterBoard2020B.jpg

Current Functionality of Simulation:

At the end our Autumn 2019 Capstone, we were able to deliver a demo that demonstrated communication between all components; high-level board (Arduino Due), low-level board (Arduino Mega 2560) with shield running low level code, router board (Arduino Due), and the instance of Carla. The instructions on how to reproduce the demo will be given later in this document. The primary roadblock that prevented further development into this project was the state of the CAN system of the Elcano trike (particularly the transceiver board which seems to have a flawed design). These are the current features of the simulator.

  • Implementation of USB serial communication between router board and computer running simulator.py. Both entities (Computer and ArduinoDue) can send and receive data through USB connection.
  • Implementation of a cyclometer. Router board can receive the current speed of the simulated vehicle in Carla and convert it into an interrupt-based cyclometer pulse, based on the wheel dimension of the Elcano. Also considers the random error of cyclometer present on the Elcano trike. This can be debugged with the built-in LED on the router board.
  • Implementation of NMEA GPS sensor. Carla is able the output NMEA GPS data to the router board 10 times per second, effectively simulating the GPS sensor on the high-level board. Router board is successfully able to output it to UART serial which is how it is transmitted to the high-level board.
  • Implementation of timing structure. Computer running simulator.py waits for data to be interpreted from the router board while the router board executes its update 10 times per second. This is the desired speed for both sensor data and actuation data to be updated.
  • Implementation of vehicle control updaters through throttle, brake, and steering data. The computer running simulator.py can take desired throttle, brake, and steering and convert them into corresponding commands for Carla using Carla API. These commands move the vehicle within Carla. This is the primarily what the demo shows.

Demo Guide (How to Set-Up)

The following is a guide to set up the Elcano simulator.

Materials

  • 1 Assembled Elcano Simulator Printed Circuit Board (PCB)
  • 2 Arduino Due
  • 1 Arduino Mega 2560
  • 1 Box (200 x 150 x 70 mm)
  • 2 Voltage Level Converter (included w/Simulator PCB)
  • 1 SD Card (w/map files)
  • 1 USB AB Cable (for Arduino Mega)
  • 1 USB Micro Cable (for Arduino Due)
  • 1 computer capable of running Carla

Pre-installation

Use the following instructions to install the required software to run the simulation.

  1. Python
    • Install Python 3 (if not installed already)
    -experiments used version 3.7.6
    • In a terminal, move to .\Simulation\
    • Run the command “pip3 install -r requirements.txt” to have pip install all the required python libraries for the simulator
  2. CARLA
  3. Arduino Libraries
    Download these Arduino libraries prior to running the simulation:
  4. Elcano Repositories
    The Elcano repositories are needed to run the simulation:

Run Simulation

Once all the requirements are installed, use the following instructions to run the simulation.
Router PCB Connections.jpg

Step 1: Program all Arduinos

  1. Upload router_board_v1.ino to the router Arduino Due programming port
  2. Upload High_Level.ino to the high level Arduino Due programming port
  3. Upload Drive_By_Wire.ino to the low level Arduino Mega 2560

Step 2: Run CARLA

  1. Open a command prompt (type cmd in the windows search box)
  2. Go to the directory where CARLA is installed (CARLAUE4.exe)
    cd C:\Users\bobross\Documents\ElcanoInstaller\CARLA\WindowsNoEditor
    -You may have it saved to a different directory
  3. From the CARLA directory, type:
    CARLAUE4.exe
    Or, to run in low resolution mode:
    CARLAUE4.exe -ResX=720 -ResY=480 -quality-level=Low

Step 3: Run Simulation

  1. Connect your PC to the native port on the Router Board Due
  2. Open a command prompt (type cmd in the windows search box)
  3. Go to the Simulation directory
    cd C:\Users\bobross\Documents\Simulator\Simulation
    -You may have it saved to a different directory
  4. From Simulation directory, start the Simulator UI
    .\Simulation\start.bat
  5. By default, the Simulator UI will populate local running CARLA settings
    • For control via the router board, leave mode as Auto
    • To control via manual keyboard input, change mode to Manual.
    • If running a network-based CARLA server, enter IP and Port of CARLA into the respective boxes. (i.e. 192.168.1.1, 2010)
  6. Click "Connect to CARLA"
    • If Auto mode, it will pop up a selection of COM devices, select routerboard native port COM to begin and press go.
    • If Manual mode, client will start with keyboard controls (WASD)
  7. Observe behavior in simulator.
  8. If car gets stuck, kill current simulator.py process and restart.

Debug/Logging

  • High Level uses the Programming Port to display debug messages over serial monitor
  • Router Board uses the Programming Port to display debug messages over serial monitor

Out of Scope Functionality Tested/Implemented

  • Our project required us to delve into pieces of code within the Elcano project outside of our scope. Understanding of how sensors are read by the high and low-level boards is necessary for the simulation of the sensors. We were able to fix some issues regarding the CAN communication between the high and low-level boards. The modified low-level code was not pushed to the main low-level code folder in the repository on Github, however, it is included within the simulator documentation. The following were implemented.
  • Unification of the proposed CAN message structure and the actual CAN message processing on the low-level code for CAN messages with ID 0x350 (High-level drive instructions). These are CAN messages the high-level will send to the low level to facilitate autonomy.
  • Fixing the DAC output voltage by using the MCP48x2 library by Jonas Forsberg.

To Do (Software):

High Priority:

  • Simulate gyro sensor
  • Simulate accelerometer/magnetometer sensor.
  • Simulate wheel angle sensor.
  • Implement steering actuation when low-level CAN processing of steering command is fixed.
  • Initialize CAN with high-level board code and incorporate in demo instead of separate CAN transceiver.

Medium Priority:

  • Simulate ScanseSweep.
  • Implement receiver board when board is fully fixed.
  • Incorporate Elcano trike physics to simulated vehicle.

Low Priority

  • Simulate camera sensor.

Bugs Within Simulator

  • Occasionally the simulator and router board will desync and the NMEA GPS messages received by the router board will be faulty. This is usually followed by a crash of simulator.py. If not, simply press Ctrl^C and restart simulator.py.
  • If simulator.py unexpectedly terminates, the created actors and sensors in Carla will not be properly destroyed. This usually does not cause problems for the simulation but proper termination in the future would be safer. For now if it is causing problems simply restart Carla.
  • Carla will never stop outputting sensor data even if asked to stop. This is a bug with Carla and not in our scope.
  • Most malfunctions can be temporarily fixed by restarting the simulator.py instance.

Other Elcano Bugs Found

  • While we were able to fix the code for the low-level CAN interpretation of the throttle and brake commands from high-level CAN messages, the PWM steering response was not fully tested. We were unable to get the low-level board to output the correct PWM for the sent CAN messages and this needs more testing.
  • The receiver board cannot get CAN to initialize. Voltage testing confirmed that the CAN components (MCP2515 and MCP 2550) are receiving the correct operating voltages (5v). Also, multiple CAN libraries besides the suggested Seeed Studio library were tested with no success (SparkFun CAN library was one of the tested libraries). Possible problem is incompatibility between the 5v CAN components and the 3.3V SAMD21 board. Another possible problem is design flaw of pcb.
  • Low-level processing of CAN messages for ID 0x350 is faulty. Bit shifting and produced output do not match the CAN guidelines on the wiki. This was fixed in the low-level code included in the simulator folder but not pushed to the Elcano Github repository.

Additional Notes

  • Currently, the order that you output data from the router board to the computer running simulator.py is non-trivial. Data is interpreted based on the order which it is received.
  • Carla sensor listening was flawed at the time this was made; .stop() would not stop the sensor from executing its attached function at each tick of Carla. Therefore speed was implemented without the use of sensors, and instead uses a command that accesses the speed only when needed. This is the suggested method until sensors are fixed. Certain sensors such as NMEA GPS, however, are only implementable using Carla sensors. Additional information can be found on Carla Documentation.
  • In PowerShell, in CarlaUE4 path, run Carla as: ***Start-Process CarlaUE4 -ArgumentList “—quality-level=Low”*** to lower gpu load.
  • Headless mode can be enabled for complete removal of graphics rendering, however, visual debugging is extremely useful.