Simulation of sensor data for Milliampere

Posted on April 25, 2019

Background

The autonomous ferry Milliampere has been equipped with a sensor rig providing 360 degree coverage of both radar, lidar, optical and infrared sensing. These sensor will, together with the software in the ferry’s multi-target tracking system and its collision avoidance system, play a safety critical role. It is therefore important to test that the sensor combinations are able to provide all the information needed to avoid accidents. Since it is impossible to encounter all possible situations in physical experiments, this analysis must to a large extent rely on simulations. To conduct such simulations with efficience and convenience, systematic software for simulation of sensor data is needed.

Eskil Hatling Hølland has just had a breakthrough with the infrared layer of the Milliampere sensor rig.

The physical principles for the different sensors are rather different, and the simulation must take this into account. The easiest sensor to simulate is perhaps radar, where one can place blobs at target locations, and corrupt these with false alarms and misdetections to increase the realism. For lidar one must simulate how the lidar beams hit the surfaces of the obstacles. This is straightforward in principle, but can be computationally intensive. Simulation of optical and infrared data is significantly more complex.

However, modern game engines such as Unity and Unreal, as well as OpenGL API’s such as Three.js, provide support for sophisticated rendering of artificial camera images. In an ongoing MSc project at ITK, camera simulation by means of Unreal is being explored as a tool for verification of SLAM methods for drones. Independently of this, a group at the EIT village Autoferje has explored Unity as a simulation environment for Milliampere.

Scope

The primary goal of this combined specialization and MSc project is to develop software that makes it easy to generate realistic simulations of sensor data for Milliampere. Several secondary goals may also be pursued if time permits. This includes integration of sensor fusion and collision avoidance software in the simulator and experiments with how performance is affected if sensors are moved around, additional sensors are added etc. Development of sensor fusion and control systems software is outside the scope of the project. Nevertheless, the project may be geared more towards the development of closed-loop software, including both multi-target tracking and SLAM, if this is desired by the candidate.

Simulation of Milliampere in Unity done by the Simulation Group in EIT Village Autoferje.

Proposed Tasks for the 5th year project

  1. Make an informed choice between the different simulation tools available.
  2. Obtain or construct CAD models of the most important elements in the surroundings.
  3. Construct motion scenarios using simple motion models and basic guidance methods. Simulate the scenarios.
  4. Simulate how the sensor data at Milliampere will look for the simulated scenarios. The simulations should at least involve one passive sensor (e.g., camera) and one active sensor (e.g., lidar).
  5. Analyze the realism of the simulations.
  6. Write report.

The critical challenge that should be solved to make this a really good specialization project is to propose performance measures that allows the candidate to quantify the evaluation in point 5 above.

Proposed Tasks for the master thesis

The project work aims to be extended into a master thesis for the spring of 2020. The following tasks are proposed for the MSc project:

  1. Simulate the remaining sensors on Milliampere.
  2. Integrate the sensor fusion and possibly also motion control system in the simulator.
  3. Implement suitable performance measures to assess whether the sensor fusion is reliable.
  4. Run simulations and evaluate performance for a wide range of scenarios.
  5. If time permits, experiment with different placements of the sensors, both on the ferry and on the sea shore. This is especially important since the full-scale ferry currently being designed will have cameras and lidars placed at different locations than Milliampere.
  6. Write report.

Contact

Main supervisor Edmund F. Brekke