The challenge
Multisignal processing for mobile robotics
Development of electronics for processing multiple sensor data for use in a mobile robot
Efficient use of many sensors:
Embedded vision for multi-signal processing
Surround-round vision solutions and robotic vision applications rely on the combination of extensive signal data. The challenge in developing such solutions is to process and evaluate them quickly and efficiently. Embedded systems can process data from intelligent sensors as well as raw data. Linux is widely used as the operating system.
For this, hema electronic relies on the combination of FPGA technology and ARM processors, for example with Xilinx Zynq Ultrascale+ devices. They contain up to six ARM processors and a large FPGA logic. Up to 800 I/O pins are available. The FPGA logic then takes over the pre-processing of the sensor data, which is provided as raw data or minimally processed by the sensors.
As a rule, corresponding IP cores are available, with which very versatile interfaces can be realised and sensors can be directly connected and configured. The FPGA logic performs the reading of the data streams and mainly the pre-processing, while the ARM processors combine the individual results into an overall picture or overall result and pass them on to a higher-level system via the standard interfaces (e.g. Ethernet, USB). This enables the development of applications that bring together data from different sensors - for example, to merge images from several cameras or to visualise the position and size of objects identified by radar on a camera image. As a rule, no information is lost through pre-processing and the processor has access to all captured data. Thus, the data from a camera sensor can be used for several applications - for example, both to count people and to transmit a complete image of the situation. The data depth of camera sensors is thus optimally utilised.
The mainboards of such embedded vision systems are developed to fit each application exactly and thus have all the necessary interfaces for sensor connection and peripheral devices. hema has developed the embedded vision platform for this purpose. The modular concept includes hardware as well as middleware and a comprehensive software framework. Customers select their required interfaces and functionalities and thus configure their individual electronics, which are then produced within only six weeks. Currently, there are already over 45 Building Blocks to choose from in the hardware library, which includes interfaces such as USB, CAN, Ethernet and Wifi / Bluetooth as well as numerous special video and sensor interfaces.
Computing power and memory are provided by System on Modules, which are equipped with FPGA logics and up to six ARM processors as well as memory as DDR or Flash according to customer specifications. The EMC-critical components around the processors are already tested and integrated here. This modular structure reduces design risks as well as effort and costs in development. Thanks to a standardised connector, different system-on-modules can be used with the same motherboard. This facilitates upgrades and enables different product variants. If SoMs or their components are discontinued, the design also makes it easy to replace them with functionally identical new modules. This extends the life of the main electronics. Series qualification of the electronics can be done in a very short time on the basis of the prototype.
Applications
Electronics for connecting and processing different sensor data
- Video sensors
- Radar sensors
- Lidar sensors
- Other technical sensors
Usable for:
- Sensor fusion and streaming
- Monitoring
- Navigation
Possible areas of application:
- Mobile robots
- Special vehicles
- Drones
- Intralogistics
- Remote controlled systems
My experiences in the cooperation with hema:
Partnership:
- Because I have a strategic partner on an equal footing.
Strategy:
- Because the solutions enable strategically long-term product series.
Platform:
- Because modular concepts and platforms deliver strategic advantages.
Future-proof:
- Because I can pick up my project again and continue it at any time.
... because working with hema is simple, binding and effective
Current Use Cases: References for Embedded Vision
Multi-signal processing: Embedded vision system for the connection and data fusion of numerous sensors
Video Distribution Unit: Development of an Optronic System for Wheeled Vehicles in Defence Applications
Retrofit:
Economical modernisation of an existing infrastructure without replacement