{"id":5220,"date":"2021-10-01T11:58:07","date_gmt":"2021-10-01T09:58:07","guid":{"rendered":"https:\/\/mission-embedded.dev.zold.at\/?p=5220"},"modified":"2023-03-29T11:53:56","modified_gmt":"2023-03-29T09:53:56","slug":"real-time-safety-critical-ai","status":"publish","type":"post","link":"https:\/\/mission-embedded-backup.dev.zold.at\/en\/real-time-safety-critical-ai\/","title":{"rendered":"Bringing AI Analytics closer to Sensors for Real-time Safety-critical Systems"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Bringing AI Analytics closer to Sensors for Real-time Safety-critical Systems<\/h1>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

2021-05-20 | Omair Sarwar, PHD | Academic and professional communities around the world are researching different aspects of Artificial Intelligence<\/a> (AI) and share their results (i.e. AI models) highlighting improved accuracies, evaluated using public or private . However, the process of deploying such AI models to<\/strong> real-time safety-critical systems<\/strong>\u00a0often faces a huge set of challenges.<\/strong><\/p>

\u00a0<\/strong><\/p>

Real-time<\/a>\u00a0means a late response of a system is considered wrong, where the response time is lower for most of our use cases. Safety-critical<\/strong> means a failed system response would cause severe injuries (or even death) to human-beings or huge damage to the infrastructure. The main reason is that academic datasets are usually collected under controlled environments and does not cover all the scenarios which the AI model could come across in real-life. Moreover, such AI models have access to huge computational resources, and do not have any real-time or safety-critical constraints.<\/p>

\u00a0<\/p>

The process of deploying AI models on real-time and safety-critical use-cases is a challenging task. The main objective behind this process is to find an AI model with an optimal trade-off between accuracy, latency, and memory footprint, so that the optimized AI model can run on a standalone embedded system. We call this process AI model optimization for real-time safety-critical embedded systems<\/strong>. In this blog, we discuss different strategies for AI model optimizations and highlight ME expertise in this domain.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t

Challenges for Real-time Safety-critical AI<\/b><\/h2>

There are different challenges for a real-time safety-critical AI system, which need to be addressed during design, development and testing phases of an AI project: <\/span><\/p>

\u00a0<\/span><\/p>

\u00a0<\/p>

  1. Due to real-time & dependability constraints, raw data cannot be sent to a cloud backend for analytical purposes and must be locally processed, preferably on the node itself.<\/span><\/li>
  2. The computational latency of an AI model on the node should be lower than the system response, otherwise the system would fail even if the model computes and predicts the right decision.<\/li>
  3. Any effort to improve latency should not affect the AI model\u2019s accuracy, otherwise the whole objective of deploying it would be undermined.<\/span><\/a><\/li>
  4. The AI model should be capable of making the right decision under extreme-designated conditions and disturbances, which can also occur instantaneously, such as entering a tunnel on a sunny day.<\/li>
  5. Finally, the optimal sensor(s) selection and developing the optimal pre-processing pipelines to remove unwanted noise in the data are important pre-requisites to fulfill the overall objectives of a real-time safety critical AI systems.<\/li><\/ol>

    \u00a0<\/span><\/p>

    <\/p><\/div>

    <\/p><\/div><\/div>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
    Figure 1: Pruning to optimize an AI model. (Left) A fully connected Multilayer Perceptron Model (MLP). (Right) an optimized MLP with pruned neuron and connections which consequentially requires less computations.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t

    Optimization of AI models <\/b><\/h2>

    Optimization of AI models is essential for real-time safety-critical systems because an optimized AI model will not only take less time for inference, but also occupy less memory with negligible (or no) effect on the accuracy.<\/span><\/p>

    There are two strategies to develop an optimized AI model for a given use case: <\/span><\/p>

    (1) optimization-aware-training, <\/span><\/p>

    and (2) post-training optimization.<\/span><\/p>

    \u00a0<\/p>

    The optimization-aware-training technique searches an optimal model architecture, i.e. the AI model\u2019s width, depth, and number of channels in each layer, during the <\/span>training<\/span> process. Moreover, these strategies can also be constrained to finding the optimal weight quantization of the AI model. <\/span><\/p>

    The post-training optimization strategies usually take an off-the-shelf AI model and apply transfer learning to train it first for the given use case, irrespective whether the model is complex or not. After the training, the AI model is optimized using different techniques, e.g. weight-quantization, weight clustering, channel pruning, network scaling, shunt connections. As an example, please see Figure 1 for post-training optimization.<\/span><\/p>

    \u00a0<\/p>

    Optimization-aware-training techniques are complex and require a high level of expertise, while post-training optimization procedures are comparatively simple but may not give the optimal results for the given use case. Once the AI model is optimized, it is deployed in the field and tested against the required acceptance criteria. The AI model optimization process is usually an iterative process as it requires to find tune hyperparameters for the given application. \u00a0<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

    \n\t\t\t\t
    \n\t\t\t\t\t\t\t

    ME expertise for Real-time Safety-critical AI<\/strong><\/h2>

    ME has successfully completed several projects involving AI for real-time safety-critical applications and has filed several patents.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

    \n\t\t\t\t
    \n\t\t\t\t\t\t\t

    ME areas of expertise:\u00a0<\/strong><\/p>