First-of-its-kind system leverages video footage and sensor data from hydraulic excavators
TOKYO, Mar 25, 2026 - (JCN Newswire) - Sumitomo Heavy Industries, Ltd. (SHI) and NEC Corporation (NEC) will begin joint development in April 2026 of a system that automatically identifies near-miss incidents at construction sites and generates reports as part of promoting safety. The system utilizes camera footage and sensor data collected from hydraulic excavators, aiming to prevent accidents.
Background
Construction industry operations are heavily influenced by unpredictable factors such as weather, geological conditions, and constantly changing work environments, resulting in frequent hazardous incidents. To enhance safety, there is growing demand for new support systems that leverage digital technologies. One such demand is for a system capable of automatically extracting, visualizing, and summarizing potentially hazardous scenes ("risk scenes") from site-specific and time-dependent video footage and work logs. However, within the construction industry, no technology is currently available for comprehensively supporting the entire process-from the accumulation of video data on construction machinery operations, to the extraction of risk scenes, the analysis and visualization of near-miss incidents, and, ultimately, the generation of reports in an end to-end manner.
Against this backdrop, SHI-leveraging its expertise in construction machinery and data analytics-and NEC, with its long-cultivated capabilities in video recognition and cutting edge generative AI technologies, will jointly initiate the development of a first-of-its-kind system for the construction industry that automatically extracts near miss incidents and generates corresponding reports based on camera footage and sensor data captured from hydraulic excavators.
Overview
The new system will utilize an extraction AI model, trained on real world hydraulic excavator data accumulated on the SHI Group's ICT/IoT common platform "SHICuTe" (Note 1), to first identify and extract risk scenes from recorded video footage. These risk scenes, together with operational data from the hydraulic excavators, will then be analyzed using NEC's proprietary technology that combines video recognition with generative AI (Note 2), and stored as multimodal data incorporating temporal and spatial information. Based on this data, along with SHI's expertise in construction-site machinery operations and human workflows, the system will cross reference hazardous and prohibited behavior data-which are defined by accidents, construction equipment failures, and operations requiring particular attention-as well as company specific data. Based on these matching results, the system will automatically identify the risk scenes that should be reported and automatically generate high quality near miss reports that provide a concise summary of the circumstances surrounding each incident.
Prior to this initiative, a technical proof of concept was conducted in September 2025 to verify a system that automatically extracts near miss incidents and generates reports from video footage captured by cameras mounted on hydraulic excavators. The results confirmed that, based on the risk scenes extracted from the footage, the system was able to report near miss cases-including potential accident scenarios and their associated circumstances. Building on these outcomes, the joint development starting in April 2026 will aim to further expand the types of near miss incidents that can be identified and enhance the report generation capabilities in line with customers' safety management needs, thereby contributing to the realization of safer construction sites.
Outlook
In fiscal year 2026, technical development and validation of the system will be advanced by utilizing on site data and safety management expertise from SHI, together with AI technologies and advanced technology consulting services provided by NEC, with the aim of achieving practical implementation in fiscal year 2027. Looking ahead, the companies plan to broaden the system's applicability beyond scenes where physical contact between workers and machinery may lead to occupational accidents to include unsafe conditions that may not be readily recognized by workers, as well as considerations for site specific operational rules, thereby further expanding its scope of use.
SHI and NEC will continue to combine their respective strengths to develop and validate new technologies, contributing to the realization of safer construction sites.
(1) SHICuTe is a common platform that enables cross functional development of capabilities across the product lines of the SHI Group. It collects and stores various operational data from the group's products connected online. "SHICuTe" is a registered trademark of Sumitomo Heavy Industries, Ltd.
(2) "NEC uses generative AI (LLM) and video recognition AI to automatically generate explanatory text from video - Applied to drive recorder videos, cutting accident report creation time in half -" (Announced by NEC on December 5, 2023) https://www.nec.com/en/press/202312/global_20231205_01.html
About this technology
A new hope for zero construction-site accidents, born from a fusion of construction machinery data and LLMs - A collaborative innovation by Sumitomo Heavy Industries and NEC URL: https://www.nec.com/en/global/rd/technologies/202512/index.html
About NEC
The NEC Group leverages technology to create social value and promote a more sustainable world where everyone has the chance to reach their full potential. NEC Corporation was established in 1899. Today, the NEC Group's approximately 110,000 employees utilize world-leading AI, security, and communications technologies to solve the most pressing needs of customers and society.
For more information, please visit https://www.nec.com, and follow us on LinkedIn and YouTube.
Source: NEC Corporation
Copyright 2026 JCN Newswire . All rights reserved.
© 2026 JCN Newswire
