Simulation companies have created a Sensor Consortium to raise awareness of sensor issues in visual simulation system design.

By Chris Blasband

For many years, the definition and concept of how to correlate and interface visual and sensor systems have been hotly debated topics within the visual simulation and modeling and simulation communities. A major contributor to the controversy is the fact that visual and sensor simulations have completely different phenomenology. Visual databases, in general, do not contain the proper geometry and information to produce a proper sensor scene. Typically, the databases provided from IG vendors are thrown over the wall, and most applications require that the sensor simulation provider spend numerous man-hours of database development work to generate realistic sensor imagery.

To overcome these issues, the sensor community must raise awareness of proper techniques for interfacing visual and sensor systems for training applications. This is optimally achieved through the development of a Sensor Consortium to educate the training community through open communication, discussion and the creation of educational materials defining the requirements to efficiently interface real-time visual and sensor training applications. One of the main goals is to raise awareness of the critical importance of having sensor experts involved in all phases of a project, especially in the database generation process.

In the past, it was considered extraordinary for a company to have sensor simulation capability. Today this capability is essential. Nearly all military vehicles and many civilian vehicles have advanced sensors onboard. The military is relying more and more on advanced sensors to deploy its weapons and execute critical missions. Civilian airlines are beginning to incorporate infrared sensors to aid in landing under poor weather conditions and decreased visibility. As sensors become more and more sophisticated, and test flights more and more expensive, there is an increasing need to simulate sensor performance as well as train pilots and operators on the use of these advanced devices.


For years, the training and simulation community has developed one database for the visual out-the-window scene, a second database for the infrared scene, a third database for the night-vision goggle scene, a fourth database for the radar scene, and so on for each sensor being simulated. This became a logistic and bookkeeping nightmare for systems integrators and those responsible for maintaining the trainers and laboratory environments.


The training community solved this problem by using the same static database for both the visual and sensor systems. Material classification became a critical part of the database development effort and correlation was obtained by material classifying all visual textures. It was not long before flaws in this solution became apparent.


For example, a synthetic aperture radar (SAR) image developed directly from a visual database used texture to represent air conditioning units on the tops of buildings. The rooftops should look very speckled—or noisy—in the SAR image due to the radar returns from the air conditioners directly and the interaction of the radar beam with the air conditioners and the rooftop. But, as it turns out, this simulation did not always produce the correct results.


Geometry is the most critical factor for radar. Since texture was used to represent the air conditioning units, the visual database had to be updated with 3-D polygonal features for the radar simulation. That change allowed for a realistic SAR image.


Database issues such as these have plagued the training community for a long time and have wasted man-hours and project dollars. Through education, discussion and eventually input to open standards, the Sensor Consortium will help alleviate many of the problems faced by those responsible for integrating and maintaining training systems.


The charter of the Sensor Consortium is to educate the training community through open communication, discussion and the development of education materials. The consortium’s mission statement is “Raising the awareness of sensor issues in simulation system design.


The consortium’s goal is to educate the community to help make the real-time training development process more efficient and productive. The objective is to help make companies more efficient. It must be emphasized that the goal of the consortium is education and not to influence a company’s development of its internal products and intellectual property or affect its ability to develop business. With inputs from the training community, it is hoped that, in the future, the consortium will help guide the development of accepted open standards for interfacing visual and sensor systems. This will greatly benefit the training community.

Invitations to become a member of the consortium were sent to those who are active in the real-time visual and sensor simulation community and have detailed knowledge of the subject areas.

The role of each member of the consortium is to actively participate in the annual consortium meeting and openly present ideas and information that will educate the community on what is required to develop correlated visual and sensor simulations; contribute papers, materials, etc. to the Web site; and become active participants at relevant conferences such as the Interservice/ Industry Training, Simulation and Education Conference (I/ITSEC), ITEC and IMAGE.

Members are also expected to vote for steering committee members, and, as much as possible, help develop and implement the materials and ideas developed by the consortium.

A steering committee was formed in May 2004 and consists of 10 members from the real-time training community. The members are chosen experts in the field of visual and sensor simulation. The goal was to fill the committee with a diverse group of professionals from private organizations and government labs. The committee was kept small to allow for maximum efficiency. A discussion of how and why members were chosen was presented at the first consortium meeting, held during I/ITSEC 2004.

In order to foster communication within the consortium and the community as a whole, the Web site Sensorconsortium.org is being developed. This will provide a place where professionals in the industry can go to get the latest information on the consortium. All minutes from consortium and steering committee meetings will be kept on the site as well as educational materials, information about future meetings, links to important sites dealing with sensor simulation and relevant matters having to do with the consortium. Members of the steering committee will maintain the Web site.

The Sensor Consortium will meet once per year—most likely in Orlando, FL, during I/ITSEC.






Leave a Reply

Your e-mail address will not be published. Required fields are marked *