{"id":18716,"date":"2022-04-05T10:47:59","date_gmt":"2022-04-05T09:47:59","guid":{"rendered":"https:\/\/exmachina.ch\/?p=18716"},"modified":"2023-03-21T10:54:03","modified_gmt":"2023-03-21T09:54:03","slug":"mixed-reality-for-the-training-%ef%ac%81eld","status":"publish","type":"post","link":"https:\/\/exmachina.ch\/en\/tech\/mixed-reality-for-the-training-%ef%ac%81eld\/","title":{"rendered":"Mixed reality for the training field"},"content":{"rendered":"

One of the most important and sensitive challenges in the sta\ufb00 training is the creation of an environment in which the user can learn in a safe and isolated manner from the company\u2019s production domain.<\/p>\n

The structure may differ depending on the type of training, but in any case it is necessary to reproduce an ecosystem as similar as possible to the real case in order to not confuse the user during his actual entry into the practical world. Training can require considerable costs and resources depending on the job (practice with the maintenance of a car engine, practice with a surgical operation, etc\u2026) thus representing a demanding challenge that must be overcome in the best possible way. Would it be possible to simulate the training that could lead to a simpli\ufb01cation of the infrastructure dedicated to the training? Mixed reality could be a valid solution to create a job simulator in a reliable and credible way.<\/p>\n

But what is mixed reality?<\/p>\n

Augmented, virtual and mixed reality<\/strong><\/p>\n

Before explaining what mixed reality is, it is good to mention two other conceptually opposite realities that form the mixed reality.<\/p>\n

\"\"<\/a><\/h3>\n

Augmented reality<\/strong><\/p>\n

Maybe this term doesn\u2019t ring a bell to someone, but there is a mobile app that is a very good example of what really is the augmented reality: Pok\u00e9monGo.<\/p>\n

This app allows you to \ufb01ght and capture different types of Pok\u00e9mon by simply walking around the streets and cities. A very special feature of Pok\u00e9monGo is the ability to place a creature on \ufb02at surfaces such as tables or bedside tables and this is a synthesis of what augmented reality does.<\/p>\n

This technology is mainly based on the real world which is represented, for instance, through the phone\u2019s camera. The real world is then \u201caugmented\u201d via virtual components (like Pok\u00e9mon). The application, thanks to image analysis, manages to understand if in the video there are surfaces where a Pok\u00e9mon can be placed to give the feeling that the virtual avatar is actually on the table. There are other techniques that can be added to give more realism and coherence to the avatar, such as adding his shadow on the \ufb02at surface. However, it is important to note that there can be no interaction between the real world and the virtual one. All augmented reality does, is add some value to the real world.<\/p>\n

\"\"<\/a><\/h3>\n

Virtual reality<\/strong><\/p>\n

On the other hand there is the virtual reality where instead of using the real world as a foundation, it creates a new whole virtual world. To achieve this, there are special helmets (Oculus Rift, HTC Vive) with the aim of \u201cdeceiving\u201d the user\u2019s brain into believing that he is elsewhere. The person is equipped with a helmet that detects the movements of the head and two controllers that are equally detected in order to give the user an interaction with virtual components. The possibilities offered by the applications are in\ufb01nite and it is possible to obtain a remarkable realism (even knowing that the user is conscious that what is presented is fake) thanks to the accuracy of the movements captured by the device. In virtual reality there is therefore no trace of real elements.<\/p>\n

\"\"<\/a><\/h3>\n

Mixed reality<\/strong><\/p>\n

Mixed reality is the union of the characteristics of the two previously described realities. The real world is the main scene where the user operates but, unlike augmented reality, there is a deeper knowledge of what surrounds the person and an interaction between real and virtual elements can be offered.<\/p>\n

Microsoft has been working for years on a helmet called HoloLens that allows you to work with mixed reality.<\/p>\n

The Microsoft\u2019s helmet uses semi-transparent lenses to be able to show virtual components on the glass in the form of holograms placed on the real world. In order to place virtual objects in an intelligent and plausible way, the helmet has several sensors placed around the head with the sole purpose of mapping the surrounding area and converting it into a digital format. This process is called \u201cSpatial Mapping\u201d and is a key feature in the world of mixed reality. In other words, \u201cSpatial Mapping\u201d allows applications to be informed about what surrounds the user (walls, tables, chairs, \ufb02oor, etc\u2026). The better are the obtained information, the better is the quality of each hologram placed around the user because they will be positioned in the world in a more accurate way.<\/p>\n

\"\"<\/a><\/p>\n

In the \ufb01rst version of the HoloLens, the user had only one possible gesture in order to interact with the digital components, thus limiting the naturalness of the user\u2019s interaction with the virtual world.<\/p>\n

\"\"<\/a><\/p>\n

With the arrival of the second version of Microsoft HoloLens, interaction with digital components has become richer and more natural thanks to the recognition of \ufb01nger and hand movements. Because of this new feature, the user is able to manipulate virtual elements instinctively without having to learn how to use the helmet.<\/p>\n

\"\"<\/a><\/h3>\n

Simulation using mixed reality<\/strong><\/p>\n

So how can mixed reality be used to simulate an environment in which a person can be formed?
\nThe \ufb01rst step is to inform the helmet about the area where the training will take place. In case the \u201cmorphology\u201d of the training area is not complicated and particular (a normal room with easily recognisable objects), the \u201cSpatial Mapping\u201d process is used. However, in most cases, you have to work on a speci\ufb01c object or machinery that must necessarily be reproduced as precisely as possible in a digital format (for example a car engine). In these cases, you have to rely on another technique.<\/p>\n

So let\u2019s take an example: training a mechanic on the maintenance of a particular car engine.<\/p>\n

\"\"<\/a><\/p>\n

You might have a really car engine where you will need to perform manual operations. The simulator inside the HoloLens can inform the user about these operations step by step by indicating the position of the components to be worked on and also showing the action to be taken. Other demonstration elements can be added such as a screen in which you can play a video with also an explanatory text. In order to do all this, the helmet must know exactly the type of engine, its dimensions and the position of each internal component. In this case, the helmet may not be able to create an accurate and detailed virtual version of this engine, especially due to the fact that some components are inside and invisible from the outside. In this case, you can rely on an accurate representation of the engine in a digital format that can be placed over its real counterpart. With this approach, the helmet interacts with the digital model (invisible and places exactly like the real engine) while the user only sees the interactions with the real engine. Finally, thanks to the tracking of hand movements, the application knows exactly how the user is interacting with the engine and can intervene in case he is doing something wrong or in case he needs help.<\/p>\n

Let\u2019s take another example of training in the \ufb01eld of medicine \/chemistry: the manipulation of a \ufb02uid using two test tubes.<\/p>\n

During the various processes necessary for the creation of a drug, it is necessary to carry out manipulations of \ufb02uids in a sterile and controlled environment. A special infrastructure allows users to transfer \ufb02uids within a vertical air stream that allows to limit the contamination of these \ufb02uids.<\/p>\n

\"\"<\/a><\/p>\n

The operations and movements that take place inside the vertical \ufb02ow are very limited and must take place in very speci\ufb01c ways in order to not \u201cbreak\u201d the air \ufb02ow, thus affecting the quality and purity of the \ufb02uid. Working in these conditions requires a heavy training phase to accustom a person to move correctly. The vertical \ufb02ow is invisible to the human eye and it is therefore more complicated to instruct the person on the consequences of his movements without visual feedback. Using mixed reality, however, it is possible to simulate the physical behaviour of the vertical \ufb02ow and to visualise its movements in order to facilitate the learning of this physical phenomenon.<\/p>\n

The task of the simulator inside the HoloLens helmet would be to detect the movements of the user\u2019s hands within the infrastructure to in\ufb02uence the vertical current. Additionally, the simulator has to inform the user if he is correctly proceeding or if it is necessary to correct his actions. At the end of the procedure, the user can be evaluated based on his behaviour by showing a report of the correct or wrong operations in order to improve.<\/p>\n

Final considerations<\/strong><\/p>\n

Mixed reality is still a relatively new and underused technology that is therefore slowly evolving. However, from the results obtained so far, it seems a very promising solution, especially if used in the \ufb01eld of education. Microsoft is spending a lot of resources to advance in this world and with the advancement of the quality of movement tracking (not just arms and hands!) And the computing power, it is more and more possible to approach the virtual world in a natural way thus helping the application ergonomics. We are still a long way from using this technology in everyday life, but little by little we could begin to opt for this solution event for small things, receiving excellent surprises in exchange.<\/p>\n","protected":false},"excerpt":{"rendered":"

How create an environment in which the user can learn in a safe and isolated manner.<\/p>\n","protected":false},"author":33,"featured_media":18956,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[33],"tags":[],"class_list":["post-18716","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech"],"_links":{"self":[{"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/posts\/18716","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/users\/33"}],"replies":[{"embeddable":true,"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/comments?post=18716"}],"version-history":[{"count":0,"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/posts\/18716\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/media\/18956"}],"wp:attachment":[{"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/media?parent=18716"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/categories?post=18716"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/exmachina.ch\/en\/wp-json\/wp\/v2\/tags?post=18716"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}