3D reconstruction system for capturing moving objects of human body both inside and outside.
Research Purpose
1.Background of research
Detailed observation is necessary to reveal the secret function of human body. Although there are a lot of techniques exist, such as diagnostic radiography and MRI, 3D information of human body is now attracting many people. For example, if you can retrieve accurate and dense 3D shape of muscle and subcutaneous fat or small bumps of the gastrointestinal inner wall, it would be important materials for medical diagnosis.
2.Research objectives
This research is for developing techniques for measuring 3D information of both inside and outside of live human body, such as (1) super-high-speed measurement to observe minute change of human body surface, (2) ultra-small scanner for endoscope to scan inside the human body, and (3) 3D data and motion capture technique with transmission system for remote surgery.
3.Research characteristics (incl. originality and creativity)
Past 3D measurement technique was unable to measure the moving object accurately. Although intensive researches are conducted to capture the data, no practical solution has been proposed yet. Since our technique is specialized for moving object, awarded several times and patented, there is high possibility to solve the problem to achieve high precision and fast human body scanning.
4.Anticipated effects and future applications of research
It is expected that super-high-speed human body shape measurement can reveal new life function, such as automatic facial expression recognition and ultra-small scanner for endoscope will contributes to highly reliable diagnosis on gastrointestinal sickness. Further, 3D scanning of live person can be used for remote surgery and remote diagnosis.
Project #3
| NEDO | |
|---|---|
| Project Title | Development of fundamental technologies for AI systems that evolve together with humans |
| Period | 2020--2025 (5years) |
| Leader | Ryusuke Sagawa |
| Institution | Advanced Industrial Science and Technology (AIST) |
1.Background of research
In the field of image recognition in deep learning, there are many case studies of transfer learning based on ImageNet. Many papers have also proposed transfere learning methods for medical images, and have achieved high accuracy diagnostic results. Since the accuracy of transfer learning depends on the quality and quantity of teacher data used for retraining, it is necessary to collect as much high-quality training data as possible.
For having AI technologies to be widely applied to medical imaging, it is essential to make AI technology dependable, regardless of the number of patients, cases, and tests. Especially for rare diseases and cases, where there are few physicians who can diagnose them, AI assistance is important. If we can improve the diagnostic support capability by AI, we can expect to be able to use existing medical devices as hardware, but with new technology for observation. However, for this purpose, transfer learning is important to compensate for the lack of training data.
2.Research objectives
In this project, we construct a quasi-general-purpose trained model as a base for a diagnostic support model that can be used as a basis of special medical purposes. Among them, our group will be in charge of sub-project of "Research and development of technology to improve endoscopic diagnosis by combining 3D endoscopic shape data and 2D endoscopic image database" in order to contribute to the construction of quasi-general-purpose trained models.
3.Research characteristics (incl. originality and creativity)
In this sub-project, we are developing a method to estimate and predict 3D information from 2D images by combining 3D data obtained by constructing a 3D/2D endoscope image database. We also aim to develop a supportive technology to provide useful 3D information for diagnosis in the observation using a conventional endoscope.
4.Anticipated effects and future applications of research
By using data accumulated in this sub-project, construction of "AI-based diagnosis technology" with less effort can be expected.
Project #2
| KAKEN Grant-in-Aid for Scientific Research (A) | |
|---|---|
| Project Title | Robust and precise 3D endoscope system based on pattern projection and deep learning |
| Period | 2018--2023 (5years) |
| Leader | Ryo Furukawa |
| Institution | Hiroshima City University |
1.Background of research
We have been developping an active stereo 3D endoscope that can measure a wide range of tumor size and shape. As a result, we have achieved a system for 3D measurement of an object by simply inserting an ultra-small laser pattern projector into the forceps of an ordinary endoscope and acquiring the image. However, there are some problems to apply the present system to actual medical purposes, such as ensuring the stability of measurement under the harsh environments insice the digestive tract, obtaining color information of the target tissue, improving the accuracy of measurement results, dealing with complicated shapes such as the folds of the digestive tract, and realizing the measurement of the actual interior of the human body.
2.Research objectives
In order to make this system usable in the medical field, we will carry out the following research projects.
1: Improvement of the measurement system to operate reliably in a humid environments with mucus and gastric juice, such as inside the gastrointestinal tract.
S2: Simultaneous acquisition of color and distance information with the 3D endoscope.
3: Improving output shape resolution and distance accuracies.
4: Development of an depth-estimation algorithm based deep learning to imprve measurement stabilities.
5: To enable the system for actual measurements inside the human body.
3.Research characteristics (incl. originality and creativity)
We have developed an active stereo 3D endoscope with a micro-sized pattern projector and demonstrated the feasibility of 3D measurement. In this study, we are constructing a system that can be usable in an actual biological environment by combining the existing achievements and deep learning methods.
4.Anticipated effects and future applications of research
If a practical three-dimensional endoscopic system can be realized through this research, it can be applied to robotic surgical systems and endoscopic diagnostics.
Project #1
| FUNDING PROGRAM FOR NEXT GENERATION WORLD-LEADING RESEARCHERS | |
|---|---|
| Project Title | Development of real-time 3D scanning system for both inside and outside of human body |
| Period | 2011--2016 (5years) |
| Leader | Hiroshi Kawasaki |
| Institution | Kagoshima University |
1.Background of research
Detailed observation is necessary to reveal the secret function of human body. Although there are a lot of techniques exist, such as diagnostic radiography and MRI, 3D information of human body is now attracting many people. For example, if you can retrieve accurate and dense 3D shape of muscle and subcutaneous fat or small bumps of the gastrointestinal inner wall, it would be important materials for medical diagnosis.
2.Research objectives
This research is for developing techniques for measuring 3D information of both inside and outside of live human body, such as (1) super-high-speed measurement to observe minute change of human body surface, (2) ultra-small scanner for endoscope to scan inside the human body, and (3) 3D data and motion capture technique with transmission system for remote surgery.
3.Research characteristics (incl. originality and creativity)
Past 3D measurement technique was unable to measure the moving object accurately. Although intensive researches are conducted to capture the data, no practical solution has been proposed yet. Since our technique is specialized for moving object, awarded several times and patented, there is high possibility to solve the problem to achieve high precision and fast human body scanning.
4.Anticipated effects and future applications of research
It is expected that super-high-speed human body shape measurement can reveal new life function, such as automatic facial expression recognition and ultra-small scanner for endoscope will contributes to highly reliable diagnosis on gastrointestinal sickness. Further, 3D scanning of live person can be used for remote surgery and remote diagnosis.
