Analyse de texture multi-modalité pour l'imagerie de télédétection

par Chenguang Liu

Projet de thèse en Traitement du signal et des images

Sous la direction de Florence Tupin et de Yann Gousseau.

Thèses en préparation à Paris Saclay , dans le cadre de École doctorale Sciences et technologies de l'information et de la communication (Orsay, Essonne ; 2015-....) , en partenariat avec LTCI - Laboratoire de Traitement et Communication de l'Information (laboratoire) et de Télécom ParisTech (établissement de préparation de la thèse) depuis le 01-11-2016 .


  • Résumé

    The new generation remote sensing sensors have specific properties : they provide data with improved resolution, with an increased number of channels, and with high temporal frequency. One of the main challenge is the joint use and combination of such data, be it optical or radar (SAR) data. Among these challenges different difficulties may be investigated. The simplest one is the exploitation of SAR multi-temporal series of a single sensor. Radiometric and geometric calibration is usually highly reliable and the main difficulties are linked to the speckle phenomenon and the variability of strong scatterers in urban areas. The second context is the combination of SAR or optic data taken with different sensors and different viewing conditions. In these situations, variable resolutions, viewpoints and radiometric responses have to be taken into account. The third and most complex situation is the joint use of multi-temporal optic and multi-temporal SAR data. In this situation the physical mechanisms leading to the image signal are fundamentally different and should probably be dealt with using machine learning techniques. Scientific challenges The PhD project can be subdivided in the following three main parts. The first part deals with multi-temporal SAR images. Their all-time all-weather capacities make them unavoidable in multi-temporal analysis, especially in the case of major disasters. The second part focuses on texture modeling, due to its utmost importance in image analysis. The developed methods will be general enough to be applied either to SAR images or optic / multi-spectral data. The third part is dedicated to multi-temporal series of remote sensing images provided by heterogeneous sensors, and especially in the case where SAR and optical information are combined. Multi-temporal SAR images In the first situation of a time series acquired by a SAR sensor, the main difficulties are linked to the noise level and the variability of the strong scatterers in urban areas. The joint use of a data-base of images should lead to new and more reliable approaches at the low level (for instance through likelihood tests defined on subsets of images or analysis of similarity matrices). To overcome the strong varibility of SAR images taken by different sensors or with different incident angles, a promising approach is to define structures at a higher scale. In particular, grouping methods of local descriptors taking into account the local relationship should lead to more reliable change detection frameworks. Indeed, the task of change detection for heterogeneous images is currently investigated in our group through local descriptors and a contrario methods permitting the detection and grouping of local descriptors having undergone some changes. Explicit relationships between descriptors could be integrated in higher level primitives. This could be done through graph structure and tolerant graph-matching methods. This would hopefully enables one to more accurately classify changes, especially in high resolution images of urban areas.Temporal texture modeling One particular aspect of remote sensing images whose time monitoring is of interest is texture. Indeed, time series of textural features may indicate vegetation changes or large scale urban changes. These approaches have been mostly applied, in the context of remote sensing imaging, to global texture attributes that are very sensitive to geometrical deformations and contrast changes (such as co-occurrence matrices or filter bank responses). Now, one of the most powerful approach to the indexing of texture images, developed in computer vision contexts, is to extract local descriptors having a controllable degree of invariance, and then to compute bag-of-words (bow) or other type of marginal statistics on these descriptors. In the context of remote sensing, this has the great advantage of permitting a high degree of invariance, both to radiometric and geometric changes. Thanks to these invariances, descriptions that are robust to resolution variations, changes in view points and weather conditions can be foreseen. Developing temporal models in this context will be the second objective of this thesis. This in particular necessitate the design of very stable local descriptors extraction schemes, dictionary learning techniques over large series of images and specific change detection algorithms. Joint modeling of SAR and optical images The third methodological aspect of this thesis work will concern the simultaneous use of SAR (possibly with multiple angles) and optical images. In the framework detailed above, the first task will be to develop local descriptors and statistical characteristics (e.g. bow) that could be computed in a unified way for the different modalities. Among them, SIFT-like descriptors are an obvious option and we will draw on previous experience acquired in our group on the development of SAR-adapted descriptors. In a different direction, Gaussian Mixture Models of patches with invariance properties should also be investigated. In this context, a particular care should be given to the patch space in which models will be computed. Then, some learning phase will be necessary to fully exploit such an indexing scheme. First, this will enable us to reduce the dimension of the descriptors and possibly to better understand their structure. Second, this learning stage could also permit to address situations in which one of the modality, e.g. the optical one, is not available. Eventually, this step of the research plan will be integrated in the framework of temporal texture modeling, in order to allow for a multi-modality temporal analysis as explained above. Applications The recently launched ESA Sentinel sensors (SAR sensor with Sentinel-1 and optic with Sentinel-2) provide a huge amount of data allowing the creation of long time series never acquired before. The learning of the link between such data with the methods previously mentioned could help defining new change detection approaches between heterogeneous images to evaluate damages and provide rapid mapping tools. Such approaches rely on highly efficient low-level features and big enough training sets.

  • Titre traduit

    Multi-modality texture analysis for remote sensing imaging


  • Résumé

    The new generation remote sensing sensors have specific properties : they provide data with improved resolution, with an increased number of channels, and with high temporal frequency. One of the main challenge is the joint use and combination of such data, be it optical or radar (SAR) data. Among these challenges different difficulties may be investigated. The simplest one is the exploitation of SAR multi-temporal series of a single sensor. Radiometric and geometric calibration is usually highly reliable and the main difficulties are linked to the speckle phenomenon and the variability of strong scatterers in urban areas. The second context is the combination of SAR or optic data taken with different sensors and different viewing conditions. In these situations, variable resolutions, viewpoints and radiometric responses have to be taken into account. The third and most complex situation is the joint use of multi-temporal optic and multi-temporal SAR data. In this situation the physical mechanisms leading to the image signal are fundamentally different and should probably be dealt with using machine learning techniques. Scientific challenges The PhD project can be subdivided in the following three main parts. The first part deals with multi-temporal SAR images. Their all-time all-weather capacities make them unavoidable in multi-temporal analysis, especially in the case of major disasters. The second part focuses on texture modeling, due to its utmost importance in image analysis. The developed methods will be general enough to be applied either to SAR images or optic / multi-spectral data. The third part is dedicated to multi-temporal series of remote sensing images provided by heterogeneous sensors, and especially in the case where SAR and optical information are combined. Multi-temporal SAR images In the first situation of a time series acquired by a SAR sensor, the main difficulties are linked to the noise level and the variability of the strong scatterers in urban areas. The joint use of a data-base of images should lead to new and more reliable approaches at the low level (for instance through likelihood tests defined on subsets of images or analysis of similarity matrices). To overcome the strong varibility of SAR images taken by different sensors or with different incident angles, a promising approach is to define structures at a higher scale. In particular, grouping methods of local descriptors taking into account the local relationship should lead to more reliable change detection frameworks. Indeed, the task of change detection for heterogeneous images is currently investigated in our group through local descriptors and a contrario methods permitting the detection and grouping of local descriptors having undergone some changes. Explicit relationships between descriptors could be integrated in higher level primitives. This could be done through graph structure and tolerant graph-matching methods. This would hopefully enables one to more accurately classify changes, especially in high resolution images of urban areas.Temporal texture modeling One particular aspect of remote sensing images whose time monitoring is of interest is texture. Indeed, time series of textural features may indicate vegetation changes or large scale urban changes. These approaches have been mostly applied, in the context of remote sensing imaging, to global texture attributes that are very sensitive to geometrical deformations and contrast changes (such as co-occurrence matrices or filter bank responses). Now, one of the most powerful approach to the indexing of texture images, developed in computer vision contexts, is to extract local descriptors having a controllable degree of invariance, and then to compute bag-of-words (bow) or other type of marginal statistics on these descriptors. In the context of remote sensing, this has the great advantage of permitting a high degree of invariance, both to radiometric and geometric changes. Thanks to these invariances, descriptions that are robust to resolution variations, changes in view points and weather conditions can be foreseen. Developing temporal models in this context will be the second objective of this thesis. This in particular necessitate the design of very stable local descriptors extraction schemes, dictionary learning techniques over large series of images and specific change detection algorithms. Joint modeling of SAR and optical images The third methodological aspect of this thesis work will concern the simultaneous use of SAR (possibly with multiple angles) and optical images. In the framework detailed above, the first task will be to develop local descriptors and statistical characteristics (e.g. bow) that could be computed in a unified way for the different modalities. Among them, SIFT-like descriptors are an obvious option and we will draw on previous experience acquired in our group on the development of SAR-adapted descriptors. In a different direction, Gaussian Mixture Models of patches with invariance properties should also be investigated. In this context, a particular care should be given to the patch space in which models will be computed. Then, some learning phase will be necessary to fully exploit such an indexing scheme. First, this will enable us to reduce the dimension of the descriptors and possibly to better understand their structure. Second, this learning stage could also permit to address situations in which one of the modality, e.g. the optical one, is not available. Eventually, this step of the research plan will be integrated in the framework of temporal texture modeling, in order to allow for a multi-modality temporal analysis as explained above. Applications The recently launched ESA Sentinel sensors (SAR sensor with Sentinel-1 and optic with Sentinel-2) provide a huge amount of data allowing the creation of long time series never acquired before. The learning of the link between such data with the methods previously mentioned could help defining new change detection approaches between heterogeneous images to evaluate damages and provide rapid mapping tools. Such approaches rely on highly efficient low-level features and big enough training sets.