TREMA

Transforming real estate management with AI

Started
December 1, 2024
Status
Completed
Share this project

Abstract

The collaboration with our industry partner, Xelidoni, specialized in damage assessment for home rentals, aims to automatically detect property damage from images or videos captured with handheld cameras.

Currently, inspections in the home rental industry involve detailed documentation and expert estimations, often taking several hours per property and costing hundreds of Swiss francs per inspection. Our goal is to develop data-driven methodologies to significantly reduce inspection times, lower costs for property managers, and enhance accuracy in the damage appraisal process.

In this InnoCheque project, the SDSC focused on identifying effective data acquisition strategies, highlighting the shortcomings of public benchmarks, analyzing the current dataset provided by Xelidoni, assessing the limitations of state-of-the-art damage detection methods, discussing their feasibility in the home rental context, and exploring future research directions.

People

Collaborators

SDSC Team:
Isinsu Katircioglu
Thibaut Loiseau
Alessandro Nesti
Mathieu Salzmann

PI | Partners:

Xelidoni SA:

  • Joana Morais
  • Thomas Siakam

More info

description

Motivation

In the home rental industry, current damage assessment practices rely heavily on human inspectors, involving manual documentation and expert judgment. This process is time-consuming—often taking several hours per property—and incurs significant costs, especially when conducted at scale. To address these inefficiencies, our goal is to develop an AI-based damage assessment system that leverages visual recognition and anomaly detection techniques to automate inspections, reduce appraisal times from hours to minutes, and lower costs for property managers without compromising accuracy.

Proposed Approach / Solution

We explored the feasibility of leveraging AI to automatically detect damages incurred during home rentals by performing the following tasks:

  1. Review of state-of-the-art damage detection methods and relevant public datasets: We explored three main damage detection approaches, including supervised defect classification, unsupervised anomaly detection, and language-guided anomaly detection.
  2. Analysis of Xelidoni’s dataset and preliminary experiments using AnomalyCLIP: We propose building a dedicated dataset comprising approximately 5,000 normal and 500–750 anomalous images, covering a balanced range of high-, medium-, and low-priority indoor damages, annotated with anomaly labels and some bounding boxes. To ensure robustness and compatibility with public benchmarks, the dataset will include diverse scenes under different lighting conditions, varying damage severities, multiple viewing angles, and high-resolution images.

In addition, we tested AnomalyCLIP on normal and defective images of furnished flats, achieving strong zero-shot detection but encountering minor false positives on reflective surfaces. Since the model was trained on industrial datasets, fine-tuning it on a dedicated home damage dataset could improve performance. Future work could explore pretraining strategies and refining CLIP-based methods for better differentiation of defect types.

Figure 1: Anomaly detection using a pre-trained CLIP-based visual-language model (AnomalyCLIP). Given an RGB image and a text prompt, the zero-shot anomaly detection approach computes anomaly scores, overlaid on the RGB image, to identify damaged regions in the image.

Impact

The AI-based damage assessment has the potential to transform the home rental industry by automating a traditionally time-consuming and costly inspection process. By leveraging visual recognition and anomaly detection techniques, the project aims to reduce inspection times from hours to minutes while lowering associated costs for property managers. This approach not only improves operational efficiency but also enhances the consistency and accuracy of damage appraisals across rental properties.

Gallery

Annexe

Cover image source: Adobe Stock

Additional resources

Bibliography

  1. Lis, K., Nakka, K., Fua, P., & Salzmann, M. Detecting the Unexpected via Image Resynthesis. In ICCV, 2019.
  2. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021.
  3. Pantoja-Rosero, B. G., Oner, D., Kozinski, M., Achanta, R., Fua, P.,  Perez-Cruz, F., & Beyer, K. TOPO-Loss for continuity-preserving crack detection using deep learning. Construction and Building Materials, 344, 2022.
  4. Zhou, Q., Pang, G., Tian, Y.,  He, S., & Chen, J. AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection. In ICLR, 2024.
  5. Zhu, J., & Pang, G. Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts. In CVPR, 2024.

Publications

Related Pages

More projects

MAGNIFY

In Progress
Machine learning Assisted larGe scale quaNtIfication of building energy FlexibilitY
Energy, Climate & Environment

SPI-GreenFjord

In Progress
Energy, Climate & Environment

SPI-PAMIR

In Progress
Energy, Climate & Environment

Fore[st]cast

In Progress
Nowcast and predict forest condition
Energy, Climate & Environment

News

Latest news

First National Calls: 50 selected projects to start in 2025
March 12, 2025

First National Calls: 50 selected projects to start in 2025

First National Calls: 50 selected projects to start in 2025

50 proposals were selected through the review processes of the SDSC's first National Calls.
AIXD | Generative AI toolbox for architects and engineers
January 22, 2025

AIXD | Generative AI toolbox for architects and engineers

AIXD | Generative AI toolbox for architects and engineers

Introducing AIXD (AI-eXtended Design), a toolbox for forward and inverse modeling for exhaustive design exploration.
Smartair | An active learning algorithm for real-time acquisition and regression of flow field data
May 1, 2024

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data

Smartair | An active learning algorithm for real-time acquisition and regression of flow field data

We’ve developed a smart solution for wind tunnel testing that learns as it works, providing accurate results faster. It provides an accurate mean flow field and turbulence field reconstruction while shortening the sampling time.

Contact us

Let’s talk Data Science

Do you need our services or expertise?
Contact us for your next Data Science project!