Jiaxi
YU

Multimodal framework for post-flood building damage assessment

Objective

Develop a multimodal deep-learning framework that improves post-flood building damage assessment by combining visual evidence with disaster-domain contextual knowledge.

Background

Segmentation models can be limited by ambiguous damage appearance and scarce labeled data. Large language models offer complementary contextual priors.

Method

Fine-tuned Qwen2.5-7B on a disaster-specific dataset, integrated it as a text-understanding branch with a segmentation model, and fused textual priors with visual features.

Responsibilities

• Developed and implemented the experimental workflow.
• Fine-tuned Qwen2.5-7B.
• Implemented multimodal fusion and conducted evaluation.

Achievements

• VLM adaptation: fine-tuned the Qwen2.5-7B text branch within LLaVA to generate structured damage descriptions for three of four damage categories.
• Multimodal fusion: achieved F1 0.9456 and recall 0.9637 for binary damaged classification, exceeding ChangeMamba by 18.48% and 23.27%, respectively.
• Major-damage sub-task: achieved recall 0.9223, a 22.13% improvement over ChangeMamba.

Benchmarking attention mechanisms and consistency regularization for post-flood building damage assessment

Advisor: Tomohiro Fukuda

Objective

Evaluate attention mechanisms and image-level consistency regularization for post-flood building damage assessment, and develop a semi-supervised approach that reduces reliance on large labeled datasets.

Background

• Post-flood damage assessment requires timely, reliable building-level labels; however, pixel-level annotation is costly.
• The effects of consistency regularization and different attention modules in this setting had not been systematically established.

Method

Benchmarked deep-learning modules for change detection, developed and optimized a prior-attention module, and evaluated image-level consistency regularization in semi-supervised learning.

Responsibilities

Independently completed literature review, framework design, implementation, experiments, analysis, and manuscript preparation.

Achievements

• Conducted systematic semi-supervised-learning evaluation across 140 configurations and 5–50% label ratios; pseudo-label consistency yielded a 4.84% Kappa gain at 5% annotation.
• Designed SPADANet, a lightweight U-Net with prior attention, achieving 9.22% higher recall and 29% fewer destroyed-to-no-damage misclassifications than change-detection baselines.
• Published in International Journal of Disaster Risk Reduction (JCR Q1, IF 4.8).

DOI ↗ Code ↗

Social-media and bitemporal satellite-image fusion for flood damage detection

Advisor: Tomohiro Fukuda

Objective

Improve automated detection of flood-damaged areas when post-event remote-sensing imagery is sparse.

Background

Satellite images may be unavailable immediately after a flood, while social-media images are timely and low-cost but are rarely combined with bitemporal satellite data.

Method

Developed a model that fuses bitemporal remote-sensing and social-media images with an improved encoder.

Responsibilities

Independently completed study design, data processing, model development, experiments, analysis, paper writing, and conference presentation.

Achievements

• Developed a bitemporal image transformer with CNN feature fusion; achieved a 2% F1-score gain over baseline on the Midwest-flooding dataset with 3.04M parameters.
• Published and presented at CAADRIA 2024.

URL ↗ Oral presentation ↗

Mixed reality for remote inspection systems

Purpose

Build high-fidelity 3D digital assets for remote building and facility inspection in mixed-reality environments.

Method

Reconstructed image sequences using Structure-from-Motion toolchains (COLMAP and MicMac), and explored Neural Radiance Fields (NeRF) for reconstruction and rendering.

Responsibilities

Contributed to the 3D reconstruction pipeline and compared SfM and NeRF outputs.

Achievements

Generated high-fidelity 3D models for subsequent MR development and compared reconstruction quality and efficiency across methods.

Commercial renovation of Zhongguancun Pedestrian Street

Advisor: Xianfeng Li

Objective

Develop a renovation strategy that reactivates Zhongguancun Pedestrian Street.

Method

Conducted site analysis; used Rhino and Grasshopper for parametric design exploration; and developed models and visualizations with SketchUp, Revit, Lumion, and Adobe Creative Suite.

Responsibilities

Independently completed site analysis, developed the design strategy, and produced final drawings, diagrams, and renderings.

Achievements

Delivered a complete renovation proposal and a professional design portfolio.

Historical evolution of public-art materials in Beijing

China Agricultural University · Advisor: Xianfeng Li

Objective

Investigate how material choices in public art in central Beijing changed over time and relate to modernization, industrial capacity, and economic development.

Method

Conducted fieldwork and built a SQL database of artwork dates, materials, and artistic characteristics; applied correlation and statistical-significance analysis in SPSS.

Responsibilities

Led field data collection, data organization, database development, and data analysis.

Achievements

Produced a structured database and preliminary evidence linking material use to industrial production capacity and economic development.