The Borg Lab

4D Cities

Unlocking the Urban Photographic Record through 4D Scene Understanding


There is a growing need for novel ways to access the exponentially growing archives of historical imagery. It is imperative to go beyond cataloging, indexing, and keyword driven databases, to a paradigm where the computer at least partially understands the content of images. Pushing the state of the art in scene understanding and 3D modeling will enable radical new ways to view and experience historical and/or temporally varying imagery. The research described here aims at building time-varying 3D models that can serve to pull together large collections of images pertaining to the appearance, evolution, and events surrounding one place or artifact over time, as exemplified by the 4D Cities project: the completely automatic construction of a 4D database showing the evolution over time of a single city.
 
To learn more, check out the videos and slides from our CVPR 2010 talk, "Probabilistic Temporal Inference on Reconstructed 3D Scenes" below:


Lower Manhattan: 3D Point Cloud and Buildings (7MB)
Lower Manhattan: 3D Point Cloud Over Time (15MB)

Slides: [PDF] [PPT]

This project is partially funded by NSF and a gift by Microsoft Research.

People

Frank Dellaert
Grant Schindler

Related Papers

4D Cities: Analyzing Visualizing and Interacting with Historical Urban Photo Collections
Schindler G, Dellaert F
Journal of Multimedia
2012
4D Cities: Analyzing Visualizing and Interacting with Historical Urban Photo Collections, Schindler, Grant, and Dellaert Frank , Journal of Multimedia, (2012)
image/png iconSchindler12jmm.png
Probabilistic Temporal Inference on Reconstructed 3D Scenes
Schindler G, Dellaert F
CVPR
2010
Probabilistic Temporal Inference on Reconstructed 3D Scenes, Schindler, Grant, and Dellaert Frank , CVPR, (2010)
image/png iconScreen Shot 2012-09-04 at 6.08.30 PM.png
Detecting and Matching Repeated Patterns for Automatic Geo-tagging in Urban Environments
Schindler G, Lublinerman PKR, Liu Y, Dellaert F
CVPR
2008
Detecting and Matching Repeated Patterns for Automatic Geo-tagging in Urban Environments, Schindler, Grant, Lublinerman Panchapagesan Krishnamurthy Roberto, Liu Yanxi, and Dellaert Frank , CVPR, (2008)
image/png iconSchindler08cvpr.png
Inferring Temporal Order of Images From 3D Structure
Schindler G, Dellaert F, Kang SB
CVPR
2007
Inferring Temporal Order of Images From 3D Structure, Schindler, Grant, Dellaert Frank, and Kang Sing Bing , CVPR, (2007)
image/png iconSchindler07cvpr.png
4D view synthesis: navigating through time and space
Sun M, Schindler G, Kang SB, Dellaert F
SIGGRAPH '07: ACM SIGGRAPH 2007 posters
2007
4D view synthesis: navigating through time and space, Sun, Mingxuan, Schindler Grant, Kang Sing Bing, and Dellaert Frank , SIGGRAPH '07: ACM SIGGRAPH 2007 posters, p.194, (2007)
Rao-Blackwellized Importance Sampling of Camera Parameters from Simple User Input with Visibility Preprocessing in Line Space
Quennesson K, Dellaert F
3DPVT
2006
Rao-Blackwellized Importance Sampling of Camera Parameters from Simple User Input with Visibility Preprocessing in Line Space, Quennesson, Kevin, and Dellaert Frank , 3DPVT, (2006)
image/png iconQuennesson06-3dpvt.png
Line-Based Structure From Motion for Urban Environments
Schindler G, Krishnamurthy P, Dellaert F
3DPVT
2006
Line-Based Structure From Motion for Urban Environments, Schindler, Grant, Krishnamurthy Panchapagesan, and Dellaert Frank , 3DPVT, (2006)
image/png iconSchindler06-3dpvt.png
Automatic Acquisition of 4D urban Models and Proactive Auditory Service for Enhanced User Experience
Oh SM, Schindler G, Dellaert F, Walker B, Lindsay J
Young Investigators Forum in culture and Technology
2006
Automatic Acquisition of 4D urban Models and Proactive Auditory Service for Enhanced User Experience, Oh, Sang Min, Schindler Grant, Dellaert Frank, Walker Bruce, and Lindsay Jeffrey , Young Investigators Forum in culture and Technology, (2006)