PhD thesis · University of Edinburgh · 2022

[Thesis] Generative Factorization for Object-Centric Representation Learning

Li Nanbo

Paper

How can a model discover the objects composing a scene from images alone? My PhD thesis studies generative factorization as a route to object-centric representation learning, recovering scene structure without object-level supervision and separating properties of the scene from the conditions under which it is observed.

From latent variables to scene structure

Three graphical models: a single latent variable generating an image, multiple object latents generating an image, and scene latents separated from viewpoint and other observation factors. — Three generative graphical models illustrate progressively more structured explanations of an observation X: a single latent Z, object-wise factors, and scene factors separated from viewpoint V and other influences. View full size · Original PDF

Three complementary directions

MulMON aggregates multiple views into a shared set of object representations. Predicting new viewpoints encourages these representations to resolve ambiguities in individual images.
Latent duplicate suppression (LDS) addresses redundant object representations, helping the model avoid explaining the same object with multiple latent components.
DyMON extends object-centric learning to dynamic scenes, factorizing changes caused by camera motion and by object dynamics.

Together, these directions connect scene decomposition with generative prediction: representations can be queried from new viewpoints and, in the dynamic setting, across time. The thesis develops this progression from multi-view observations towards structured models of changing scenes.