Normative models of spatio-spectral decorrelation

Abstract

In line with the efficient coding hypothesis, the early visual system aims to minimize spectral and spatial redundancies arising from overlapping opsin sensitivities in retinal photoreceptors (PRs) and highly correlated structure in natural scenes. Encoding color information, or spectral information independent of intensity, requires comparing activities across different types of PRs. Mounting evidence shows that several species across the animal kingdom, such as the fruit fly Drosophila Melanogaster, have an uneven proportion of PR types in their retinas. However, it is unknown whether this uneven proportion is optimized for objectives relevant to the early color processing of natural scenes, as previous studies have modeled spectral and spatial processing in the early fly visual system independently. We built a collection of models incorporating both spatial and spectral information to solve tasks relevant to the fly’s early visual system, such as predictive coding at the level of PR inputs for spatial decorrelation in the retina as well as spatial and spectral decorrelation at the level of PR outputs via color opponency mechanisms. Using this framework, we asked how varying the ratio of the fly’s two main PR types changed performance accuracy on these tasks. From this normative approach, we were able to conclude that the optimal ratio of PR types to best solve these tasks aligns with the experimentally observed distribution and showed this for multiple opsin sensitivity profiles determined within and across labs. Moreover, shuffling either spatial or spectral information in the input natural scene predicted an even PR type ratio, suggesting that biologically observed PR type ratios are optimized for spectral and spatial decorrelation. Altogether, these results suggest that natural scene statistics may have shaped the ratio of PR types in the fly retina through evolutionary mechanisms, providing important implications for understanding sensory systems in an ecologically relevant context.

Conference Presentation

Normative models of spatio-spectral decorrelation predict observed receptor distributions
Ishani Ganguly*, Matthias P Christenson*, Rudy Behnia
CoSyNe, 2022