Beauty in the Mind of the Beholder

Over four weekends in 1958 and 1959, 40 of the world’s most creative and accomplished architects gathered in Berkeley, California to participate in a psychology study. They assembled at Berkeley’s Institute for Personality Assessment Research (IPAR), which was then at the forefront of the burgeoning field of the psychology of creativity.

At IPAR, the researchers interviewed each architect, took notes on their social interactions, and administered a battery of personality, intelligence, and creativity assessments. One such assessment was the Mosaic Construction Test, created for the study by IPAR members Frank Barron and Wallace Hall.

The Mosaic Construction Test is a test of visual creativity. The test uses a set of colored cardboard squares in a palette of 22 different colors and a rectangular frame accommodating 8x10 squares. Participants are asked to produce a composition which fills the frame completely. The resulting “mosaics” are assessed by a panel of expert judges for originality and artistic merit.

Beauty in the Mind of the Beholder explores the duality between data (passive sensory input) and code (intentional activity) — or, put differently, between mind and matter. In Beauty, a selected subset of mosaics created as part of the IPAR creativity study are used as the parameters of an artificial neural network. Like the panel of art professors who evaluated the mosaics at IPAR, the network is then trained to evaluate images for their aesthetic merits. During training, the colors of the original mosaics shift in subtle ways as the network iteratively adjusts its parameters to improve the quality of its judgments. The resulting mosaics function both as visual artifacts which are subject to human judgment, and as active participants in the process of visual criticism. This process of appropriation brings the midcentury fascination with the newfound science of creativity into dialog with contemporary efforts to quantify aesthetic judgment and creativity using machine learning.

For technical viewers, the similarity of the trained convolutional filters to the original mosaics demonstrates the widely-held assumption that the loss surfaces of deep neural networks have many local optima distributed through the parameter space, and that the closest local optimum to the initial parameter setting can be very close indeed.

While Beauty can be documented in the form of grids of still or moving images, the artwork itself consists of the parameters of the neural network, along with the code necessary to execute it.

Acknowledgments

Thanks to Elizabeth Peele at the UC Berkeley Institute of Personality and Social Research for providing images of the architects’ mosaics. For more details on the IPSR study, see Pierluigi Serraino’s excellent book The Creative Architect: Inside the Great Midcentury Personality Study.

Technical details

The network is a variant of the MobileNet architecture, trained for Neural Image Assessment (NIMA) on the Aesthetic Visual Analysis (AVA) dataset, a collection of images from dpchallenge.com. The mosaics are decorrelated using PCA whitening before being injected into the network, and the weights from each step of training are restored for display using the inverse of the transformation matrix derived from the PCA of the original mosaics. The mosaics are used as convolutional filters with an 8x10 kernel in the lowest level of the network.