This project was created to explore the usefulness of a MATLAB program which labels components of photographs with labels from a given set of 47 tags (such as "tree", "white-horse", "perspective-building", "front-building"). The paper on the original project and the image set can be found here.

The goals of this project was to find similarities and highlight differences in image content and composition, which would reflect the success of the image tagging and sorting. I wanted to ask if content or tag similarity was a useful metric for comparing visual similarity. I hoped that by sorting the images by tags and blending images based on sort order I could create interesting relationships between the content of the images.

I started with a list of the image file names and their corresponding tags which came from the MATLAB program mentioned above. I created a python program to reorder the lines in this csv file to maximize tag-closeness of subsequent images. This created a list of images that are content-wise most similar to the images before and after them, according to their assigned tags.

I then wrote a program in processing to showcase the newly ordered images using a mix of blending techniques. This program takes two of the re-ordered images at a time and overlays the first image on the second using one of three methods (screen, add, or lightest) at random. It then removes the first image and blends the second image with a third, the one which appears directly after the first two in the file. This repeats indefinitely in the animation produced by processing, looping when the file ends. The video above showcases two loops of the file, though originally the program ran continuously and projected into a public space at UChicago.