The Wisdom of Crowds

Fig. 1: Franz Marc’s Yellow Cow (1911), in Munich’s Lenbachhaus

Emergence can be defined as a network effect, and it is little wonder that the concept plays a vital role in the theory of net-based media art. [1] The aim of this essay is to show that emergence can also be observed in research processes, even when the scholarly aspect might, at first glance, not be readily apparent.

Crowdsourcing is often introduced in situations of scholarly need where huge amounts of data have to be processed and where the task is too great for unavailable specialists. [2] In order to proceed, the problems must be subdivided in smaller parts, so that they can be approached even by laypersons. This can prove difficult; at other times, it is relatively simple.

Artigo is a crowdsourcing game that invites people interested in works of art to assign them descriptive annotations. These annotations are validated and registered in a database as soon as at least two people submitted them. Such a method helps to avoid deliberately false input. Within the last eight years, 9 million descriptions have been collected.

Most of these annotations, also called tags, making Artigo part of what is generally defined as social tagging, are very simple concepts that in no way seem to correspond to sophisticated scientific descriptions.

Fig. 1: Franz Marc’s Yellow Cow (1911), in Munich’s Lenbachhaus
Fig. 1: Franz Marc’s Yellow Cow (1911), in Munich’s Lenbachhaus

Franz Marc’s famous Yellow Cow (1911), for example, has received tags like
WALD PFLANZEN BLAUER REITER EXPRESSIONISMUS MALEREI WOLKEN RIND SPRUNG ÖL FARBE GEMÄLDE NACHT TANZEN WEIDE OHREN FARBEN BRAUN BEWEGUNG ROSA MARC EUTER KUH ORANGE SCHWARZ FRANZ KÜHE FARBIG ROT TIER KAMPF TIERE HÖRNER HIMMEL LILA GELB FLECKEN KALB OCHSE HORN SCHWANZ FRANZ MARC BLAU FORMEN FLECK HUF RINDER HUFE BÄUME GELBE KUH STIER SPRINGEN SIGNATUR GRÜN BUNT

Several of them are somewhat advanced, like “dancing,” and others show art historical knowledge in identifying the artist’s name and the movement to which he belongs (“Expressionism”). Most of the annotations, however, are simply descriptions of the colors and objects present in the painting. The appropriateness of the descriptions implies that the method to collect them is also appropriate. Taken together, the tags demonstrate a wisdom not present in any single word, but only in the collection. The whole is more than the sum of its parts: that is also the definition of emergence.

Such an insight, whose importance for any form of historical thinking cannot be overestimated, became clear to me when I wanted to find out about painters similar to Adolph Menzel, a nineteenth-century German realist. What is similarity? Of course, we have a generic idea of what similarity can mean; in his latest book, Douglas Hofstadter defines the analogy, which is nothing but a similarity, as “fuel and fire of thinking.” [3] In art history we would call, say, François Boucher and Jean-Honoré Fragonard similar painters, since both work in the Rococo style, and Boucher and Jacques-Louis David as dissimilar ones, since David belongs to the Neoclassical movement, commonly defined as a rejection of Rococo.

Fig. 2: François Boucher, The Toilet of Venus (1751), in New York, Metropolitan Museum of Art
Fig. 2: François Boucher, The Toilet of Venus (1751), in New York, Metropolitan Museum of Art
Fig. 3: Jean-Honoré Fragonard, The Swing (1767), in London, Wallace Collection
Fig. 3: Jean-Honoré Fragonard, The Swing (1767), in London, Wallace Collection
Fig. 4: Jacques-Louis David, Saint Roch and the Virgin (1780), in Marseille, Musée des Beaux-Arts
Fig. 4: Jacques-Louis David, Saint Roch and the Virgin (1780), in Marseille, Musée des Beaux-Arts

We do so because art historians have an idea of the relationships between historical styles, and because we see different and sometimes opposed painterly aspects in works of art, e. g., in the looseness of brush stroke or in the intensity of color. But we could also try to find a more ‘objective’ way of defining the similarity of painters, possibly based on mathematical reasoning, and that is what the data collected in Artigo allows us to do.

We can take all the annotations attributed to different paintings and compare them. This has the advantage that many eyes might see more than two. Tens of thousands of players have contributed to Artigo over the years; the more identical tags are shared between two works of art, the more similar those works can be called.

The following illustration shows the degree of similarity between Adolph Menzel and various other artists [4]:

Fig. 5: Similarity search in Artigo with reference to Menzel
Fig. 5: Similarity search in Artigo with reference to Menzel

The closer other painters are to Menzel, the more their artworks resemble his, based on their shared crowd-sourced tags. Conversely, other painters from, say, the seventeenth-century are further removed from Menzel and someone from earlier periods may actually be positioned outside the frame of the diagram. Consequently, all those represented in the diagram have a very high degree of similarity.

Looking more closely at these artists, one can see that, with a few exceptions, they are all from the nineteenth-century, and mostly naturalistic, as opposed to academic, painters. This corresponds to what we know about Menzel. Of course, there are exceptions, such as Albrecht Dürer, who was working around 1500. Dürer could be close to Menzel because of the bourgeois and realist character of some of his works, which is also true for William Hogarth, an eighteenth-century English genre painter. The case of Dürer is especially difficult to explain and renders the results somewhat doubtful.

All in all, however, the conclusiveness of the procedure seems convincing: Simple phenomenological descriptions of artworks produced by laypersons can say a lot about their historical and aesthetic character. In order to be even more convincing, a larger study of a more extensive group of artists would be necessary. In addition to measuring the similarity of artists, it could also be fruitful to investigate the similarity of artworks themselves. [5]

What is the added value of such research, especially considering that the results are not a hundred percent convincing? Like many other projects in digital humanities research, they can serve as recommender systems for analyzing huge amounts of data. The enormity of art historical databases means that researchers are in need of tools to help them find the pieces relevant to their work among tens of millions of entries.

Searching for similarities thus helps researchers to find paintings close to the ones they are studying. Such methods do not solve any scholarly problems, but they inspire researchers by allowing them to encounter items chosen not at random, but on the grounds of historical evidence. If, among the first 50 artworks retrieved in a search, 30 can stimulate the research process instead of only five, this is a clear improvement in efficiency. Even if a Dürer comes up, he does not distort the bigger picture.

This article was written by Hubertus Kohle and published on On_Culture: The Open Journal for the Study of Culture 1 (2016)
http://www.on-culture.org/journal/perspectives/the-wisdom-of-crowds/
http://geb.uni-giessen.de/geb/volltexte/2016/12072/