Layout in print media

The previous section established that the analysis of layout, that is, how the semi-otic modes combine and interact on a page has been particularly challenging for multimodal research. As I will show below, the previous research may nevertheless inform the work undertaken in this dissertation.

I will first address the work of Royce (1998, 2007) on intersemiotic comple-mentarity in Section 2.5.1, followed by a brief discussion of the stream of research represented by O’Halloran (2008b) in Section 2.5.2. I will then consider generic

structures in print media in Section 2.5.3, before finishing with a discussion of covers as a specific page type in Section 2.5.4.

2.5.1 Intersemiotic complementarity

Royce (1998) presents one of the first applications of Kress and van Leeuwen’s vi-sual grammar to explore the co-occurrence of verbal and vivi-sual modes on a page.

Royce (1998, p. 27) proposes that the relationship between the modes is char-acterised by intersemiotic complementarity, which is synergistic in nature, as the verbal and visual do not merely co-occur on the page, but they also constantly interact with each other (see also Royce 2007). Furthermore, Royce (1998, pp.

28-29) suggests that the complementarity between the verbal and visual can be approached by combining the visual grammar of Kress and van Leeuwen (1996) with the systemic-functional theories of cohesion. According to Royce, this enables identifying the participants, activities and circumstances present in both language and image, and describing their interrelations using the lexico-semantic sense re-lations of repetition, synonymy, antonymy, meronymy, hyponymy and collocation (1998, p. 31). In this aspect, he takes a step forward in incorporating the analysis of language and image into a notion of multimodal cohesion.

At this point, it is useful to briefly consider the notion of cohesion. Bateman (2008, pp. 145-146) has pointed out that descriptions of multimodal cohesion rarely go beyond superficial observations, unless the analysis is supported by a notion of structure. I propose that cohesion should be viewed as a feature of multimodal semiosis, which arises primarily from meaning-making to consolidate the subject matter realised using multiple semiotic modes. The notion ofcoherence, in turn, is a property of multimodal structure, which ties in with the expectations of the reader and is therefore of higher relevance to this dissertation (see Chapter 4).

Nevertheless, from the perspective of contemporary multimodal research the holistic approach of Royce is remarkably novel in several aspects: the proposed analytical framework is not only limited to language and images, but also accounts for the typographic features of a page. Royce notes that the typographic properties are subject to conventions arising from the particular functionalities of the text in question. For example, the body copy is seen as an orthographic whole, “realised by various typographical conventions” (1998, p. 41). Royce uses similarities in typographic realisation to define structuralunits in the multimodal text: a similar approach can be found in the earlier work of Waller (1987) and later in Bateman (2008, p. 120).

Royce (1998, p. 44) also suggests that the specific page analysed in his study can be divided into four balanced sections and continues by elaborating the rela-tionship between these sections:

In many texts visuals often compete to varying degrees with the verbal aspect of the text for page space. Here however, there is no sense of competition but a clear demarcation and balancing in terms of spatial allocation.

The first statement is somewhat assumptive: it would be more accurate to describe that from a visuo-spatial perspective, language and image occur physically close to each other. Whether they are in competition with each other is difficult to establish and shifts the focus away from the actual use of layout. The second statement can be deemed more accurate: it describes how the modes occupy their own space, in which the semiotic modes realise various functionally motivated elements, such as header, copy, visuals and logo. The question that should follow these two statements should ask why these configurations occur in this artefact, and what their function and structure are in contrast to other configurations in the artefact? This is precisely the question that this dissertation seeks to answer in Chapter 7 by deploying the framework developed here. And as we will see below, in many aspects this contribution resembles the one also envisaged by Royce.

Finally, Royce (1998, p. 46) makes an important statement regarding multi-modal analysis of page-based texts and a step towards answering the aforemen-tioned question:

[The WMI text] is an amazingly rich source of complexity in mul-timodal meanings and intersemiotic complementarity, and that there are further areas which would be rich avenues of exploration, and levels of delicacy in analysis which could be developed further.

Underlining the possible analyses on other levels of delicacy, Royce (1998, p. 46) continues:

One area which may be very interesting is how the visual and verbal modes realise intersemiotic complementarity at the level of a multi-modal text’s discourse structure, and even in terms of texts as realisa-tions of a particular generic structures.

Royce’s observations regarding future multimodal research are interesting from a contemporary perspective, as the research has often focused on image-text interac-tion in great detail, while the abstract levels of semiotic organisainterac-tion have received less attention in multimodal research (see however Lemke 1999, 2000). A full-page advertisement in The Economist exemplifies just one page type: within the higher level of organisation of a multimodal artefact, various layouts (and types of intersemiotic complementarity) are likely to be found within the entire magazine.

Instead of looking only at specific, hand-picked examples of image-text inter-action, the research needs to be directed towards the larger units formed by the

modes and the relations holding between them. Thus, Royce’s call for expanding the scope of multimodal analysis towards the more abstract levels is on point: it is a prerequisite for a more comprehensive account of multimodal structures in print media. But at the same time, such an account has to be constrained by the notion of structure to retain the capability to compare data (cf. Bateman 2008, p. 145).

In conclusion, the work of Royce (1998) has been mainly followed up in stud-ies of interrelations between semiotic resources (see e.g. Royce 2007; Liu and O’Halloran 2009; Bowcher 2012; Bowcher and Liang 2013). It has also been incor-porated into more extensive models of multimodal meaning-making, such as the framework presented in O’Halloran (2008b), which I now proceed to discuss.

2.5.2 Analysing print media using SF-MDA

O’Halloran (2008b, p. 450) presents an elaborate theoretical framework for the analysis of print media, which incorporates several previously introduced theoret-ical concepts, such as rank scale and metafunctions (see Sections 2.3.2 and 2.3.3) into a framework known as the Systemic Functional-Multimodal Discourse Analy-sis (hereafter SF-MDA). Following a Hjelmslevian tradition, SF-MDA incorporates two distinct strata: content and expression. Figure 2.6 shows how the rank scales for language and image are positioned in a corresponding relation on the content stratum. The “linguistic, symbolic and visual elements” engage in intersemiosis, which is described using intersemiotic mechanisms (O’Halloran 2008b, p. 453).

The meanings formed on the content stratum are then realised on the expres-sion stratum by cross-functional systems, such as colour and typography (see Lim 2004a).

O’Halloran (2008b, pp. 461-469) exemplifies the use of the SF-MDA framework by analysing a printed public service announcement for AIDS awareness. The analysis proceeds from the abstract semiotic levels towards the content stratum, beginning with the generic structure of the advertisement (see Section 2.5.3) by establishing the primary functions of the visual and verbal elements and the layout areas that they occupy. The rank scale for image is used to identify a Scene, which acts as the Locus of Attention in the generic structure, which “embeds the central idea of the advertisement” (Cheong 2004, p. 165). O’Halloran (2008b, pp. 464-468) then proceeds with a detailed analysis of the advertisement, which outlines how the meanings emerge as a result of the intersemiotic mechanisms.

O’Halloran convincingly shows that SF-MDA is capable of describing print media advertisements on various levels of analytical delicacy with a high number of systems and functions described in O’Halloran (2008b, pp. 457-459) providing the theoretical backdrop. At the same time, the high number of intersecting systems and functions warrants caution, especially from the perspective of theory-building.

Despite their high descriptive capacity, the question is whether the systems and

IDEOLOGY GENERIC MIX REGISTERIAL MIX CONTENT

Stratum

DISPLAY Stratum

INTERSEMIOSIS

Mini-Genres, Items and Components (Linguistic, Visual and/or Other)

LANGUAGE VISUAL IMAGES

INTERSEMIOSIS Discourse Semantics

Discourse Intervisual Relations Work

INTERSEMIOSIS Grammar Clause complex

Clause

Word Group / Phrase Word

Scene Episode Figure Part INTERSEMIOSIS

Materiality

Typography/Graphology and Graphics

Figure 2.6: SF-MDA framework for printed texts: language and visual imagery (Reproduced from O’Halloran 2008b, p. 456)

functions have been mapped to a sufficient degree and subjected to empirical scrutiny. The linguistic systems and functions are likely to be more developed in this aspect, whereas the analysis of the visual systems may have been largely exploratory (see e.g. Lim 2004a; O’Halloran 2008b).

To conclude, I suggested above in Section 2.4.2 that the analysis of visual grammar has to be supported by a more abstract framework, in order to make statements about the generic structure of print media advertisements. However, these generic structures need to be positioned in their context of occurrence — in this case a men’s magazine (O’Halloran 2008b, p. 469) — which adds another level of analytical delicacy: the multimodal artefact. Several different generic structures are likely to be found within an artefact, whose semiotic properties are shaped by

their function. While remaining within the SF-MDA framework, I now proceed to describe one possible approach to describing these generic structures.

2.5.3 Generic structures

Cheong (2004, p. 164) presents a study of five print advertisements. Based on the analysis, Cheong proposes a generic structure for print advertisements: Lead ˆ (Display) ˆ Emblem ˆ (Announcement) ˆ (Enhancer) ˆ (Tag) ˆ (Call-and-Visit Information), where ˆ indicates ‘followed by’, while parentheses imply optionality.

The aforementioned elements of the generic structure allocate the verbal and visual elements into functional clusters, which work together in the print advertisement.

Cheong (2004, pp. 175-176) acknowledges the challenges in describing print me-dia advertisements due to the constant flux in advertisement design. This seems reasonable, as the fluctuation in design conventions has found support in research elsewhere (see e.g. Kostelnick and Hassett 2003). In relation to the data of this dis-sertation, Molina and Esteban (2006, p. 1051) observe that the tourist brochures are also constantly redesigned. Cheong (2004, pp. 175-176) points out that due to the structural evolution of the advertisements, defining the labels for the required and optional elements is challenging. If this holds true for other multimodal arte-facts as well, then we need new approaches to describing the generic structures, which are not based on the principle of linearity (see Section 4.1.1).

As the advertisement for Guess (Cheong 2004, p. 190) shows, advertisements may be minimal in terms of language-image interaction, which leaves the adver-tisement rhetorically open to interpretation on the level of a page. Instead, the advertisements draw on the meaning potential of the visual to represent the ad-vertised entity. In this case, what could be roughly characterised as the semiotic mode of fashion photography, is responsible for communicating with the reader.

This is precisely the area of research to which SF-MDA can contribute to with its high descriptive capability.

Finally, unlike full-page advertisements, the content pages can be far more constrained in their expression, especially in terms of the organisation of language and images in a layout.⁹ Although layouts also evolve over time, they may lack the freedom of expression available to print media advertisements. I will return to the topic of advertising in Chapter 6, in which I consider its implications to modelling the artefact structure. The current discussion continues with another page type that may be found in the tourist brochures, that is, the cover page.

9Legal requirements may also affect layout and design conventions, as Berazhny (2012) and Kostelnick and Hassett (2003, pp. 110-112) have observed.

2.5.4 Covers as contact texts

A multimodal artefact may combine multiple page types, of which each may ex-hibit variation in the multimodal structure according to its function. For example, Held describes magazine covers as “omnipresent contact texts” (2005, p. 173), emphasising the dual function of cover pages as advertisements for the magazine and as a “window” into its contents. According to Held, magazine covers are a

“contact-and-advertisement genre”, which “achieve their global meaning on var-ious levels, viz. page composition, layout, visual images and language” (2005, p. 193). Yet attaching the concept of genre to magazine covers should warrant caution, especially as the constraints and the features of the proposed genre are not outlined. A more accurate description would be to suggest that the covers constitute one possible page type in the particular multimodal artefact, which in turn is affected by other constraints that demand the inclusion of this type of page in the artefact.

Held (2005, p. 177) also acknowledges the relationship between the magazine cover and its contents:

... the visual and textual elements are full of cataphoric reference and indices which obviously drag the reader ‘inside’ while simultaneously controlling and supporting his/her navigation.

Here Held relates the magazine cover to the contents, which both co-exist in the same artefact. In terms of layout design, it is reasonable to expect that several layout configurations may be found within a single multimodal artefact. Therefore, the configuration of the deployed semiotic modes needs to be accounted for in each case. The variation exhibited by different page types can then be related to how layout is used to present the content. This, in turn, sets further requirements for understanding how the pages work both individually and together in an artefact:

in order to achieve a complete picture of a multimodal artefact and its structure, the analysis needs to be able to take on all page types. Achieving this capability is the main goal of Chapter 3, which is presented shortly after the final remarks.