Discourse semantics in the tourist brochures

As Section 3.3.3 established, the semiotic resources become interpretable only in context, that is, within unfolding discourse (Bateman 2011, p. 21). In short, the discourse semantics guide the processes of inference and interpretation and therefore provide the multimodal “cue structures” described by Holsanova and Nord (2010, p. 83). To begin the investigation of page-flow discourse semantics in the tourist brochures, it is necessary to consider the function of the brochures.

Molina and Esteban (2006, pp. 1051-1052) outline the generic functions of the tourist brochures as follows:

A brochure could be defined as a leaflet whose main aim is to mo-tivate people to visit the destination advertised and which includes photographs and the description of the most important monuments and views of the location, as well as information about its services.

rhetorical

Figure 3.11: An extract from Suomenlinna Seafortress in Helsinki (1988) Previous research (Hiippala 2007, 2012a,b; Kv˚ale 2010; Francesconi 2011) has shown that the functions described above by Molina and Esteban (2006) may be realised using different multimodal structures. Therefore, I now continue with an example of the discourse semantics, before proceeding to discuss how to distin-guish page-flow from other semiotic modes — such as text-flow accompanied by images — in order to gain access to its discourse semantics.

Figure 3.11 shows an extract fromSuomenlinna Seafortress in Helsinki (1988), which describes a location within the fortress island, along with a representation of its discourse semantics. Firstly, the example itself needs to be considered. In linguistics, the extract’s brief verbal description might be described as a “colony text”, because its meaning is not arise from participating in a larger linear linguistic structure, but it is perceived as an independent entity (Hoey 2001, p. 75). Note how the colony text as a non-sequential entity ties in with page-flow, which provides the discourse semantics for interpreting non-sequential organisations of multimodal meaning. Following Kv˚ale (2010), this combination of verbal and visual elements could be described as an “image-text-complex”. Thus, what needs to be made explicit is how the example promotes the specific interpretation that the verbal and visual elements belong together.

The right-hand side of Figure 3.11 shows a “back-and-forth” diagram, which outlines the discourse semantics for the example on the left-hand side (cf. Bateman 2011, pp. 28-29). The diagram shows two different domains, which are (1) layout space and (2) rhetorical relations. To begin with, the two entities e and e’ exist within domain (1). Let us assume that in the current example,e is the photograph, while e’ is the accompanying instance of text-flow. As the diagram shows, a relationship of spatial proximity holds between the two entities, that is, they are placed close to each other.

There also exists a mapping relation z, which connects the entities e and e’

in domain (1) with their counterparts z(e) and z(e’) in domain (2). In domain

(2), which handles the rhetorical relations, the photograph z(e)and text-flowz(e’) are both considered units with a rhetorical function. As the diagram indicates, a rhetorical relation holds between the entities z(e) and z(e’): they are united by a rhetorical structure.

What makes all of the above meaningful is that the mapping relation z joins the two domains. This allows the spatial proximity of the elements to be translated into rhetorical unity.

The discourse semantics provide a powerful tool for making explicit how the verbal-visual combinations are to be interpreted within page-flow, but their limited reach needs to be acknowledged. In Figure 3.11, the spatial proximity in layout translates to integration in the rhetorical structure. Yet the image-text-complex represents only one possible configuration of multimodal structure, which means that the discourse semantic interpretation presented in Figure 3.11 does not hold for all instances of page-flow or for the semiotic mode in general. Therefore, an understanding of page-flow in the tourist brochures requires a careful study of the brochures’ multimodal structure. As I will show in the subsequent chapters, the layout and rhetorical layers of the GeM model provide the means to distinguish between the different semiotic modes and their structural configuration.

This concludes the discussion of the three strata of page-flow: the page as a material substrate, the semiotic resources that a page may carry and finally, the discourse semantics. In short, a major challenge for this dissertation is to map the discourse semantics that are required for interpreting the tourist brochures, while connecting them to the patterns observed in the semiotic modes.

Additionally, a model of artefact structure has to be able to formulate cer-tain constraints that set limits to what counts as a tourist brochure (cf. Bateman forthcoming). We may then begin to outline the factors that enable us to recog-nise the multimodal artefact as a tourist brochure and to invoke the discourse semantic models that are required for its interpretation. However, for the purpose of making generalisations about the multimodal artefacts and the dynamics that affect them, the notion of genre is required. The following chapter, which deals exclusively with genre, provides the final part of the theoretical framework of this dissertation. However, I shall first provide some concluding remarks to the chapter at hand.

3.6 Concluding remarks

The aim of this chapter was to discuss the applicability of the Genre and Multi-modality model to the task undertaken in this dissertation: modelling the structure of a multimodal artefact. By progressing through the analytical layers of the GeM model, I concluded that the model is suited for describing various types of

multi-modal structure. Given that the data of this dissertation is heterogeneous in terms of structure, manifested in different page types and their multimodal configura-tions, the GeM model provides an appropriate point of departure for studying the data.

I also pointed out that none of the analytical layers of the GeM model — the base, layout, rhetorical and navigation layers — are capable of describing the multimodal structure of an artefact alone. Instead, the so-called cross-layer analyses are required to pinpoint the relevant semiotic choices that characterise the tourist brochures as an artefact.

Last but not least, I dedicated a significant part of the discussion to the semiotic modes. The stratified model of a semiotic mode — with its strata of a material substrate, semiotic resources and discourse semantics — is a powerful tool for explaining how the multimodal phenomena described using the GeM model op-erate. It is important to keep in mind, however, that these semiotic modes are abstractions. If we want to understand how the semiotic modes work, a delicate description using the data provided by the GeM model is required, as Chapter 7 will show.

To conclude, we now possess a framework that enables us to deconstruct and compare multimodal artefacts, and to store this information into a multimodal corpus. The next step is to consider what can be done with the data, and how the data should be interpreted? In the following chapter, I address the notion of genre and how it may help us to understand the data.

Chapter 4 Genre

In the previous chapters, I examined and outlined the requirements for a theo-retical framework capable of modelling the structure of a multimodal artefact. I concluded that a structure-driven approach to multimodal analysis is the most effective choice for the task at hand. The chosen framework — the Genre and Multimodality (GeM) model — fulfils a key requirement for comparing data, that is, the capability to segment the studied artefacts into analytical units. By build-ing on these analytical units, it becomes possible to compare the structure of multimodal artefacts, provided that the framework is applied consistently to the data.

The questions that follow the collection, analysis and compilation of the data into a corpus are naturally related to data analysis:

• What should we be looking for in the data?

• If we look for patterns, how can they be found?

• If patterns are found in the data, which theories can be used to explain them?

These are the questions which I seek to answer by deploying the notion ofgenre to support the analysis of the data. At this point, however, it should be noted that the following discussion will not touch upon matters such as the representativeness of the data, its analysis and visualisation: they will be addressed in Chapter 5.

The chapter begins with a discussion of genre as a theoretical concept and the challenges of its application in Section 4.1. The remaining chapter has a dual focus, which may be divided along the lines of theory and practice. Sections 4.2 and 4.3 focus on rhetorical, linguistic and multimodal studies of genre, in order to outline the social and communicative functions of the tourist brochures. Section 4.4, in turn, approaches genre from the perspective of document theory and information design, and provides a practical counterpart to the preceding theoretical discussion.

Finally, Section 4.5 brings the aforementioned perspectives together into a model of a multimodal artefact.