The aim of this study is to inspect how verbal and visual facets of a discourse assimilate to create meanings. It aims at exploring how the cohesive devices subsidize to found link between the verbal and visual aspects of discourse. The emphasis of this study will be the exploration of the cohesive devices providing the incorporation of verbal and visual means present in Primary level books printed by PTBB. This research will also investigate how the authors use different approaches to create a meaningful ideational association between verbal and visual aspects of discourse.
1.1 Incentive for the Research
”Children learn more effortlessly and quickly through a visual along with a vocal text than simply using an isolated semiotic mode for their erudition. To inspect whether this multimodal phenomenon is helpful for the students of primary level, was the motivation of this reading. This study was motivated by the idea of discovering those topographies, devices and approaches that make a multimodal text easy to comprehend. This research was accompanied to analyze the amalgamation of visual and verbal discourse as an effective meaning making source.
Pakistan came into being in 1947. After Sixty four years we are unable to adapt a policy of education on equal basis. Without education we cannot achieve the development goal. In Pakistan there is illiteracy rate is very high due to different education system. The country is one but education system is different in all over the country. In this system we cannot compete the global challenges. Pakistan has 60 to 70 million youth. The youth want to work hard and play a role for the development of Pakistan, but need to give them policies which are able to use the youth energy for the betterment of country. So therefore if we give them different education system like English and Urdu Medium then the performance of every student is affected.
Education is an important part of any person life. Educations usually refer to the engagement of people formally or informally in the learning of new ideas and skills. Education provides information and knowledge to people for their better understanding of the world. Educated people get more respect and approval in the society. Formal education was not very common in previous centuries and only informal education was given to children for basic learning. Now days because of the rapid development and progress of the world, education has got a very important position and appreciation among people of the world. Education is considered as an instrument of social change and development.
1.2 Contextual of the Study
The current study is about the multimodality in discourse which contains different semiotic modes. Semiotics is the education of sign system in which linguistics is one kind of semiotic means used for meaning making. Halliday and Hassan (1989) define semiotics 2.
Broods learn more quickly when they are essential to read and yield text involving different modes of communication i.e. visual and verbal. Now in recent time, books are transformed. We see this change in the semiotic schemes which are done by the textbook originators.
Now the books encompass of colorful images on almost every leaf along with the text explanation. Designers of the textbooks use assets which are helpful in manufacture meaning for the educational dedications. In recent times, designers of a volume include writer, illustrator, copyreader, and other specialists. All kinds of professionals have their precise semiotic modes for generating a meaningful multimodal text. These modes and genres perform definite pedagogical societal work. All of the professionals are now transporting their contribution to design a multimodal textbook (Jawitt, 2008).
In the textbooks, images and inscribed texts stand equally with each other so that neither of them enjoys supremacy. To clutch the meaning successfully, we need to scrutinize how the two modes (visual and verbal) of communication work in integration or in segregation. The current study will debate how these two modes work together. The works done on 3.
Multimodality is the junction of different semiotic modes existing in a society/culture (Van Leeuwen, 2005). It is the association of dissimilar semiotic modes i.e. visual, verbal, music, dance, paintings, art, sculpture etc. It is a kind of discourse which includes several different semiotic modes. Different works have, in current times, verified the significance of studying multimodality in the textbooks. The existing study also focus on the multimodal cohesion in the Primary level textbooks distributed by the Punjab Textbook Board.
MasoumehMahdizadegan Attar (2014) has worked on inter-semiotic cohesion analysis of multimodal fundamentals in Iranian English Textbooks. In his research he has explored that how visual and verbal discourse is signified in the textbooks for the children and explored what are the cohesive devices that work among these two semiotic resources at textual level of ideational meaning.Kress and Van Leeuwen (1996) have calculated the science text books containing images, drawings, photos etc. to turn out the graphic grammar.
The present study debates the inter-semiotic cohesion analysis in primary level textbooks published by PTBB. Cohesive devices have been sightsaw by Liu and O’Halloran (2009) based on Halliday’s (1985) Systemic Functional Theory.
There is a big difference between the educational level of developed and developing countries of the world. In developing countries like Pakistan although literacy rate is low but now the trend are changing and are accepting the importance of education. This low literacy rate can be because of different reasons. The acceptance of education is increasing the number of enrolment of students in educational institutions. This enrolment is affecting by the drop outs and not continuing studies by the students in Pakistan. Continuity of studies is a big issue. Many students enrolled in schools and colleges don no complete their degrees and left their education.
Since then, the government of Pakistan strived to promote education in a big way.
Initially the Pakistan’s entire system of education was state-run. With the growing demand for quality graduates, many Pakistanis were prompted to seek university degrees abroad in places like the United States, Great Britain and Australia.
However, the growing demand for higher education fast outpaced the establishment of new public universities. During that period, the system could accommodate only 25 percent of the high school graduates who applied to higher education institutions.
In 1979 a government commission came out with the findings that there was an acute poor participation rates at all levels of education, the public sector could no longer be the country’s sole provider of education. By the mid-1980s, private educational institutions were allowed to operate on the condition that they comply with government-recognized standards.
Until 1991, there were only two recognized private universities in Pakistan: Aga Khan University established in 1983; and Lahore University of Management Sciences established in 1985. By 1997, however, there were 10 private universities and by 2002, this number had doubled to 20. As per records taken in the year 2003-2004, Pakistan had a total of 53 private degree granting institutions.
The rapid expansion of private higher education is even more remarkable if we look at the number of institutions established on a year-by-year basis. In 1997, for instance, three private institutions were established; in 2001 eleven new private institutions were opened; and in 2002 a total of 29 private sector institutions came into being. The private sector contributes only 0.5% on education of GDP. It means only one sixth of total resources. The private schools and private educational institution can be made only for generating profit.
“The medium of instruction employed by the educational institutions is predominantly Urdu (65%). This percentage is higher for public institutions (68%) compared with the private sector institutions (57%).” (NEC 2006).
The public sector accounts for around 64% of all enrolments and dominates the structures of all stages of education. While the overall share of the private sector in total enrolment is around 36% its enrolment share is 42% in pre-primary education. Primary stage 32% middle stage 33% high 30% and higher secondary 18%. Technical/ vocational (52%) Vocational/Polytechnics (57%), Non Formal Basic education (61%) and Deeni Madras (97%) (ibid).
The private sector, role has expended in recent years. There are several causes of this relative growth. But it is partly reflection of short-coming growth of the public sector and provides quality of education.
Today our education system does not offer the environment and opportunity of one and equal system of education. We have a very divisive education system which has created a huge gap among the nation and penetrated deeply into our culture. Thus different tiers have been created in our system over a period of 60 years to facilitate the hold of the elite over the governing of our nation. There are many systems working in the country, resulting in social division and conflict.
The study aims to:
1) Analyze the inter-semiotic nature of meanings present in Primary level books published by PTBB.
2) Conduct the Multimodal analysis of the verbal and the visual modes at the ideational levels of meaning in texts and images, respectively.
3) Explore the inter-semiotic complementarity of the visual and the verbal semantics at ideational (logical relations, experiential meaning) level.
1.4 Research Questions
1) How are the verbal and the visual modes expressed at ideational (transitivity) and representational level of meaning, respectively, in the texts and images in Primary level textbooks published by PTBB modes?
2) Which categories of cohesion are employed to cohesively link verbal and visual modes in Primary Level Textbooks published by PTBB modes?
The current study deals with the analysis of inter-semiotic cohesion between verbal signs and visual images in Primary Level Textbooks by PTBB. A sample of 59 pictures has been selected from Primary Level Textbooks of English, General Science and General Knowledge by applying criterion sampling techniques. Both of the modes, verbal and visual, were analyzed by employing Halliday’s (1978, 1985 and 1994) SFL and Kress and Van Leeuwen’s (2006) multimodal frameworks. To study the Inter-semiotic cohesion between the visual and the verbal modes, Liu and O’Halloran’s (2009) proposed categories for the Inter-semiotic cohesion were used.
Liu and O’Halloran’s (2009) framework is based on Halliday’s (1985) Systemic Functional Linguistics (SFL). SFL (1985) views the language as a system not as a structure. It is systemic in the way that it provides a system of paradigmatic choices for making meaning. Language is viewed as Functional because it performs the functional purposes. Function is performed through the three Meta-functions given by Halliday (1985) which are: Ideational, Interpersonal and Textual.
Halliday (1985) has proposed three meta-functions of language which are Ideational, Interpersonal and Textual meta-functions. According to Halliday (1985), Ideational Meta-function is the function used to convey information and to express the content of the language. Interpersonal Meta-function examines the relationship between participants of a communicative act, whether it is spoken or written. Textual Meta-function studies the relationship of text with context and its intended meanings.
This study will discuss the ideational meta-function at logical and experiential levels. Yu Liu and O’Halloran (2009) model analyses the inter-semiotic cohesion between visual and verbal modes at ideational level by employing inter-semiotic categories of logical and experiential meanings. The current study analyses the inter-semiotic cohesion of 6
visual and verbal semiotics by utilizing Logical and Experiential categories identified by Liu and O’Halloran (2009). Logical meanings have one cohesive device i.e. Implication Sequence and experiential meanings have six cohesive devices: Antonymy, Hyponymy, Polysemy, Meronymy, Collocation and Correspondence, to analyze texts or images at the ideational level of meaning. For verbal analysis, Halliday’s (1985) lexico-grammatical feature of transitivity is employed and for and visual analysis Kress and Van Leeuwen’s (2006) framework for representation has been utilized respectively.
1.6 Delimitation of the Study
Halliday (1985) has pointed out three meta-functions of the language for analyzing the text. However, in this study the researcher will be only dealing with the ideational function of the language to analyze the data taken from Primary Level Books of PTBB. Interpersonal and textual meta-functions will not be examined. This is the limitation of the study that it only focuses on examining the ideational meta-function and it does not discuss the other functions which are interpersonal and textual. The study is delimited to the representation meta-function (Kress and Leeuwen, 2006) and Transitivity (Halliday, 1985) for examining the visual and the verbal modes, respectively.
English is the official language of the country and also International language. In Pakistan lots of private English medium schools are working. English is international language as you can find people who understand that language in all parts of the world. It is also the language of the internet and international media. Although urdu is the official language of Pakistan but English is the language which is formally used in all private and government offices. One reason of the importance of English in Pakistan and India is that both remained under British control for long time and consequently adopt the language of their previous masters. Now English is compulsory in all schools in Pakistan and its importance along with its usage is increasing day by day.
Significance of the study The previous research works on multimodality have shown the significance of studying multimodality in textbooks. However, very few studies have been done to analyze the cohesive meaning in the textbooks for the Primary Level in Pakistan. The theoretical significance of the study is that it has analyzed the verbal signs and visual images by using two frameworks; one is by Halliday (1985) for verbal text analysis, which is further 7modified by Yu Liu and O’Halloran (2009) for analyzing the inter-semiotic cohesion and the other framework is by Kress and Van Leeuwen (1996, 2006) for analyzing visual images. The study can be helpful for the course designers to understand how illustrations and text should be put together to create effective meaning. The practical importance of the study is that it can help the course designers to establish effective relation between linguistic signs and visual images at ideational level by understanding these cohesive devices (see chapter 2 and section 126.96.36.199.1) most comprehensively. This research has the pedagogical implications that it can help the teachers to teach the intended meanings to the students effectively. Teachers can easily and effectively teach the learners, by understanding the processes and cohesive devices involved in meaning making and cohesion in the text and images. This study can also contribute to the existing work on multimodal literacy specifically the work on multimodality in textbooks.
1.8 Organization of the work
This study consists of five chapters, in which the first chapter casts background of the study, multimodality in textbooks, research objectives, significance and limitation of the study; chapter two consists of the review of the literature of the study; chapter three comprises methodology, data sampling etc.; Fourth chapter deals with the analytical process of the study and the fifth chapter concludes the study. 8
The following chapter discusses the literature related to this study. The signs and sign systems have been discussed in section (2.1). The concept of Multimodality is discussed in the section (2.2) to explain how visual and verbal semiotic signs integrate to produce multimodal meanings. The chapter, briefly, discusses the framework of Systemic Functional Linguistics (SFL) as purposed by Michael Halliday (1985) which provides the backbone for the verbal analysis of the texts. Section (2.9) explains Kress and Van Leeuwen’s model (2006) for analyzing visual grammar of the images. Section (2.7) discusses O’Halloran’s model (2009) for inter-semiotic cohesion which has been used to understand the interdependence of the visual and the verbal modes.
Semiotics is the theory of signs and symbols. Human beings need to unravel their existence in the society and this leads to the creation of signs and sign system such as language, art forms, paintings, images etc. People interact with each other through this sign system. Semiotics is not simply a study of signs but it is the study of “sign system” in which language is one kind of semiotic systems which is used for meaning making, but Halliday indicates that there are many more ways of meaning making modes that exist in any culture (Halliday ; Hasan, 1985, p. 4). For Halliday, semiotics is not just a study of signs but it is the study of sign system i.e. the study of meaning in its most general sense (Halliday ; Hasan, 1989, p. 4). Semiotics is the study of these signs and sign system 9
within a society and the rules governing these signs in every culture. Sebeok (1994, p. 5) has defined semiotics as “the antique doctrine of signs”.
There was a study Performance of Secondary School Certificate (SSC) Graduates of English and Urdu Medium Schools: A Comparative Study by Nisar, and Ijaz (Year?). The major purpose of the study was to investigate the difference in performances of students of English and Urdu medium schools at Higher Secondary School Level. All the students enrolled during 2005, 2006 and 2007 in Government Degree Colleges in Rawalpindi constituted the population. Two degree colleges (one male and one female) were randomly selected and 730 students enrolled during 2005, 2006 and 2007 in F. Sc program were included in the sample. Data were collected through personal visits to the sample colleges. Collected data were tabulated and analyzed by using t-test for significance of mean difference. The results of the study revealed that students of English medium schools performed better than the students of Urdu medium schools in the subjects of English and Physics whereas no significant difference was observed in the subject of Urdu. It was recommended that medium of instruction for science subjects at secondary level should be English. There was an also another study with the title of “English-medium instruction for subject courses in tertiary education: Reactions from Taiwanese undergraduate students “By Yu-Ying Chang Department of Foreign Languages and Applied Linguistics Yuan Ze University. This paper reports part of the results of a research project, the main goal of which is to evaluate the implementation of English as the medium of instruction (EMI) for content courses at a private University in northern Taiwan. The perspectives of both students and teachers were examined in the project. This paper, however, reports mainly the results obtained from the students in terms of the following four aspects: students’ reactions to the EMI subject courses, influence of `English-medium instruction on the students, difficulties that students encountered in their EMI courses, and their English language learning needs observed during the research process.
There was study by Gillani et al. (Year?) aimed at analyzing the effects of medium of instruction on students’ achievement in the subject of English at secondary level. The participants of the study were 310 students of grade X. Sample was randomly selected from 12 boys and girls schools. A multiple choice-item test in subject of English was used as research instrument. It was found that in most cases the performance of the English medium school students was better than that of the Urdu medium students. On the whole, in some cases, by keeping in view the difference of means, the study showed almost the similar performance of both groups but apparently, the students of English medium school were better in subject of English.
Semiotics is the study of signs and symbols as elements of communication (Random house of electronic dictionary 1992). Ferdinand de Saussure (1983) defined semiotics as “the science which studies the life of sign within a society”. Here we need to define that what is a sign? Signs are communicated through modes. Modes are the different shapes of the societal and cultural resources for making the meaning.
Semiotic modes are the sources of representing the experience of the world by the humans. Any semiotic mode should have the ability to represent the entity and its relationship to the outside world. There are different choices given by the semiotic modes through which the relationship of the objects with other objects and processes is represented. These modes can be used to describe how people are linked socially, how the world around and inside us is alike and how different entities in a semiotic system are connected (Kress, 2009). Modes offer the resources for the representation, for example in writing grammatical, lexical, typographical which includes size, font etc. Signs exist in the form of words, pictures, art, music, flavors, objects etc. People think only in the form of signs (Peirce, 1931-58). According to Ferdinand De Sassure (1983) a sign is the association of signified and signifier. He says that sign is composed of these two parts in which signifier is the form that a sign takes and signified is the concept that the sign represent.
Sign must always have both of these parts. A sign is a combination of signifier with a particular signified. Both of these signified and signifier is “psychological” (Sassure, 1983, pp. 12, 14-15, 66; Sassure, 1974, pp. 12, 15, 65-66). Semiotics is the study of these 10
Khawja et al (2002) conducted a research in Pakistan; their interest was in different school system. This study was conducted in summer to observe the capacity of students. They saw the performance of students through tests in different schools. They assumed that primary education in English and Urdu medium affects the performance of students. The education system in Pakistan is generally divided into five levels. We exactly behave like western countries, adopting their rituals and traditions. Same is the case with education system of Pakistan. The education in Pakistan is carried in two major languages comprising of English and Urdu. The amendment in the course material each year has devastated the whole educational infrastructure of our country. Even we are not able to determine the language of syllabus whether it should be in Urdu or English.
In spite of Urdu, being the national language of Pakistan, the English syllabus is given preference. The inferiority complex is driving us to ignore our national language. The children, on the other hand, also find it very difficult to focus on studies because of regular alteration in the syllabuses. The new syllabus is more tough and equivalent to that of higher education. Private education exists at all levels of education in Pakistan. The medium of instruction is usually English and the quality of education varies. Some schools provide traditional Islamic education. So due to expensive education many people are kept beyond the reach due to unavailability of resources. Therefore people send their children in low level of school and this division affects the performances of students.
SaraIjaz and Asadnisar (2011) conducted that study, the major purpose of the study was to investigate the difference in performances of students of English and Urdu medium schools at Higher Secondary School Level. All the students enrolled during 2005, 2006 and 2007 in Government Degree Colleges in Rawalpindi constituted the population. Two degree colleges (one male and one female) were randomly selected and 730 students enrolled during 2005, 2006 and 2007 in F.Sc program were included in the sample. Data were collected through personal visits to the sample colleges. Collected data were tabulated and analyzed by using t-test for significance of mean difference. The results of the study revealed that students of English medium schools performed better than the students of Urdu medium schools in the subjects of English and Physics whereas no significant difference was observed in the subject of Urdu. It was recommended that medium of instruction for science subjects at secondary level should be English.
Iram gull gillani, Khalid khurshid, nabibuxjumani,andfazalurrehman conducted this study. A comparison of student’s achievement in the subject of English. (2010).
The present study aims at analyzing the effects of medium of instruction on students’ achievement in the subject of English at secondary level. The participants of the study were 310 students of grade X. Sample was randomly selected from 12 boys and girls schools. A multiple choice-item test in subject of English was used as research instrument.
It was found that in most cases the performance of the English medium school students was better than that of the Urdu medium students. On the whole, in some cases, by keeping in view the difference of means, the study showed almost the similar performance of both groups but apparently, the students of English medium school students were better in the subject of English.
Aziz marjankhattak (2005) comducted this study to check the effect of medium of education during school on performance of students in medical college. This study was conducted to know the role of Urdu and English language as medium of education during Primary and Secondary school in Pakistan and its impact on the results of university MBBS professional examinations. Material and Methods: This study was designed and conducted during September to December 2005 in Gomal Medical College, D.I.Khan. Students of MBBS classes, from sessions 2001 to 2004, were provided with a proforma. The first professional Part-1 examination was expressed as M-1 and other university examinations as M-2, M-3 and M-4 respectively. The medium of learning either Urdu or English during Primary and Secondary level and attempts in their university examinations were asked. The statements regarding medium were randomly checked and the university level performance was verified from results available in the student affairs section of this college. The data was tabulated and statistically analyzed by x2 test. All the 189 students studying in second to fifth (final) year of MBBS classes in Gomal Medical College were included in this study. Among these 47 were in Second year, 57 in Third year, 46 in Fourth year and 42 in Final year. In Second year, students passing in the first attempt in M-1 examination from Urdu medium were 9 out of 19. In Third year, passing M-I and M-2 in first attempt were 26 out of 44 from Urdu medium. In Fourth year, in first attempt of M-1, M-2, M-3 examinations were 39 out of 72 from Urdu medium. In Fifth year, students passed in M-1, M-2, M-3 and M-4 examinations in first attempt coming from Urdu medium schools were 44 out of 56. The rest of the students from each class were from English medium schools. The results of students from Urdu and English medium schools were compared with each other by x2 test for the number of students passing in first and second attempts, the p-value was found to be insignificant (p ; 0.5). There is no effect of language as a medium of education during Primary and Secondary school upon the results of university professional MBBS examinations.
signs.Halliday (1978) defined semiotics as a social semiotics that social semiotics is a theory in which the meanings of the text depend upon the social situation i.e. context. Lemke (1990) defined social semiotics as:
A theory of how people make meaning. It asks how we make sense of and to one another and how we make sense of the world. It concerns itself with everything people do that is socially meaningful in a community: talking, writing, drawing pictures and diagrams, gesturing, dancing, dressing, sculpting, and building – in effect, everything (Lemke, 1990, p. 186).
Semiotic modes are significant in the way that incorporation of words and images is effective meaning making resource in any communication. Different semiotic resources combined to make an effective message because no single mode of language makes the communication effective and clearly meaningful when it is used in isolation. Distinct semiotic modes help avoiding the vagueness of the words. We see semiotic modes as alternatives of each other. Visuals provide the illustration of the verbal and the verbal gives the explanation of the visuals.Hodge and Kress’ (1988) have focused on the interrelation of the semiotic system and society and studied that how semiotic system is functional and social in its use. Such approach makes the multimodal discourse more coherent by creating a relation with the context.
Multimodality refers to the different modes used for the human communication; they include: written, verbal and images. Multimodality focuses on the study of the relationship between different modes of communication, whether they are written, verbal, 11
visual or auditory. Mode is any kind of communication, which a person uses to interact with the other. So, a multimodal discourse is a kind of discourse which includes several different modes.
In the early years of schooling, children produce images and then write about that illustration. In early years of schooling, the text books provide such illustrations, and later it gives way to written and verbal texts. The convergence of verbal and visual aspects of a discourse is called multimodal phenomenon. Multimodal is the combination of different modes of semiotics in a communicative act. A multimodal system involves several different modalities. It is based on social semiotics in the way that meaning is not only created through language but it involves the other modes of communication as well i.e. art, paintings, pictures, music, sculpture, gestures, décor, dresses etc. (Halliday, 1975 & 2003). Meaning making is social in the sense that three social semiotic meta-functions i.e. ideational, interpersonal and textual meta-function (see, section 188.8.131.52) of the language are present in every act of communication.
According to Van Leeuwen (2005) multimodality is the combination of different semiotic modes of communication present in a culture. Multimodality not only refers to the products but also to the processes involved in the production of multimodal discourse.
2.3 Discourse Analysis
Stubbs (1983) defined discourse analysis as the language use beyond the sentence or utterance. He also defined this term as “the interrelatedness between language and society”. Discourse analysis is concerned with the study of language in use beyond the sentence and it focuses on talk and interaction between participants (Hymes, 1972; 12
Coulthard, 1977; Berry, 1981; Tannen, 1984; Sinclair and Coulthard, 1992; Schiffrin, 1994). Halliday and Hassan (1976), Van Dijk, (1977), De Beaugrande and Dressler (1981) has described discourse analysis as “the cohesion in written texts under the banner of text Grammar”.
Brown and Yule (1983) have defined discourse analysis as “the way how human beings use the language to communicate with each other or more particularly, how addressers construct the linguistic message for their addresses and how addresses work on that linguistic message to interpret it”.
According to Stubbs (1983), discourse analysis can be defined as “study of organization of the language above the sentence or clause level”. Discourse analysis can also be defined as “discourse analysis is concerned with the language use in social context or it is concerned with the interaction or dialogue between the speakers. Linguists study discourse analysis to find out how language works to understand different activities of humans.
2.4 Multimodal Discourse Analysis
It was late 1980, when Halliday’s discourse analysis seriously started working on analyzing meaning making process beyond the language.
Multimodal discourse analysis is an emerging epitome of discourse studies which studies the language with combination of other semiotic resources. Discourse analysis is not just concerned with the text but in multimodal discourse analysis it is explored how different modalities i.e. image, text, art, music etc. integrate to create meaning.
Multimodal discourse analysis studies the language when different modes of communication are converged to create meaning. For example verbal, visual, music, 13
gestures, written etc. are some distinct modes of communication in language. These are all semiotic resources. Semiotic resources are the agency of making meaning. When these resources combine together, a semantic relationship exists between them. A multimodal discourse analysis is analyzing the meaning emerged by combining different semiotic resources.Multimodal discourse analysis is the analysis of a discourse which is made up of different semiotic resources which are used for constructing meaning e.g. language, images, music, gestures, art etc.
Halliday (1978, p.123) has defined these semiotic resources as “the system of meaning forming the reality of a culture”. Semiotic resources are the resources of meaning i.e. language, visual images, architecture etc. which integrate with sensory modalities in a multimodal discourse and events and it is called multimodal phenomenon. Lemke (2002, p. 322) has described the interaction between the different semiotic systems as “the image and text re-contextualize each other and they influence our understanding of each in isolation and both together”.
2.5Overview of Multimodal Discourse Studies
Kress and van Leeuwen (1996) have described multimodality in the way to challenge the readers to study different forms of language and how these forms converge to create meaning in a semiotic system. Kress and Van Leeuwen (1996) have studied the visual communication in newspapers, children’s drawings, magazines and textbooks.
According to Kress and Leeuwen (1996), there was a time when all the genres e.g. music, art, dance etc. had been using their own distinct methods, styles, vocabulary and premises. Monomodality was carried on by all these genres at that time. More recently 14
different magazines, universities, organizations etc. have started producing documents incorporating illustrations with script. So theses genres are crossing their own boundaries with the passage of time and getting united with other forms in a semiotic system to create the meaning. So this desire of crossing the boundaries instigated twentieth century semiotics. Kress and Leeuven (1996) were the specialists of images, still standing in the world of monomodal disciplines; they stressed at the same time, that the same meaning can be expressed in the different semiotic modes. Lawrence Sipe (1998, p. 107) expresses that “visual and verbal texts stand equally with each other.”
Yu Liu and Kay L. O’Halloran (2009) have done a research on inter-semiotic logical relations in different semiotic resources to explore the nature of inter-semiotic texture. They have presented how the cohesive devices between verbal and visual aspects of language work to create meaning. Martin (1992) introduced collocation as text forming resource which contributes to the formation of inter-semiotic text in multimodal discourse.
O’Halloran (2005) has worked on mathematical discourse which involves language, mathematical symbols and the visual images and they are organized in a specific way that allows the integration of these three semiotic resources.
Kress and van Leeuwen (1996) have purposed the grammar of visual images in educational context and they utilize systemic functional linguistics formulated by M.A.K. Halliday (1978, 1985, and 1994). Van Leeuwen (1999, pp. 1-4) has worked on sound, speech and music. O’ Tool (1994, 1995) have done a research on visual semiotics in Art 15
forms i.e. sculpture, architecture, paintings etc. He has also adopted the systemic functional modal given by Michael Halliday.
Kress and Leeuwen viewed semiotics as social because according to SFL model the language is influenced and interpreted by the structure of a society. Halliday claimed that there are other sources of meaning making as well other than the language. These different modes may comprise art forms, dance, architecture, sculpture, modes of dress and modes of exchange (Halliday ; Hasan, 1985, p. 4). The semiotician Barthes (1977, pp. 38-41) has examined in his essay “The Rhetoric of Images” that verbal text and images in photographs and advertisements are in either of the two relationships; image-text dependency or image-text co-operation.
Both visual and verbal modes work together to create a cohesive message for the readers or viewers. Cohesion is viewed as “the relations of meaning that exists within the text and defines it as a text” (Halliday ; Hasan, 1976, p. 4). Halliday and Hasan are considering here the different modes of communication in terms of text.
Royce (1998) has applied a framework for analyzing page-based multimodal text which is extracted from the finance department of the magazine naming The Economist Magazine. He has analyzed that how the visual and verbal modes complement each other for making effective meaning. The sample text is titled as Mountains Still to Climb extracted from The Economist Department. 16
2.6 Frameworks for analyzing multi-model cohesion
Different frameworks have been used to analyze the verbal signs, visual images and their inter-semiotic cohesion. For this study, the frameworks used for the analysis of the two semiotic modes (visual and verbal) and their inter-semiotic cohesion, are following:
• Verbal Mode: Halliday’s (1985) Model of SFL
• Visual Mode: Kress and Leeuwen’s(2006) Model for Visual Grammar
• Inter-semiotic Cohesion: Liu Halloran’s (2009) model for inter-semiotic cohesion
Following sections discuss these models briefly.
2.6.1. Verbal Mode: Halliday’s (1985) Model of SFL
Systemic functional linguistics was first purposed by M.A.K. Halliday (1985). It is a functional theory of language. He has described that language always performs a social function because it is a social process. Language is social in the way that whatever we speak or write is always related to society or context. Even if we are writing something for ourselves or about ourselves, we will use the language in the way related to the society.
It is a theory used to analyze the text and context of the language. According to him, language is a social phenomenon and it is related to the study of language in social context.
Halliday (1978) considers “social” as the term suggesting two things; one is that social refers to social system which is synonymous to the culture in the social meaning and the second is that it refers to a relationship between language and society. In a social 17
structure, SFL perspective relates the language to any of a particular aspect of the human experience (Halliday and Hassan, 1985, p. 4). This relationship of language with society viewed the language as a system of paradigmatic choices made against the other available options and other ways of communication which human beings have developed over the time. SFL viewed language not as a structure but as a system because it provides the inter-related set of options for making meaning (Halliday, 1994, p. 15). So this is “systemic” in the sense that it consists of a choice system providing the participants different choices and ways to verbalize their messages and to convey their ideas. Language is “functional” in the sense that it performs functional purposes which are the three meta-functions of language i.e. ideational; interpersonal; textual. These meta-functions are realized in the language used by the speakers when they make the lexico-grammatical choices to represent the on-goings in the world. So SFL is a social and context based theory which views the language from functional and systemic perspective.
Halliday (1978, pp.16, 21, 27-29, 109) has made four claims about the language:
• Language is functional in the sense that what can be done with it.
• Language is semantic because it is used to generate meaning.
• Language is semiotic because it is used to create meaning by selecting the option from the available set of items.
• Social and cultural context actuate the language to be generated and exchanged. 18
184.108.40.206 Context in SFL: Genre (Cultural Context)/ Register (Situational Context)
According to Halliday (1991), language is related to the society, that it is a social semiotics through which people interact with each other and through this people interchange their meaning and language perform some functions in a social context. He has analyzed the context in following three terms; field, tenor and mode.
It is the topic of the language, the subject matter of the language. Field is about what is occurring, to whom, when, where and for whom it is occurring.
It refers to the reader of the text and the social relation that exit between the members in a speech situation. It talks about the people involved in the communication and their relationship with each other. Tenor can be at formal or informal level.
It describes how the language is being used in a particular speech situation. It depicts what is the medium of the language. Language can be written or spoken. It can also be used in an emphasizing way i.e. informative, persuasive etc.
Halliday (1991) has given the concepts of cultural context and situational context. Whatever the text will be written or spoken, it is product of a particular culture and situation. Both of these contexts lie outside of the language. Language can only be understood through its relationship with the environment in which it is used. Halliday (1987) has described the cultural and situational context in his SFL. According to him, environment of the language seen as text, the text and its component parts are viewed as 19
the context of situation and on the other hand, context of the culture is described as, environment of the language seen as a system, its grammatical categories and its lexical items.
Context of Situation is the concept of explaining how text is related to the society and social processes in which it locates and consists of the three components mentioned above; Field, Tenor and Mode (Halliday, 1978, p. 10). Context of situation is the systemic relationship of the social environment and the organization of the language. Each text requires a different use of the language in each different situation which is called register.Register refers to the varieties of language applied according to the different social use e.g. scientific, journalist, religious etc.So register refers to these verities of language in different situations (Todd ; Hancock, 1986). Field, Tenor and Mode are the variables of register. Context of Culture (Genre) lies beyond the context of situation (Register). It gives the linguistic resources that can be used in any text produced in a particular situation.
Register realize the context of situation and genre corresponds the context of culture. According to Martin (2001, p. 155), a Genre is a goal oriented activity in which the speakers are engaged as the members of culture. Genre is involved in every kind of the things we do. Genre has the effect of cultural context on the language.
Genre is a broader level in which there comes register and language. 20
Language, register and genre (Martin, 2001, pg. 156)
Following picture is showing the relationship of the three meta-functions (ideational, interpersonal and textual) of language with register and genre
According to Systemic Functional Linguistics (SFL), language is interpreted as a social semiotic system. Halliday focused on the function of the language.
220.127.116.11 Meta-functions 21
Halliday (1985) has identified three functions of language to interpret the semantic relation between visual and verbal aspects of the discourse; these are the meta-functions of language. These are the major functions of language as described by M.A.K Halliday. Semiotic resources provide tools for creation of these three meta-functions. Through these meta-functions, people utter their intended message that is related with the outside world. A huge network of meaning exists in this world have a reflection of these three meta-functions in its existence. These meta-functions are ideational, interpersonal and textual.
18.104.22.168.1 Ideational Meta-function
This is the function of the language to represent “what is going on in the world”, what our experience of the world is and what lies within us or inside us or the main idea of the text.
This meta-function is related to the language in the way that language is used to express the physical, psychological and the social experience of the world. It represents outer and inner experience at clause rank to construe the meaning. This is the content of the language and it provides the domain. According to Halliday (1985) this function is used for the expression of content and to convey the information. The content of the text is the focus of this function and the information is conveyed effectively making it quite comprehensible. Ideational meta-function connects the cognitive realities of the world with language and gives it a meaning. It is this function of the language that the writer or the speaker embodies their experience of the real world in the language (Halliday, 1973, p. 106). These meanings are accomplished through the “transitivity system” in a process which is realized athrough a verbal group, participants are realized through a nominal group and the circumstances through an adverbial group. Ideational meta-function 22
consists of further two meta-functions: experiential meta-function and logical relations. Experiential meta-function is for the representation of the events, happenings, and the entire on-goings in the world and on the other hand, logical meaning is the study of the ways through which a clause is connected to the sentences and paragraphs (Edward Haig, n.d.). Both of these meta-functions combine to construe the ideational meta-function of the language. Concisely, ideational meta-function is the sum of all the real, fictitious or cognitive realities of the world expressed through language. The ideational meta-function can be realized through the representation by clause in the language. Clause is used to represent the experience i.e. to describe or state the events in the real/unreal world.
Clause analysis is the focus of the transitivity process. “Transitivity is the grammar of experience (Halliday, 1985).” Halliday (1981) also defined transitivity as “grammar of clause”, through which a particular range of ideational meaning can be expressed i.e. experiential meaning. Transitivity makes the clause more comprehensible as the reader knows what processes comprise the clause. This analysis of the language makes the reader understand the structure of the language and the deeper meaning intended by the author. According to Halliday, this system deals with the concept of process, types of participants and circumstances which are semantic categories to represent the phenomenon of the real world represented through the language structures and images. Each different process requires different participants. Transitivity system can examine the clauses efficaciously. It is a complex phenomenon than the traditional concept of transitivity in grammar, which is concerned with the differentiation of transitive with intransitive verb (Edward HAIG, n.d.). Transitivity In the traditional grammar is 23
described as a system of transitive and intransitive verb i.e. presence or absence of an object (Halliday, 1976, p.159). However, according to SFL, transitivity is the system of transmittal of message through process, events, circumstances, consciousness or relations (Halliday, 1985, p. 53).
According to Halliday, process is the event or the action that is being depicted; participants are the entities involved in the action and the circumstances are where, why and how of the event.
Different kinds of processes are following:
Kinds of process
We will be dealing with following processes of transitivity:
I. Material process (this is the process of doing which are the physical actions (O’Donnell, 2011) and it involves two participants; actor and goal. It consists of an action verb representing an action or happening. The one who is doing the action or the doer is labeled as actor, process affects the Goal and the circumstances give the detail about the verb. In this process, one entity may be affecting the other. Actor is usually realized by a nominal phrase.)
He Is playing Football In the lawn
Actor Material process Goal Circumstance ”