Research & Evaluation in PR: Individual Research Portfolio Introduction In their guide to evaluating public relations, Watson and Noble (2007, p. 57) argue that “[t]he emphasis on improving programme effectiveness strongly indicates that the information collected on previous activity is used as feedback to adapt the nature of future activities, and therefore argues for a formative...perspective on public relations evaluation.” This insight into the extent to which research and evaluation can be useful in forming the basis of PR analysis stresses the importance of adopting specific processes within the existing theoretical framework and critiquing the results in line with pre-determined objectives. The research processes and methodologies that are used within the arena of public relations are extensive and those employed largely depend on the results required but this course has enabled the personal development of several different types so this critical reflection will provide an assessment and evaluation of those to determined the extent to which they have been successful. As such, this research portfolio will examine six entries – the literature search, document analysis, media content analysis, questionnaire, focus group and expert interview – in order to assess the research objectives established prior to each activity. It will utilise theory to examine why each one has been used and assess the extent to which it facilitated the ability to meet those objectives. This will be done with a view to concluding that the research process in relation to both the dissertation and the Southampton Solent University (SSU) International Students project have enhanced my knowledge of the theoretical framework relating to PR and facilitate the development of the relevant research portfolio. Literature Search The literature search was a vital element of the dissertation research process because it was specifically undertaken to locate materials that were relevant to the topic of traditional versus 1
digital PR, which was presented in the question “To what extent are digital public relations activities beginning to surpass traditional public relations and what factors have contributed to this in the United Kingdom?” It was decided that the literature search process would follow that advocated by Ridley (2012) in adopting a dual approach. The first part of this approach was locating the relevant sources for review with the second part drawing on the information discovered to compose a thorough review of the literature so as to build upon that solid academic foundation in the dissertation at a later date. The research objective was to create a comprehensive literature review that provided an insight into the prevalence of digital PR, the factors that have contributed to it and the extent to which further growth is expected. The search element was comprised of three processes – a library search, an article database search and a conventional Internet search. The former involved a search of the library database to determine texts that could be pertinent to answering the question whereas the article database (Jstor) search required a keyword search to find articles that related to both traditional and digital PR. After cursory searched were complete, in depth assessment of the suitability and authority of the material was performed in order to identify texts that were relevant and valuable. However, the Internet search was performed in attempt to access reports that have been published online and grey literature, or “[d]ocuments that are outside of traditional bibliographical controls and are not readily available via conventional channels and which are therefore difficult to identify and obtain” (Hart, 2001, p. 30). Very little material was found in this particular niche despite considerable time being spent on it. The criteria for evaluation in this particular entry related to the level of material found and its authority and credibility. Although the level of grey material available was limited by the nature of the topic, the fact that data in white papers, news reports and business reports yielded extensive information about the development and progress of digital PR aided this 2
particular process and enhanced the final detailed literature review that was completed. As such, the objectives established for the literature search were achieved and enabled it to facilitate further document and media content analysis. Document Analysis Document analysis is a key element of research in the field of public relations for a variety of reasons, not least because it draws upon the motivation and intent of documents that are pertinent to research in addition to summarising their content (Bowen, 2009). According to Daymon and Holloway (2005), qualitative document analysis is a useful source of narrative data that can verify relationships that form the basis of hypotheses and discover emergent patterns that can be utilised for critiquing existing thought in addition to formulating innovative conclusions. This particular perspective of document analysis was adopted in this particular project and was actively used to inform the process, particularly regarding what I was looking for within the primary documents that were discovered. In effect, the use of document analysis was geared towards discovering patterns that have been identified between traditional and digital forms of PR, particularly with reference to companies that have selected the latter for their campaigns. As such, the document analysis that was performed during the initial literature review was able to identify specific case studies that could be employed in a comparative capacity, with technology oriented companies like IBM, Facebook, Twitter and Microsoft offering significant contributions to the overall review. The main objective of identifying such patterns was necessary to provide the dissertation with an innovative yet valuable insight into the way in which public relations is evolving. The criteria of evaluation as to whether this particular result was a success was based upon the theoretical framework that governed the approach. For example, Potter (1996) advocates that the information derived from document analysis can heighten the credibility of the 3
results of further investigation (in this case the literature review as a whole) and Altheide (1987) noted that the method made provision for the emergence of concepts. As the patterns discovered via document analysis underpinned case studies that supplemented the findings of the literature review and identified several concepts that are associated with digital PR, it is logical to conclude that it was used successfully. However, again, it was limited by the lack of grey literature found and therefore the extent to which document analysis could be used was relatively disappointing. It is anticipated that, if more grey literature could be located, then the success in achieving the established objectives could have been enhanced somewhat. Media Content Analysis A further qualitative research method, media content analysis has some similar strengths to document analysis in that it can “triangulate and point out discrepancies in data being collected in alternate ways, suggest questions for participants and identify analytical categories� (Love, 2006, p. 84). However, it is entirely different and nature and in terms of what it is trying to discover. Instead of analysing the purpose of documents, it seeks to examine how certain issues and meanings within the media via a five step process – establishing objectives, selecting samples of texts, determining units of analysis, identifying categories of analysis, and coding content (Theaker, 2013). This framework was used for the purposes of the dissertation and actively provided a structure for the analysis performed. As such, the success of this element of research was determined in terms of whether each step was fulfilled. It adopted a statistical approach that could offer an insight into the tone and content of various media reports that were perpetuated by both traditional and digital PR methods. Although conventional media content analysis looks at a specific story or element of coverage and assesses it in terms of the tone of coverage, specifically whether it is negative, 4
positive or neutral (Watson & Noble, 2007), this purpose was altered to a degree in order to provide useful content analysis to contribute to the determination of the extent to which digital PR is taking over from traditional models. As such, the method was utilised to determine the extent to which traditional PR methods and digital PR methods alike were able to yield a positive, negative and neutral impact in various circumstances, thus differentiating between the two in terms of their overall impact. As a research method, it was highly successful in achieving this objective as a result of the generation of statistics that actively supported the literature review. There were limitations associated with media content analysis though, which largely stemmed from the nature of the dissertation question. It was necessary to establish case studies in order to implement it effectively and data was processed manually, thus significantly enhancing the degree to which inaccuracies were likely to occur. If performed again, media content analysis would utilise technology to produce a greater range of accurate statistics (Fekl, 2010), but that should not diminish its achievements in this context. Questionnaire/Survey Morris and Goldsworthy (2011, p. 205) state that it is vital for all public relations campaigns to identify unique and innovative areas of business and services that appeal to potential consumers in order to avoid getting lost in the “barrage� of information that is presented on a daily basis. They also advocate the use on research to identify new areas of the market and its consumers (Morris & Goldsworthy, 2011). One means of doing so lies in the administration of a questionnaire or survey to a campaign’s target market and this is exactly what was conducted here. Presenting a questionnaire directly to Southampton Solent University international students was deemed desirable in gauging their reactions to the program from within in addition to assessing their personal needs and perspectives. The objective of this
5
exercise was to identify selling points of the program in addition to formulating a greater understanding of exactly what it is that international students are looking for. The questionnaire was composed based upon the conclusions drawn by Matt Haig (2000) in his assessment of PR online. As such, it had short and easy to understand questions, which was an important consideration based on the fact that international students were the chosen respondents, with a multiple choice format in order to obtain pertinent data. However, there were also open questions incorporated in order to obtain quotes, opinions and “...the full richness and complexity of the view held by the respondent� (Denscombe, 2010, p. 165). It was determined that the combination of the two was vital in producing the information necessary to conduct an effective PR campaign. Indeed, the evaluation of the level of success that this technique had was performed on the basis of the fact that it revealed a wealth of information that would be useful to a multifaceted campaign. The insights produced were informative and effective in revealing what a potential SSU student is looking for. Focus Group, or Experiment, or Observation The focus group was perceived as one of the most integral elements of the SSU international students campaign as a result of its potential ability to yield copious amounts of useful information that could be utilised in line with Morgan’s (2013 n.p.) ideas regarding the value of content and repurposing material to enhance digital PR campaigns. For example, she noted that: ...one way to incorporate content marketing into your mix is to repurpose content you already have (press releases & bylines) into fresh content. A single press release can be re-purposed into a Slideshare presentation, a blog post, a guest blog post, a Pinterest infographic, a LinkedIn news
6
update, a post on a relevant trade publications Facebook page, an editorial opinion article for the local newspaper’s website and more. The focus group was conducted with these principles in mind and the level of success achieved is assessed on the basis of it achieving the objective that it is possible to formulate a comprehensive campaign by the information gathered. The focus group was structured so as to create an informal environment in which ideas and opinions can flow freely. This helped to negate the difficulty of the directive influence of the interviewer, which can be problematic in terms of imposing limitations on the information produced (Kamberelis & Dimitriadis, 2013). Although a loose structure was defined in order to ensure that all issues identified in advance were covered, the flow of the focus group was not set in stone. The students involved were encouraged to discuss any issues and ideas they felt were relevant. This extended the ensuing process of analysis but effectively ensured that the discussion was comprehensive and captured the students’ wants and needs. In effect, using the established knowledge on how to run focus groups really helped to devise the most appropriate research space possible. In line with this and the objectives established previously, it was entirely possible to deem this particular research method a success. The information discovered most definitely facilitated the composition of a multifaceted and highly targeting PR campaign. Expert Interview The undertaking of interviews with experts on the topic of the SSU international student program was the most complex facet of this particular project because it demanded a degree of professional conduct that was not required nor desired in the focus group. Obtaining information from experts was vital in lending credibility to the outcomes and products of the campaign and therefore was a research method that must achieve its objective of providing 7
information that was conducive to formulating a base upon which it would be possible to launch a professional yet appealing campaign. Indeed, experts are able to offer an insight into the world of attracting international students that the students themselves would not because they could provide an insight into the wider topic, rather than one specific to SSU, from a detached perspective. Aronson et al (2010) advocated a preparedness for interviews comprising predetermined questions that would facilitate pertinent quotes and facts in addition to the discovery of background information that could be used for any public relations campaign that was undertaken. In this case, that campaign would relate to SSU international students and therefore input from experts in the area was required to ensure that the campaign would appeal to its target audience. With this in mind, the following questions were formulated: •
How does SSU currently market its degrees to international students?
•
What groups or segments of international students exist?
•
Why are international students important to the future of the university?
•
What web sources exist about this?
•
How does SSU’s international student body compare with other universities’ international student body?
•
How do other universities market themselves to international students?
•
Why international students consider the UK to be a good place to get a degree?
•
What are international students looking for in a UK university?
Upon reflection, all questions were fully answered and therefore the process of interviews achieved the objective in that they provided useful data that enhanced knowledge of the topic 8
in general in addition to giving me an insight into what had previously worked in this PR niche. The theory was critical here though in that it dictated exactly how the interviews would be conducted and so facilitated the gathering of information that may have otherwise been missed. Conclusion In conclusion, this critical reflection has effectively been able to identify the research methods used during the course of the dissertation and SSU international students projects, critique the successes and failures formed on the basis of those methods and evaluate the processes associated with them so as to learn those techniques and enhance methods for future public relations projects. In each case, it is clear that forging a reciprocal relationship between the method and the objective, specifically ensuring that they were equally suitable in pursuing the required information, has ensured that all aims and goals have been met throughout. The literature search, for example, was particularly successful in providing a foundation on which independent analysis could be constructed and legitimate and credible conclusions drawn. In addition, the focus group achieved its objective of producing quotes, information, ideas and concepts that could effectively be drawn upon within a multifaceted public relations campaign in line with Morgan’s (2013) advocating of repurposing information were possible. It is possible to identify strengths and weaknesses within each research method and the critical analysis has certainly highlighted areas for improvement but it must be stressed that the portfolio itself is just as invaluable for its learning outcomes as the two projects have been. Whether producing theoretical insights into the world of digital PR or creating an effective campaign to promote Southampton Solent University’s international program, the use of all six entries in this portfolio will enhance my ability to work within the field of public relations and is therefore important to my career opportunities. As such, it is possible to conclude that the work here has enhanced my knowledge of the theoretical 9
framework relating to PR and enhanced my personal and professional development as a consequence. Bibliography Altheide, D., (1987). Ethnographic Content Analysis. Qualitative Sociology, 10, pp. 65-77. Aronson, M., Spetner, D. & Ames, C., (2010). The Public Relations Writer's Handbook: The Digital Age. 2nd Edition. Hoboken: John Wiley & Sons. Bowen, G., (2009). Document Analysis as a Qualitative Research Method. Qualitative Research Journal, 9:2, pp. 27-40. Daymon, C. & Holloway, I., (2005). Qualitative Research Methods in Public Relations and Marketing Communications. Abingdon: Routledge. Denscombe, M., (2010). The Good Research Guide For Small-Scale Social Research Projects. New York: McGraw-Hill International. Fekl, R., (2010). Review. 2nd Annual Summit on Measurement, Barcelona. [Online] Available at: http://www.communicationcontrolling.de/aktuelles/meldungen/measurement-summit2010.html [Accessed 25 March 2014]. Haig, M., (2000). E-PR: The Essential Guide to Public Relations on the Internet. New York: Kogan Page Publishers. Hart, C., (2001). Doing a Literature Search: A Comprehensive Guide. London: SAGE. Love, P., (2006). Document Analysis. In F. Stage & K. Manning. Research in the College Context: Approaches and Methods. New York: Routledge, pp. 83-96.
10
Morgan, C., (2013). What is Digital PR? Social Media Today. [Online]. Available at: http://socialmediatoday.com/carriemorgan/1399891/what-digital-pr [Accessed 25 March 2014]. Morris, T. & Goldsworthy, S., (2011). PR Today: The Authoritative Guide to Public Relations. Basingstoke: Palgrave Macmillan. Potter, W., (1996). An Analysis of Thinking and Research About Qualitative Methods. Mahwah: Erlbaum. Ridley, D., (2012). The Literature Review. London: SAGE. Theaker, A., (2013). The Public Relations Handbooks. Abingdon: Routledge. Watson, T. & Noble, P., (2007). Evaluating Public Relations. New York: Kogan Page Publishers.
11