Day 1: Is your thesis good enough? Examination and the PhD

Dr Cally Guerin

Welcome to the first post for our new mini-course, ‘Is your thesis good enough?’. This Pomodoro Break is designed to give you short, targeted information while you are having a coffee break. Each day this week we will explore topics related to the final document you submit for examination, your thesis.

If you didnā€™t get a chance to join our coffee meeting on Monday morning, please introduce yourselves to the group via the Comments function at the bottom of the post. Where are you up to in your studies – do you have a definite submission date yet? How do you imagine that day of submission?

One of the big challenges of the PhD is that, ultimately, it must be examined, invoking many of the fears we have regarding examinations from past educational experiences. This is quite different from submitting a research article to a journal, where we might expect some formative feedback on how to revise and improve the paper before a final decision is made to accept it for publication.

This high-stakes examination is as much about your identity as a scholar being allowed into the academy as it is about the quality of the research itself. For some, holding onto the thesis and NOT submitting is a means of holding onto the identity of PhD candidate ā€“ after all this time, if Iā€™m no longer a PhD candidate, then who WILL I be? Maybe the job I hoped for is not out there waiting for me; maybe the brilliant career everyone expects me to live out may not be available; maybe my days will become empty without this big project to keep me connected into the University environment. (More on researcher identities in a future Pomodoro Break course.)

What is ā€˜doctoratenessā€™?

One of the mysteries surrounding PhD examination relates to the expected standard. What counts as ā€˜doctoral levelā€™? The Australian Qualifications Framework (AQF) identifies this for us at Level 10 (scroll down to p63).

This can sound very general, though, and relating these broad skills, knowledge and application to the thesis itself can help conceptualise what you are aiming for. But having a sense of what you are aiming for is useful: this blog by Claire Aitchison is useful for moving into an evaluative mindset regarding your own work.

ACTIVITY: Components of doctorateness: The sum is greater than the parts.

Trafford & Leshem (2009) explain that reaching ā€˜doctoratenessā€™ requires a synergy of all 12 of the following elements, but if you had to choose your top 3, which would they be? Use the comments function below to explain why you think these are the most important.

1.Ā Ā Ā Ā Ā  Contribution to knowledge
2.Ā Ā Ā Ā Ā  Stated gap in knowledge
3.Ā Ā Ā Ā Ā  Explicit research questions
4.Ā Ā Ā Ā Ā  Conceptual framework
5.Ā Ā Ā Ā Ā  Explicit research design
6.Ā Ā Ā Ā Ā  Appropriate methodology
7.Ā Ā Ā Ā Ā  ā€˜Correctā€™ data collection
8.Ā Ā Ā Ā Ā  Clear/precise presentation
9.Ā Ā Ā Ā Ā  Full engagement with theory
10.Ā  Cogent argument throughout
11.Ā  Research questions answered
12.Ā  Conceptual conclusions

What is really being examined?

Is it the person or the thesis that is being examined? When you read the ANU criteria provided to examiners, youā€™ll see reference to ā€˜The candidate is expected to haveā€¦ā€™ and ā€˜The thesis must beā€¦.ā€™ Youā€™ll also notice that these criteria are rather vague and general, even though the requirements are pretty much the same at every university in Australia.

ACTIVITY: If you were examining a PhD thesis, what would you be looking for? What makes a research project worth a PhD? What do you regard as the most important elements? Imagine you had to assign points for the main criteria listed by ANU ā€“ what weighting would you give to each of these 4 criteria? Are there other criteria you would like to add to this list? Use the comments function to tell us what you think.

Tomorrow weā€™ll look at what makes a thesis big enough ā€“ what is a ā€˜substantial contributionā€™?

References

Vernon Trafford & Shosh LeshemĀ (2009)Ā Doctorateness as a threshold concept,Ā Innovations in Education and Teaching International, 46:3,Ā 305-316,Ā DOI:Ā 10.1080/14703290903069027.

23 Comments

  1. Hi all,

    If I had to choose the top 3 of all 12 of doctorates synergy elements, then I would choose:
    1. Contribution to knowledge
    2. Conceptual framework
    3. Appropriate methodology

    I think those three elements that characterise and differentiate a scientific PhD work from other typical investigation work.

    Cheers,
    Depi

    1. Thank you for being the first person to add a comment, Depi! There’s a nice logic working from the conceptual framework through the methodology to reach the contribution – and contribution is where we get to see the ‘originality’ aspect we’ll explore on Thursday.

      1. Activity 2 – Assessment Criteria

        I agree with Rosie – all four criteria are important in assessing a PhD thesis. I would give them evenly at 25% each. However, I am curious how the examiner would examine the criteria of independent research if the research is part of a big project. For example, a PhD student is part of the research team working on a big topic in the same area?

        1. Good point, Depi. I think it’s really important to make clear statements in the thesis about what you have done, including being explicit about what you’ve done in co-authored papers. ‘Independent research’ doesn’t have to mean you made it all up from the beginning, but your literature review, discussion & conclusion sections are where you can demonstrate taking ownership of the research, where you have opinions about what matters in the field, why the work in your own part is valuable and offer ideas about future directions for research. Not sure if that helps, but we can talk more about this.

  2. Hi All!

    Activity 1- Selecting Top 3 Criteria

    2.Ā Ā Ā Ā Ā  Stated gap in knowledge
    3.Ā Ā Ā Ā Ā  Explicit research questions
    4.Ā Ā Ā Ā Ā  Conceptual framework

    I think if I have these three sorted, the other points will flow somewhat naturally: itā€™ll be apparent how my knowledge contribution is unique if I state a gap in the knowledge, clear research questions will (hopefully) lead to a logical process and answered questions, and a good conceptual framework must have a strong and clear engagement with theory as well as a sound methodology, through which a thesis structured around a cogent argument can be grafted.

    Activity 2 – Interpreting the Assessment Criteria
    If I were marking a PhD (in Humanities!) I’d probably look for any signs that the Scholar is interested in the work they are doing, and I’d seek out signs that the Scholar is able to orient themselves within their field in a conversational way, that invites the reader to participate in their research journey, and enjoy the conversation that the thesis has presented. I think I’d also seek out a clarity of thought: the clearer the thesis is to follow, the more able I think I’d be to make an assessment about the analysis conducted, as well as get ease and pleasure from marking it!

    In terms of the assessment criteria provided, I think the personal criteria is about my ability to show that I am able to learn, and that I have developed as a researcher through the PhD process. Iā€™m also asked to locate this learning through my standpoint within a particular academic field and context, from which I might be able to step into the work of academia as a professional.

    Comparatively, the marks for work itself seem to be moreso about conduct, and the structure and communication of information. They seem to ask the writer to consider and present research practices as cohesive professional tools the academic uses. Iā€™d probably divide them evenly at 25% each! The only thing I pause at is 1. stating that the candidate is expected to have made a substantial contribution to learning – Iā€™d like to see the criteria be more specific and state whose learning it is.

    1. Hi Rosie – thanks for being an early starter šŸ™‚
      Nice to see how you are lining up the parts of the thesis to make sure it all comes together in the final document.
      I’m also interested to see your focus is a little different from Depi’s response, which is a great reminder that there are lots of differences between disciplines as well as between projects.

      Assessment Criteria: I like your idea of conversation in the document – to me, this ability to see yourself as part of the ongoing discussions and debates in your field is a key part of taking up the identity of ‘scholar’. Your query about a ‘substantial contribution’ is useful, and we’ll look at this a bit more tomorrow. However, the rather vague criteria are part of why it can feel rather unsettling at times. But this is all the examiners get (apart from also requiring a statement of the HDR’s contribution if the thesis includes jointly authored papers).

    2. Hi Rosie

      Agree completely about the need to engage the examiner — it would be a long slog for someone to read a long, dry, uninteresting thesis. However, I’m reflecting on your point about showing that the candidate has grown as a researcher. I’m not sure how that would work in some fields — perhaps, its more suited to qualitative methods. Interested in your thoughts.

  3. Activity 1 Components
    Yes all the components seem important however these three stand out for me.
    1. Contribution to knowledge – to me this is the essence of the PhD and to do this successfully you will cover off many of the other points including these two below.
    11. Research questions answered – we have to have focused research questions and answer them.
    6. Appropriate methodology – however to make the contribution and answer the questions we need to investigate the issue in a way that is suitable for the topic and context.

    Activity 2 Assessment Criteria
    All these aspects seem important to me but if I have to weight them I would guess the following with number 1 having highest because as commented above to me this is key to the PhD. 1. 40% 2. 20% 3. 20% 4. 20%

    1. Constructive alignment (Biggs was right!) – matching up the criteria with the assessment process DOES matter šŸ™‚
      But it is also interesting to me that examiners are assessing the candidate, but at present doing so only through the written document.

  4. I really resonate with Rosieā€™s comments, above. The gap (2), research questions (3) and framework (4) would probably be my top contenders, too, with the rest falling into line behind those. ā€œResearch questions answeredā€ would be the last one on my list, because to me the project is about the broader context and research(er) development, rather than an explicit ā€œanswerā€.

    To that note, I think thereā€™s a strong (disappointing?) emphasis on clarity of the final output as quite a linear recount, rather than any explicit space to acknowledge the roundabout journey we might take in our projects.

    I totally agree that itā€™s important to be able to communicate understanding, contribution, methods, and results etcā€¦. But I feel like the bulk of my own personal development as a researcher has been when the whole project has fallen over and Iā€™ve had to shift my thinking or methods to accommodate something unexpected. These dead ends or change in direction donā€™t often make sense to include in the final thesis, despite being where the bulk of my own ā€˜learningā€™ has happened.

    An incorporation of growth mindset, flexibility, lateral thinking, and/or determination – some of those more ā€˜humanā€™ elements of research – would hence be interesting to see in the assessment criteria.

    Within the four criteria given, I think this comes most under ā€œan account of the research doneā€, so #3 gets my highest allocations of 30% each. 25% each goes to #1 and #4, and with an increasing push for inter- and cross-disciplinarily in research, broader framework of discipline(s) gets the lowest remaining 20%.

    1. Hi Hannah – it’s an interesting question about how much of the final output reveals all the twists and turns along the way (especially if you consider that the assessment criteria talks about where the candidate has got to on this journey). There is discussion about how much of that ought to appear in the thesis itself – overcoming problems in the research can be an important aspect of the contribution to knowledge. But it is also useful to think about the idea of ‘thesis’ (and perhaps hypothesis) as the statement/argument/theory that is being put forward in the document. The conventions of the thesis genre vary between disciplines, and examiners are likely to expectations of what this looks like at doctoral level. Very useful questions for us to ponder…

      1. A thoughtful reply, thanks Cally. I’m wondering if some of these factors are better explored in the oral examination, rather than just submission of a document. Is that true?
        It’s an interesting line to consider between the problems that ‘contribute to knowledge’ vs ‘contribute to development of the final thesis’ in whatever form it takes.
        PS oops, just noticed a typo in my OP – “#3 gets my highest allocation of 30%.”**

        1. Hmmm, I’m not really sure whether the oral exam discussion allows for more of this. I guess so, but I’m also thinking about how the academy is usually focused on ‘success’ and the idea of examination being a gate-keeping function that is deciding who gets credentials and access to the inner circle.
          Important considerations with implications for how we choose to ‘play the game’.
          Do others have opinions about how they see themselves negotiating this part of the process? I have a strong feeling that the underpinning philosophical approaches of disciplines and their methodologies/theoretical frameworks are likely to determine the boundaries here.

  5. Activity: Componenets of doctorateness
    Iā€™d choose (1) contribution to knowledge, (2) cogent argument throughout, and (11) research questions answered. Contribution to knowledge has always been a core feature of what a PhD means to me. Doing research that contributes seems a fundamental part of the PhD. Cogent argument throughout shows thought and understanding of the material, and an ability to communicate this effectively. Answering the research questions also seems a fundamental part of the PhD, as stating what was learned shows that the research was thought through, can be communicated clearly to others, and can be used in the field .

    Activity: % weighting
    1. 28
    2. 24
    3. 24
    4. 24

    1. Hi Sally – I like your % weightings with just a fraction more for 1. Do you want to tell us more about this feeling that carrying out independent research and making a substantial contribution to knowledge is slightly more important than the other criteria?

  6. Hi everyone

    I am 4th year of a part time PhD in psychology. Like many others, COVID has derailed my plans for this year (personally and professionally). I’ve collected most of my data but can’t complete the rest at the moment, so am concentrating on writing for the rest of the year.

    To me, at the moment, the key requirements for a thesis are: a cogent argument; originality; and a contribution to the field.

    But if I was examining one, I would be looking for a thesis that is easy to read, that takes the reader systematically through the research and arguments. That is, it should be easy to mark. Examiners are busy people, theses are long, and the easier it is to read, the easier it is to mark.

    1. Hi Toby – I’m so pleased to see someone put this focus on an easy to read thesis! Absolutely agree with you – give examiners what they need to be able to say ‘Yes, give that person a PhD!’. šŸ™‚

  7. Activity 1: I think mine would be original contribution, cogent argument and conceptual conclusions… and if I could choose a fourth probably conceptual framework (and I imagine methodology come within that conceptual framework). But oh so hard to choose! All are so important. And I love Toby’s comment about being easy to read … so important but so difficult!
    Activity 2: At first I thought that the top two should carry more weight than the bottom two, so I would allocate at least 70% to the top two (perhaps 40% substantial contribution, and 30% broader framework) and 30% to the bottom, but then I realised that the account of the research is the bulk of the thesis so am I shooting myself in the foot????

    1. Your comments show us how these decisions are not as straight forward as they appear at first. My main reason for presenting these ideas is so that they are sitting alongside you as you work towards preparing your final document for submission.

  8. In examining humanities or social science thesis I do not look for perfection in spelling and grammar, though many do.

    I look for coherent arguments which means sound reasoning. No romance and poetics, they are a waste of time when universities are meant to serve the community.

    Strong methodology.

    Relevance. This is the most important part of the originality criteria.

    Broad awareness of the cultural context of the topic. This is one area in which many researchers fail, even the best.

    I had a twitter discussion with a man who supported an article out of the Fenner School which argued that growing rice in the MRDB was the best crop when water is available. As we know, the growing of rice is one of the major causes of water shortages in the lower parts of the MRDB.

    In other words, I look for common-sense approaches that do not isolate a research problem.

    1. Good points, Aileen (and common sense always seems useful!). I think of the thesis as demonstrating a kind of T-shaped knowledge – a broad understanding of the field itself and what is relevant to the project, with a deep dive into a very specific topic in that field.

      There is some research into examiners’ responses that indicates lots of them do tend to care about proofreading, so it’s worth trying to get that as good as possible. My take away from this is that a really neat, tidy document makes examiners feel confident that the work has been careful, thorough and scholarly in other ways too.

      1. Thanks Cally, T shaped knowledge is a useful image to keep in mind. As is the point on proof reading. Poor presentation does annoy me, believe me. But I try to not let it affect my perspective of what I consider more important elements of a thesis.

Leave a Reply

Your email address will not be published. Required fields are marked *

*