A day late and a dollar short: EQE appeal outcome favours the brave

A respectable cohort of this blog's readership is made up of those who are training to qualify as European Patent Attorneys, while many more readers are qualified attorneys with a deep interest in the examination which governs this qualification, perhaps due to their activities in training or supervising trainees who will be sitting the European Qualifying Examination (EQE), or perhaps simply because they want the qualification to be a fair test of fitness to practice.
The strapline reads:
Because sometimes ... there
aren't second chances

Update: According to at least one comment below the Exam Board has changed the result of every candidate affected by having given the "wrong" answer to questions 15.2 and 17.3, not just those who appealed, for which the Board should be commended. 

The post below is mostly unchanged as the IPKat believes the points made about the unsuitability of the exam format for the subject-matter being examined, and the problems arising from undue haste in issuing and then reissuing results, are still relevant.

First of all, a health warning: the IPKat's knowledge of this story is based on a couple of accounts which may not represent the complete story, or the situation as it applies to all affected candidates. If any readers can add to this story, which is still at the breaking news stage, by clarifying, correcting or supplementing the understanding set out below, this blog post will be corrected accordingly. 

An anonymous correspondent alerted the IPKat yesterday to a development which appears to have favoured candidates who appealed a fail result. The timing of this development is unfortunate to say the least, leaving non-appealing candidates at a disadvantage and one day too late to file an appeal. Once alerted to the story, the IPKat found corroboration in a blog post by Laurence Lai on the IPCopy blog.

The issue is about the results for the pre-examination paper, which were sent out at the end of March. There was surprise and controversy over a few of the answers which the examining committee deemed to be correct in the official marking scheme. Indeed the official marking scheme was published, then changed and republished leaving candidates in confusion over whether they had passed or failed (see the comments on this blog post over at Delta Patents to watch as it unfolded in real time). 

The expert tutors in Delta Patents (whose training materials are widely used by candidates and whose exam preparation tutorials are very well-regarded) were similarly confused by the official answers. Shortly after the exam was held, they had published their own suggested solutions, but in several instances the answers they provided were deemed incorrect by the examination committee's marking schedule.

The deadline for appeals expired this week on either the 6th or 7th of May for most (but some candidates who received their letters more than ten days late could perhaps still appeal today or on Monday). Just yesterday (the 7th) the Examination Board apparently wrote to candidates who did appeal based in the answers to questions 15 and 17 notifying them that their result had been changed to a "pass", but without elucidating the reason for the change.

But what of those candidates who did not appeal? Well, unless the Examination Board has written to them also with happy news (and the IPKat has not heard of any such felicitous cases, but hopes they exist), they would now seem to be stuck with their fail grades. It appears that they too have had their results changed (updated following comments below).

"Fine," one might say in a logical but cold-hearted manner, "if someone is not unhappy enough to appeal, then they implicitly accept the decision. One can't complain about the outcome for others who pursued a remedy that one chose not to pursue oneself." That might be true if dissatisfaction was the only factor in deciding to appeal, but it ignores the financial reality that an appeal costs €1,200 and many of the candidates are not earning salaries providing that sort of disposable income, especially for a process whose outcome is uncertain.

Furthermore, if non-appealing candidates have no further recourse (and the IPKat hopes this is not the case), one really has to question the timing of the letters that issued this week. 

Did it really take two whole months for the Exam Board to conclude that there was an error which would require them to allow any appeals based on the error? And if the Board knew that it was obliged to allow such appeals, was it not incumbent on the Board to spread this news in good time, to allow other deserving candidates in the same categories the chance to appeal?

The IPKat also wonders whether the Board can still exercise the power to revise a candidate's marks according to Article 6(5) REE after the result has been notified, but suspects not:

Article 6

Duties of the Examination Board
[...]
(5) The Examination Board shall scrutinise the marks for each paper proposed by the Examination Committees and decide whether a candidate should pass or fail the examination. The Examination Board may revise candidates' marks or instruct the Examination Committees to re-mark their papers according to a revised marking sheet.

The IPKat is aware that the Board has had its knuckles rapped in the past for acting ultra vires when attempting to remedy problems with a paper, and is probably very careful nowadays not to stray beyond its remit. Such a knuckle-rapping happened in decision D 17/07, when the Disciplinary Board of Appeal made it clear that the correct response to an error that comes to light in a marking scheme is to re-correct the papers and that it is not permissible to simply award every candidate 10 bonus marks across the board. 

This Kat is also aware from his own experience on an examination committee how it can occur that candidates spot arguments or points that the entire committee missed when setting the exam. When he was on such a committee, the system was set up to catch those missed points while the marking schedule was still in draft and to re-mark every paper with such answers in mind (it may still work this way). Equally, he knows that candidates (often many candidates) will present arguments that appear attractive but which the entire committee will thrash out exhaustively before deciding that the answer is definitively wrong and should get no marks. When that happens, they can stick to their guns in clear conscience - exams are not an exercise in democracy among the candidates, after all. 

Yet those safeguards don't appear to be in place for the pre-exam, or if they are in place, they don't appear to be working well. This can be seen from the fact that the results were notified on a website then taken down and reissued, with some candidates getting a different mark twice on the same day, crossing the pass/fail threshold as the day unfolded. Even after this happened, the answers deemed correct were entirely controversial, as now appears from the allowance of the appeals. 

The problem of undue haste
This suggests that one part of the problem may be haste. The emphasis on getting results out quickly is fine until the integrity of the exam starts to suffer, and from the outside it looks as if the committee, Board or Secretariat could have done with being allowed a little more time with the announcement of this year's marks. The  Supervisory Board might need to look at the time given to get the results right.

However a bigger issue is probably the unsuitability of the multiple choice format to nuanced subjects like claim construction and novelty analysis. There's probably little difficulty in asking strictly legal questions in a multiple choice format, but when it comes to the claim construction questions which are at the heart of these appeals, one suspects that the examiners might not have made the same determinations on which answers were true and which were false if they'd had the benefit of understanding the reasons why a candidate chose the "wrong" answer. With a tick on a multiple-choice answer sheet, such reasoning is not evident to the examiner. 

If any readers, whether they are candidates or members of the Examination Board or Committees, can clarify the situation, their comments would be very welcome.

A day late and a dollar short: EQE appeal outcome favours the brave A day late and a dollar short: EQE appeal outcome favours the brave Reviewed by David Brophy on Friday, May 08, 2015 Rating: 5

20 comments:

  1. It is really not correct for the examining board not to correctly check the answers before they are published.
    It is now the second time in a row that this happens.
    The usefulness of the pre-exam has always be put into question.
    Since the pass rate of the real exam has not improved, one wonders about the use of the pre-exam.
    Candidate for the real exam fail mainly because they are not arguing correctly, especially when it comes to added matter or the correct use of the problem solution approach. What is done in claim analysis is not enough to sieve out the good from the bad candidates.
    There is also too much text to read for candidates not having as mother tongue one of official languages of the EPO. The way the questions are drafted are sometimes quite tricky, and for those not mastering the subtleties of the language they have chosen to sit the pre- exam or the real exam, an unnecessary hurdle.
    This should be one more nail for the coffin of the pre-exam.

    ReplyDelete
  2. Woe is me... it's not that the pre-exam is not useful - rather that multiple choice is not an appropriate mode for a professional examination.

    And to think that the EPO (who hold all the cards when it comes to the EQE - read the Regulation) are now proposing multiple choice FINAL papers marked by only a single marker.

    But you won't have heard about that because there doesn't appear to be an incumbent Examination Board or Supervisory Board, the memberships of the previous boards having now expired.

    Beggars belief.

    ReplyDelete
  3. a little perpective here: this concerns 16 candidates out of 796 (see previous IPKat post on the topic). Or actually less since some apparently appealed. If you have a problem with efficiency gains leading to that level of collateral damage, then you probably have an even bigger issue with other efficiency improvements at the EPO.

    ReplyDelete
  4. @Old Man:
    At least this interesting comment by Joeri Beetz suggests that those not passing the pre-examination would have had very little chance the year after:

    Joeri:
    In 2012 and 2013, you only needed 50 marks to pass. Nowadays you need 70. They didn't really change the exam. Passing rates in the main exam for candidates that scored <70 in the pre-exam are 0% for both the 2012 and the 2013 group. The decision to raise the bar to 70 marks seems to be a logical step.

    So it seems the pre-exam is not completely useless.

    ReplyDelete
  5. @Manneken-Pis: please help me - what earlier post? Moreover, 2 correct answers can add up to 4 points.

    ReplyDelete
  6. Sorry, but the article is not correct. All candidates who would have passed with the change of the required answers, got a pass.

    ReplyDelete
  7. http://ipkitten.blogspot.nl/2015/03/breaking-news-uncertainty-over-eqe-pre.html

    ReplyDelete
  8. @Manneken-Pis, UXY & Anon 07:26:-

    There are two issues which are in danger of being confused. Question 20.2 was ambiguous and the exam committee recognised this, giving every candidate credit for that answer. The marks were however revised a couple of times with candidates crossing the threshold in the process. That was what Darren's earlier post was primarily about.

    Separately, questions 15.2 and 17.3 were controversial. It seems clear that the marks for these answers were not corrected and candidates had to appeal. Those who appealed have been given a pass result. If all affected candidates have had their result changed, appeal or not, that's great but the IPKat hasn't heard this to be the case.

    Are you saying that non-appealing candidates have had a letter telling then that they have been deemed to have passed, contrary to an earlier fail letter? That's what the appealing candidates have received in the last few days, and it would be very interesting to know that this occurred across the board.

    Finally, as regards acceptable error rates, I do think that 16 candidates getting the wrong result is unacceptable, myself. Just a personal view.

    ReplyDelete
  9. I received a letter friday telling me my grade went from fail to pass because of my answers to questions 15.2 and 17.3. So apparently they did send a letter to everyone that got affected by the appeal.

    ReplyDelete
  10. @Anon at 10:18:-

    That's great to hear, thanks for the update.

    ReplyDelete
  11. I took/passed the pre-EQE in 2014 and paper D in 2015.

    The Pre-EQE gave me a nice head start for paper D preparations, so I did find value in taking the pre-EQE, even though I grumbled about it for the 6 weeks I studied for the pre-EQE. So perhaps the pre-eqe can help nudge the paper D pass rate for first-time takers.

    For pre-EQE and paper D candidates: The epi offers a pre-EQE course that is also very helpful for paper D. The epi course was my outline, and presents the EPC and PCT in an engaging manner. epi course + delta patents questions + past exams should equal a pass.

    ReplyDelete
  12. If everyone affected has received a letter correcting their result, have the people who appealed received a refund of their appeal fees?

    ReplyDelete
  13. If everyone affected has received a letter correcting their result, have the people who appealed received a refund of their appeal fees?

    In case of rectification by the Examination Board, the appeal fee is reimbursed (Art. 24(3) REE).

    If the Disciplinary Board of Appeal allows the appeal, or the appeal is withdrawn, "it shall order reimbursement in full or in part of the fee for appeal if this is equitable in the circumstances of the case" (Art. 24(4) REE).

    So need for a substantial procedural violation in these cases.

    ReplyDelete
  14. Anon @ 23:53

    Here the decision was rectified by the Examination Board, and so the full appeal fee will be reimbursed.

    This was confirmed to appellants in the letter they received, the text of which can be found on the DeltaPatents pre-exam blog.

    ReplyDelete
  15. I'm all for candidates being able to appeal an errorneous answer, but with a multiple choice paper where no "working" is provided showing how a candidate reached a certain conclusion, what is to stop a candidate doing an "ex-post facto" analysis of the correct answer, helped by peers and tutors, and then filing an appeal based on arguments that the candidate didn't even consider during the exam?

    Indeed, this appeared to be the case in may of the appeals filed, where multiple appelents grouped together to discuss strategy.

    ReplyDelete
  16. But the same applies for all questions in a multiple choise test; you can reason incorrectly on all questions and still ace the test.
    The questions that were appealed were wrongly corrected and it was therefore right to appeal them.
    And what more, I think that in this particular case it was obvious to most people writing the exam what the issue was..The questions where not complicated.

    ReplyDelete
  17. But it is true for all questions in a multiple choice test that you can answer it correctly but on incorrect grounds.

    It should be no different when appealing. I for one did not write in the appeal how I reasoned on the exam but why the answer I gave must be corrected as correct.

    The appealed queations where not complicated questions, they were quite basic and I hope that everyone within the range of 70 points of the exam would have had it correct if not for the missing facts of D1.

    ReplyDelete
  18. Exactly, which is why having a multiple choice paper where no working can be provided to judge what the candidate's thought process is not a suitable medium for an exam of this nature.



    ReplyDelete
  19. All candidates had their marks upgraded, if they had passed but for the cardboard issue. As is now confirmed in the Addendum to the Examiner's Report recently published on the EPO website.

    ReplyDelete
  20. Today I noted that for the C exam of 2017 in decision D0025/17 2 new answers were send to the examination board for a fair scoring. With the preEQE 2015 as precedent, would this in fact mean that all candidates who failed their exam based on scoring zero points for these 2 specific claims should be reevaluated? Do you think it would still be possible for C 2017 candidates, even though they did not appeal, to object to their fail grade?

    ReplyDelete

All comments must be moderated by a member of the IPKat team before they appear on the blog. Comments will not be allowed if the contravene the IPKat policy that readers' comments should not be obscene or defamatory; they should not consist of ad hominem attacks on members of the blog team or other comment-posters and they should make a constructive contribution to the discussion of the post on which they purport to comment.

It is also the IPKat policy that comments should not be made completely anonymously, and users should use a consistent name or pseudonym (which should not itself be defamatory or obscene, or that of another real person), either in the "identity" field, or at the beginning of the comment. Current practice is to, however, allow a limited number of comments that contravene this policy, provided that the comment has a high degree of relevance and the comment chain does not become too difficult to follow.

Learn more here: http://ipkitten.blogspot.com/p/want-to-complain.html

Powered by Blogger.