It’s been a few weeks since the Ireland report regarding expert evidence in family proceedings was published. Apologies for not getting to it before. It coincided of course with a rash of articles about George Hibbert, one such expert who is the subject of outstanding complaints and disciplinary proceedings. I wrote about the George Hibbert coverage here.
Before we look at the substance of the report it’s worth giving some consideration to how it came to be written in the first place. The first reference I can find to it is in the Family Justice Review, who seem to have received a draft copy of the report. The FJR report says:
3.142.A report to be published shortly by the Family Justice Council has examined the quality of expert psychologist reports in a sample of private and public cases. It points to serious issues both with the quality of reports and the qualifications of those carrying them out. We are not surprised in view of the concerns we heard expressed throughout our work about the quality of reports generally. We recommend that studies of the expert witness reports supplied by various professions be commissioned by the Interim Board, subsequently the Family Justice Service.
3.143.Agreed quality standards for experts in the family courts are clearly needed and we recommend that they should be developed. The FJS should lead this work. Meeting the standards could be a requirement for payments to be approved by the LSC. Criteria could include adherence to set timescales, membership of appropriate professional bodies and completion of specified court focused training, peer review and continuing professional development.
That was November 11, but the report was actually only published in mid March 12. It appears to have been part-funded by the Family Justice Council but published by the University of Central Lancashire.
The report’s author, Professor Ireland is a Forensic psychologist and academic at the University of Central Lancashire. Judging from the CV type information on the UCLAN website, Prof Ireland’s main area of experience appears to be around offenders and secure environments. I cannot see anything that suggests a particular experience in relation to family proceedings per se (although of course in many families there is much overlap between family and criminal justice). At the time Professor Ireland’s report was published she was a member of the Family Justice Council’s Experts Committee. I know this because I checked it at the time of publication, but I cannot provide a link because the Judiciary website no longer lists an Experts Committee (although there are various orphaned references to that committee, making me confident it is not a figment of my imagination – see minutes of FJC meetings in April and July last year here and here, and the Press release issued by UCLAN which quotes a Heather Payne, who is described as the chair of the FJC experts committee. Dr Payne is a Cons. Paediatrician and member of the FJC).
To the best of my recollection the FJC Experts Committee as listed on their website in mid March was comprised entirely of experts, some medical, with only one lawyer named (a female QC whose name I did not note). We’ll come back to that. Again, to the best of my recollection, at least one of the reviewers who assisted with the research was also a member of the FJC experts committee.
So that’s where it’s come from. But what have we actually got? The report that has been published is a summary report only. It is unclear if a more substantive report is in the pipeline. The author herself acknowledges the limitations of the study which was only ever intended to be a preliminary study to inform decisions about whether more robust work ought to be done (although you’d be unlikely to pick up on the qualified nature of its conclusions from the press coverage). But much of what Professor Ireland reports reflects my own experience of variable quality in the psychological reports that courts rely upon, and I agree with many of her sensible proposals.
I’m a bit puzzled though by a couple of things. In particular the assessment of experts reports by reference to the Civil Procedure Rules and something called the Daubert standard. I know that the Family Procedure Rules 2010 only came into force in the early part of 2011, but prior to that the Family Proceedings Rules applied and there was a previous PD on experts which was applicable from 2008 and was itself quite comprehensive and prescriptive (the files sampled were from between 2009-2011). Thankfully there is some commonality between the Civil Procedure Rules and the Family Procedure Rules, but I don’t understand why the author has assessed against what appears to be inapplicable criteria – the Family Procedure Rules and Practice Direction (even the previous incarnations) are far more detailed than the CPR and more tailored to the particular requirements and purpose of family proceedings). It suggests a lack of knowledge of the culture and practice of Family Proceedings (one could argue this might bring much needed objectivity from an observer not steeped in the prejudices and complacencies of a hermetically sealed environment, although I would tend to take a different view). We know that in November 2011 the report was still in draft stage, so I would have thought there was ample opportunity for the report to have at least made reference to the FPR – at the very least acknowledging that a new regime had recently been implemented which may have impacted upon practice since the timeframe from which the sample cases were drawn.
I wonder if the conspicuous absence of any reference to the governing framework that is applicable to this field is attributable to those involved in the commissioning and conduct of this research being more experienced in criminal and civil / PI fields than in family? I can go no further than wondering because the relevant information about the membership and makeup of the FJC experts committee is no longer publicly available, and it may well be that the CV available on the UCLAN website for Prof Ireland is not comprehensive. Regardless of which set of rules the reports were assessed against it is pretty clear that a significant number of reports assessed fell far short of either set of criteria . The report appears to question whether there should be more stringent rules and guidelines in place for the commissioning and admissibility of expert evidence, but I think that prior to that question comes the query as to whether or not the courts are consistently applying the guidance already in place, and if not why not? My experience and the qualitative data set out in the summary report suggests they are not consistently applying the guidance.
The Daubert standard is a reference to an approach to expert evidence applied in courts in (for example) the US. For the purposes of this post the wikipedia reference will do. In essence the Daubert standard says that if expert evidence is to be relied upon in court proceedings it must be based upon sufficient facts or data, be the product of reliable principles and methods, and of the application of the principles and methods reliably to the facts of the case. Although there have been prior calls for a test along these lines to be introduced in this jurisdiction (Law Commission Mar 11) this has not yet happened. Again, it seems odd on one level that the current practice in the courts is being assessed against criteria adopted only in other jurisdictions, although no doubt the author of the report would say that the findings of it demonstrate just why we need some kind of reliability test akin to the Daubert standard in this jurisdiction. Again, if one cross checks the previous and current PDs applicable to family proceedings these should go some distance towards that aim – if consistently applied and enforced.
The report states that:
Of those experts completing the reports, over one fifth were unqualified and were not registered either with the Health Professionals Council (HPC) or the British Psychological Society (BPS). Of the qualified psychologists, over two thirds were clinical psychologists, with the remaining either educational or forensic psychologists. Just under one fifth of the sample referred to themselves as consulting psychologists. Nine tenths of experts did not report a current assessment or treatment practice, with expert assessments their primary work.
I carried out an entirely unscientific exercise myself earlier today to see how many of the psychological experts I could think of were registered with the HPC. Most were, a couple were not (although these were people I had not heard of recently so may simply have retired), and one was listed as an Educational Psychologist which came as something of a surprise given that my experience of him was as an expert assessing risk in relation to a parent. All of the experts I would “rate” were appropriately registered (I did not check the BPS). Most of the experts I regularly encounter are to my knowledge in clinical practice, but those who are best known are often also the most busy. It is not uncommon for them to be so booked up that an unknown name is put forward. I don’t doubt that there will be some who are not in clinical practice or who do probably far more of these expert reports than is healthy.
The problem with these striking figures however is that the report relies upon the data provided in the CVs on the court file, presumably separately indexed in the bundle (rare) or annexed to the body of the main report (rare) or set out in summary form as a paragraph incorporated into the start of the report (more common). There is no consistency of information provided on CVs, which in my experience range from the ridiculously lengthy to the totally useless. Often the focus is (understandably) upon the experts experience qua expert and it may well be that clinical or other expertise is not explicit or foregrounded in some CVs where perhaps it should be. It is highly likely that at least some of the files sampled will have included no CV – they often circulate at court but don’t make their way into the bundle. This is clearly an area for improvement and I wonder if there should be a template for CV information required prior to instruction of an expert in family proceedings, so that experts are able to set out their stall in a coherent way and courts and parties can make good choices about the appropriate expert at the outset. I agree that it should be a requirement that any party who wishes to instruct a psychologist should be permitted to instruct only an expert who is registered both with the Health Professions Council and the British Psychological Society.
The report also raises concern about the quality of the reports – inappropriate use of or reliance on psychometrics, insufficient analysis of data, failure to link data to opinion, and most concerningly a lack of actual psychology.
Fifty-three reports were indicated to have problems in linking the final opinion to the body of the report. There were 126 reports in total, so this represents 42% of the sampled reports.
“To illustrate examples concerning problematic opinions, comments from raters on the assessed reports included:
- – “No backed up opinion, one paragraph for each instruction question”;
- – “Very little opinion; Thirteen lines of opinion and does not address thequestions”;
- – “Three pages of psychological assessment only”;
- – “Poor, just has no opinion and refers to facts so readers can make their own mindup”;
- – “Opinion very weak, report mainly a description of an interview that has littlefocus”;
- – “Fails to answer any of the instruction questions”;
- – “Huge assumptions expressed in the final opinion, not linked to facts e.g. sexualassault [unproven] may have led to angry acting out of Mrs X emotional turmoil”;
- – “Psychometrics used but not referred to”.
- – “Bases opinion on personality using the wrong measure”. “
These are highly concerning ratings, albeit qualitatively based. But for all that I’d quibble with the extent to which this is a problem I agree entirely that it is a problem. As has already been noted on the Suesspiciousminds blog there is a frightening diversity in the quality of psychological reports. Good reports – which I think are the majority – are enormously useful, but there are some real stinkers. You wait months and when they arrive they help no-one. Some seem to regurgitate everything they have read and then bolt on a paragraph of “opinion” at the end, whilst others operate on the basis of factual assumptions that are neither accepted nor proven. It is not uncommon for me to get to the end of a report and immediately go back to the start to see where I missed the actual psychology. Several thousand pounds and a few observations in the absence of the application of psychological theory is neither value for money, nor purposeful delay and nor does it aid the justice of the case. There are a few repeat offenders where the same stock paragraphs seem to reappear in each report, setting out general psychological principles.
That said I struggle slightly with some of the criticisms made by the rates that suggest experts have strayed into the remit of the judge by making a recommendation as to whether or not a parent can provide good enough parenting or on the question of rehabilitation. It’s unclear from the report whether or not the reviewers had letters of instruction (I would guess not if relying on the bundle, although most experts set out the questions in the main body of their report), but psychologists are often asked in terms to offer such recommendations.
I share the anxiety expressed about the use of expert witness commissioning companies, and think the suggestion that experts should be in clinical practice as well as undertaking expert work is sensible. I also agree that there should be further thought given to mechanisms to enable peer review and assessment of the quality of expert reports to be undertaken.
There is a recommendation for feedback to be given to experts. There is in fact a provision in PD 25 for experts to be notified of the outcome of proceedings and the use that has been made of their report, although how consistently this is applied I don’t know.
As ever, the issue for me is not in identifying the problem (inconsistent quality of expert evidence), or what needs to be done in order to remedy that (we already have a good template in the FPR and PD25), but in how we translate that into practice in the pressured environment of the family justice system. Nobody has the time or resource to implement this stuff. And whilst Ireland’s report has generated some punchy headlines and given grist to the mill of those who would happily condemn all experts, it can only go some distance in helping us in working out how to fix this on the ground. Everyone, from every branch of the system, has so much to fix – this is one more thing on a long old To Do List.
Yes, I did wonder whether they had come at it with a civil mindset, and the emphasis on the CPR rather than the regulations that expressly deal with experts in family cases troubled me.
I agree totally that before we rush to introduce a whole new raft of rules, it would be worth scrutinising why the Practice Direction on experts which is very detailed and very robust is rarely followed, and in practice it seems to me that most lawyers only connection to the Practice Direction is when paperclipping it to the Letter of Instruction to the expert.
(I’d suggest that the answer might well be that “Case Management” too often consists of focussing on time targets and has to be done by Judges who are massively overloaded and just have to get through the list, rather than being done by a Judge who has been allowed the proper time to prepare and scrutinise the case to see that the detail of active case management is taking place. I wonder if Justice Ryder’s move towards a genuinely inquisitorial system will cater for the massively increased burden that will place on the Judge to drive the process)
Thanks Lucy for such an informative and considered review of the Ireland report, this answered many questions I had about it. It would be great if somebody could do a similar study of CoP experts taking into account these methodological issues. Years ago I used to be an assistant psychologist in neuropsychiatry, and so much could hinge on the fine detail of the way the test was conducted I would be concerned about experts not embedded within clinical teams offering supervision and peer review.
Interesting that pinktape finds most psycholgists who are commissioned to write court reports are still in clinical practice as one of the points I did recogise in the Ireland report was that a number had become professional witnesses. Balancing clinical work with court work is not easy. There is a significant problem in getting NHS Trusts to engage in providing these services, as mentioned in the note I wrote in response to the Ireland report for Family Law Week.
How long will courts be able to rely on these reports in any event, with all the pressures from the LSC and the recommendations of the FJR?
Experts in clinical practice – thats just my personal experience – may not be representative. And I certainly don’t think all are in clinical practice – although as Ireland suggests they really should be.
Lucy
I would be interested in your view of Camilla Cavendish’s follow-up article in The Times in which she speaks of a concerted letter writing campaign by expert witnesses, and the abusive language used, as well as what looks like an attempt to make sure any further review is carried out by ‘one of their own’ rather than someone like Professor Ireland who has the benefit of being objective, because she is not a family law expert witness.
Can’t offer a view as it is behind a paywall…Who is said to have written to whom?
The follow-up article:
“My attack on ‘expert’ witnesses in family cases brought a vitriolic response — and a sad story that proves my point
‘Salacious”, “confused”, “unfounded”, “biased”. The response to my last column has been so extensive — and vitriolic — that I feel obliged to return briefly to the subject of the so-called “experts” whose opinions influence decisions made by courts to take children into care.
It is quite natural that so many psychologists and psychiatrists have written to defend themselves against what I admit was a hard-hitting article. We columnists are paid to provoke. We expect verbal attack, and it helps to keep us honest. But what has surprised me — and the Times letters editor — is the co-ordinated and unpleasant nature of some attacks on the study that I cited into the quality of expert witnesses, and the lead author of that study, Professor Jane Ireland.
This government-commissioned report, the first of its kind, examined 127 reports made by psychologist expert witnesses in cases at three major courts. It found that a majority were written by career witnesses who did not practise but did only court work; a fifth by people who commented beyond their expertise, and a fifth by people who were not qualified. While the sample is small, if it is remotely representative it brings into question the reliability of many expert psychologist reports. Since the most draconian decision the State can make is to take a child from its family, this is alarming.
Few of the psychologists who have written in have expressed concern that Professor Ireland might be right. Instead, some allege that she is biased against career expert witnesses and determined to bring the profession into disrepute. Yet Professor Ireland is a sensible and respected academic who acts as an expert witness herself and hardly stands to gain by denigrating her own profession. I suspect some dislike her pointing out that full-time court work can pay three times as much as the average psychologist earns.
It is quite possible that the study may not be representative and that some of the experts deemed unqualified by Professor Ireland, because they do not belong to the main professional bodies, may be perfectly good. But that does not undermine the argument for more scrutiny. Claims that poor expert opinions are weeded out early on, under cross-examination, are sadly not always true — and in any case should not exonerate poor evidence.
Another claim made by the critics of Professor Ireland is that forensic psychologists should not judge the work of other types of psychologist. But if you read the Ireland Report, you can’t help feeling that you don’t need a PhD to see there is something wrong if an expert assesses a person he has not met, or cuts and pastes from previous reports, or comments on a parent’s mental health despite having never worked in that field. The campaign to discredit Professor Ireland has the bitter flavour of a group determined to ensure that any future research is carried out by someone who reflects their perspective. I hope that the Government is wise to this.
Other people — parents, doctors and lawyers — have written to say that their experience does tally with Professor Ireland’s findings. Several have contacted me to claim that they were “assessed” by professionals who never actually met them or their children.
One of these is Lucy Allan, an articulate Conservative councillor who sat on Wandsworth’s fostering panel until she went to her GP seeking antidepressants one day — and ended up battling to stop her son being removed.
[edited (see comment below)]
Since Haringey Council failed to prevent the tragic death of Baby P, the number of children being taken into care has accelerated. Figures released yesterday by Cafcass, the Children and Family Court Advisory Service, showed that applications to place children in care have risen by 10 per cent since last year, to a record 10,199. I struggle to believe that every single one of these children would be better off in care than with their families.
The mantra of “suspect everyone” has gone too far. In 2009 the BMA issued guidelines telling doctors to contact social services with any suspicion of abuse or neglect. That document was an understandable reaction to the high-profile failures of paediatricians to get help for Victoria Climbié and Baby P. But it also spelt the end of patient confidentiality.
The consequence for Lucy Allan has been a “permanent fear of seeking medical help. I could not go back to the NHS doctor because I was told: ‘If it happens again we will take further action.’ So I did not seek treatment.”
What kind of world have we created in which women dare not talk to their doctors? Where people who purport to have children’s best interests at heart scramble to libel a modest academic, rather than risk a spotlight being shone on their industry? What is everyone trying to hide?”
Gladiatrix – I’m not sure if your post is your own words or the text of a Times article (I’m guessing by Camilla Cavendish)? I’ve deleted a large chunk, largely for reasons of space (the bit about Lucy Allan whose story is easy to find elsewhere for those who are interested – she was on This Morning earlier this week), and partly because I don’t really want to reproduce an entire Times article.
At any rate whoevers words these are I agree with much of the points of principle about the importance of scrutinising the quality of expert evidence and the fact that the Ireland report raises serious questions that need to be given further consideration, even if there are weaknesses in it.
Yes it is the article, I thought the first sentence starting “Follow-up article” made that clear, apologies if that was not the case.
I thought so, but wasn’t sure. Thanks.
Up until the “Lucy Allen” paragraph, where we descend into the usual polemic, I actually agree with Camilla Cavendish’s article (a sentence I never thought I’d type) . I think it is a shame to use a phrase like “so-called experts” at the outset, if you are trying to have a proper constructive debate, rather than an inflammatory slanging match, however.
(Psychologists, psychiatrists and social workers do possess an expertise in the field, and have a contribution to make, so it is nonsense to describe them as “so-called experts”. There is a valid but separate argument as to whether this expertise means that one should listen uncritically to what the expert has to say, and whether any individual expert has done a good piece of work)
What I think is a real shame is that the Government, in passing the statutory instrument setting cost caps, have allowed the LSC to drive Independent Social Workers out of business, whilst creating lots more work for psychologists at over four times the hourly rate of ISWS BEFORE doing any sort of comparison study of whether ISWs offer comparable value in terms of timescales, quality, and value for money to psychologists.
I would argue that if you picked out 100 psychological reports at random, and 100 ISW reports, the quality of one set would outshine the other – and it wouldn’t be the ones who are getting paid 4 times as much per hour.
I don’t know whether Ireland’s report is perfect, and I suspect that it isn’t, but it doesn’t seem unreasonable to me that with so much delay being attributable to the instruction of experts, so much weight being given to their opinion and so much money being spent on them that there should be independent scrutiny and assessment of whether they are doing the task well.
There is a lot I disagree with on a factual basis in this post. However, the fundamental point I would make is that if the court accept evidence that is so unreliable that an expert is struck off for providing it, it raises questions as to the quality control in the judicial process. The responsibility this falls with all of the advocates involved as well as the judge.
John,
I am unsurprised we have differing opinions but what’s factually inaccurate?
Lucy
[…] relevant reading on this blog: Experts Upon Experts, and Experts, the Press and a Sloppy Approach to Evidence Based […]
[…] ease of reference other relevant reading on this blog can be found here: Experts in Family Courts, Experts Upon Experts, and Experts, the Press and a Sloppy Approach to Evidence Based […]