How do you rank international arbitration practices?
It seems as if it should be easy. After all, they compete head to head, and somebody with understanding of the process pronounces a winner at the end.
Except … well, imagine three firms. Firm A goes up against firm B in a big case and A wins. Next, firm B goes up against firm C – pitching for work. B wins.
So applying school-boy logic, A must be better than B, and B must be better than C?
Possibly. The A B C order works, as long as long things are pretty simple. They seldom are. For example, let’s say A “wins”, but is given only a fraction of the amount it was asking for. In fact, let’s say it received less less than its client was offered before the case began. Is that still a win?
Or what about B and C’s pitch? It may have been for a government. Now, those are famously price sensitive shoppers. So what if B “won” because it low-balled its fee – should we still regard it as more capable of the two? It may become so, through working on the case, but should we infer that from the result of the pitch? Not necessarily.
These sorts of scenario are much closer to what really happens than you might think. The following true story shows just how difficult it can be to draw inferences from the facts of a case.
Some years ago – a leading international arbitration firm of the day, Coudert Brothers – represented an Asian conglomerate in one of those apparently minor arbitrated disputes that sometimes crop up, between a client and its Middle Eastern agent. Except – and here’s the rub – it turned out, once the surface of the case was scratched, that the agent was seeking vested rights in most of the Asian firm’s businesses. That had the effect of projecting the case’s possible impact forwards ad infinitum.
The result from a single contract claim - seeking $50 million - could ripple across the conglomerate’s balance sheet, because of similar terms in a number of contracts. Making matters worse, its major corporate loans all contained acceleration clauses. Any legal award of more than $5 million and they could fall due for earlier repayment. So the outcome of an apparently small case had the ability to affect the company’s entire financial foundation.
The conglomerate tried to settle – for $16 million – but was rejected and had to go to a full trial.
After a “pretty bloody fight” according to some of those who remember involving the discovery of forged documents the arbitrators awarded the claimant $2 million and declared the contract terminated.
“A silly summary of the case,” says Jan Paulsson, one of the lawyers for Coudert, “would have been that we lost a little case.” In reality, he notes, the client breathed a vast sigh of relief. It treated its arbitration lawyers – and their spouses – to an all-expenses paid trip to the Seoul Olympics, which it was sponsoring.
If “achievements”, then, don’t help, how about simply looking at the value of work on a firm’s books? That’s an approach used in some surveys. Again, it’s a decent idea - those with most to lose chose most carefully). But again, it is hard to wield in practice.
The Coudert Brothers case shows one problem the huge claim disguised as something more run of the mill. But it’s negative image is more common – a huge claim that’s really quite piffling. Some will view that as harsh – but it’s quite notable how few “billion dollar claims”, especially in the public international law/ICSID arena – have so far produced anything like a billion dollars for the claimant. The largest ever ICSID award remains US$867 million (won by White & Case against the Czech Republic) . Otherwise, around US$ 150 million is more normal. Even allowing for strategic exaggeration, that’s a big difference. A few years ago, three claims against Turkey – for a purported US $30 billion in total –were slung out as entirely frivolous (albeit after an almighty legal battle incurring millions of dollars on both sides). The arbitrators went so far as to plant heavy costs on the losers, in part to deter similar frivolous claims.
While a new wave of matters about the expropriation of hydro carbons may challenge these glib assertions, thus far investment arbitrators have been loath to hand down huge awards.
It seems that, when it comes to assessing the capabilities of international arbitration firms, some of the obvious metrics only take you so far.
* * *
Is there a better way to take the pulse of a practice?
A few years ago, Global Arbitration Review journalists and the magazine’s editorial board decided to ponder that very question.
That led to a suggestion, what about a survey built on merits hearings?
(Click for larger image)
For those entirely new to this discipline, the merits hearing is the closest that arbitration has to the client’s day in court. For the combatants, it is the moment they convene in whatever hotel room or law firm office or arbitration hearing centre they’ve agreed will host them for a pre-set number of days of oral submission, arguments and – possibly - cross examination.
That means, for the senior lawyers – who are probably the advocates on the case - plenty of time alone pressing a metaphorical cold towel to their head.
For the younger lawyers it means less of that and more solving logistics – they have to marshal all the materials and make everything run like clockwork. Sometimes their tasks are as mundane as ensuring the hotel can provide food at the strange hours of the day or night … surprisingly important on a big case.
If it's a very important case, the whole team may engage in extensive dry runs and dress-rehearsals, in front of arbitrators hired specially for the purpose.
So why did the incidence of these things - merits hearings – emerge as a good shorthand for the health of a practice?
Well, as metrics go, the merits hearing has a number of advantages.
First, it’s innate. Just as fissile material emits radiation, so an active arbitration team can’t help but produce merits hearings.
Second, it’s fair. Although some firms have complained that it places them at a disadvantage –“our cases tend to settle”– there is no evidence to suggestions that one firm has a superior settlement rate to any other firm. Furthermore in a practice of any size, the partner who is going through a phase where all his cases seem to settle is probably offset by a partner whose cases are doing the exact opposite.
Third, it’s checkable. International arbitration is covered to a large part in a cloak of confidentiality. But firms can reveal the information needed to prove a merits hearing has indeed occurred – as in, language, dates, claim size, opposing firm, chairman of the tribunal etc etc – which allow us to establish a) the fact of existence and b) and what order of magnitude it was, without contravening any rules.
Fourth – and related to the above – it’s not as prone to inflation or manipulation. That’s not to say it can’t be inflated – but as just explained, it’s checkable. That’s in stark contrast to self-certified statistics such as the number of international arbitration specialists within a practice, which is difficult to scrutinize.
Fifth, experience is gained in merits hearings. Any advocate will tell you that there’s no better gymnasium for the legal muscles than going into the crucible of a hearing, big or small.
Of course not everybody agrees with this approach. As mentioned, a number of firms think they have a better ability to settle cases than others, and they lose out.
Others think our approach discriminates against their practice. This comment is usually expressed by firms that work on a few large ICSID matters. Those kinds of large cases suck in lawyers and generate major fees for firms, but produce little to show in terms of hearing days.
To the extent they don’t produce a lot of merits hearings, those firm’s are right. This formula is tilted against them. On the other hand, these days merits hearings are only a starting point. We now include other elements so firms that work on complex matters get credit for that (see “methodology” for more).
It’s also hard to think of a metric that ultimately flatters everybody. If a practice is both low volume of merits hearings and that’s not offset by the value at stake, then it’s hard to know what else to bring in. Perhaps a head-count of lawyers working full time on arbitrations, but hard to police.
That’s not to say the merits hearing is a perfect lens.
Or that in the future we won’t discover a better one – an encouraging suggestion is billable hours. But for now, the merits hearing is a good starting point for a survey, and a useful way to begin to break down each one’s fundamental character.
To quote one of our editorial board members. “The number of merits hearings instantly creates a picture of how busy a practice is and gives you a fairly good indication of its size without having to reference the number of self-proclaimed arbitration lawyers.”
“It is also the metric least likely to be manipulated. Yes, there are cases where a tribunal may bifurcate the quantum phase – so you end up with two hearings – but this is marginal stuff.”
After acknowledging some of the flaws just mentioned, he said “Admittedly, no metric is perfect, but this approach is likely to be the next best.”
* * *
The first GAR 30 – which announced itself as “what the Fortune 50 is to the Fortune 500” – came out four years ago. So this is the fifth edition. Since the first edition, a number of tweaks have been introduced to the overall ranking formula.
In brief, at different times in the preceding four editions we’ve included columns that score practices according to the reputation of their individuals (Who’s Who research data) and by the arbitrator work they attract.
We’ve also added a weightings intended to reward appearances on chunkier cases ( (in response to one of the comments above about being too generous to the high volume, low value practices). We’ve called these in the past bet-the-company cases, for ease.
This year, the formula for the ranking is a little longer than last year – as in it adds together more components. In particular, we’ve reintroduced a role for both the elements mentioned above - how individual members of the practice are (Who’s Who research), and arbitrator appointments.
(That’s in part why firms such as Herbert Smith and Lalive – who have quite a few individuals included in our International Who’s Who of Commercial Arbitration) – are riding a little higher in our rankings.)
We’ve also included additional ways of filtering for high-value work. (See ‘methodology’ for more.)
Overall, the idea is that, a bit like upping the resolution on a printer, the more lines of information used, the crisper and clearer the image.
The firm’s are presented in the “30” according to their final ‘score’ – representing the sum of their relative performance in all relevant columns. These scores are developed using t-scores, which calculate the elative to the average under each heading. But we don’t print the final score as it doesn’t indicate much beyond position in the ranking.
That said, it’s worth nothing that this year the spread of numbers from Baker & McKenzie at number nine to Arnold & Porter at number 45 was a smooth continuum. If we had to break the top 50 firms down into different bands the logical split would be a pyramid with three tiers – with around 35 firms forming the base.
What kind of practice does well in the GAR 30? As so often, the formula for success is easy to say, harder to do. The glib answer to the question is – a well rounded one. It’s notable for instance that only the top two firms – Freshfields Bruckhaus Deringer and White & Case – do well in all three broad categories that the formula looks at - reputation/arbitrator appointments; volume of cases/ merits hearings; and value of the work.
For others, and this is evident from the next five firms, the secret is out performing the norm in one particular segment.
In the case of Shearman & Sterling, Wilmer Hale and King & Spalding for example they outperform when it comes to value of work (also true of Skadden Arps, and Curtis Mallet-Prevost), while Clifford Chance, Hogan Lovells and Baker McKenzie generate way more merits hearings than is normal.
On to this year’s results.
The 30 this year has 16 climbers, including five “chart entries”, and 10 fallers. Four firms stay the same.
It recorded more merits hearings in total than a year ago – up 30 per cent. Looking at these 30 firms (rather than the 30 in the ranking last year), they recorded 750 hearings over the preceding 24 months compared to 530 when sampled before.
The five new entrants for the record are Curtis Mallet-Prevost; Kim & Change; Derains & Gharavi; Orrick; and Chadbourne & Parke. The highest climber is Curtis Mallet-Prevost, up around 20 from a year ago.
The number one slot is, again, Freshfields Bruckhaus Deringer – making it the fifth time in a row. Below that, however, there’s been a bit of change. White & Case has overtaken Shearman & Sterling, and King & Spalding, has overtaken Wilmer Hale.
A little lower down Lalive and Schellenberg Wittmer are both substantially up, as is Clifford Chance and Norton Rose.
The industries that have been fuelling some of the higher firms is also worthy of note. For Freshfields, the biggest contributor to its merits hearings score was energy, followed by (quite a way back) banking and finance, and construction, with real estate and transport-infrastructure also in the mix.
For White & Case on the other hand, construction provided that most hearings, followed by energy (including power generation). Both firms though worked in at least 16 different industry sectors each.
Freshfields’ most frequent opponents (albeit not that frequent) were De Brauw Blackstone Westbroek, Shearman & Sterling and Herbert Smith, while White & Case’s were Salans, DLA Piper and Baker McKenzie.
Telecoms disputes were also slim on the ground, in contrast to some previous years.
It was also obvious from several firms' submissions – and this could prove to be the beginning of a trend – that high value arbitrations are tending more and more to pullulate and produce inter-related cases and follow-on claims. Wilmer Hale and Shearman & Sterling were but two firms that have clearly been led by one underlying dispute into numerous proceedings.
Similarly, there seems to be the inkling of a trend in the value at stake. Despite the caveat at the top of this piece about taking the figures that fly around especially in investment claims with a pinch of salt, the overall value at stake across all hearings according to the daa collected has rocketed. It went up to US$206 billion from US$96 billion (a figure that inevitably sees a few disputes counted twice in any year (if opposing firms on a particular case are both inside the GAR 30). At the same time, though, there was no change in the number of what we simplistically call bet the company cases in the 30. There are 46 in this year’s chart, versus 47 a year ago.
It may be that the wave of arbitrations about gas pricing that’s been underway in Europe explains part of the rise, and the spread of resource nationalism.
Just why is Freshfields number one?
There are many theories. There’s the “right time right place” school of thought, which believes that the practice took advantage of a unique moment to build a dominant position, which it has skilfully managed.
Others think it benefits from particularly strong internal practice management that ensure nearly every arbitration wherever it originates in the firm finds its way to the international arbitration department. The inside view is that none of its competitors have as many good international arbitration people on the ground in as many places – which makes acquiring cases easier.
That’s only partially true – firms such as White & Case, Debevoise & Plimpton and Clifford Chance have - at times been in a similar position.
More important, perhaps, and in contrast to those firms, Freshfields has enjoyed a steadiness and cohesiveness at the top of the practice – the current leaders of the practice have been with the firm for a decade or more. Those senior figures have also shown a willingness to relocate to other locations for the good of the practice and the practice has grown from within rather than by adding laterally. Where others have lost figures and rebuilt (or in some cases not) Freshfields has grown steadily yet consistently, in the process becoming wider-spread yet still close-knit.
A lot of credit for Freshfields’ rise to this level of stability and dominance goes to Jan Paulsson, who himself chose to remain with the firm to shepherd through the next generation of talent.
Members of the practice who’ve worked at other firms also mention the fertile atmosphere into which younger lawyers are placed. One told GAR he was encouraged, even expected, to understand that arbitration is a vocation, as much as an area of specialisation. If he asks to be allowed to accept a speaking opportunity, he knows that at Freshfields it’ll be looked at with seriousness and compassion, which is not the case at other shops he’s experienced. Asked where that attitude comes from, he said “It all starts with Jan.” (This view is by no means universal. Some who have moved to the practice have found it limiting in comparison with the room to roam they used to enjoy.)
Which of the challengers to Freshfields seems to have the best chance of overtaking it, at some point?
Well, the number two firm is now White & Case. Does its rise in the 30 this year mean things are headed in the right direction?
They may do. But in truth White & Case’s improved position owes as much to changes in the ranking formula and an unusually low number of merits hearings a year ago as to obvious expansion in its practice. In particular this year, we started to add in jurisdictional hearings to the count – which helped lift it.
While White & Case looks a lot like Freshfields in some ways – in partner infrastructure, in its presence on some of the world’s biggest cases – in others it looks different. It hasn’t enjoyed the same level of stability in the practice, and there’s less of a common patrimony at the top. Recently, for example, it lost its head of international arbitration in Germany. And, when the two firm’s performances in the International Who’s Who of Commercial Arbitration are compared, the White & Case lawyers receive fewer votes on average (while still doing well).
That said, there’s no doubting White & Cases’s ambition – which has seen it expand the number of brand-building activities it partakes in, and hire a global practice manager. If it now took part in a sequence of incredibly high value merits hearings … who knows. It might take the number one slot. It wasn’t impossibly far behind in this year’s final score.
Otherwise four obvious prospects stand out.
The first is King & Spalding, which from 18 two years ago has moved up to fourth.
But again, this year’s small climb owes much to the inclusion of the Who’s Who column in the formula. That said, whereas that was mentioned as a neutral fact in the context of White & Case, in the context of King & Spalding, it helps to show a positive – namely how well the practice has done at bringing in additional highly-regarded names, while also nurturing its associate crop. Once upon a time the firm had barely one lawyer in the International Who’s Who.
Likewise, once upon a time the firm was synonymous with treaty work. In this year’s submission, only two of its 10 merits hearings were treaty-based.
That said, its caseload is nowhere near the Freshfields level.
How about Shearman & Sterling?
In the short term, it looks probably the best bet.
It’s international arbitration group has had an incredible 18 months. The figure for value at stake in merits hearings [columns three and four] starts to tell the story. (note, the Freshfields score in this column has had one outlying case removed; also removed from Curtis Mallet-Prevost’s figures). What doesn’t come across though is the pace at which its core team in Paris and Germany have been slamming through high value hearings.
Covington & Burling, Crowell & Moring, Vinge, Homburger, Fulbright & Jaworski, Latham & Watkins, Squire Sanders Dempsey, Cleary Gottlieb Steen & Hamilton, Watson Farley & Williams, Python & Peter, Gide Loyrette Nouel, Sidley Austin, WongPartnership, Bae Kim & Lee, Arnold & Porter.
Between 6 September 2010 and 9 June 2011, it was in merits hearings nearly non-stop - 96 days worth in all (and that excludes a few jurisdictional days). Eleven of the 16 or so cases being handled were fights about more than US$1billion.
During that phase, billable hours “exploded” according to practice manager Nanou Leleu- Knobil (It’s most regular opponents were Latham & Watkins, Wilmer Hale and Freshfields Bruckhaus Deringer.)
Although Shearman & Sterling doesn’t at this point have anything like Freshfields partner mass, the quality of its work is pretty much unsurpassed. If it can find a way to convert that base into additional partners, who knows what things might look like in a few years.
And next year the Yukos merits hearing is due to hit the GAR 30 formula. It alone could propel the firm to the top, at least for one year.
Then there’s WilmerHale and Debevoise & Plimpton, which share some of the characteristics of Shearman & Sterling. They’ve both opened in in new jurisdictions (Debevoise & Plimpton in Hong Kong, WilmerHale in Germany). Having people on the ground in new locations ought to bring in new clients and in time to expand their work.
It has to be said that neither firm is exactly taking a huge risk in either location. They’re using existing partners who will be based in those new location less than full-time. (In contrast to Freshfields, who at the time of press, had just announced the relocation of Lucy Reed to Asia, full time.)
Realistically, it could be Shearman & Sterling on top next year, and then Freshfields back on top the year after, especially if the value of the Yukos claim proves to be less the current US$105 billion figure.
The bigger jumps occur lower down the ranking where Lalive, Schellenberg Wittmer, Salans, Norton Rose Curtis Mallet-Prevost, and Clifford Chance, make the largest leaps.
How much credit each should get varies. Some are moving up because of the adjustments in the ranking formula (the Swiss firms get a big boost thanks to their large number of arbitrator appointments), while others appear to be getting over a blip in their figures (an abnormally low number of merits-hearings for that particular firm) to return to the more usual level (Clifford Chance in particular). But they all clearly have practices in rude health.
Norton Rose, however, may be one to watch. It has been steadily building a much bigger case load for several editions. This year’s higher position is a clear further step arising from amalgamating that high volume practice – fuelled by its big global footprint –with a reputationally very strong team from Ogilvy Renault in Canada (all of whom were tutored by the leading arbitrator Yves Fortier CC, OQ, QC). It will be interesting to see how far the combination goes.
DLA Piper and Kim & Chang both also have cause to be pleased, as both have risen substantially. But n each case it remains to be seen if the improvement will sustain. In DLA Piper’s case, researchers noted that it did well in part because of a number of high-value hearings that were all facets of the same gas-pricing dispute in Eastern Europe,. That’s now over so it might fall back next year. Kim & Chang, for its part, could have finished even higher but was held back by inadequate details regarding some of its hearings. It will need to be more transparent in submissions, and in particular to include clear durations for hearings and dates they occurred, to sustain its score.
Its appearance, though, means that for the second year running, the 30 features two indigenous Asian firms: Rajah & Tann again made the top 30, for a fifth year in a row
Meanwhile Bae Kim & Lee from South Korea finished just outside the 30 (see box 31-45).
For some firms success in the GAR 30 must seem as simple as turning up, to quote Woody Allen. For others – it must be a frustrating experience. Skadden Arps for example enjoyed an enchanted year – appearing at the heart of events on the pages of our newspapers, and obtaining some great results. It won one client – Ron Fuchs - one of the largest ever payouts to an individual at ICSID (and then got him out of a Georgian jail). It also scuttled BP’s plans for a US$16 billion merger with Rosneft. Skadden Arps’ 11th place ranking may not be quite how it feels about its 2011.
Likewise Allen & Overy, whose 12th place belies its practice’s golden reputation. And ditto Arnold & Porter and Fulbright & Jaworski and Baker Botts.
We wish it were otherwise. But if you follow a a metrics based approach there’s only so much one can do, once scrutinising the data is over.
At least all those firms continue to take part. As ever some firms each year decide not to, or else drop the ball when it comes to providing a sufficient level of detail in their questionnaires to make their work checkable, and allow us to therefore include it in the final spreadsheet. Whether that’s deliberate, we never know. We recognise that it’s hard, if you’re a practice manager, to guess exa ctly where you’ll finish in the 30 year on year – especially if you have a practice that’s a little prone to yo--yoing. It’s a long questionnaire to complete and all told not the most comfortable exercise for firms who like to micromanage their image. Still, as the great Wayne Gretzky said: “You miss 100 per cent of the shots you don’t take.” Our thanks to all of the firms who did decide to put themselves on the line. They deserve tremendous credit. In the process, they help to render the world of arbitration more transparent. The next GAR 30 will be out in early 2013.
The GAR 30 is based on a score composed of the sum of various T-scores. The T-scores cover things such as the number of merits hearings participated in two years, the number of lawyers rated highly enough to gain entry to the Who’s Who of Commercial Arbitration, and the value that’s been at stake in the firm’s merits hearings, among other things.
Here we explain more about the columns in the spreadsheet that count towards the final score (and that don’t) and the “accounting” principles our researchers use.
The GAR 30 chart
No. in Who’s Who Legal – the number of members of a firm who achieved inclusion in the 2012 International Who’s Who of Commercial Arbitration. (New to the formula for 2012.)
Pending cases as arbitrator – the number of cases in which a member of the firm is sitting as an arbitrator. Ignores CAS matters, and a few industry-specific bodies. (New to the formula for 2012.)
Merits hearings (any rules) – the number of merits hearings in which the firm participated as counsel or co-counsel.
Value in (merits) hearings – the sum of claims and counterclaims that were argued during merits hearings.
Value in four highest merits hearings – exactly that.
Jurisdictional hearings – the number of jurisdictional hearings the firm took part in. (New to the formula for 2012.)
Pending cases as counsel – the number of cases the firm has on its books as counsel or co-counsel. (Not part of the GAR 30 formula.)
Value of current pending portfolio – the value of claims in the previous column. (Not part of the GAR 30 formula.)
The last two are included for cosmetic purposes only; we include these because the information is always interesting. From a ranking perspective, however, it’s not necessarily reliable.
The GAR 30 formula
The GAR 30 formula uses:
- “pending cases as arbitrator”;
- “merits hearings (any rules)”;
- “merits hearings (sexy rules)”;
- “ratio sexy rules: all rules (merits hearings)”;
- “value in (merits) hearings”;
- “value in four highest merits hearings”;
- “no of hearings with US$1 billion or more at stake”;
- “matrix ratio value: volume”; and
- “jurisdictional hearings”.
- “Accounting” policies
When deciding whether to include a particular reported matter and what value to assign it, use the following rules:
- “If in doubt, leave it out” – ie, items such as merits hearings that were insufficiently described (as in, missing, in particular, either state date, duration, name of opposing firm, value at stake, arbitrators or at least chairman of the tribunal) do not count; nor items that appeared to be duplicates.
- “One dispute in front of numerous multiple panels” (different arbitrators) – count it.
- “One dispute, numerous merits hearings” – count matter once only.
- “Disagreement among opposing firms re the amount at stake” (confined to larger disputes only) – select one firm’s version and apply that to all participants.
- “An award has been handed down in a large case” – substitute the amount arbitrators have awarded wherever the case is mentioned in questionnaires.
- “A firm played a co-counsel role on major case” – apply a discount when including that case in some of the columns (chiefly, cases of more than US$1 billion).
- “Use of barristers on a major case” – ditto.
- “Value of a case so high it is distorting the table” – treat the claim as an outlier. Either assign it a lower value equal to the next highest case in the column, or else use independent research from financial markets reports to establish a realistic value for the case. Apply that to all firms working on it.
The research period for all columns was 1 August 2009 to 1 August 2011.
Q&A on methodology
What’s the source for this data?
All figures used in the GAR 30 are supplied by the firms who want to be in the survey (119 firms sent in data). To count, each matter must be accompanied by sufficient information that it is checkable. If in doubt, we leave it out.
How did the “Bet-the-company hearings worth more than US$1 billion” and “Value in four highest hearings” columns come about?
These columns were introduced to level the playing field a bit between volume and value.
Do you accept the values asserted in ICSID claims at face value?
Initially, yes. But, with claims of all stripes, once there is a known award, that award becomes the value of the case for our purposes.
Why don’t you include ICSID jurisdictional hearings?
We used not to. The policy, however, has changed. See formula above. We now include all jurisdictional hearings.
We had more matters in the bet-the-company US$900 million range than you have counted – why?
What do you do in the following scenarios?
(i) law firms on either side disagree about the value at stake;
(ii) no exact value is given other than a statement “worth billions”;
(iii) requests for declarative relief;
(iv) baskets of Public International Law matters before unique claims tribunals;
(v) test cases;
(vi) hearings “happening soon”; and
(vii) acting as co-counsel.
Some of these have already been dealt with ((i) and (vii)). Otherwise, with (iii) requests for declarative relief, we assessed items individually; with (iv) baskets of public international law matters, we tend not to treat as equivalent to a “bet-the-company”, but would include in the total value in all merits hearings column; with (v) test cases, we took the same approach as with declarative relief; and with (vi) hearings “happening soon”, it’s a simple case of waiting until next year’s survey. Throughout, we have tried to use common sense and caution. A general statement of our approach would be, as before: “If in doubt, leave it out.”
These figures seem entirely self-reported. How do I know that they’re accurate?
We require firms to supply sufficient information about each hearing that can be verified.
Do you weight some columns in the final score?
Yes, we put a weighting on the number of lawyers in the Who’s Who’s and on the number of bet-the-company cases. This is largely to spare the blushes of certain firms; however, they are both important scores so it’s no bad thing to turn up their effect a bit, and it does level the playing field between firms that seek to specialise in quality and those who turn a high volume of low-value work.
Lots of my cases settle – do you take the rate of settlement into account?
No. The GAR 30 regards settlement as affecting all practices equally. International arbitrations do often settle, but it is less common than in litigation. In addition, it would be curious for all the partners in a supposedly big and active arbitration practice to experience a slump in merits hearings at the same time.
Z-scores and T-scores
A Z-score, also known as a standard score, measures an individual’s performance against the average performance of a group in the same activity. It does so by expressing the difference between the performance and the average in a standard unit (the group’s standard deviation). In English: it means you compare a bunch of different results without hitting problems to do with weighting.
What do we mean? Well, consider a class taking three papers: two easy (every student scores between 65 per cent and 85 per cent) and one incredibly difficult, in which the average score is a mere 10 per cent, but one student scores 30 per cent. If one simply added the percentages, the student who scored 30 per cent may not receive due recognition (depending on how they did in the earlier tests). Their “over-performance” may be drowned out. Thus, Z-scores and T-scores avoid this. A T-score takes a Z-score and converts it to a position on a scale of 1 to 100.