Reactions to the First Artisans Cup

Basically one must rank order the list of scores to find the high and low values to eliminate. I've tabulated the scores for trees #1 thru #9 to illustrate the Median score, average of the middle three scores (highest and lowest scores dropped), versus the average of all 5.
[table=head]tree a b median d e ave(mid3) ave(all5)
1 38 38 41 43 45 41 41
2 35 36 37 39 43 37 38
3 36 37 40 40 44 39 39
4 36 49 50 50 50 50 47
5 40 45 48 49 52 47 47
6 36 40 42 45 48 42 42
7 36 40 40 43 49 41 42
8 33 40 53 54 55 49 47
9 36 42 45 46 53 44 44
[/table]
One would think that the median and average of the middle three would be very similar, but it makes a noticeable difference when one of these three values is quite different from the other two. Here is the ranking order (first place at the top) of these 9 trees according to each 'scoring system':
[table=head]median mid-3 all-5
8 4 8
4 8 4
5 5 5
9 9 9
6 6 6
1 7 7
7 1 1
3 3 3
2 2 2
[/table]
Rather than trying to figure out how to make your favorite tree come out on top (i.e., practice 'decision theory'), think about which better represents intents. I favor the median.
 
Last edited:
Oso,

Your system doesn't solve the problem I'm addressing. For whatever reason, Boon did not give high scores. So his opinion as to the relative merit of tree Number 1 vs tree Number 3 was not taken into account. Because his scoring was discarded on both trees. In fact his ranking was discarded on most of the trees. So, in effect, the Cup was judged by only 4 judges. (I haven't studied the scores in detail, but I noticed that David DeGroot seemed to score high, so his judging was largely disregarded, too.

Let's say the average score of Judge 1 was 50 points, the average of judges 2, 3, and 4 was 40 points, and the average of judge 5 was 30 points. Within each judges scores is a relative ranking. But if judges 1 and 5 score are discarded, their rankings didn't count.

Perhaps, since all this is "grading on the curve", the average score of all judges should have been determined, and a factor (a standard of devievence) determined so that all judges rankings were "standardized".

Then the standardized scores could be compared for each tree, the high and low elimated.

I know that sounds like a lot of work, but it would be a snap in an excel spreadsheet.
 
For whatever reason, Boon did not give high scores. So his opinion as to the relative merit of tree Number 1 vs tree Number 3 was not taken into account. Because his scoring was discarded on both trees. In fact his ranking was discarded on most of the trees. So, in effect, the Cup was judged by only 4 judges. (I haven't studied the scores in detail, but I noticed that David DeGroot seemed to score high, so his judging was largely disregarded, too.
Ahhh... Sorry, I wasn't paying attention:oops:. You make a very good point!

We could rank order each judge's scores first, then find the median ranking of each tree. That would 'normalize' across the judges and we would then be looking for which tree do the judges think is the best, next best, etc. (as opposed to which is the highest scoring tree).
[table=head]tree w d l p m average of rank median rank
3 1 4 2 2 2 2.2 2
2 5 1 4 1 1 2.4 1
7 2 5 6 3 3 3.8 3
1 3 3 3 4 7 4.0 3
6 4 2 7 6 5 4.8 5
9 6 8 5 5 4 5.6 5
4 8 6 9 8 6 7.4 8
5 7 7 8 7 8 7.4 7
8 9 9 1 9 9 7.4 9
[/table]
One would need to consider more results, but median rank by judge, then median of the ranks across judges would appear to deliver on intents. In this example, I would think tree #2 is indeed the best - three judges thought so. Wheras, only one judge thought tree #3 was the best, but three thought it the second best tree.

Again, I prefer to work with medians, but I've listed the average of the ranks just for fun.
 
My good friend and artisans cup exhibitor John Kirby said this about the scoring:

“I think the Judges did a marvelous job. There will always be complaints, concerns, extra analysis. This was an open, thoughtfully conducted and carefully executed process. The fact you know it was a success is that people are still talking about it and there is still buzz. I had a tree in the show, a really good tree, it didn’t place but I have no doubts about the process. Sure we can debate normalization, dropping scores to eliminate outliers, judges using the full range of scores available, questions about can you score a show that will satisfy 100% of the people who see it. I will say this, the trees that won were awesome trees, the judging worked, yet there will never be a perfect system of judging. What I like about this is that the results are open and transparent, subject to Monday morning analysis, this is an unusual and refreshing outcome of the Artisan’s Cup.”

This was taken from the bonsai bark blog. http://bonsaibark.com/2015/10/07/visceral-wonder-other-comments-on-judging-bonsai/#more-50619
Kirby is a quality guy.

Ironic that you quote him as I feel he was the most harmed by the "throw out the highest and lowest"!

Boon rated his tree as best. His top tree. He gave it a 44. Yet, it was still the lowest score of all the Judges! So, it didn't get counted. Does that seem right?
 
Ok, I'm slow...

I just read the Judging Rubric, and looked at the scores. One thing that jumped out to me is that Boon tended to give the trees a lower score than the other judges. My first thought was that it didn't matter, he still "ranked" the trees.

And then I noticed the Rubric. The highest and lowest rankings for each tree were discarded. And the remaining 3 scores were averaged to give each tree a score.

Hmmm...

On the face of it, that seems reasonable, but then it occurred to me that maybe that wasn't such a good idea.

Since Boon's scores tended to be lower than the other judges, his score was dropped from the scoring of the trees most often. Which means his ranking didn't affect the scoring very much.

I haven't run the numbers, but it appears that David DeGroot's scores ran the highest, so most often his scores were dropped, too.

So, in effect, the Cup was judged by the other three judges.

Again, I haven't run the numbers, but I wonder how the rankings would stack up if the highest and lowest were not dropped?
Way differently--I scoped it this way for fun.
 
Crust, are you saying that you have run the numbers using all 5 judges scores?
 
I got the ranks backwards in post#105 - rank-9 would be first place and rank-1 would be 9th place.
Correcting this, the results using median of the judges rank of the trees, versus the ACup scoring system placements is as follows.
[table=head]tree by median(ranks) by AC_score
1 7 6
2 9 9
3 8 8
4 2 1
5 3 3
6 5 5
7 7 6
8 1 2
9 5 4
[/table]
 
Just my opinion but it doesn't seem right that a judge is allowed to have a tree in the competition.
 
Maybe the Artisans Cup should be called the Ryan Neil show instead.
 
I'm not questioning anyone's integrity, Ryan is a great guy I'm certain. But if you want to rid yourself of all uncertainty, he needs to get a jury of anyone but himself for the entrance of trees, the judges were fine, but also not be involved in any tiebreakers.

People will make all sorts of accusation, when you find out he had worked on a large number of the trees entered it raises debate for sure.

In the end it is probably hard for anyone to think amazing trees got passed over for lesser trees. An amazing tree won in a subjective art form.
 
When I was a kid about 7 years old, my dad ran public relations for a big company. He was in charge of managing holiday events. The plant's annual Christmas Party was one of those events. The party was for 1,500 people and there were prize drawings for the kids. The top prize was a shiny new bicycle. My brother and I not really smart enough to think anything of it, threw our ticket stubs in with about 500 others in a fish bowl hoping to win candy or something -- we'd NEVER win the bike, we just weren't that lucky.
Dad called the holder of "ticket # 9768" or whatever, up to get the prize. My brother who was 8 at the time had the winning ticket. He didn't know any better and went up to collect it.

My dad was mortified and redid the drawing. Nevertheless he was accused by more than a few people of fixing the drawing -luckily his boss was confident in my dad's integrity.

All this taught me that appearances matter, no matter how hard you think they don't--even if you win a bike, or something.
 
Can anyone tell me how to get access to the judges comments. I would love to read these for my own personal enrichment. Are they posted yet?
 
Back
Top Bottom