Jump to content


Photo

A New Raiding Metric


  • Please log in to reply
31 replies to this topic

#21 terjekv

terjekv

    Von Kaiser

  • Members
  • 38 posts

Posted 08 May 2008 - 08:16 AM

I guess my question would be, are the tools currently available to analyze raid performance really inadequate? I would think that just through WWS and general observation it'd be pretty apparent who's doing their job and who isn't. Also barriers to change aren't nearly as large in WoW raiding as in your example, the NBA.

I'm sure it'd be nice to have more precise tools, but I don't think the additional information you'd glean would be nearly worth the amount of effort it would take to model something like this.


There *is* information missing though, I miss proper time slicing, modeling of specific traits over time (how did the raids dps look over the course of the fight, how did the different bloodlust groups do during bloodlust and so on) and of course, yes, I'd like to compare this data to previous data to see how people are progressing. I'd also like to know exactly how much that feral druids aura contributed to the group he was in. I'd also like to see what selected sets of people were doing when the tank hit 42 health but lived through it. When did the last heal from each of his designated healers land? And then I'd want the possibility to move to their timeline and see where their time was spent, maybe I'll find that a stupid warlock did something that made him scream out for healing and drew the healers attention.

Finding this information isn't hard given the proper tools. The tools aren't even *that* hard to write. They just require an interface that doesn't think of itself as a glorified damage meter.

Yes, I've been working on something down this path, but no, I don't really have anything to show people unless you wish to see perl stubs. I could use a few proper combat logs though, and, well, I'd also like to see chat logs for the same raids at some point.

#22 Yenadar

Yenadar

    Piston Honda

  • Members
  • 106 posts

Posted 09 May 2008 - 04:56 PM

I am actually willing to tackle a few of these ideas. Specifically the Retard Report, Deaths Averted, and Consumables.

It should be fairly easy to accomplish, especially due to the new combat log data, and especially if more than 1 person is running it, and it is syncronizing through the mod channel. If it is also saved from week to week, you can easily show a 'vs last week' comparison, and '4 week average' or something, and tie it into DBM or BW boss engage and boss death alerts for easy distinction. A bit of finess and you could easily make it similar to WWS, being able to look at the whole night,or individual boss kills/attempts.

Retard Report:
It would require information from each boss of the specific abilities, then watch for damage from them, and start tallying up the damage against players as it occurs. Even abilities that people will take a bit of damage from them, and the retard check is how fast you move away, is simply looking at how much someone took. Even random ones, like Mother. If someone gets it every single time, they will definately be high on the damage, but looking at the average per application is easy to see if the average is acceptably low. Granted, this can also be inferred with awareness and gut-reaction, but other bosses not so much. For example, Archi's doomfires. Damage taken from doomfire isn't quite accurate, but you could 1) uses of pvp trinket during fear (buff gained between fear gained and fear faded), 2) doomfire gained during fear, 3) doomfire gained without fear. Here the mod would look for the application through the high damage first hit (given a range, to account for talent reduction)

Deaths Avoided:
Simple enough. A few mods already do the same math with Ardent Defender saves. +1 to a healer when A) the size of their heal is greater than the amount of health remaining on the target and B) the next incoming hit is larger than the health value the tank was at, and C) that incoming hit was within a set period of time after the saving heal.
Can also show the differences between tanks. A tank with a really high # of saved counts vs a tank without puts into numbers healer gut reaction that "tank A seems to take more damage or is harder to heal". Only should be given to raid leaders, since you don't want healers competing to top this meter.

Consumables:
Really only useful on DPS burn fights like Brutallus, but I know my guild leadership would find use for it, since they already look to WWS the next day to check consumable usage counts. That plus the constant "The cauldron is still up!!! Make it disappear!!! Who doesn't have one yet!!!" Would be able to be quickly solved. (Similar to how Blizzard fixed the age old Raid Warning: "LOOT THE DOG!")

Are any of these as needed as a simple meter or WWS? Probably not. Would they still find use to raid leaders? I think so.

#23 clavicle

clavicle

    Glass Joe

  • Allied Members
  • 5 posts

Posted 09 May 2008 - 06:51 PM

That plus the constant "The cauldron is still up!!! Make it disappear!!! Who doesn't have one yet!!!" Would be able to be quickly solved.


Well, assuming the entire raid is using oRA2 or CTRA, you can use their item check commands to see who's missing it.

#24 Mosely

Mosely

    Glass Joe

  • Members
  • 4 posts

Posted 11 May 2008 - 08:39 AM

I assume you are looking for a way to capitalize on the objective nature of WWS to indicate who is "good" and who is "bad" in your raid. There is only one situation where I could see such information being useful: for a raider to gain leverage when applying to a new guild.

I believe that the only meaningful way to judge a player is in fact subjective, and any statistical approach to quantifying a raider's performance is bound to be flawed. Anyone willing to pay attention can quickly develop a sense for each raider's strengths and weaknesses - no need for an advanced WWS report for that. As a result, I am a firm believer that character references are the best raiding metric. Unfortunately, you will not find those in a WWS parse.

#25 Sajukar

Sajukar

    King Hippo

  • Members
  • 655 posts

Posted 11 May 2008 - 01:19 PM

Well, assuming the entire raid is using oRA2 or CTRA, you can use their item check commands to see who's missing it.


Unless, like a large amount of people, they have item checks disabled.

#26 Ghando

Ghando

    Bald Bull

  • Guild Members
  • 1,341 posts

Posted 11 May 2008 - 07:16 PM

Just the other day I was listening to a sports podcast where the host and his guest were talking about baseball statistics; specifically, which stats are the most creative/important and which stats were going to be the "next big thing." I feel like this thread is much the same way. We've gotten so good at gathering data and WWS reports are so crucial to getting the most out of your guild's performance. On top of this, encounters are tuned so tightly (kudos Blizzard!) that as a top-end guild you can't reasonably get by without these metrics. I really like the idea of generating new statistics/metrics by which you can measure raider performance, and as a healer I think you need to be willing to get creative when putting these metrics together.
I write a humor blog: http://idropthings.blogspot.com

#27 Phlis

Phlis

    Don Flamenco

  • Members
  • 491 posts

Posted 11 May 2008 - 10:31 PM

A "reaction time" metric could be interesting, just by comparing time stamps.

One easy thing: The amount of time a curse is on a player during archimond.

Something a bit more complicated: A Player dies, looking back at the time stamps, was their a heal being cast on that player? If so, how soon before he died was it being cast? When would it have landed if the player had lived 1 second longer?

Number of Ticks from Consecrates, Blizzards and Flamestrikes on Council.

There are a number of things which require quick reaction from players in individual encounters which could be measured in this way.

#28 Ellyh

Ellyh

    Don Flamenco

  • Members
  • 324 posts

Posted 12 May 2008 - 06:16 AM

A "reaction time" metric could be interesting, just by comparing time stamps.

One easy thing: The amount of time a curse is on a player during archimond.

Something a bit more complicated: A Player dies, looking back at the time stamps, was their a heal being cast on that player? If so, how soon before he died was it being cast? When would it have landed if the player had lived 1 second longer?

Number of Ticks from Consecrates, Blizzards and Flamestrikes on Council.

There are a number of things which require quick reaction from players in individual encounters which could be measured in this way.


I'm not sure how well this metric can be measured by a mod. Does a friendly player get notified of the start of a cast by another player? if not you would require every single person to be running the mod and comparing timestamps would be exceedingly tricky because of latency. Already healers often see a heal land on their screen but then get informed that the target has died because of latency.

Also without a full log of everyone's actions and conditions you cannot determine what is slow healing and what is simply triage and cutting losses. Sometimes the person dieing is simply not high on the priority list. For example on council towards the end you are down to one raid healer due to random deaths and that healer has the choice of saving a rogue on interrupt duty or random hunter 3. I would heal the rogue and hope the hunter can pot or bandage. To really determine how things were going you would need to also analyse the hunter and the rogue to see if there was something they could have done to survive/take less damage. Simply giving the metric that the raid healer took 3 seconds to start casting on the hunter is not of any value in and of itself.

#29 Ish

Ish

    Von Kaiser

  • Members
  • 41 posts

Posted 12 May 2008 - 12:15 PM

Sounds like an awesome idea, problem with recruiting and maintaining a solid raider core is that you cannot always see the small details regarding a raiders performance, a new tool to evaluate your raiders would be most helpfull

#30 webmeister

webmeister

    Von Kaiser

  • Members
  • 33 posts

Posted 14 May 2008 - 01:46 AM

In reply to a few people above - the raw numbers and reports are never going to give you the full story. Even WWS can be very misleading.

For example, mage X only shows 2 decurses for an entire Archimonde kill. Maybe nobody near him got cursed, maybe he was letting a nearby druid do all the work. What would be more useful to me in this instance is seeing how good each person is at actually decursing - if people are taking 2-3 ticks of the curse before it's getting removed, clearly that isn't good enough.

The consumables report is something I would also like to see focused on who uses health pots/stones and why. Believe it or not, there are still people in my guild who don't use health consumables even when they're going to die; why, I don't know. But it's still an issue for us, and we're working on Illidan.

If anyone is actually thinking seriously about putting these sorts of things together, send me a PM!

#31 caelus

caelus

    Glass Joe

  • Members
  • 12 posts

Posted 16 May 2008 - 10:08 PM

I've been tossing around a few ideas for this sort of thing recently, since my professional life has so far focused on the research and analysis fields. I definitely think WWS needs some sort of analysis tool that can generate reports rather than just numbers.

A few ideas I've had include:
- a consumables report. Who consistently uses potions/healthstones versus who doesn't. Who dies while their pots/healthstones aren't on cooldown (ie they don't use them). Who uses flasks/elixirs/oils/food buffs etc.
- a "retard" report -- people who consistently take avoidable damage (eg Supremus volcanos, Doomfire, etc).
- a decurse/dispell report. Dispell counts by themselves are fairly meaningless; I want to see who is taking damage from dispellable abilities, organised by who eventually dispelled them. This shows you how quick/attentive your dispellers are (or aren't).

Yes, I'm aware that most of this information can be extracted from WWS with time and patience, but having tools that could automatically do this would be a massive help.

I'd put something like this together myself, but I don't have a lot of programming experience :(

Interested to hear other people's thoughts.


It is hard to objectively judge by any of the above results. Can you say someone that has taken less mana pots a worst player? It could be just so that he didn't need it, the consumables report won't tell the difference of needing and not using from not needing and not using.

The retard report is not entirely accurate either. Say Supremus volcanoes like you've listed, no one can control who it spawns on. If it spawns on top of your best player three times, no matter how fast he is, he will take a tick or two, are you going to call him a retard now?

Thirdly, the dispell report, if you have a very fast and dedicated dispeller he is going to dispel a very large portion of the total count. Any "semi-fast" dispellers won't be able to get the dispell off, can you call them bad dispellers just because they are slower than one person but faster than half the world?

Raid performance measured by numbers alone are very sketchy, say heal meters, the lowest healer may not be your worst healer. Reliability is the important attribute, whether each raider is well capable of completing his task with colours when he is required to be under the spotlight, and that numbers and reports can never really tell.

#32 Chewy

Chewy

    Von Kaiser

  • Members
  • 30 posts

Posted 18 May 2008 - 09:14 AM

Our Illidan flame tank went afk during phase one. Does this new metric account for this? Wouldn't that count negatively against the healers?




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users