Jump to content


Photo

SimulationCraft for Shamans


  • Please log in to reply
135 replies to this topic

#121 GrayMatter

GrayMatter

    Glass Joe

  • Members
  • 16 posts

Posted 03 July 2011 - 09:13 PM

As for discrepancies within EnhSim and SimC, the impression I got is that EnhSim just adds a blanket "latency" between all types of actions, and that this latency is typically set to something like 300ms? If that's correct, that's probably a very large part of the discrepancy right there.


Could you give any indication as to how much of a difference the enh 4 piece bonus makes to the overall result?

#122 Mindrila

Mindrila

    Von Kaiser

  • Members
  • 65 posts

Posted 03 July 2011 - 10:32 PM

The issue with the particular brain lag and reaction times I ended up having to use in order to get the sim to match the dummy test is that we still don't feel like it accurately models what is actually going on. 1200 milliseconds is a long time and we don't think our volunteer shaman actually took that long to notice that his MW stack was above 2. I talked to him about it, and what he said was that he often decided *not* to cast LB at MW2 because he saw that SS or LL was about to come off cooldown. Modeling that sort of behavior gets really complex, you'd have to add all sorts of conditionals based on the cooldowns on all the various spells above LB in the priority list. [...]


Actually I was thinking about that a bit, and maybe it would be a start to check whether a cooldown with higher priority has a certain amount left to come off, and then wait for that rather than to do anything lower in the priority list. This would be applied to all cooldown based abilities, which are actions which can be predicted pretty well (unless the cooldown is variable through some mechanism).

So there could be a check like if any action with higher priority has a cooldown smaller than some time then wait. Maybe this would already model the behaviour of players quite well. I also think a functionality like this is important as I find myself also postponing certain actions because Lava Lash or something with higher priority comes of cooldown in a second. Actually I'm not sure if this behaviour is beneficial for my DPS or not, so it would be another nice parameter to tweak. Maybe that's also not too hard to implement (at least it doesn't sound so, but it depends on the actual SimC implementation).

#123 Rouncer

Rouncer

    Deeper Shade of Blue

  • Allied Members
  • 4132 posts

Posted 04 July 2011 - 12:01 AM

As for discrepancies within EnhSim and SimC, the impression I got is that EnhSim just adds a blanket "latency" between all types of actions, and that this latency is typically set to something like 300ms? If that's correct, that's probably a very large part of the discrepancy right there.


Looking at it a bit more carefully, the only way I can get EnhSim to be within a couple of percent of SimC is to run 0ms of latency/reaction time so it would seem to be something else going on. That's with taking the Earth Elemental Totem damage out of the equation since EnhSim doesn't model it and considering my raid leaders love for things that can taunt I'm not sure it's something I want to model. Normal latency/reaction time shows about a 5% difference between the two simulators.

Biggest discrepancies that are constant no matter the latency/reaction time seem to be searing totem where SimC is doing about 300 more dps (7.5%) and Lightning Shield where EnhSim is doing about 500 more dps (48%). Also when I get the latency in EnhSim to a level where the LB and LvB are being cast at a similar level, 50-75ms of latency, SimC is doing around ~6% more white and Stormstrike damage, ~3% more WF damage as well as about 13% more Flame Shock damage even though they have pretty similar action time intervals.

Seems like it would be worth checking on how SimC is modeling Lightning Shield since 3% of overall damage seems very low to me. Do you know if it is modeling a check on both hands for Stormstrike? There can only be one proc per stormstrike but each hand gets a chance to proc a Lightning Shield orb. Searing Totem timing could probably use a check as well. For EnhSim we probably need to check on Flame Shock since I think we were having some issues with how it was modeling haste.

#124 Zakalwe

Zakalwe

    The Chairmaker

  • Members
  • 1466 posts

Posted 04 July 2011 - 01:17 AM

Could you give any indication as to how much of a difference the enh 4 piece bonus makes to the overall result?

You can do this yourself by downloading SimC and removing the 4 piece bonus, either by equipping different gear or simply setting tier12_4pc_melee=0 at the end of the profile.

Actually I was thinking about that a bit, and maybe it would be a start to check whether a cooldown with higher priority has a certain amount left to come off, and then wait for that rather than to do anything lower in the priority list. This would be applied to all cooldown based abilities, which are actions which can be predicted pretty well (unless the cooldown is variable through some mechanism).

So there could be a check like if any action with higher priority has a cooldown smaller than some time then wait. Maybe this would already model the behaviour of players quite well. I also think a functionality like this is important as I find myself also postponing certain actions because Lava Lash or something with higher priority comes of cooldown in a second. Actually I'm not sure if this behaviour is beneficial for my DPS or not, so it would be another nice parameter to tweak. Maybe that's also not too hard to implement (at least it doesn't sound so, but it depends on the actual SimC implementation).

You can do this yourself by tweaking the action list. To ensure you don't cast LB@MW2+ when there's less than a second left of your lava lash cooldown, for instance, you would modify the the current line to read:

actions+=/lightning_bolt,if=buff.maelstrom_weapon.react>1&cooldown.lava_lash.remains>1

Looking at it a bit more carefully, the only way I can get EnhSim to be within a couple of percent of SimC is to run 0ms of latency/reaction time so it would seem to be something else going on. That's with taking the Earth Elemental Totem damage out of the equation since EnhSim doesn't model it and considering my raid leaders love for things that can taunt I'm not sure it's something I want to model. Normal latency/reaction time shows about a 5% difference between the two simulators.

What is this "normal latency/reaction time" of which you speak? My concern isn't as much what exact value you use for this option, it's that there's such an option in the first place, if it does indeed do what I got the impression it does. Now that the spell queue affects all spells and abilities, even following the GCD of an instant cast, the only way network latency affects DPS at all is the aforementioned "cooldown lag". There shouldn't be any latency modeled in between, say, a LvB cast and a LB/MW3 cast, or between a LL and a SS when both are off cooldown, etc. If EnhSim has a crude "latency/reaction time" option that simply adds a fake delay between each ability used, that's obviously very very different from what we do, and much less realistic.

As for the Lightning Shield and Searing Totem questions, I'll forward those to our shaman guy.

#125 Zakalwe

Zakalwe

    The Chairmaker

  • Members
  • 1466 posts

Posted 04 July 2011 - 12:41 PM

Seems like it would be worth checking on how SimC is modeling Lightning Shield since 3% of overall damage seems very low to me. Do you know if it is modeling a check on both hands for Stormstrike? There can only be one proc per stormstrike but each hand gets a chance to proc a Lightning Shield orb.

Turns out we were only checking once - it's fixed now, thanks for that.

#126 Mindrila

Mindrila

    Von Kaiser

  • Members
  • 65 posts

Posted 04 July 2011 - 08:45 PM

You can do this yourself by tweaking the action list. To ensure you don't cast LB@MW2+ when there's less than a second left of your lava lash cooldown, for instance, you would modify the the current line to read:

actions+=/lightning_bolt,if=buff.maelstrom_weapon.react>1&cooldown.lava_lash.remains>1


Thanks for pointing that out, I did some testing with it and it turned out, that only waiting for lava lash might result in a DPS increase. I tried some different combinations and in the end my best result was using:

lava_burst,if=dot.flame_shock.remains>cast_time+travel_time&cooldown.lava_lash.remains>1.5

Waiting in case of lighting bolt did not increase DPS, probably because it's a shorter cast and the DPS gained while casting seems to be the same as the DPS gained through getting lava lash off as fast as possible. This was also the case for the melee profile in case someone wonders.

Waiting in case of lava burst increased the DPS from 36619 to 36667, so it's nothing major but maybe noteworthy. (I was using SimC rev 8988 and the given profile and 25000 iterations).

#127 ziff

ziff

    Piston Honda

  • Members
  • 235 posts

Posted 04 July 2011 - 11:44 PM

I can help with checking out EnhSim to see exactly what EnhSim is doing but I'd like to understand how to get the paperdoll stats out of SimC's output. Looking at Shaman_Enhancement at the Shaman_Enhancement_T12H setup, I'm assuming the unbuffed stats are a good starting point. I do have some questions though.

By unbuffed, do you mean the stats are only from gear and base stats? So this does not include any raid buffs, I assume given the Raid-Buffed column. But does this include buffs given from the shaman themselves, like Strength of Earth totem or Windfury totem? How about personal buffs like the armor type bonus or Flametongue Weapon? How about buffs based on talents like Unleashed Rage? What about primary talents like the melee hit bonus from Dual Wield? Does the Raid-Buffed column include debuffs like Critical Mass or does it only include direct buffs to the player?

The reason I'm asking this is because I can't make much sense of the Raid-Buffed column compared to the Unbuffed column. For example, the Mana for Raid-Buffed is listed as 25595 but for Unbuffed it's listed as 25490. So where is the 2126 mana buff from Arcane Brilliance? Melee Haste is listed as 7.25% for both columns, but where is the 10% melee haste buff from Windfury totem? I see Spell Haste go up by 5.37% from Unbuffed to Raid-Buffed. I assume that the 5% is from raid buffs like Moonkin Form, but where is the extra 0.37% coming from?

I'm assuming this is all just the way the Stats section is displayed but I'd like to get a sense of it so I can do an apples-to-apples comparison between the two sims.

In saying all that, I've been staying on the sideline watching this discussion between SimC and Rouncer. I really like that you are trying to look at the Enhancement shaman's priority list in a completely new way, and I think it's going to lead to better overall DPS both with or without the caster weapon. But there was two comments that I want to address:

Look, you don't fix a simulator by tweaking a single crude dial (EnhSim's "latency" setting) until you get the output you want. I submit that Simcraft has a very clear and explicitly defined relation to what we try to simulate, while EnhSim does not, it just tries to tailor certain details about its output to match WoL reports while not actually caring about the mechanics of how it arrives at that output.

Are you seriously suggesting that EnhSim doesn't care about how it implements it's mechanics? Especially when SimC has brought over some of the models I've worked on for EnhSim, like Flametongue Weapon or Lava Lash? I find it ironic in the same thread that states that SimC is going for +1/-1 damage variance, you find a bug where Static Shock is missing half its potential procs.

Both sims are attempting to simulate the blackbox that is WoW, as accurately as they can. Some of the mechanics are figured out via data-mining, which SimC has done a phenomenal job with. Some mechanics are figured out through in-game testing, like the Lava Lash example above. And some are just done with best-guesses based on combat log parses, like the ICD of Flametongue Weapon or the Fire Elemental cast mechanics. As inaccuracies are found in either sim, they will be fixed so we more accurately model what is happening on Live. Now, as Althor suggests earlier, is EnhSim's reactive cooldown based action model with a coarse reaction time accurate enough in a world of command queuing? Probably not, but I'd really have to sit down and think on the proper action model before proposing a solution in EnhSim. Especially since that area of the code is probably pre-Lich King. But calling out either sim as not caring about what it's simulating just distracts from the argument at hand, which is why isn’t the proposed caster priority queue viable in a raid setting.

I do find it interesting throughout all this, that if I punch in an approximation of the two gear sets and priority queues into EnhSim-2.4.10 I see about a 1K+ damage increase with the caster set-up versus the melee set-up. So while the absolute numbers and PPM of each action don't match between the two sims, the potential of the new suggested priority queue with that gear seems pretty strong. As I get a fully matched up set of gear and stats from SimC, I'll post the two runs just for comparison purposes.

There was also this comment from above:

We did find a bug in EnhSim yesterday that was stopping LvB from being cast as often as it could be even if trying to hardcast.

I'm assuming from the comment that you found a bug or multiple bugs in the code itself. I never got a ticket or a PM though so I can't be sure what exactly you ran into. Now I did fix a problem with Lava Burst casting when it was interacting with the Flame Shock dot, Changelist # 59954. But without knowing what build, config or source file you were looking at I can’t be sure I fixed the problem you ran into. Could whoever found the bug in EnhSim go ahead and try EnhSim-2.4.10 and see if the problem still repros? In the future, if you run into any bugs in EnhSim please make sure you either enter a ticket or PM me. Especially if it’s blocking you folks from doing some comparison work.

Finally, the enhancement shaman community is in the enviable position of having two completely viable simulators to test out rotations, gear setups, etc. As was stated earlier, this is not a competition between the two sims. Both sims have their own goals, theories and implementations. This diversity will allow the enhancement shaman community to come up with new gearing choices, priority setups and casting mechanics, and more importantly break out of established conventions and thought processes. So let’s keep the focus on game mechanics and not the sims themselves.

#128 Rouncer

Rouncer

    Deeper Shade of Blue

  • Allied Members
  • 4132 posts

Posted 05 July 2011 - 12:10 AM

And some are just done with best-guesses based on combat log parses, like the ICD of Flametongue Weapon


The existence of flametongue's ICD was postulated through combat log parses but I did establish it's specific value as 0.15s through testing during Cataclysm's Beta. It was a bitch to test too. Synced different speed weapons with enough haste to get their separation to exactly what I needed and then see what specific speed differential caused it to lose the 2nd proc.

#129 Zakalwe

Zakalwe

    The Chairmaker

  • Members
  • 1466 posts

Posted 05 July 2011 - 12:14 AM

I'll have to take a closer look to answer the specific questions about the unbuffed/buffed stats in the report, but in general, yes, unbuffed is supposed to be gear + base stats while raid-buffed is supposed to be all buffs including the ones that the actor itself supplies.

That comment of mine is clearly taken out of context - I'm responding to what Rouncer says in the post immediately above it, and my post is talking specifically about the concept of a single blanket "latency" option, so it should be obvious that it was not meant as a general statement about all aspects of EnhSim. I have to confess I've personally never used EnhSim, so I know very little about it beyond what I've gleaned from the discussion here.

As for the +1/-1 damage variance mentioned by Althor, that's referring to absolute damage done per cast/proc/swing/whatever, not proc rates or percentage-wise breakdowns of damage done per ability. The former is obviously much easier to test than the latter, as the latter has all sorts of interdependencies with rotation, reaction time, RNG, etc, and claiming to have tested those things down to near-zero variance would make no sense.

The bug Althor is referring to was something we found while I was discussing things with Rouncer last week - I pointed it out to him and it was fixed a couple of days later. So yes, it's that fix you refer to, #59954. At the time I had the impression Rouncer was somehow attached to the EnhSim project, so I didn't think to report it beyond mentioning it to him in a PM.

#130 Zakalwe

Zakalwe

    The Chairmaker

  • Members
  • 1466 posts

Posted 05 July 2011 - 12:48 AM

The reason I'm asking this is because I can't make much sense of the Raid-Buffed column compared to the Unbuffed column. For example, the Mana for Raid-Buffed is listed as 25595 but for Unbuffed it's listed as 25490. So where is the 2126 mana buff from Arcane Brilliance? Melee Haste is listed as 7.25% for both columns, but where is the 10% melee haste buff from Windfury totem? I see Spell Haste go up by 5.37% from Unbuffed to Raid-Buffed. I assume that the 5% is from raid buffs like Moonkin Form, but where is the extra 0.37% coming from?

The spell haste is as expected - the raid buff stacks multiplicatively with haste from gear, not additively, and 1.0725 * 1.05 = 1.126. The melee haste thing is also as expected, because there's a difference between attack speed and melee haste, and the stat report only includes the latter while windfury totem buffs the former.

The mana thing I'm unsure about, I'll have to look into it. Could be a bug, though obviously not one that would affect enhance much :)

#131 ziff

ziff

    Piston Honda

  • Members
  • 235 posts

Posted 05 July 2011 - 01:55 AM

The existence of flametongue's ICD was postulated through combat log parses but I did establish it's specific value as 0.15s through testing during Cataclysm's Beta. It was a bitch to test too. Synced different speed weapons with enough haste to get their separation to exactly what I needed and then see what specific speed differential caused it to lose the 2nd proc.


Oh I remember when you did that and I definitely appreciate you doing that investigation. Coming from somebody who blew a ton of hours trying to come up that silly Lava Lash damage model, I can definitely relate to the effort that took. I was just trying to illustrate the level of inaccuracy of simulation as we get further down into combat log parse, especially when it comes into things like ICDs.

That comment of mine is clearly taken out of context - I'm responding to what Rouncer says in the post immediately above it, and my post is talking specifically about the concept of a single blanket "latency" option, so it should be obvious that it was not meant as a general statement about all aspects of EnhSim. I have to confess I've personally never used EnhSim, so I know very little about it beyond what I've gleaned from the discussion here.

Yeah, I read you comment the same way. I just got a couple of IMs from people mentioning it which is why I commented on it. It's definitely one of the larger gaps between EnhSim and SimC.

As for the +1/-1 damage variance mentioned by Althor, that's referring to absolute damage done per cast/proc/swing/whatever, not proc rates or percentage-wise breakdowns of damage done per ability. The former is obviously much easier to test than the latter, as the latter has all sorts of interdependencies with rotation, reaction time, RNG, etc, and claiming to have tested those things down to near-zero variance would make no sense.

My comment was more associated with the feedback I was getting through the IMs. Like you stated earlier, you didn't intend the comment to be about general accuracy so just ignore my original comment.

The bug Althor is referring to was something we found while I was discussing things with Rouncer last week - I pointed it out to him and it was fixed a couple of days later. So yes, it's that fix you refer to, #59954. At the time I had the impression Rouncer was somehow attached to the EnhSim project, so I didn't think to report it beyond mentioning it to him in a PM.

Glad to see that I fixed the issue. I'd didn't hear anything from Rouncer, which is why I wasn't sure if I fixed the issue you were seeing. Rouncer had stepped away from therorycrafting for abit, but I'm glad to see him back looking into the area.

The spell haste is as expected - the raid buff stacks multiplicatively with haste from gear, not additively, and 1.0725 * 1.05 = 1.126. The melee haste thing is also as expected, because there's a difference between attack speed and melee haste, and the stat report only includes the latter while windfury totem buffs the former.

The mana thing I'm unsure about, I'll have to look into it. Could be a bug, though obviously not one that would affect enhance much :)

Ack, I wasn't looking at my code and I forgot about spell haste being multiplicative. Got tricked since it was so close to a 5% additive bonus. I see what you mean by the attack speed buff, since SimC deals with ranged attacks from other classes. Just in case I'm missing something, are you still treating it as a multiplicative melee haste buff in the end. Or is their some other mechanic you are trying to model?

I guess the only thing I'd like clarification on is if the unbuffed section includes the armor bonus stat buff, the AP/Expertise self-buff from Unleashed Rage and the melee hit bonus from Dual Wield. From looking at the stats, I'm guessing that it's a no to the armor bonus and Unleashed Rage, but yes to the melee hit bonus from Dual Wield. Then I should be able to use the numbers from that output pretty easily.

And thanks for looking into this so quickly.

#132 Althor

Althor

    Great Tiger

  • Members
  • 792 posts

Posted 05 July 2011 - 02:31 AM

There was also this comment from above:
I'm assuming from the comment that you found a bug or multiple bugs in the code itself. I never got a ticket or a PM though so I can't be sure what exactly you ran into. Now I did fix a problem with Lava Burst casting when it was interacting with the Flame Shock dot, Changelist # 59954. But without knowing what build, config or source file you were looking at I can’t be sure I fixed the problem you ran into. Could whoever found the bug in EnhSim go ahead and try EnhSim-2.4.10 and see if the problem still repros? In the future, if you run into any bugs in EnhSim please make sure you either enter a ticket or PM me. Especially if it’s blocking you folks from doing some comparison work.


Yeah that was the bug we found. I think it was Zakalwe who passed it onto Rouncer who passed it onto you.

#133 Zakalwe

Zakalwe

    The Chairmaker

  • Members
  • 1466 posts

Posted 05 July 2011 - 09:19 AM

I see what you mean by the attack speed buff, since SimC deals with ranged attacks from other classes. Just in case I'm missing something, are you still treating it as a multiplicative melee haste buff in the end. Or is their some other mechanic you are trying to model?

I guess the only thing I'd like clarification on is if the unbuffed section includes the armor bonus stat buff, the AP/Expertise self-buff from Unleashed Rage and the melee hit bonus from Dual Wield. From looking at the stats, I'm guessing that it's a no to the armor bonus and Unleashed Rage, but yes to the melee hit bonus from Dual Wield. Then I should be able to use the numbers from that output pretty easily.


Well yes, for enhance it should act like a multiplicative melee haste buff in the end. I believe the main difference between the two concepts is energy regen, which obviously doesn't apply here.

As for the other buffs, you're correct in your guesses - basically unbuffed will include anything that isn't accompanied by an actual buff (aura) on your player. The idea is that you should be able to look at the unbuffed column and see what you see on your paper doll when standing around in Orgrimmar, and look at the raid-buffed column and see what you see on your paper doll when in combat against a raid boss, after everyone has had a chance to get their buffs up but before any random stat procs (trinkets etc) have occurred.

#134 Delboy

Delboy

    Glass Joe

  • Members
  • 8 posts

Posted 05 July 2011 - 12:19 PM

Not sure if this is the right place to post but the Ele Shaman Tier 12 Heroic BIS list includes heroic items that dont exist in the game. These are:

[Rippling Flamewrath Drape]
[Firemend Cinch]
[Infernal Signet of the Avengers]
[Fiery Quintessence]

If this isn't a good place to post this - where can i let the simc guys know?

#135 Zakalwe

Zakalwe

    The Chairmaker

  • Members
  • 1466 posts

Posted 05 July 2011 - 12:26 PM

Not sure if this is the right place to post but the Ele Shaman Tier 12 Heroic BIS list includes heroic items that dont exist in the game. These are:

[Rippling Flamewrath Drape]
[Firemend Cinch]
[Infernal Signet of the Avengers]
[Fiery Quintessence]

If this isn't a good place to post this - where can i let the simc guys know?

In general, the best way to let us know is always to open a new issue, here:
http://code.google.c...eport from user

As for the gear you're referring to - most that was fixed a while ago, but it looks like we may have forgotten to swap out the cloak. Thanks.

#136 Delboy

Delboy

    Glass Joe

  • Members
  • 8 posts

Posted 05 July 2011 - 02:00 PM

Thanks for the response I'll post the defect there for any other issues.

Are you sure though that they were fixed a while ago though as this is present in the Patch 4.2.0 Release 1 version which is the latest isnt it?




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users