Jump to content


Photo

SimulationCraft Model Development


  • Please log in to reply
608 replies to this topic

#1 dedmonwakeen

dedmonwakeen

    Bald Bull

  • Members
  • 1,302 posts

Posted 02 October 2008 - 06:12 PM

SimulationCraft is an event-driven multi-player DPS simulator with modules available for all classes.



#2 sp00n

sp00n

    Bald Bull

  • Members
  • 1,836 posts

Posted 07 October 2008 - 03:28 PM

Are there already any melee mechanics available? Or would they have to be completely (re-)written?

I stumbled across the postings in the mage WotLK threads, and I'd really like to see how rogues compare to the caster numbers.

That said, I have exactly zilch knowledge of C++, so unfortunately am of no help you. I'm not sure if Aldriana has any C++ knowledge, but including his RogueCalc - which is written in Python - into a larger simulation tool would be the ideal solution I guess.

Stopped Playing


#3 Skellum

Skellum

    Bald Bull

  • Members
  • 1,607 posts

Posted 07 October 2008 - 03:40 PM

Is there any chance of formatting the output a bit? I love the bar graphs but I hate the fact that it doesnt include any specific information about the sample data its collected from. By modifying the Simcraft input I can give high crit and haste rates with low +dmg rates and start shunting this all about a bunch of forums,non-EJ, in a rabble rousing attempt.

It would be really amazing if the output automatically provided a link to the data source or was linked to it to where excluding the source data would be difficult to do and be more effort than the average troll would put in to creating a disruption with it.
My Name Is Skellum, And I wear a Silly Hat.

#4 Kalle

Kalle

    Von Kaiser

  • Members
  • 85 posts

Posted 07 October 2008 - 03:51 PM

I found some problems with warlocks in r688:

  • Unstable Affliction was changed to 15s duration, it now has only 5 ticks, new coefficient is 1.0
  • Shadow Embrace debuff lasts for 12s ingame, not 15s
  • Improved Soul Leech has two ranks ingame, restoring 1%/2% of total Mana
  • Duration of Demonic Empowerment is different for imp and for felguard now
  • In trigger_backdraft you wrote "sim -> add_event( this, 15.0 );" and "e -> reschedule( 12.0 );", the second one should also be 15s, shouldn't it?
  • Deathcoil is an affliction spell ingame, not destruction
  • The <25% multiplier for Drain Soul is only present for rank 6 ingame
  • Haunt's crit bonus was lowered to 50%
  • Soul Fire doesn't get base_execute_time / 3.5 for its coefficient. Quick testing shows something between 1.112 and 1.182, probably the 1.15 mentioned here.
  • Dark Pact is not affected by Shadow Mastery ingame
  • Chaos Bolt cannot miss, but I see partial resits - not sure on the mechanics yet
  • Metamorphosis doesn't trigger the gcd ingame
  • Melee attacks from pets may be glancing hits ingame
  • I retested Master Conjuror in build 9038 and the DD/DoT damage increasing portion of fire-/spellstone is still not affected by it

In imp_pet_t:
    virtual void player_buff()
    {
      imp_pet_t* p = (imp_pet_t*) player -> cast_pet();
      warlock_pet_spell_t::player_buff();
      if( p -> buffs_demonic_empowerment )
      {
	player_multiplier *= 1.20;
	player_crit       += 0.20;
      }
    }
Demonic Empowerment only increases the critchance for the imp. I suppose, the player_multiplier part should be removed.

Regarding your combat rating interpolation, WoWWiki has formulas on that.

In the default raid_80.txt, the destruction warlock does not use Corruption but has glyph_corruption=1. For me it was a dps increase to include Corruption in the action list.

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".

#5 Mearis

Mearis

    Mr. Sandman

  • Members
  • 5,455 posts

Posted 07 October 2008 - 03:53 PM

Sent you a PM

#6 dedmonwakeen

dedmonwakeen

    Bald Bull

  • Members
  • 1,302 posts

Posted 07 October 2008 - 05:21 PM

Are there already any melee mechanics available? Or would they have to be completely (re-)written?

I stumbled across the postings in the mage WotLK threads, and I'd really like to see how rogues compare to the caster numbers.

That said, I have exactly zilch knowledge of C++, so unfortunately am of no help you. I'm not sure if Aldriana has any C++ knowledge, but including his RogueCalc - which is written in Python - into a larger simulation tool would be the ideal solution I guess.


This is actually my third attempt over the past three years to model DPS. The first attempt was for Shadow Priests and it used the standard "formulation" method of enforced rotations, etc. This led me to simulation and I opened up support for all caster classes. For WotLK I did a complete re-write to create an architecture that would support melee classes as well.

So far I only have Enhancement Shaman written........ which is somewhat superfluous because there already exists two very good Enh-Shaman simulators! I do intend, barring personal catastrophe, to have all the classes implemented. The class currently in development is Paladin.... with Hunters to follow. (I wanted to get through all the mana-based classes first.)

Aldriana is a very good developer with exceptional insight into the Rogue class, but I highly doubt he will be interested in contributing to SimulationCraft. There are some things that simulators are good at..... and there are some things that simulations are NOT good at.... and Aldriana understands this quite well. Provided you have invested sufficient thought into your model, formulation is much better at delivering consistent EP values. He and I have had many conversations on this subject and we understand the need for both Formulation and Simulation tools.

My thoughts on the subject can be found here: FormulationVsSimulation - simulationcraft - Google Code

All that being said..... I'd be THRILLED to have someone like Aldriana on the team. Detailed knowledge of WoW mechanics is FAR more important than deep familiarity with obscure/obscene features of C++. Indeed, if I knew there was a motivated individual with sufficient experience waiting in the wings, I would change the order in which new classes are added to the sim just to get them involved sooner...........

Is there any chance of formatting the output a bit?


By default, the tool generates TONS of text output...... but I have no way of enforcing people to post that along with the graphs. The SampleOutput page (found by clicking through my sig) has both the charts and the detailed text, as well as a link to the config file. It is capable of generating HTML and WIKI format for the charts..... and I do intend to add the detailed text output to the HTML and WIKI. However, I still can't guarantee that people will host an entire HTML file as opposed to just posting a GoogleChart http link.

#7 Battlemaid

Battlemaid

    Glass Joe

  • Members
  • 21 posts

Posted 07 October 2008 - 05:26 PM

It would be really amazing if the output automatically provided a link to the data source or was linked to it to where excluding the source data would be difficult to do and be more effort than the average troll would put in to creating a disruption with it.

At least, the ethalon SimCraft report: LatestReport - simulationcraft - Google Code doesn't suffer from such issues since it includes links to input and output artifacts of SimCraft in the wiki page text.

However, it doesn't inline simulation artifacts right into HTML, but rather uploads those to the server and stores them next to the report web page. The simplest idea would be to include input/output files in the HTML text, but toggle its visibility via JavaScript. Tho this still leaves the charts images by themselves being incomprehensive and susceptible to frauds that you described.

#8 dedmonwakeen

dedmonwakeen

    Bald Bull

  • Members
  • 1,302 posts

Posted 07 October 2008 - 05:42 PM

I found some problems with warlocks in r688:

  • Unstable Affliction was changed to 15s duration, it now has only 5 ticks, new coefficient is 1.0
  • Shadow Embrace debuff lasts for 12s ingame, not 15s
  • Improved Soul Leech has two ranks ingame, restoring 1%/2% of total Mana
  • Duration of Demonic Empowerment is different for imp and for felguard now
  • In trigger_backdraft you wrote "sim -> add_event( this, 15.0 );" and "e -> reschedule( 12.0 );", the second one should also be 15s, shouldn't it?
  • Deathcoil is an affliction spell ingame, not destruction
  • The <25% multiplier for Drain Soul is only present for rank 6 ingame
  • Haunt's crit bonus was lowered to 50%
  • Soul Fire doesn't get base_execute_time / 3.5 for its coefficient. Quick testing shows something between 1.112 and 1.182, probably the 1.15 mentioned here.
  • Dark Pact is not affected by Shadow Mastery ingame
  • Chaos Bolt cannot miss, but I see partial resits - not sure on the mechanics yet
  • Metamorphosis doesn't trigger the gcd ingame
  • Melee attacks from pets may be glancing hits ingame
  • I retested Master Conjuror in build 9038 and the DD/DoT damage increasing portion of fire-/spellstone is still not affected by it


THANK YOU! I will make the requisite changes ASAP!

In imp_pet_t:
Demonic Empowerment only increases the critchance for the imp. I suppose, the player_multiplier part should be removed.

Regarding your combat rating interpolation, WoWWiki has formulas on that.


Only the Imp is getting buffed. Pets are considered "players" by the core architecture. I probably should have used the term "actor". The pet_t class is a sub-class of player_t. The spell_t class holds the calculated set of buffs/debuffs/etc for each spell. The dynamic values associated with an "actor" are all of the form "player_xxx".

I'm afraid I was very cowardly with my implementation of Ratings-conversion with the hope that most people would run with either level 70 or level 80 players. I will look into the information and put it on the TODO list, but it may get prioritized a bit low for now.

In the default raid_80.txt, the destruction warlock does not use Corruption but has glyph_corruption=1. For me it was a dps increase to include Corruption in the action list.

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".


The Destruction profile has been in constant flux due to the recent code reviews to ferret out the remaining problems with Warlock support. Corruption keeps getting put in and out depending upon who touched the profile last and their particular opinion in the "Is the extra spell worth the benefit."

The statistics idea is very interesting. I supposed an option to simulate for a certain number of CPU seconds (as opposed to iterations) might also be useful. I'll see what I can do.

PS: Sorry for the odd "two topics per quote-segmentation". This is already a back-to-back post and if I made it too granular I'd likely pick up another infraction.

#9 dedmonwakeen

dedmonwakeen

    Bald Bull

  • Members
  • 1,302 posts

Posted 07 October 2008 - 08:05 PM

@Kalle: I've checked in fixes for all the issues you highlighted. Thanks again for such a detailed review. I'll have a new download available later tonight.

The statistical requests will take a bit longer. I have several ideas bouncing around and I need to decide which avenue to pursue.

#10 Battlemaid

Battlemaid

    Glass Joe

  • Members
  • 21 posts

Posted 07 October 2008 - 09:09 PM

"simulate until estimated error < x% for all players".

Back then we had a discussion about variance on our boards (check the post and several following posts here: shadowpriest.com • View topic - SimulationCraft).

General idea is that currently a sequence of averaged simulation output parameters (be it dps or proc uptime) converges too slowly with increasing number of iterations. Mentioned posts attempt to address this problem, however things didnt move further than some not implemented ideas - we had lots of stuff to do at that time.

By the way, while we're talking LatestReport - simulationcraft - Google Code got an update with the latest fixes by Dedmonwakeen. Check it - maybe you'll find some more corrections.

#11 dedmonwakeen

dedmonwakeen

    Bald Bull

  • Members
  • 1,302 posts

Posted 08 October 2008 - 04:28 PM

SimulationCraft version r698 available for download.

Summary of changes: RecentUpdates - simulationcraft - Google Code

Sample Output: SampleOutput - simulationcraft - Google Code

Note: Improved Divine Spirit no longer stacks so DPS will see a minor drop across the board.

#12 dedmonwakeen

dedmonwakeen

    Bald Bull

  • Members
  • 1,302 posts

Posted 08 October 2008 - 07:52 PM

Can you add statistics to the results? Would be nice to know how many iterations will be neccessary to know for sure if talent/glyph/spell x is a damage increase. I would also like an option "simulate until estimated error < x% for all players".


I've recently checked in code that reports standard deviation for player DPS.

In addition, I report "convergence" data: I take the average DPS when the sim has executed 90% of the iterations. I compare this to the final average DPS to report a convergence metric.

Shortly I'll add an option for controlling the runtime via this convergence metric.

#13 Kalle

Kalle

    Von Kaiser

  • Members
  • 85 posts

Posted 09 October 2008 - 03:16 PM

Thanks for the quick implementation of my fixes/suggestions.

My question with the quoted part from imp_pet_t was: Why are you doing multiplier *= 1.2 if the buff only increases the critchance?

When I talked about statistics, I thought about error estimates. Currently you calculate the standard deviation your model produces. With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame. Sure, you only have to divide by the square root of the number of iterations to get the estimated error, but writing something like (3000 +/- 10) dps would make it easier to understand for people not familiar with statistics, I think (10 dps being the estimated error of the average value 3000 dps). This Google chart thing doesn't support error bars, does it?

#14 Battlemaid

Battlemaid

    Glass Joe

  • Members
  • 21 posts

Posted 09 October 2008 - 04:00 PM

With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame

You'll be surprised but even with average_dmg=0 stddev doesn't differ from average_dmg=1. Results for 9 runs of the same config with 10^2 sims, stddev taken from Druid_Balance stats:
average_dmg=0: 3x times 3.0%, 3x times 3.1%, 2x times 3.2%, 1x time 3.3%
average_dmg=1: 4x times 3.1%, 1x time 3.2%, 3x times 3.3%

This Google chart thing doesn't support error bars, does it?

This is what gcharts can do for emulating error bars:

http://chart.apis.go...c6d9fd&chb h=20

#15 dedmonwakeen

dedmonwakeen

    Bald Bull

  • Members
  • 1,302 posts

Posted 09 October 2008 - 05:47 PM

My question with the quoted part from imp_pet_t was: Why are you doing multiplier *= 1.2 if the buff only increases the critchance?


Why? Because the code is bugged and I failed to understand your very clear explanation the first time. :( Fix incoming...........

When I talked about statistics, I thought about error estimates. Currently you calculate the standard deviation your model produces. With average_dmg=1 I don't find that number particularly interesting, because it is something completely different from the deviation you'll see ingame. Sure, you only have to divide by the square root of the number of iterations to get the estimated error, but writing something like (3000 +/- 10) dps would make it easier to understand for people not familiar with statistics, I think (10 dps being the estimated error of the average value 3000 dps). This Google chart thing doesn't support error bars, does it?


Thanks for the feedback.... We're still feeling our way through this stuff.

The standard deviation value I have there is: ( std_dev / avg_dps )

What you are looking for is more like this? ( std_dev / sqrt( iterations ) )

GoogleChart provides a freakish amount of functionality with several options for error bars, I believe. Getting to my current level of understanding almost triggered a brain aneurysm, but I'll jump back into the fray.......

#16 Pitbuller

Pitbuller

    King Hippo

  • Members
  • 626 posts

Posted 10 October 2008 - 02:40 PM

Dedmowakeen: I suggest you use this spec at 80lv and this at 70lvl. Do shamanist focus work in your sim? It dind't lower downtime at all. How about convection?

Rotation:
actions=flask,type=relentless_assault/windfury_weapon,weapon=main/windfury_weapon,weapon=off/auto_attack/lightning_bolt,maelstrom=5/stormstrike/strength_of_earth_totem/windfury_totem/shamanistic_rage/spirit_wolf/earth_shock/lava_lash/searing_totem/water_shield/bloodlust,target_pct=35
This give 4733dps.
Static shock was dps loss becouse mana starving but I bet there is some mana gain problem and it's usable when those are fixed.

Also lava burst isn't castable with maelstorm weapon anymore.

What wrong with searing totem? Only 1500damage per execute? Should be over 15000/250dps.
Sim cast earth shock once per 12s. Cd is still 6s without reverberation. No flame shcok was used.

Or what I do so wrong?
Slow, slower, shaman weapon.

#17 Zurm

Zurm

    The Ultimate in /facepalm Technology

  • Members
  • 1,353 posts

Posted 10 October 2008 - 03:22 PM

I'm currently coding the Death Knight DPS module for Rawr... my C++ knowledge isn't to a point where I'd be comfortable owning a module but I'm more than willing to help someone who does with the math/theory and suggestions on how to model certain mechanics.
Back, semi-casual, and proud of it.

#18 ravenndude

ravenndude

    Von Kaiser

  • Members
  • 32 posts

Posted 10 October 2008 - 04:20 PM

Any chance you can add Glyph of Icy Veins for mages?

#19 Erdluf

Erdluf

    Great Tiger

  • Members
  • 968 posts

Posted 10 October 2008 - 05:48 PM

I'd like to see some support for fights which aren't pure tank & spank. For instance, player input containing

movement_period=20
movement_duration=1.7

might mean that every 0 to 40 seconds (zero to movement_period*2), the player must "move" for 1.7 seconds (cancelcast, and then use only instant-cast spells for the next 1.7 seconds).

pushback_period=5

would mean that a caster gets a pushback event (takes a hit causing spell to be delayed) every zero to ten seconds.

Just as latency favors spells with long cast times, movement and pushback favor spells with short (or zero) cast times. I think simcraft could be a great tool for quantifying that benefit.

Discuss? Thanks!

#20 dedmonwakeen

dedmonwakeen

    Bald Bull

  • Members
  • 1,302 posts

Posted 10 October 2008 - 06:36 PM

Dedmowakeen: I suggest you use this spec at 80lv and this at 70lvl. Do shamanist focus work in your sim? It dind't lower downtime at all. How about convection?

Rotation:

actions=flask,type=relentless_assault/windfury_weapon,weapon=main/windfury_weapon,weapon=off/auto_attack/lightning_bolt,maelstrom=5/stormstrike/strength_of_earth_totem/windfury_totem/shamanistic_rage/spirit_wolf/earth_shock/lava_lash/searing_totem/water_shield/bloodlust,target_pct=35
This give 4733dps.
Static shock was dps loss becouse mana starving but I bet there is some mana gain problem and it's usable when those are fixed.

Also lava burst isn't castable with maelstorm weapon anymore.

What wrong with searing totem? Only 1500damage per execute? Should be over 15000/250dps.
Sim cast earth shock once per 12s. Cd is still 6s without reverberation. No flame shcok was used.

Or what I do so wrong?


Pitbuller, thank you very much for the feedback. Enh Shaman hasn't gotten much oversight compared to the other classes..... and I didn't want to distract to much from Enhsim's efforts.

The plumbing for Shamanistic Focus seems okay...... I'll dig in. Downtime comes into play due to waiting on duration, cooldowns, and resources. Unfortunately, I do not distinguish between those in the report..... so it is unclear just how much is due to lack of mana.

Thanks for the LvB catch. I also need to update the Maelstrom mechanics to PPM. I saw so many reports on the subject..... but I must have missed the final analysis on #PPM and the affects of haste.

Searing Totem: First, I haven't implemented the new ranks. Bah! Second, I neglected to call player_buff() during the execute stage which means it isn't picking up the spell power/crit., just the target_debuff() stuff. Bah! Sorry dude, stuff like that shouldn't escape into the field. :(

I don't see Flame Shock in your action list...... Also, since Earth Shock has a lower priority than say Stormstrike, perhaps it is possible that there may be stretches of time (between shamanistic rage) where you are severely mana limited??? Basically.... you'd never get to Earth Shock because Malestrom Weapon and Stormstrike are eating up the mana.

I need to do a full review of my mana costs for Shaman spells.... perhaps they are just too high.

I'm currently coding the Death Knight DPS module for Rawr... my C++ knowledge isn't to a point where I'd be comfortable owning a module but I'm more than willing to help someone who does with the math/theory and suggestions on how to model certain mechanics.


Thanks! It may be a while before we get to Death Knight, but I will certainly remember your offer when the time comes.

Any chance you can add Glyph of Icy Veins for mages?


Definitely. It will be in the next release for sure.

I'd like to see some support for fights which aren't pure tank & spank. For instance, player input containing

movement_period=20
movement_duration=1.7

might mean that every 0 to 40 seconds (zero to movement_period*2), the player must "move" for 1.7 seconds (cancelcast, and then use only instant-cast spells for the next 1.7 seconds).

pushback_period=5

would mean that a caster gets a pushback event (takes a hit causing spell to be delayed) every zero to ten seconds.

Just as latency favors spells with long cast times, movement and pushback favor spells with short (or zero) cast times. I think simcraft could be a great tool for quantifying that benefit.

Discuss? Thanks!


We've talked about this amongst the development team multiple times. It really isn't very difficult to insert events that cancel (or push back) current casts and resubmit a "player_ready" event some number of seconds in the future.

The only real hard part is coming up with a reasonable set of controls. As you imply, we would want to be able to tie probability to these events. minimum lengths, cooldowns, etc. If you can hammer out a comprehensive set of controls that are not unwieldy, I'm sure we'll find the time to implement them.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users