I'm cross posting this cause it is quite probable that many players don't head into the Rules form, not wanting to deal with what can come when passionate gamer geeks start debating what a rule means XD

We have been collecting game results to start building a comprehensive statistics listing on how effective various heroes are, and how deadly villains and environments are - among many other statistics. It is entirely for fun (at least for me! lol) and I am certain with time, many interesting thing's will come of it.

The Project Thread

Check it out  Here: https://sentinelsofthemultiverse.com/forum/topic/the-best-heroes-a-statistical-community-effort-2798

Due to increased real life commitments, I haven't been able to keep this updated. We are STILL taking data! Don't stop! Arathorn's version below is less frequently updated, but far more intense in its statistics. Keep putting that data in!

Alternative, awesome view by Arathorn:

http://public.tableausoftware.com/views/Sentinels1/Overview?:embed=y&:display_count=no which 

The Form to enter your own games:

to fill out your own games is linked in that thread  and also here:

https://docs.google.com/forms/d/1zCJwaHbUeS9VqbIhqZj2zAJ1TuWq5E0lnHWndheq1IM/viewform#start=invite

**Please Note, due to the huge number of calculations I have that run of that sheet to make my life easier in updating the Tracking Report, that form can take a while to run through its submit -20-30 seconds is not uncommon. Please be patient and only hit the Submit Button Once.

**Please Note the Second: I have done what I can to fix this - but it still takes 10-15 seconds or so to submit. Please be patient.

**PPlease Note the Third: Now on new  Google Sheets, which should be far far far faster.

The Official Monthly Report

Thanks to the hard work of Arathorn, a very comprehensive comparison study is being undertaken with the results. Since I could not continue to handle the work with my Real Life commitments, Arathorn has graciously taken over. He updatest his fairly regularly, and it has GREAT information - Including a page with all the raw data on it! Amazing report, and great information if you trust these sorts of things ;)

Fiew it here: http://public.tableausoftware.com/views/Sentinels1/Overview?:embed=y&:display_count=no

The Naive Bayes Difficultly System

Thanks to the hard work of mkgrey, the stats are being inputed into a naive Bayes Difficulty calculation. This works by a point system - the more points, the harder the game, the less the easier. Of course more data is needed to make this really effective, so like above, keep putting in more games!

Find it here: http://x.gray.org/sentinels-of-the-multiverse-difficulty-scores.html

rkcr took this model and put it into an android randomizer app. See it here: https://play.google.com/store/apps/details?id=com.idunnolol.sotm&hl=en

Want to see some general data of your Entries? Or Specific Hero/Villain data?

**Please Note: This no longer works. Its going to be back but not for a while.

Want to Comment about the Project?  

If you don't mind leaving this thread for the official updates, that be great. Head over to Here to leave comments and suggetions and anything else you want to leave, so we can keep it all in one place (The official thread on the Rules forum) Thanks!

Want old information for Comparison?

Head to this Dropbox to get the information. You can even download the raw data (including all my formulas for sorting out information) if you want to find some specific values yourself that I don't compute! (ie, you want to see how many times The Wraith has played with Legacy and Omnitron X against Baron Blade... you can! I leave it up to you to write your own formulas tho :P)

Over 4000 Entries as of 7/5/2013!

(actually 4138- we're clearly averaging between 600-700 entries a month, or 27.4 perday)

July 2014 - Removed outdated report links, updated Arathorns report to be the Official Report.

July Update - 4138 entries!

June Update - 3489 entries!

May Update Blurb - 2823 entries!

April Update Blurb - 1817 entries!

March Update Blurb - 934 entries!

Take a look! Have some fun! and certainly, put your own games into the tracker for next month!

*Noting Feburary Update. Will note this thread sooner   in march :p

Okay! The March update is on the sheet! Go check it out!

Some interesting developments this month! The Chairmain has lost his top spot of the Most Dangerous villain. Excluding Promos, Matriarch has gotten a nice lead... but if you keep the promos in Cosmic Omnitron had decimated the heroes this month, surging forward into the 50% danger range!

Baron Blade has fallen down into the high 20% range, showing that as we get more games that fluke of him being super dangerous is leveling out. Poor Akashbuhuta still rounds out the bottom of the villain pile though for wins, despite being the second most villain played. Well, I guess thats a good thing - we like the earth still standing.

Heroe wise, Tempest, Legacy and Wraith remain the most popular heroes to play - and Legacy (excluding Promo, playtesting) remains the most powerful hero. Bunker has fallen to the least likely to win a game with a mere 63.29% win rate, followed by Absolute Zero at 64.32 - wraith has jumped from her bottom slot into the solid middle - the high 60's range.

A new stat was added on the Most Incapacitated column - if said hero was incapacitated, did the team still go on to win? Again, excluding Promo and Playtesting heroes, you msot certainly want Unity on your side, with a 25.45% chance that if she gets incapacitated you're still going to win the game... but not haka, with a mere 10% chance to still win the game after he goes down... Maybe that just means he's just that crucial to victory?

Rook City remains our Top Contender for most dangerous enviroment, and if you want an easy day hit up the Realm of Discord.

Cool stuff happened this month - The Self Destruct Sequence went off in Wagner Mars Base! As well as Omnitron getting his first defeat with Distributed Code still in play.

Looking for an Easy Game?

You Could try Legacy, Ra, Tachyon and Tempest, in the Realms of Discord against Akash'Bhuta (straight percentages)

Really want to stack the Deck? Akash'Bhuta has never defated a tea of 3, and never in ... well a lot of environments. So the above should be pretty dang easy!

Looking for a Hard Game?

Take on the Matriarch in Rook City, with Unity, Mr Fixer, and Nightmist -- Excluding Promo's those 3 heroes and that enviroment is where the Matriarch is the strongest!

0:23:13 and 0:23:22 , 3/19/13

Are duplicats (sorry, didn't know you can't hit back)

The Tracker has been updated for April! Go check it out!

Wow! Shattered timelines is here and amazingly what changes we see now! You can tell people did a lot of playing with all the Shattered Timelines stuff at once - Chrono Ranger and OX both have over 20 games with ST Villains but less than 10 each with the others. Still the games are getting out there and we're seeing some fun new things!

Heroes

For instance, now that people have got their hands on Team Leader Tachyon she has quickly become a favorite - shooting up to 140 games in the last month, and taking the lead above even Legacy for Heroes with more than 100 games - 85.21% victory rate, almost 3% more than Legacy! looks like Tachyone makes a better Freedom (#) team leader than the ol Parsons!

AbZero is still having trouble, hovering around the 65% range, but Wraith has shot up into the 70s, which seems to be the Median for heroes. That said, Even Expat and Fixer, long said to be the 'Weakest' are only about a percent and a half from hitting that middle ground where it seems that most heroes are found.

Eternal Haka continues to be the worst hero to drop out of a fight - if he does, his team has never won once he's hit incap. Excluding the low value promo heroes, Chrono Ranger and Omnitron-X have over a 35% win rate after gettting Incapped. If someone has to go down, these two seem to give the best!

Villains

As most expected, Iron Legacy is now dominating the difficulty chart, with a 50.94% loss rate against him. Cosmic Omnitron is still right behind him, with both around the 50 game range. Chairman and Matriarch arent quite giving up their Rank 4 status however, giving a strong showing at 47%.

Ambuscade has apparently turned into the jokester of the bunch, droppiong down to a mere 9% loss rate against, with Miss Information and Kismet dropping below Akash too - I guess Choas Bound had a good month in destroying heroes. I do hope to see the Assistant pull up a little stronger, and Kismet as well - Both have been brutal in the games /ive/ played against them so Im hoping they see a stronger showing in months to come!

Environments

Rook City remains dominate, and Time Cataclysm is the new set's current favorite with over 100 games. Silver Gulch proves to be the most dangerous, but Ive yet to see anyone Lost in TIme... however we have had 2 Self Destruct sequences go off! (Hate to break it to a one of you guys, that /is/ a loss :P)

Other

4 Heroes seems to remain the sweet spot for getting the magic number of 2/3rds wins (What we can now reliably draw as the average win rate) managing such even against Iron Legacy (65% win rate with 4 heroes)

I may be adding a few more data sets - that is, some more organization in the near future - Such as groupings by expansions to see just the expansions data, and perhaps averaging out data amongst expansions to see which expansion leads what categories. Look out for new categories to appear in the Report to be filled in next month

Also, I'm now saving copies of each months report, and the raw data on my computer. I may put them up in a DropBox or something too for general consumptions - this means ill be able to make comparisons month to month (or others will) to see how they fair

Again, if you havent seen it yet, the Raw Data sheet has a personal entry tracker - go to the last sheet, put in your username, and change the second field to on, then go back a sheet to see your entries. NOTE! if you put your username in differently, well youll be SoL - if you add a space to the end accidently, it will read it as a different username. There isn't much I can really do about it, unfortunately. Try not to mess up your Username!

Oh, and i went through -all- the points and removed some duplicate games i found, so a few of the smaller datapoint groups (ie, some promo heroes) may see a few games lost, but for the most part no difference will be noted.

Wanna Try a game no one has done yet?

Wraith: Rook City, and The Eternal Haka (and one more) against Baron Blade in the Final Wasteland has 0's all around in there

Or Bunker Engine of War and Wraith; Price of Freedom  (and someone) against Spite in Silver Gulch 1883 is another set with a  lot of 0s'. Wanting a challenge? Try looking for the lowest Percentage groupings and try your luck!

Not that statiscally matters much, I wonder how many of those 4 Heroes are played by 2 players.

Its just from what I have seen 4 minds colliding is much differnet than 2 people having to manage 2 Heroes.

Nice data.  I took it and generated a naive Bayes win/loss predictor from it and translated that into a "difficulty points" system to estimate how hard any given game is:

http://x.gray.org/sentinels-of-the-multiverse-difficulty-scores.html

Fantastic!  The data is obviously not completely solid yet though.  No way Miss Information is the easiest villain or that Mad Bomber Blade is only slightly more difficult than his normal version.

I love it! But spiff is right, several of them still need a /lot/ more data to be properly analyized.

Yeah, I agree some of it seems a bit off (particularly Miss Information), but in the data she has (by some margin) the highest win rate (93%) for the heroes, and over more the 50 games.  But, I suspect there's some sample bias, which is hard to correct for, but I also suspect that some villains feel harder even though their actual win rates are quite comparable to others.

I'll run it again in a couple months once there's more data.

Added a Dropbox link ( https://www.dropbox.com/sh/9t6pyl7zb76wony/ynMJKqGZWP ) to get the archived data that ill be storing. This includes the Raw Data up to that point, if people want to fool around with custom searches, like Specific teams and such.

Once the data gets to the point where we think it's reliable, I'd love to add this to the Randomizer so that when it shows you the random lineup for a game it can also give you an indicator of how easy/hard that game is likely to be.

That would be cool.

Anyone from the company that makes Sentinels Sidekick care to comment on the plausability of adding functionality to Sentinels Sidekick so that it could facilitate reporting the results of a game?

While waiting for the May monthly report to come out I downloaded a copy of the raw data spreadsheet and started pulling some info out of it. I was looking at Environment/Villain pairs. The number of games per pair is still too low to be able to make any statements about the results but I was surprised that there are some pairings that have never occurred.

Akash'bhuta has never appeared at The Block despite there being 151 games played there.

The Chairman has never appeared at The Final Wasteland after 166 battles fought there.

Spite has never participated in a battle at Silver Gulch after 140 battles occurred there.

The Chairman has not appeared in the Time Cataclysm. There have been 206 games there.

Iron Legacy has been played twice at the Pike Industrial Complex and once at the Ruins of Atlantis. He has never been defeated at either location.

Cosmic Omnitron has never wreaked havoc at Rook City after 209 games played at Rook City. La Capitan has only menaced the city twice but won on both occasions.

Looking forward to the official report :)

FYI, I've built an updated version of the naive Bayes difficulty model (and made it so I'll be able to update it more frequently), and updated the web page.  There's no huge changes, but I added some data on some advanced villains.  In general, there's not enough data to draw firm conlusions about villains on advanced mode, but in a few cases, they (preliminarily) appear to be so much harder than their base villain it was worth mentioning.

Both Miss Information and The Ennead rise in difficulty profoundly on the advanced setting, seemingly.  But, those conclusions are based on 18 and 40 games respectively, so it would be good to get more data.

It's interesting, given people's initial reaction, that basic Miss Information continues to be the easiest villain (tied with Gloomweaver), with only 6 hero defeats in 86 games.

The gap between Baron Blade and the Mad Bomber version has widened meaningfully with more data.

http://x.gray.org/sentinels-of-the-multiverse-difficulty-scores.html

Update for everyone 5/5/13

This is a very busy week for me at work. I thought I'd find some time to finish this out over the weekend, but didnt.

Tonight, I got the Villians, Indivdiual Villans, Heroes, and Environmentpages updated with the current data. I saved the data I am working with to update the Individual Hero Page soon. It probably will be weds.

But the data that everyone really wants, how 'effective' a hero is or how 'dangerous' a villain is there and ready to go.

Interesting tnote, as Heores start to hit the 350-500 game marks, Legacy has certainly lost his top mark to TlT and even The Scholar, and Yes. The Dreamer was more dangerous than her previous information indicated, jumping up to a 32.09% loss against rate.

also, it seems a little less than half the games played are solo. Interesting.

As I said, Ill be updating the 'Indivdiual Hero Page' come Weds.

Hey all - I took mkgray's score system and integrated it into an Android app for randomly selecting game setups: https://play.google.com/store/apps/details?id=com.idunnolol.sotm

I did a test run of it last night and it worked great; the first mission was a bit too easy (at around 75%) and then we bumped it up to a lot harder difficulty (40%) and it was quite challenging.

Enjoy!

That is very cool. Thanks!

May Update

Time Time time!

May found me far more busy than I expected. Work, Progress, and a lot of other things in life delayed the Update for 5 days. Even then I wasn't able to finish it. However, I have moved forward with a diffrent plan.

The Individual Villain and Individual Hero pages on the report have been removed. I have instead replaced them (well at this time just hero) look up function simlar to the look up your username function on the Statistics page. If  you go to that same link for the Username look up, you can also select from a drop down menu of Villains and Heroes, selecting one will allow you to view the stats for that hero or villain individually,just like  the stats on the individual pages. 

This is kinda awesome in two ways - 1) I don't have to do nearly as much work and 2) We can pull information I wasn't pulling before. You can see now that you can look up the actual individual advanced rates for each hero rather than just the generic. And thats just the start. I can probably  roll in some formula's to find out other aspects, like the best 2 pair combos of heroes, or other things that may be of interest to the avid gamer.  

More still needs to be added - i need to add the Team Size and the Position trackers to Heroes, and do all the Villain  formulas. But over the course of this month Ill get them added. And even better, that is all live data, up to the minute with all the current entries (so you may see some weird numbers on occassion - people often put in incorrect data, such as saying Heroes Incapacitated when everyone was alive, or saying Win=Nowhen it was a yes and other things that I often have to fix.)

In other news, we had 2800 games on the tracker. (as of today were over 3k in the raw data)

That's just amazing! Amaaazing!

That's huge Lynkfox. I know we say this all the time, but thank you, thank you, thank you for the work you put in! We all really appreciate it and enjoy the fruit of your labor.

Middle of May Update!

As Pointed out on the BGG Forums, the data between Normal and Advanced for  villians really needs to be clearly seperated. Overall is kind of a useless stat. SO that being said, theyve been seperated.

https://docs.google.com/spreadsheet/ccc?key=0AqClP82b_XgZdGJKUVJRakJmZ3RMX2pFUjB3MEtBWHc#gid=1

you can now see the Normal Games first, and then the Advanced games second!

I haven't updated the By Difficulty below, but i will in the future.

In other news, the Look Up A Hero section is finished, being able to look  up Total games against a villain or just advanced (it matters less for a hero if it is adv or not, because the hero plays the same), Environment, and Position and Team Size.

as well as Flipped and Flipped Percentage

I've updated the difficulty estimator page and json data files.  No major moves since the last model, with only a few things moving more than 15 points.

Bunker, Engine of War, Eternal Haka, The Scholar and La Capitan both appear to be harder than previous estimates.

Freedom Tempest and Horus of Two Horizons both appear to be more difficult than previous estimates.

Another interesting milestone is that there's now enough data for the many of the Advanced villains to validate the assumption that the advanced side is harder than the basic side.  For some of the villains with fewer than 30 matches recorded, there's still an inversion, presumably due to sample selection issues (players who know what they're doing are more likely to choose Advanced villains).

We still need more Advanced games with several villains, many of them already-hard-on-Basic (The Chairman, Iron Legacy, The Dreamer, Cosmic Omnitron, and The Matriarch) but a couple not-too-hard (Spite and Mad Bomber Blade), all of which really need more games to estimate their difficulty.

Additionally, the data suggests La Capitan's advanced mode isn't much harder than Basic, which seems like we need more data.

As far as the data suggests, we can also identify the hardest possible game (using basic villains), and the easiest possible game:

Hardest: 3 players, Mr. Fixer, Bunker[Engine of War], and Absolute Zero[Elemental Wrath] vs. Iron Legacy in Rook City.  Expected win rate: 8%

Easiest (without 2 Legacies): 4 players, Greatest Legacy, Team Leader Tachyon, Dark Visionary, and Omnitron X in the Final Wasteland vs. Gloomweaver.  Expected win rate: 99%

The strongest hero team vs. the strongest villain/environment pairup should be pretty easy for the heroes, but noone has tried.  The reverse (weakest heros, weakest villains) should be more of a typical matchup, but again, noone has tried it yet.

http://x.gray.org/sentinels-of-the-multiverse-difficulty-scores.html

Another observation:

Of the games recorded, the heroes have never lost if the net difficulty was below -276 (and there have been 38 games played with lower difficulty than that).  A single game of TLT, DV, O-X and Tempest lost to Miss Information with a difficulty of -276.  That's the only loss below difficulty -224, out of 150 recorded games.

Similarly, the heroes have never won if the net difficulty was above 203 and only two matches (out of 46) have ever won with a difficulty of 172 or higher.

Hmm... If I did my calculations right, Bunker, Fixer, and Fanatic against Chairman in Ruins of Atlantic is at 206 should yield a victory fairly easily. Just have to get Bunker and Fixer incapped so Bunker cna keep returning Wrathful Retribution into hand and Fixer keep putting Aegis back into play. Might take 3 Wrathful, but it is doable. As long as Fanatic gets lucky with her card draws.

I do not understand how Plague Rat can be so high.  I seriously think I can beat him with any combination of heroes trivially.  I've soloed him with just Nightmist (using H=3).  He seems Gloomweaver-level to me.  Not to mention there is no way he is above La Capitan.  She's vastly more brutal.  And Omnitron is above her?  Not a chance.  That isn't even a close comparison.  Clearly I need to submit more game.

Well to be fair, all nightmist needs is Mistform and the amulet against a villain with no minions and just a ton of one high damage one shots. I wouldn't base your opinion of the Rat around a particular hero who is perfectly built to handle that kind of situation.

I have not seen that difficulty estimator before. I really like it! Gonna book mark that and use it for my next few game nights. Great work.

Omnitron also has a bunch of hugely damaging/debilitating nuke cards that can really ruin your day, especially if they come out at the wrong time.  Tech Singularity can easily kill equipment-dependent heroes, especially on advanced or with damage bonuses in play.  Likewise Sedative Flechettes can wreck most defensive strategies.  And if you happen to get Electro-pulse Explosives played during the environment turn, you're generally going to either be in a world of hurt or TPK'd.  It's usually not the drones that kill heroes, it's the board wipes and the spike damage.

Using a single hero (even with H=3) is not indicative of a balanced game, certain cards will miss the Hero entirely and certain cards will hit that Hero more than once. If you were playing a 3 player game and it came down to just Nightmist in mist form tank for the win then you've lost 2/3rds of your team, I can't see how you wouldn't consider that a hard fight.

YMMV. Thats why we do the stats.

Also, Plague Rat does Irreducable damage, and with the right combination of infections and limited ongoing removal, he starts doing 3-5 or more damage to each hero each turn. And thats without any of his one shot damages. Ive seen him take legacy down to less than 10 hp in the space of 2 turns.

I suppose it's possible.  I just beat him with Bunker, Haka, Mr. Fixer and Absolute Zero at Pikes last night.  And I've never played AZ or Fixer, and Haka and Bunker only once.  It was a sloppy game as a result and I lost Bunker (stupidly...I miscounted damage by one) and AZ, but AZ was shut down by the "all dmg toxic" Environment Vats anyway.  But Haka and Fixer were basically impossible to kill at that point, despite both being infected.  I just find that without a +dmg environment, Plague Rat doesn't get enough damage boost of his own to make him a threat.  I actually did this game with the worst heroes I had available to see if maybe I had always lucked into a great counter hero against PR.  He still feels just a step up from Gloomweaver.

To be fair, Mr. Fixer is probably one of the best heros against Plauge Rat. For what thats worth.

Are you keeping Infected cards on incapacitated heroes?

Yes I am.  I'm wondering if I'm doing something else wrong with him (or in general), because I just can't see losing to him except to extremely bad luck or lack of experience.  Maybe I should post a play-by-play thread to see if someone catches something wrong.  Heck, I didn't even learn about Jack Knife on Mr. Fixer and how it works with Plague Rat until after that game (I used the chain tool to negate most of his one-shot damage).  But Incapacitated Bunker's return a card from trash power plus Haka with two Ground Pounds meant I was invulnerble the rest of the game.

Um, doesn't Ground Pound only negate non-hero damage? So heroes would still be hitting themselves if Infected. The thing that tends to get us with Plague Rat isn't so much that he does lots of little hits of damage (though that is one aspect). It's more that most of the damage is irreducible, so stuff like Fortitude, Stealth, and Ta Moko do nothing. Plus buffs like Galvanise become unwelcome as soon as someone becomes Infected (unless that someone is, say, Fanatic, who can build up for a Wrathful Retribution, or Mr Fixer, who can Jack Handle or Driving Mantis it elsewhere). It also doesn't help if Afflicted Frenzy comes out, because then heroes start hitting each other, and continue to do so even if they become incapacitated as long as they're Infected.

We find Plague Rat to be one of the more difficult villains, up there with the Matriarch, La Capitan, and Kismet with the villains who regularly beat us. Maybe it's easier with more heroes but we only use three.

Neat, I'll update the app at some point.  Do you mind if I have the app hit the JSON directly or should I rehost?

I've got two requests for the JSON data, if you've got time:

1. Validate the JSON so that it can be parsed without modification.  Right now it's got trailing commas so it needs a lenient parser to work.

2. Add "advanced" points to the villains that have it.  (I've done it internally in the app like so: { "name": "Baron Blade", "points": -55, "advanced": 20 }).

Thanks!

I've updated the json generator to be more strict.  If you do reasonably aggressive caching (ttl >= 1 day) feel free to hit it directly.

I've added the advanced points, but only for villains that have at least 30 games logged, and added a note on the ones with both relatively low data and suspicuous deltas.

I've also added advanced to the json, unfiltered, but I also added "advcount" which specifies the number of matches against that villain in advanced mode.  I would be very skeptical of all villains with advcount < 50 and would disregard all with advcount < 30.  More specifically, I'd use a minimum difficulty for advanced villains as basic+75, unless there's >75 games logged, and only Baron Blade has that many.

For example, even with >50 games, Apostate/Advanced appears to be barely more difficult than Apostate/Basic, which seems possible, but unlikely.

I've also updated to the latest version of the spreadsheet.  No huge moves, but Gloomweaver seems harder and Omnitron seems easier than before.

Advanced Apostate being not much harder than Basic isn't unlikely; it's a rougher start, but if you deal with the Tome at the start then it's less likely to pop out when the Runes and/or Orb are protecting it. It's probably the smallest gap between basic and advanced of any villain.

There's a problem with "scale" now - missing trailing brackets.  I usually verify my JSON with http://jsonlint.com/, helps a ton.

I'd like to think that was a game I recently logged .  For someone who was theoretically so easy he obliterated us…  though I blame Rook City, screw that environment.

Fixed the trailing bracket issue.  It should validate now.

Ill be updating the Stats tonight (Monday night)

Sorry. I got home from work tonight and pretty much collapsed into bed. 

June Stats Posted! (6/3/14)

The June Stats are up. Remember, Ive removed the individual information pages - you can look them up directly from the Raw Stats file by going to the last sheet.

You can also download the sheet for your own use from the dropbox. Don't forget that!

Poor Miss Information has dropped to the bottom of the pack, but she is still pretty brutal in Advanced mood. Speaking of advanced mode, I got a PM just as I started to update the form, Vex85 said he had 100++ advanced games to enter. So they willbe in for the next month!

And the stats have gone about and radically changed advanced wise as well - Lots of flux there, but were getting up to the 100games area per villain (or... mostly there) so that should start to look pretty good.

Iron Legacy still dominates the top, but Cosmic Omnitron is much harder than his rating suggets. The Dreamer has really coem up into her own as well. 

We also had our first 6 hero game against Iron legacy! For a win! Congrats, FryGuy! He left a comment - I thought Id share it

Iron Legacy at the Time Cataclysm using Team Leader Tachyon, Tempest; Freedom, Wraith: Price of Freedom, Bunker Engine of War, Absolute Zero Elemental Wrath, and Golem Unity. No Incaps!

 

"Jeez this fight is insane.

 

Picked Time Cataclysm because Metropolis doesn't make much thematic sense (Even Iron Legacy in his evil mode isn't targeting innocents).

 

Tach was at 14 before she took a turn. This was won mostly on the back of some nice interactions with Unity and pushing Legacy under his HIT EVERYONE side. Stealth Bot and a couple Throat Jabs basically managed to stop the group from dying outright, and Bunker lost both his ammo drops before getting to use them. Unity had Swift going, and then raptors to keep him lowish, while AZ slowly pushed him into the low 10s at which point an Omni-Cannon managed to blow him out of the air. 

 

Would have been all but impossible if Unity hadn't started with Pylon to get Swift and Stealth down at the start. Also starting with a Final Evolution that hits everyone for 3 is not nice. No chance they would have won this on Advanced."

For the "Omnitron's Devices After Omnitron died" game-end condition, the summary lists all 10 as defeats, but in fact there are 3 defeats and 7 victories with that condition.  Is that intended to be a failure-only condition rather than a generic one?

nah i probably put it in wrong. I’ll fix it tonight

In heroes tab, Team Size (Normal), Heroes = 5, games = 283, wins = 297!? 104.95%?

The 4 player advanced Matriarch was mis-entered. Correct one follows it.

Yeah i guess i never updated my 5 Hero Team Win formulas to remove the advanced wins from it as well. Its bene updated.

287 237 82.58%

July Update

July 4th delayed the update a little bit, with family obligations and what not. But the update is now available for your purusal. 

Not much to say in this one! I dont remember if I got around to correcting the Omnitron snafu yet (about devices inplay and wins/losses) but if I can keep it in my head Ill sit down and look through the code tonight after work.

Other than that, stats are still fluctuating a little bit - you can see Miss Information climbed a few ranks and the Dreamer is tearing em /up/. As usual, these new version is available in the drop box for comparison to last months. 

Enjoy!

Hey all,

I love the work that's been done on capturing a bunch of games. I took a shot at putting them into a form that allows more detailed investigation than the initial reports being created. They are up to date as of July 22. I've not been able to establish a reading connection to the output google document (yet). I think it may be possible, but I've not been able to yet.

Anyway, you can see my efforts at http://public.tableausoftware.com/views/Sentinels1/Overview?:embed=y&:display_count=no which I think provide some interesting insights (enough so I'd love to see this linked to on the front page).

Some initial insights:

• The Dreamer is the worst villain for moving up in # of heroes. She's pretty easy with 3, tough against 4, and brutal against 5. No other villains comes close. The Ennead is the only other one that is harder with more players.
• Citizen Dawn is pretty even against any # of players.
• Wagner Mars Base, in particular, seems different depending on the villain. Seems easy for the heroes to beat the Ennead there but much harder for Mad Bomber Baron Blade.
• The previous observation, particularly, but many others, are very limited by sample size. MORE GAMES, PLEASE!
• There may be some "good" and "bad", statistically, match-ups for heroes and villains, but, again, sample size. Some interesting data, at least.
• Picking heroes vs. choosing randomly didn't have as big an impact as I expected. On the order of 2-10%.

I'm sure lots more people can discover more, if they are interested.

Arathorn

(Edit: Fixed link)

i have just started going over this, but AWESOME work. I know how much time and dedication a simple thing like this takes. And it gives great info for scenario-based campaigns or how to avoid theme-traps beyond the obvious ones.

I assume the blank spots in the various amount to "no data available".

Yeah, blank spaces are "no data". Glad you're enjoying it, PePe. Any feedback or thoughts on improvements to it?

The one thing I can think of it's adding a way to match Hero/Villain/fun factor (or is it how the game felt?). Maybe environment/Villain too. Maybe it's too subjective to be useful.

Pretty awesome man.

Are you using the Report or the Raw Data that I throw down into the Dropbox folder? I ask cause i notice you have one 2 hero game - there are actually like 30-40 2 hero games entered, just that the data for them is on a seperate page. If you have one then there is one i missed pulling out... XD

lynkvox, I'm using the data from the google doc. I wasn't aware of the dropbox one. I, sadly, haven't even read all the posts in this thread. Can you point me to that location again, please? (Though I will again.)

PePe, are you looking for something like http://public.tableausoftware.com/views/Sentinels1/DifficultyViews?:embed=y&:display_count=no ? That should also be available as a tab from what I linked before.

Arathorn

if you're using the googledoc, keep doing so. The drop box lags a month behind, as I only update it when i update the report.

But if you want the raw data as an excell file to be easier to work with, you can get them at the dropbox here:

https://www.dropbox.com/sh/9t6pyl7zb76wony/ynMJKqGZWP

i also linked that doc into the top post of this thread (and finally put in a link to the naive bayes score and the android app for it as well, thanks mkgrey and rkcr)

Awesome. Yeah, I only have a tiny fraction of the two-player games.... I'm not sure I'll get those added before I go on vacation. Thanks for all the work! Great data and input method.

Arathorn

Awesome work, Thanks for the effort

I updated with games through this morning. Same link. I included all the two-player games, too. I didn't do any data corrections, so multiples may exist. 

I'm not having any luck getting a real-time connection to the google document. Google's API for accessing things is ... not as robust and complete as I'd like. It might be possible to set up a side SQL database and sync them all the time, but I'm not quite sure I have all the resources necessary to do something like that. If someone else is interested in doing this, I'm willing to help and provide ideas. It seems like it involves setting up something like Thunderbird with an account subscribed to changes in the document and then automatically downloading the new document and putting it into a SQL DB.

Let me know with other requests for reports.

Arathorn

Hey guys. 

Its been a very. busy month for me. Between getting ready for a trip to Europe next month, my work place moving entirely to a new location (a local catering company) and said work picking up quite a bit, i didnt get to update this month. 

Now the gdoc doesnt seem to be accepting responses anymore - and Im not certain whats causing that

Thanks to Arathorn keeping the data going - ill try to find some time to work out the kinks, and we really should try to find someone with SQL knowledge to get us something more robust!

I remain super excited about this project. I hope that one day (maybe in the next year? who the heck knows) I'll have the time to set up a SQL database and REST API or something, hosted on the GTG servers. That way you all could build more robust and clever frontends and displays of data.

Taking the Sentinels statistics data, adding them to an SQL database, creating a RESTful API frontend for the data, and creating clients to consume and display this data.  This sounds like all sorts of wonderful joy.

Note: there was no sarcasm in this post.  Not even in this statement declining said use.

Updated with data through this morning. At http://public.tableausoftware.com/views/Sentinels1/Overview?:embed=y&:display_count=no%C2%A0which#1

Note: I tried to get some hero A with hero B joint comparisons going, but I couldn't quite make it work. Sorry.

Arathorn

Well, while I don't have the time to really update the statistics, I did go through and update the form for Vengeance and the Dark Watch Promo's. 

Of course Vengeance complicates things quite very much. 

Very much

But such is the life of this project.

I removed all the formula's from the data, except the individual look up formulas (which have NOT been updated for Vengeance yet, be aware) that should help move things along a little bit.

Google Spreadsheets is also updating to be faster, stronger better - but sheet protections are not in yet. So i am not willing to put it up yet into the new format - because tampering is a real concern and breaking the form by too many people playing around is a possiblity.

So anyways there is that. Arathorns update isnt as often as would be awesome, but at the very least, the stats are getting collected. When someone *cough paul? cough xD kidding!* has time to put up a real SQL and API and what not stuff that be great, but at least until then the stats are being collected. :)

I wrote a script to see how the various heroes and villians are impacted by the player count (as I didn't see that done elsewhere - may have missed it) - see attached. Interestingly, some villians get much easier (with my metric - Omnitron, The Chairman, Plague Rat, The Matriarch, Iron Legacy, Voss) with more players, some stay about the same (Spite, Akashbhuta, Apostate, Dawn, Kismet), and The Dreamer gets much harder (and is actually the hardest 5p villain). I think this is mostly due to the extent of H icon usage and/or group vs individual attacks.

Similarly, some heroes get much better in larger groups (again by my metric - Visionary, Bunker, Tempest, Legacy, Wraith, Fixer) some get just a bit better (Haka, Chrono, Argent Adept) and some stay about the same (Absolute Zero, Nightmist, Omnitron-X). I think utility like villian deck manipulation and Legacy's base power gets better with larger groups, and heroes that have basically their own game plan seem to be just as good in 3P as in 5P. Some of the results for the heroes surprise me - I would have guessed Absolute Zero would add more to bigger teams b/c he gains so much when his damage is pumped. 

Sentinals_Stats_2014-02-12.xls (39.5 KB)

very cool!

I hope in the next couple of weeks to be able to work on getting the sheets updated again. No idea when this will be, but hopefully Ill be able to find the time to do it in a more effecient manner allowing for easy updating.

If anyone out there has the knowledge for converting google docs into SQL Databases, please let me know. Arathorn and I don't have that knowledge unfortunately making it harder to update using the Tabelau software.

If I get the time, I could convert the spreadsheets into a SQL schema and, indeed, would love to do so at some point, and also help you guys out with a hosting solution. I'm super busy most of the time, though, so if you happen to get to it before me, I would like to recommend going with PostgreSQL instead of mysql or something else.

Having work with both (tangentally), I second the vote for postgres over mysql.

Just out of curiosity, is there anyway to know if the document submitted correctly? I've just filled it in for the first time and after the last page it just went to a blank white screen, does it usually do that?

Not usually, it will usually say 'Thanks!'

go ahead and put it in again, i police the document for duplicates. Also, being the older googledocs, it doesn't always work properly... i hope they put in some sort of migrate tool to migrate from an older version to the new gsheets

ALSO! I've been working on updating the statistics again. Its hard going work,because in order to make it not so stupidly processive intensive, ive removed the formulas from the webdoc, to put them into an actually excel sheet. But gsheets and excel use similar... but different formulas. Most notably, array formulas are done  ENTIRELY different (its an actually formula function in gsheets, and in excell you have to end a formula by hitting ctr alt enter - and array formulas are how I am able to even begin to sort the information)

so ive had to rewrite most of the formulas I used to parse the data. Its slow going, but if you are interested in...

here is the villains as of today...

https://docs.google.com/spreadsheets/d/1tX46pXuJYoK21LVMXM8eHRkbLffUkR_-h4nP2X97hT0/edit#gid=0

Not pretty, but you can at least get an idea of where we have been going over the last few months since the last update!

I've worked with MySQL, but not PostgreSQL. I like MySQL quite well.

Lynkfox, why are you going with "loss percent" rather than "win percent" when the numbers look like the winning percentage?

Because its villains. The Heroes have won that many times, yes, but the villains lost!

HAH!

we have hit the limit of Cells allowed for an old version of Google Drive Sheets. (40,000) I am working on converting the sheet tot he new version without loosing any of the data. This may be difficult, but the new version runs faster, smother, better, and has no limit (well, 2million they say, but should be able to go above it (they say) ) 

So for the time being the response form has been disabled. When i get this figured out, I'll provide the new link to the new submit your data form.

New Submit form. Please use this form from now on!

https://docs.google.com/forms/d/1zCJwaHbUeS9VqbIhqZj2zAJ1TuWq5E0lnHWndheq1IM/viewform#start=invite

And ive put up the individual hero data.

https://docs.google.com/spreadsheets/d/1tX46pXuJYoK21LVMXM8eHRkbLffUkR_-h4nP2X97hT0/edit#gid=0

Heh, Citizen Dawn is the most popular villain by far.  Everyone wants a piece of her.

Only seven Vengeance Five games so far...not that surprising, of course.

The Heroes vrs specific Villains has been posted on the work book

https://docs.google.com/spreadsheets/d/1tX46pXuJYoK21LVMXM8eHRkbLffUkR_-h4nP2X97hT0/edit#gid=928215062

The link in Sentinels Sidekick isn't working right now, but we will fix it pretty soon.

I took the latest data in the worksheet (might be the old one, I dunno) and created a few minimal reports at http://public.tableausoftware.com/views/SotMVengeance/VillainWinPct?:embed=y&:display_count=no

I think I have improved my turn-around cycle by about 15X, so I should be able to do more things and update more quickly now. 

Lynkfox, where is the new data being stored? And does it have 2-6 hero stuff all together?

Arathorn

FWIW, I don't have a stake in the MySQL vs. PostgreSQL debate. I've used 'em both. Either will work fine for me to link up with, once I get all the necessary expansions into SQL instead of doing them manually. I made good progress on that in the last couple weeks (slower for this update, hopefully faster to do new updates). But I lost a bit of data I'd gotten manually. I need to figure out how to get # of heroes accurately with my new ingest mechanism going.

I've not used the new gsheets at all. I'm glad we won't be running into space issues.

Also, BTW, if it's tough to get all the new data together with the old data, I can pretty well do ingests from multiple sources, once I get a few things ironed out.

Paul, though, you could make me hugely happy with a SQL DB getting all the information in. HINT HINT! :P

Arathorn

Its the same data lay out as the previous - limitations in the manner in which data is collected via the google sheets, unfortunately. Could probably put it through a Table converge function, maybe even on the same sheet, but not sure how much that would slow it down. the new Sheets are better for running processes, but still... over 7000 data entries really can throw a kink in it. ;)

Lynkfox, one thing that I kinda almost need is unique headers for the columns. Right now, the hero incapicated column is the same for each hero. I don't know much about getting data into google sheets from a collection questionaire, but would it be possible to change that to Hero Incap-1 and then Hero Incap-2, and the like? Else, I can't pull in the data I want (well, can't is too strong, but I don't know how to). It all gets jumbled together. Same thing with "Flip?" for the Vengeance Five.

I still think I've got some good improvements on my turn-around time. I need to test a few more times and then I'll try to get everything put all together.

Arathorn

Er. thats not easy.

The headers are set by the Form. Each header is the first line of the Form Question. Changing the name of the Form question might create a whole new column. Ill do some tetsing to see if thats the case and see if I can switch those around, but I'm not sure how flexible that will be.

Wow. That was surprisingly easy. Yay new Google Forms. the Old forms would have made new columns when I changed the titles, but these new ones successfully just rename the columns. Should be much easier for you to parse the data!

Fantastic! Thanks for doing that. I can do more analysis now. Eventually.

For now, I've just updated http://public.tableausoftware.com/views/Sentinels1/Overview?:embed=y&:display_count=no%C2%A0which#1 with data through today. It seems a bit goofy, but I've not had the chance to full validate my results. Still, overall, things seem pretty close.

A couple other things in the data collection form I'd like to see changed:

1. "Ra: Horus of the Two Horizons" switched to match the earlier "Ra: Horus of Two Horizons"
2. "K.N.Y.F.E." only has the final period for hero 1. Elsewhere, she's "K.N.Y.F.E" -- I'd like to see those match exactly. Would make life a lot easier.
3. "Young Legacy" shows up sometimes and "Youngest Legacy" sometimes. I thought it was "Young Legacy", but the important thing is consistency. Once the form is updated to always be one or the other, then we can update the stored records to all match.
4. I think there might be one place with a "Dark Watch Setbac" on the form. The rest are all Setback (with the k), but I had to correct at least one.

I plan on doing a bit more with villain/hero frequency, position order, hero incapaciation, etc. in the next week or so (I hope). I also think I have my updating process down to under an hour. We can maybe make it even faster, if you're willing to eliminate some of the text lines breaking up data.

And, yes, I think I do have all the two-player data, too.

Thanks for the collection and such!

Arathorn

Those names have now been changed on the Form, and ive gone through with a find and replace to switch them over. Should all be correct now! Let me know if you find anything else out of order.

I also replaced all the Yes, Both Sides, Yes, Side A and Yes Side B with just Yes, for advanced mode, for making it easier to parse the data through.

Thanks a million, lynkfox. I found and corrected a couple errors (that might not have been seen). But I'm still having a couple. Regardless, we've made fantastic progress in the last week!

Arathorn

I think I've fixed all my errors. And I got the updated data, so it should be nearly completely accurate at this point.

I'll try to update weekly. But feedback on what people like (and what they want to see more of!) would probably help encourage me. I still should do some Incapicitated work and some work on the Vengeful Five. Anything else? 

And, all, please USE THE TABS across the top. There are LOTS of reports!

Arathorn

For the Heroes tab, can you show not only the overall win %, but also how many entries that data is based on?  Especially with the new heroes from Vengeance, I'd like to see how often they've been played relative to the other heroes, so I know when the sample size is big enough to trust.

Also, the entry for Youngest Legacy is showing 100% win, which I'm sure is an error.

Good idea, squad. I added game count reports to a few sheets. And updated with latest data (which corrects the Youngest Legacy problem). And added a new tab (at the end) which shows SOME stuff with two different heroes. So, you can ask, does Legacy work well with Argent Adept, for instance. That's still not the clearest sheet, but it shows a bit of analysis in that regard.

Arathorn

Oh WoW! i love the ability to filter it by username. So awesome! Now I know who i haven't played against whom! :)

Thanks!  And great idea about the two hero comparison!  I'll definitely put that information to good use.

First off, thanks for making this available - it makes me unreasonably happy to have tools for slicing and dicing data.  (Why yes, I do find myself growing more and more fond of Parse, why do you ask?)

Secondly, feedback for Arathorn:

• It would be nice to have some global filtering available that extends across all reports, such as (de)selecting specific heroes and/or villains, preferably in a multi-checkbox dropdown list like you already have on several reports.  Excluding certain options in a given report one by one sometimes carries over to other reports, but not always.  Centralizing that functionality would be convenient.
• It would be nice to have the ability to mass de-select groups of heroes/villains by expansion.  For example, I have no promo cards at all.  For awhile I only had the base set and the Vengeance expansion.  While it's great to be able to look at the data on cards that I don't have, I'm usually more concerned about the ones to which I do have access.  I've just started getting some friends into the game and it's helpful to give them some broad overviews based on this data - e.g. "Taking on The Chairman in Rook City, while thematically amazing, is likely to be very painful."

Can I get access to the raw data?  I've made stats sites for race and dominion before.

http://jburnim.github.io/rftgstats/

http://councilroom.com

At long last, I have finally updated the charts again. I didn't do anything more than just update with new data. I changed computers, and it made things complicated, but I should be in a position to be quicker on updating again.

I really like the idea of being able to filter by expansion as well. I'm not quite sure exactly how I'll implement that, but I know it's doable. I just have been slammed at work and slammed at home. Home should be easing a bit, at least. 

Later all!

Arathorn

Hey guys, Matthew Gray (the guy running http://x.gray.org/sentinels-of-the-multiverse-difficulty-scores.html) doesn't have access to the raw data he needs to update his site anymore. Can we do anything to get him access to that?

Tell him to send me an email from the gdoc address he'll use to access them (lynkfox at gmail) and i'll add him up.

I think this project is awesome and I have been adding a lot of games over the last few months. I really appreciate the work that went into getting this all set up. Now that it is Mid-November and I see Arathorn was last in the forums 4 months ago I began to worry that the sheets will no longer update and any activity added is now in vain. Does anyone know if these awesome sheets will be updated? If not, is there another way to see the raw data?

Thank you

the sheets are sometimes updated randomly, but we always come back to them. It may go by the wayside for a while, but it always comes back :)

I was just hoping to update tomorrow....  It's always on my mind; life is just busy.

Updated! Thanks for the reminders, all!

Arathorn

Ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh-

*breathe*

hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh!

I can't submit statistics for games with the SS Tachyon in them, this is a disaster!

Do what I do, put it in as Tachyon and put in the comments that is it SS Tachyon, that way it can be fixed when SS Tachyon is added!

Thanks so much for the update and this project

I completely forgot to get off my butt and update it. ><

Super Scientific Tachyon

is now added as an option after Team Leader Tachyon in all relevant fields.

I jumped the gun a little bit, since we have the names of everyone from WotC.

All the environments, heroes, villains, and promo's are up on the Survey now. Playtesters, if you are using the final proofs then by all means, put the data in, but if you arent then don't :P

have fun everyone!

(if anyone see's something i forgot and or missed or see's that i missspelled something, let me know!

Forgot the end parenthesis.

I just thought of something.

how the FRACK are we going to Statistic 10 new Vengeance style villains? SON OF A....

considering I think there is something like 3003 variations of all 15... gah....

It's a kind of magic...

I was looking at the tables on the Tableaux web page in detail today, and found something that confuses me.  So hopefully Arathorn or someone else will see this and take a look.

If you look at the "Villain" tab, as of this moment, you can see that column headings are 

"Villain", "Heroes Win?", "Villain Loss Pct", "Total Number of Games"

Now look a the lines for Kaargra Warfang compared to Infinitor.  In both cases, the "Total Number of Games" is 3.  However, the "Villain Loss Pct" is 33.33% for Kaargra and 27.27% for Infinitor.  For Kaargra, this makes sense.

But for Infinitor, this seems impossible.  If there are really 3 total games, then the only possible values for "Villain Loss Pct" are 0%, 33.33%, 66.66% or 100%.  So where is that 27.27% coming from?    Now it's possible the column is mislabeled, and should actually say "Total Number of Games Lost to Villain".  If that were the case, then that would mean there are 9 total games for Kaargra and 11 total games for Infinitor, except...take a look at Progeny.  Progeny shows 3 games and 38.46%.  There is no whole number ratio with 3 in the numerator that gives that percentage.  (3/7 = 0.4286, 3/8 =0.375).  In fact, 5/13 = 38.46%.  So where is the 3 coming from?

Now look at the "Environment" tab.  It has columns labeled...

"Enivironment", "Heroes Win?", "Hero Win Pct", "Total Number of Games Won in Environment"

Now look at the line for Omintron IV.  It shows the "Total Number of Games Won in Environment" is 4, and the "Hero Win Pct" is 72.22%.  Except, again, this is not possible.  There is no whole number denominator X for which 4/X = 0.7222 (4/5=0.8 and 4/6=0.6666).  In fact, this percentage is 13/18, and there really isn't any way to reasonably manipulate those two numbers I can think of to get to 4.  

So something seems screwy.  It's easy to see the problem with the new Wrath items because the total number of games is still small.  but I have to assume that the same problem exists on all the other lines, it's just harder to see. 

I'm curious as to what might be happening.  Thanks for your time.

Hans

Look at the Hero vs. Villain Win Pct for Infinitor.  Data is available for 3 and 4 heroes, or for 4 heroes.  For 3 & 4, the win % are:

Captain Cosmic 50%
Guise 66%
Parse 100%
PW Fanatic 100%
Sky-Scraper 66%
Sentinels 100%

For 4 heroes, the win % are:

Captain Cosmic 100%
Guise 100%
Parse 100%
PW Fanatic 100%
Sky-Scraper 100%
Sentinels 100%

So the 3-hero game that was a loss had to have been Captain Cosmic, Guise, and Sky.  CC was only in one of the 4-hero games, which was a win.  Guise and Sky were in both 4-hero games (wins).  Parse, PW Fanatic, and the Sentinels were each in one of the 4-hero games.  Hypothetically that would give us this:

LOSS:
CC
Guise
Sky

WIN:
CC
Guise
Sky
Parse

WIN:
PW Fanatic
Guise
Sky
Sentinels

That's a total of 11 hero vs villain matchups, with 3 losses for a villain win % of 27.27%  Captain Cosmic's win rate is 50% (one out of two games), Guise and Sky-scraper are at 66% (two out of three) and Parse, PW Fanatic, and Sentinels are each at 100% (one out of one.)

Following your example, you seem to be saying that the percentage that is shown is not…

of games villain has appeared and lost/# of games the villain has appeared

you seem to be saying it is…

of games villain has appeared and lost/# of heroes total in those games

That could be how the percentage is actually calculated, but that number seems unhelpful.  The units of the numerator (games) and denominator (heroes) are different, so I'm not sure how you would even intepret it in a useful way.   It also isn't what is suggested by the column heading which uses the word "games".  Why would you use one denominator for the hero percentages, and a different one for the villains?  Finally, if by chance a particular villain was played more frequently with 3 hero groups, and another with 5, it would mean that the first villain would have an artificially inflated win percentage and the 2nd deflated, even if their game win percentage was the same.

Also, if that were the case, it still doesn't explain how you go from 3 of something on the Progeny line to 38.46%.  That just isn't possible.  38.46% has to be representing 5/13.  So 5 of what and 13 of what and where is the 3 coming from?

I'll provide one more example.  I looked through the underlying data to find one of the older villains whose percentage was an obvious repeating decimal where one could calculate what the underlying fraction must be.  Iron Legacy has a pct listed of 44.57%, but when you look at the underlying data you can see this is actually 44.566544567%  That's pretty clearly a repeating decimal, and some quick math tells us that fraction is 365/819.  But the number of games listed there is 439.  Where is that coming from?  819-365 is 454, so there is no help there.   What is going on?

I believe that all of the numbers in the table are by some definition correct.  But I am now almost certain that the column headings are not describing what is actually in the column.

That... is a question for Arathorn, I'm afraid. I don't understand all that stuff he does with the Tableu.

I will, however, check the actual data points for Infinitor and make sure something didnt get entered wrong.

Edit:

I think, what happened, was just a delay in all the data.

Before 1/8 (when the tableu was last updated) i see 4 infinitor games. However, two of them were on the 8th. it is quite possible that while updating or messing around with the data, Arathorn ended up with one game getting added in.

Currently, there are 30ish games with Infinitor in the data. When he updates it again, It should even out.

Nope.  I'm saying the formula is # of hero vs. villain matchups with a LOSS / # of hero vs. villain matchups TOTAL.

Think of it this way (formatting is going to be horrible, I'm sure):
[table]

[tr][h]Hero[/h][h]Villain[/h][h]Outcome[/h][h]Game ID[/h][/tr]

[tr][td]Cosmic[/td][td]Infinitor[/td][td]LOSS[/td][td]1[/td][/tr]

[tr][td]Guise[/td][td]Infinitor[/td][td]LOSS[/td] [td]1[/td] [/tr]

[tr][td]Sky-Scraper[/td][td]Infinitor[/td][td]LOSS[/td] [td]1[/td] [/tr]

[tr][td]Cosmic[/td][td]Infinitor[/td][td]WIN[/td] [td]2[/td] [/tr]

[tr][td]Guise[/td][td]Infinitor[/td][td]WIN[/td] [td]2[/td] [/tr]

[tr][td]Sky-Scraper[/td][td]Infinitor[/td][td]WIN[/td] [td]2[/td] [/tr]

[tr][td]Sky-Scraper[/td][td]Infinitor[/td][td]WIN[/td] [td]3[/td] [/tr]

The total number of records is 11.  The total number of hero losses is 3.  The total number of distinct Game IDs is 3.  The total villain win % (which is the header on the report) is 3/11 or 27.27%   If the 3-hero game would've been won and one of the 4-hero games a loss, then the villain win % would be 36.36% for 4/11, still with 3 games.

3 of something on the Progeny line is pretty straightforward too - 3 games, one with 3 players, two with 5 players, where one of the 5-player games is a loss.  13 total records, 5 of which are a loss, with 3 distinct game IDs.

Or, thats what it is - which would make some sense, though thats now how the records are stored in the statistics file, but I know Arathorn does some translating of them so they work properly in tableu. Again, unfortunately, I'm not really certain how that works out.

Hmmm…ok, that makes sense, as to what is happening behind the scenes.  It seems strange to me that that percentage would be based on matchups, but then you would list the number of actual games (instead of either the numerator or denominator of the percentage), but the math works out.

If that is what is happening, then I suggest that the labelling on the columns could use some work, or a few footnotes.   I think that percentage itself (matchups over matchups) is not very intuitive, as it seems to me most people will be thinking in terms of "If I play in a game against Iron Legacy how likely am I to win?", and that number is not quite that.  It seems odd that the percentage there should vary based on the size of the games lost, as you demonstrate in your example.  The fact that the column lists games instead of matchups increases this non-intuitive feeling for me.  On the other hand, with large numbers of games played, the proprotions should even out (assuming there is no bias in the size of games played against certain villains/environments) to be something close to the game over game percentage, where the actual numerators and denominators of the percentage are all roughly 4 times the number of actual games played.

But your explanation makes sense to me, Edge, thanks for taking the time to think this through.  In the case of Iron Legacy, it is possible that by chance the true fraction is actually a higher multiple of the reduced fraction that gives that percentage.  In other words, its actually something like 1095 winning matchups over 2457 total matchups, which 3*(365/819), with 439 game ids.

There could be Villain Win % and Villain Win % by Game as separate columns with the different calculations - that's something for Arathorn to consider.  If you look at the Villain Win % for a specific number of heroes at a time, I think it should work out to be the same number for those two calculations.  

Because villain difficulty often scales with the number of heroes (most get easier, at least one gets harder) it's probably more accurate to give an overall win rate based on matchups than by game count.  If people want better estimates of success, they need to supply more data for the question.  Asking how likely they are to win a game against a villain can often have an answer that varies wildly if they only bring three heroes instead of five.

Glad my explanation made sense :)

I've looked more closely, EdgeDS and others, and I am sure EdgeDS is right.  The percentage on the Villain tab is matchups that were losses/total matchups, and not based on number of games.  When you get the underlying data, the number of records in the underlying data (which is a list the losing matchups, with games having a number of rows equal to the number of heroes) is clearly the  numerator of a fraction that is equal to the quoted percentage.  The resulting denominator is about 4 times the number of games, which is what one would expect for the total number of matchups.  

I withdraw my objection to this as a measure, if only because it is now clear to me that the loss average by game and the loss average by matchup will always be very, very similar numbers, because really all you are doing is multplying the underlying numerator and denominator by ~4.  It will only ever be noticeably different when there are very small numbers of games.  

The only concern I now have is the labelling for when someone is trying to do what I am doing (some extra analysis on the underlying data).

Skalchemist, if you want access to the raw data, just email me lynkfox@gmail.com with a account that can access the Google Docs and I will add you to the access list.

Yeah, I've talked with Arathorn a bit on it, so he does know about it now. (In fact, it was Progeny's 3 game records I talked to him about.) I think the environment win percentages have the same issue. That nice round number of 200 games on the Mobile Defense Platform makes it stand out a bit.

I also let him know that some of the stats were pulling in 2-hero games, too, and it looks like he's fixed that.

There's also a bit of ambiguity in the case of games where the same hero is used twice. While there's no rule against it that I know of, odds are games like that are a data entry error. So then do you count it as 2 games for the hero, 1 game for the hero, or throw it out as a mistake? From the numbers I've been running, Tableau counts it as one game for the hero. (There are so few of these games - something like 39 in about 10,000 records, that it's probably not worth worrying about.)

Once I adjusted for that and removed exact duplicate records, the analysis I've been running pretty much matches up with what Arathorn is getting, aside from the hero count issue for villain and environment win percentages.

A couple other things I did notice:

• There's still one game record with "Youngest Legacy" on 3/10/14.

• It looks like there are two versions of Rogue Agent KNYFE. Judging by the stats, one of them is a lot better than the other. (I should look and see which copy I got, hopefully it's the good one. )

Let me know how the two rogue agent ones are spelled and I should be able to fix it

Did you get the link for the document, nineedge?

Got it, thanks.

The two spellings are "KNYFE Rogue Agent" and "KNYFE: Rogue Agent", with and without the colon.

I hope she still has her colon.

Might have to change the villain input to "Vengeance Style Game" which has a sub-question of " which villains did you use" in the same way people enter heroes. Input could even be stored in a different underlying table if that is easier for people's processing scripts to handle. 

I hadn't noticed that comment earlier, but since both villain order and the number of villains (anywhere from 3 to 5) matter, you get a lot more than 3003. The total turns out to be 15*14*13 + 15*14*13*12 + 15*14*13*12*11 = 395,850.

That aside, since the Vengeance 5 villains are already handled separately in the underlying spreadsheet, I'd think the sane approach would be to just continue handling it that way. From what I can tell, the main things that would need to change with that approach are:

• Figure out how to handle games using villains from the original Vengeance 5, games using villains from the new expansion, and games using a mix of the two. (Or just lump them all together to keep things simple).
• Extra tabs for all the new mini-villains.

While it'd be nice to have detailed breakdowns on winning percentages for each of the mini-villains (and breakdowns versus heroes, environments, and all that), that's something that could be added later. Full breakdowns based on each individual set of mini-villains wouldn't be all that useful, though - the sample sizes are too small to do that even with just the Vengeance 5. (143 games as of 22 January, compared to 16 possible villain selections even if you don't care about order.)

If you do decide to lump them all together, that pretty much lines up with dclietz's suggestion. Not my call to make, but I'd think splitting it up into Vengeance 5 Only/New Expansion Only/Mix of Both is way too much of a hassle to be worth it.

Except, that if you're playing with Baron Blade, he has to be first. I don't know if Villains has a similar counterpart, but Blade cannot be anything other than first.

I always interpreted the “he should go first” as a suggestion more than a rule. If you take it as a hard and fast rule, the math does get a bit messier, but it’s not too bad to handle. I’m not sure anyone here besides maybe Arathorn really wants to see those numbers…actually, come to think of it, I’m not sure even he’d really want to see them.

It’d be more of a concern for lynkfox with designing the input form, based on how strict he wants to be about that sort of thing. Right now it allows Baron Blade to show up in the later slots, and there are a handful of games where he does appear someplace other than the first slot.

Considering how rarely I play against the Vengeance 5, it’s a detail that had honestly completely slipped my mind.

Same. I put Baron Blade where ever he ends up according to the randomizer.

Agreed. I'll put Baron Blade in wherever the randomizer says. I suggest the statistics reflect that option. Also, the numbers aren't bad if Baron Blade is always first:

5 heroes, with BB = 114131211 = 24024
5 heroes, without BB = 1413121110 = 240240
4 heroes, with BB = 1141312 = 2184
4 heroes, without BB = 14131211 = 24024
3 heroes, with BB = 11413 = 182
3 heroes, without BB = 141312 = 2184

Total number of combinations = 292838

So...

First, THANKS for all the other analysis and feedback and putting up of questions (and suggestions on answers).

Second, I *think* I've addressed all of the issues with the latest version, updated today, at the regular location. I also added a "details" for heros, villains, and environments, with W-L records and percents of each, as well as totals. It looks a little cluttered, but it provides all the information. No color-coding or good sorting on those, though.

Third, I've been looking at ways to do expansions and subsets. I think I'm nearing something, but it's not quite right, so I'm not trying to unveil that today.

Fourth, the Vengeance Five are already a nightmare. If an expansion adds a bunch more...YUCK! Just yuck. I think input won't be TOO bad, since there should still be a limit of 5 of them. Still, I'm not promising any analysis of sub-categories with more in the Vengeance Five mold. Lumping them together makes the most sense, though, IMO.

Fifth, more props to lynkfox for getting all this data gathered.

Sixth, having said that, lynkfox, getting the KNYFE Rogue Agents consistent would be really nice.

Let me know if anybody has more suggestions/corrections.

Arathorn

I agree, and that's how I'm gonna do it. All the Vengeance 5 subcategory page is going to end up with all of the vengeance mold villains on it.

You're welcome! Thank you very much for doing the /real/ hard work of making the data shine!

And the Find and Replace is running /right this minute/ on the document to add the : to KNYFE: Rogue Agent… and I'm pretty sure its done!

Eeewwwww.

ah. Found the KNYFE problem, the 4th hero selection didnt have the :

and added Deadline destroying the environment for an alternate loss condition! :p

Just looking at Deadline’s games (taking advantage of the small sample size), it looks like there are some games where the heroes are listed as having won, but the game end is flagged as “HP Incapacitation (Heroes)”. There’s also one loss flagged “Incapacitated Hero Ability” - which is possible but highly improbable.

How difficult would it be to do some sort of sanity checking on that page for the alternate win/loss conditions, to make sure they’re possible with whether win or lose was selected? Only Sucker Punch/Final Dive and Incapacitated Hero Ability choices can be for either a hero win or loss, and Relic Victory can only be a win for the players. (A blank entry should also always be a hero win.)

Also, I’m sure I’ve forgotten at least once to actually mark all the heroes as incapacitated when I lose by HP incapacitation. That one may be a lot harder to catch, since there are two different pages involved.

There’s also a small typo for Wager Master’s alternate lose condition.

Also, it looks like Skinwalker Gloomweaver isn’t the only villain that can kill himself. Citizen Dawn can too, and almost did in my last game. If she goes into a turn with no more than 2 HP and plays Channel the Eclipse, it’ll kill her.

Pretty unlikely, but a well-timed Suggestion or two from Nightmist can definitely up the odds.

Lots of villains can kill themselves. V5 Baron Blade hits himself, as does Friction.  Prolotariat's copies hit him, does that count?

Spite technically deals himself the Safehouse damage when he flips.  Akash'bhuta usually dies to self-damage.

Infinitor, Kaargra Warfang, and Ambuscade all have cards that damage all targets, which could kill themselves as well.

Kaargra killing herself just flips her back to her other side, but you’ve got a good point with all the others. (Especially Akash’Bhuta.) It’s just that Skinwalker Gloomweaver killing himself is handled as a special case in the game entry form. And when Citizen Dawn nearly pulled off the same trick in one of my games, it got me thinking about it.

Unless there’s something else special about Skinwalker Gloomweaver that I’m missing, which is entirely possible since I’ve only played him once or twice, tops.

Spite damaging himself with the Safe House I should have remembered, though - especially since I had one memorable game where the Safe House blew up right after Wraith hit him with a Throat Jab. Oops…

Well, to be honest, I only put Skinwalker in there as a special case because it was on his character card. Everyone of those other options is mostly just HP damage loss.

A lot of those special conditions was because I was curious, when i started the project. I wanted to know how often TerraLuna Impulsion beam fires, for instance - it never ever came up in any of my games against Blade, but is it common? I don't think Arathorn parses those stats into his final document, but when I did it it came up supririsingly alot.

So that's why its in there. How often does a game last long enough for Skinwalker to eat himself? How often does the Mars Base Self destruct actually go off or how often do players get lost in time? I figure close to never but...

So Channel the Eclipse and Dawn killing herself is something that just struck me as 'normal play' for her. Where as Skinwalker is an effect of his character card so it made me more interested in if it happens or not. I suppose you could also classify Channel the Eclipse (and similiar) under 'Some other card took them out (Trex, Kraken, ect) sort of deal.

Makes sense. I just wasn't sure if there was some other detail about Skinwalker that I was missing.

Unless you think it's worth adding a separate category for the villain killing themselves through normal play (and I'm not sure it is, honestly), it's probably easiest to just keep filing those games under normal winning conditions.

I wanted to bump this thread to ask for updates. What are some areas that need more data? like specific villains, heroes with variants, or environments that need more play to have a good sample size? I'm not good at parsing data, but I like providing it to those who can use it. I only have the video game at the moment, but as the expansions become available in the video game, I'd love to help add the data that is least provided.

Also, other updates I'd love to hear about: How many games are we currently averaging a month? Do games submitted from the video game have their own category?

Thanks!

Edit: Okay, I found the details tabs on Arathorns list, and if I only count the things I can do in the video game, DW Expat, DW Fixer, SS Tachyon, and GI Bunker have less games than the rest at less than 325 games each, and Spite and Cosmic Omnitron are the slightly less played villains, at less than 400 games each. Nothing stood out on the environment list with only things available in the video game right now.

I stumbled across this project as a new fan of this game. Are the stats still being updated? I've been submitting games, but was curious if anyone actually is collecting them anymore. 

Anyone have a sense of how hard this project would be to generalize to other games? (I'm thinking Marvel's Legendary, which is my other favorite game at the moment)

It would be nice to hear some of the lessons learned in the process, and hopefully some reusable pieces of code or formulas.

I know entries are still put in as this Google doc gets updated occasionally based on data from it http://x.gray.org/sentinels-of-the-multiverse-difficulty-scores.html

What would the chances be of eventually getting the app to submit data to the project?  Or created their own statistics collection?

Very high. It is on their to do list.

I was wondering if there was any way I could help update the data? I'd have to be trained of course, but was curious if help was needed?

Ask arathorn, he is the one who does all the public version now. It’s still being updated just on occasion

Just wanted to say, I love this tool and have been entering my games. Hope to see it updated again sometime.

Welcome to the forums!

Hey guys. Arathorns updates have slowed, I can only assume RL is taking its toll.

If anyone would like to work with the raw data and put together some sort of Statistics display, send me a PM and I'll get you access to it to do with as you wish! Some sort of form like Arathorns would be great - I simply don't have the knowledge or the time to do such a thing. 

Let me know if you can help! Data is still being collected, and when Villians is released I will update the input form for it. New results graphs would be really needed then!

When will the Google Form be updated with the Holiday Promos?  (And VotM stuff?)

Just got the VotM Stuff in. Forgot the Holiday Promos and the VoTM Promos, so I'm off to do that right now!

Awesome, thanks!  Just entered the first game with the Holiday Promos.

I know Arathorn hasn’t had a chance to run the stats with the new batch of heroes so far, so I ran my ratty old Perl script on them just a few minutes ago to see how things are shaping up. (Standard small sample size warnings apply.)

• Desperate Naturalist looks very strong (15 wins in 16 games), as does Santa Guise (10 wins in 12 games). Scholar of the Infinite isn’t looking half bad himself, either (34 wins in 43 games).

• Chokepoint is winless in 13 games recorded so far. Almost all of those games were considered a 1 or a 2 on a difficulty scale of 1-5 (5 = hardest). Still early, but she’s looking weaker than Deadline.

For the other heroes and villains (and the environments), there’s still not a lot of data to work with yet.

As for the raw stats, it looks like there are a few Scholar entries that were actually Scholar of the Infinite (from the comments), plus a few “Infitinte” typoes still in there. Not a big deal, but it probably wouldn’t hurt to edit those in by hand where they show up.

Thanks NineEdgeFirst for doing that! I corrected the mistake on the forums that lead to the Scholar having the typo, and the 'Find and Replace' is still processing... there are LOTS Of entries! Lol.

I would love if you could post the results of your script up for all to see for everything, but I don't know how expansive that is!

Sooo.,... the new version of Google Sheets that we've been in for about a year or so, has really upped their game on processing complex formulas with large amounts of data. I got the basic formulas put back into the sheets, so I have some basic data again - nothing as complex or cool as Arathorns, but given a few more weekends I might be able to get some updated stats again.

It’s not that expansive, but the output is pretty messy since I’d only ever intended to use it for myself. Let me see if I can get it cleaned up in the next day or two, and then find someplace to upload it.

I threw the output up on Google Drive:

As I said, it’s pretty raw. If there’s any interest (and if Arathorn doesn’t have time to do a stats update soon), I can clean it up a bit. (If anybody’s really feeling brave, I can also post the link to the Perl script I’m using.)

Chokepoint is still winless in 19 games. Also, Desperate Naturalist has 20 wins to 1 single loss. (The single loss was against Cosmic Omnitron, in a 4-player game with Captain Cosmic and Bunker as teammates. Even with Tempest as your fourth that’s a rough match.)

Also looks like the Celestial Tribunal may be the new easiest environment (2 losses, 14 wins), though that's not much data, and the Adamant Sentinels are also at one loss out of 16 games.  Termi-Nation Unity is at one loss out of 9 games, not much data there but she could be the most successful version of Unity yet.

Looking forward to Vengeance/Villains villains data!

i didnt post when you first put that up, but i love that script. That is awesome stuff, great work!

I have to start addng data to this, cause some of these numbers seem way off.  Out of six playthroughs with Kaargra I have never lost, out of 3 playthroughs with Court of Blood I've lost twice.  (It's a crazy hostile enviro.)

Ah well, just means it's time to SCIENCE!

With a lot of the moving pieces unlocked with Kaargra and her ability to quickly build up her army of gladiators I'm not surprised she is that high.  I'm sure having her and the other Wrath of the Cosmos villains in the app will probably have her go down at least a little bit.

I know you're working on the new wiki but could the new OblivAeon variants for current heroes be added since they are in people's hands at this point? 

Yes. tomorrow night probably

Okay

Completionist Guise

Ra: setting Sun

Parse: Fugue State

Visionary: Unleashed

Sky-Scraper: Extremist

Freedom Five: Legacy

Freedom Five: Absolute Zero

Freedom Five: Bunker

Freedom Five: Tachyon

Freedom Five: Wraith

XTREME Prime Wardens: Argent Adept

XTREME Prime Wardens: Haka

XTREME Prime Wardens: Tempest

XTREME Prime Wardens: Fanatic

XTREME Prime Wardens: Captain Cosmic

have all been added to the form. Whew. Getting long in there. 

If you notice any spelling discrepencies, please point them out to me ASAP. Thanks!

Would it be possible to add Challenge Mode (Y/N) as an additional question?  It's an official GtG supplement, after all.  My group is close to having beaten all villains on Advanced and we'll be cycling through on Challenge next.

It is certainly possible.

The reason I never put it in in the past was because Challenge modes are outside the normal rules - they are official yes, but freely addmited by >G that they aren't in anyway even possibly balanced. The main goal of the Statistics project was to evaulate heroes vrs each other, in comparison to various villains and such. Which is why I've tried to make it clear to keep things like House Ruled Gloomweaver out and such. Unbalanced for fun challenges may not be the best way to compile that information

That being said, it wouldn't be impossible to exclude the challenge mode ones from the none challenge modes (after all, already keep advanced seperate) so its not really a big deal to put it in there.

Eh. F' it. I'll add it.

xD

Yeah, I don't see how Challenge Mode is that different from Advanced when it comes to balance.  Some villains benefit massively from their Advanced ability, and some aren't helped in any significant way at all.  And frankly, >G frequently dropped the ball when it came to estimating villain difficulty levels in the first place.

I sent you a PM, but I'd like to get read access to the data.  Since Arathorn seems busy these days, I should be able to put something together to pull the data daily or weekly using cron and the Google Sheets API, and automatically create and update statistical reports with some R scripts.  Take the human element out of the equation.

The rulebook also states that Advanced mode isn't balanced.

fair point

All right, version 1.0 of my own stats tool is up:

http://mindwanderer.net/sotm/

Features include:

• Updated once a day, automatically!
• Separate pages for basic rankings and full analyses.  The full thing has a lot of data loaded into tables that can be filtered and sorted, so it take a little while to load on a desktop, and a very long time to load (and may render incorrectly) on a phone or tablet.
• Most of the same data as on Arathorn's page.  I left out some of the analyses that didn't produce interesting results, especially any that would require me to add even more visualization libraries.  Also, for greatest flexibility, it contains no metadata that isn't in the source, e.g. what expansion a hero/villain/environment comes from.
• A few extra comparisons, such as win rates of combinations of two heroes, and win rate by team size and hero's position in the team.
• Significance testing on detailed pages by hero.  This helps sort out variance due to randomness and small sample sizes.
• This leads into one big weakness: since it's all automated, there's no element of human quality control.  Unlike Matthew Gray's naive Bayes model (which mine does something to using logistic regression--very different math, but usually similar results), it doesn't filter out anything by "not enough data."  It will always try with whatever it has.  Significance testing helps with some of that, but it's no substitute for common-sense sanity checks.  This also means that some data, like interactions, are highly subject to interpretation.

My to-do list currently includes:

• A contributor view page, which will allow people to see who's contributed how many games to the project, how many times they've used each hero, villain, and environment, and a few more details.  (It will not function as a leaderboard, since that could encourage people to do other than just play the game as normal and submit honest data.)
• A difficulty calculator using the logistic regression data.  This should be the most accurate difficulty estimator available to date.

Sooo awesome!

Hmmm… are we still having the edit issue? I want to add this link to the first post

So we were wondering... Would you be interested in adding summary data from the video game to the statistics project? There are over 800,000 finished game sessions (and counting) for which we can provide the basic setup & result. I'm not sure Google Sheets could handle that, but we could send some sample data and see how it goes.

I don't think it would integrate well with the Google Sheet, but I'd be more than happy to add it to my summary statistics.  As long as you have it in some kind of format I can parse easily, like JSON, I can work with it.  Heroes, Villain(s), Environment, Advanced, and won/lost are pretty much all I look at right now, but date would be good too, so I can make sure any data we have from the Google Form doesn't overlap it.  At some point I may add more involved data--alternate win/loss conditions, villain flips, hero incapacitation--but I'm not there yet and it's not critical.

How many unfinished games do you have?  The reason I ask is that I expect that games that are going poorly are more likely to be abandoned than games that are going well, and that could skew the statistics a bit.  But if the number of abandoned games is small (compared to the 800,000 total games) then it won't be a noticeable effect.

Also, do you have any idea how many of those games are weekly one-shot games?  That could also affect the numbers if they make up a large quanity of the games played.  Also, I'm kind of curious about how much of game activity is Weekly One-shot activity.

...Holy crap.

Also, is that data from Handelabra's own test games, or from the players? Because you'll get some duplicates for sure if it's the latter...

If that's every game ever played on the video game version ever, I can simply eliminate all the Google entries played on the video game (unless I'm looking at data that Handelabra doesn't log).  Otherwise, I can probably filter by date.

Either way, the real challenge will be improving efficiency.  Right now my tool takes about 3 minutes a day to run, and that's with 17,000 games.  I'll have to do things a very different way if that number gets multiplied by 40.

Wowowowowo

If we need to, we could probably change the Google form to store data in the same manner that the Handel abrasive team store their Data. At least I'm betting we could, then (someone other than me) could write a script to convert the old data.

All that data would be awesome, but the points above of unfinished games being stopped when they are being lost is an important part. If the data says it's about 2/3rds win rate tho, that would line up...

We can provide JSON from analytics events. You'll get heroes, villain, environment, advanced, won/lost, and "result kind" (alternate win/loss conditions). I'll get some samples together for you to have a look at the format.

How many unfinished games do you have?  The reason I ask is that I expect that games that are going poorly are more likely to be abandoned than games that are going well, and that could skew the statistics a bit.  But if the number of abandoned games is small (compared to the 800,000 total games) then it won't be a noticeable effect.

Captain Cosmic: Requital (the "Captain Infinitor" variant) isn't in the hero selection dropdown. Looks like it was missed in the update that added the other OblivAeon Kickstarter/5th Anniversary Foil Hero variants.

I am only referring to them as the "Handel abrasive team" from now on.

Wow, wouldn’t it be fun to put some of those interactive data analytics widgets like fivethirtyeight has on Handelabra’s site…

I think what I'll have to do is run a completely separate report from Handelabra's data. With only 63% of games completed at all, and no way of knowing whether a particular abandoned game was abandoned because the player anticipated losing or for some other reason, that's a very different set of assumptions.  Clearly the baseline win rates will be drastically different, and I anticipate that heroes that require longer buildup times will have much lower win rates.

Anyway, you can PM me with details about how you'd like to get the data available for analysis.

Edit: Contributor statistics have been added!  Keep track of what heroes, villains, and environments you've played with, and how well you've done with each, as well as see who's contributed how much so far!

Thats a great idea MindWanderer - then we don't have to worry about duplicates between the two as well. 

We can run the statistics on our own collected data (which has more information too, like position of the hero, ect - and that won't get diluted out with 10k games without that data, as a possible (maybe not true) example) and seperately on theirs.  Plus, as MindWanderer already said he's going to be adding a 'username lookup' function to his stat page, that can still have players put their video game data into the form so they can track their own statistics.

Handelabra (handle abrasive autospell correction team) - is this something Mind Wanderer can put up on your own site? Is that something you guys are interested in? I have no horse in that race, just a curious (and excited!) bystander.

Ever since I started this project, and the Video Game was released, I wanted to see the statistics done for the video games data. I had HOPED that you guys had been collecting the data, and I am SO glad that you were. This has me so super excited I'm beside myself.

MigrantP - Thank you so much for having this data, having the forsight to capture it, and offering it up

MindWanderer - an even BIGGER thanks for having the ability, the patience, the time, and the desire to automate the analysis of the data, so that we can all see it in (practically) real time. Thank you so very much!

Thanks for catching that. He's in now! (at the bottom. I've been debating if I should reorganize them by team? by alphatbetical? By hero? Not sure yet, still debating - i only want to do it once because reorganizing them is a pain in google forms, and if I make a new question it messes up the results sheet)

OKay.

The Hero list has been bothering me for a while. There are a few problems

1) I was not smart enough to start every hero with the hero name. This means the list is ugly looking

2) This is a problem because what is in the list is what is put into the results spreedsheet. So you can see Tempest; Sacrifice has a ; instead of a : - it got that way on 3 of the 5 fields, wasn't on the other two, and we ended up with Tempest; Sacrifice and Tempest: Sacrifice in the results when arathorn started doing them.

3) The spreedsheet is now HUGE. i could run a find and replace on all the data to make it cleaner.... I have no idea how long that would take. Might be a weekend project to do... might crash the sheet.

4) What this means is that organizing that hero list is ... difficult. Its difficult already because the only way to do it, and it has to be done for each of the 4 questions, is to click and drag them where they need to go... but the list is so long it spans multiple screens... and the google sheets doesn't scroll with the mouse holding an item.... so you have to do multiple moves for each one. Which is why F5 and Xtreme are at the bottom together, because I was like F'it.

5) Of course... the list is organized by expansion, then by hero... which is confusing.... so... ugh.

I have a few options.

1) Reorganize the lists in true alphabetical

this means things like Best of Times Chrono Ranger comes before Chrono Ranger, and all the 'The Hero' are together.

2) Organize Semi Alphabetical

- Like Argent Adept, Kvothe Six String, Prime Warden Adept, XTREME Prime Adept, Absolute Zero, Absolute Zero Wrath, Termination Zero... .ect

2b) Semi Alphabetical - but seperate out the teams - put the teams all together in order at the end. (so it would look like the above, but also like the XTREME and Freedom Five are now in the lists)

3) Rename and clean up, and do a find and replace on the 18000 entries.... i'm not even sure if that is possible...... And if I do, it's a weekend project.

I'm asking for peoples opinions. I'm dreading, but partial to option 3.  ... because I like things clean.

What are thoughts?

Additional Question: MindWanderer - will doing option 3 mess up your results page?

I'd suggest alphabetical by base hero name, then variants in whatever order you want to do.

Option 3 isn’t as bad as you think–I actually took the liberty of correcting some old inconsistencies (Young and Youngest Legacy, Vengeful Blade and Vengeful Nation Blade, and Mister “FIxer”). As long as every entity has the same name throughout the dataset, my tool will work correctly with no changes from me.

Four suggestions:

1. If we ever wanted to merge the two datasets, it would work best if the Google names matched the Handelabra ones.

2. It would be handy if the Vengeance villains had different names than their solo versions, the way Vengeful Blade does now. Right now I have to do some juggling so they don’t overlap.

3. Kvothe needs to be labeled, if not named, in such a way that it’s clear that he’s the same entity as Dark Conductor.

4. Similarly, some of the names don’t clearly match the cards (like Desperate Naturalist). You should know who’s who by just looking at the card, with no background knowledge.

Edit: Also, do bear in mind that we have 11 new heroes with a variant each coming out in the near future. If it’s that much work to reorder them, you might want to get it all done at one time.

Keep in mind these aren't the full statistics we plan to do if/when we can. Our ideal statistics server would record everything that happens in each game, so you could find out things like "in which environment is Enduring Intercession played the most" or "which hero does the most damage to Progeny" etc.

The basic data we'll be providing includes hero order.

If you're looking to update the hero/variant names - all the official names are in Sentinels Sidekick.

There's another option for you - using a database instead of a spreadsheet. It's not quite as straightforward as using Google docs though.

If I had the ability to do a database, and a form that entered into it, I would xD. Maybe thats something I'll teach myself in the future.

Good point on matching names to the Handelabra data, I'll do that. And I ran a quick F and Replace to test it on a copy of the data last night, changing Tempest; Freedom to Tempest: Freedom Six (460ish entries) only took a few seconds. So Shouldn't be too bad.

And yeah, definitely, I'll be adding the new hero names to it when I do it as well, so I don't have to mess with that later.

Our data doesn't have display names, it uses identifiers like GrandWarlordVoss or RaHorusOfTwoHorizons.

I could do that, but Google Forms/Docs has the advantage of maximum persistence and maintainability.  The data has essentially zero chance of being lost, and lynkfox always has the option of adding another administrator to work on it if it starts being something he can't do anymore for some reason.  Plus, although Google's API documentation is absolutely horrible, once you figure it out, it's easy to get the data out to anyone who wants to use it.

Hm, that makes integration slightly more difficult.  I'd have to manually create a lookup table to translate the names into their official forms. Not too onerous, though, especially if I have to add new data manually anyway.

That being the case, then, ill organize the statistics list differently: since google forms cant do tabbed lists in the same question, having it with the same names as in thr Sidekick wouldnt look good.

Probably next weekend ill work on this. Maybe this, but the wife and i have a lot of fall plans this weekend… apple farm… my first haunted house in over a decade…

Anyone paying attention:

https://docs.google.com/spreadsheets/d/1kqD_zlsapFcpb-QTjN6kQCjkwJZ17k93F-RURxt9dUQ/edit?usp=sharing

That would be the final list, in the way I feel like it should look. Easier to find whomever you are looking for, at least I think so.

If people agree, I'll make the changes as necessary. If people could also look for any spelling errors, or other wise (Don't need any more ; instead of : ! xD)

Yes, this looks good. Thoughts as I read through it:

The female Luminary variant is known as Heroic Luminary, if I'm not mistaken.

Do you really need the color in "Scholar of the Infinite"? (Also, I approve of taking out the "The". But not for the Sentinels?)

I’m still of the opinion that the names should reflect what’s on the cards, since the “official” names require the app or finding then online somewhere. The main reason to go “official” is moot, since Handelabra’s data doesn’t use them either.

If Google sheets could do Tabbed lists, hell yeah i'd go more official

So it be like

Absolute Zero

     Freedom Five Absolute Zero

but it can't... so we can do

Absolute Zero

Argent Adept

Dark Watch Expat

Dark Watch Mr Fixer

Expatriette

Freedom Five Absolute Zero

Freedom Five Legacy

for instance, but... i find that harder to find what you're looking for, because, at least for me, it goes 'Absolute Zero! Okay, where is his F5 variant... oh its way farther down.

Or... I'm looking for DW Expat... heres Expat... oh DW is higher up. hmm.

Its up in the air still though, lots of options.

As for The Sentinels vrs no The on Scholar.... Honestly, it was because I never refer to them as anything other than 'The Sentinels' in my head, where I refer to just Scholar, or Visionary, or Wraith all the time. It's probably because they are a team.

And... probably no, we don't need the : on Scholar of the Infinite. I was just trying to be consistant, but it's been bugging me too.

Oh, and a second thing!

If done the way I have it on the list, when you sort your results page MindWanderer, people can easily compare all versions of a hero to its base - It will be all grouped together automatically, rather than seperated down the list.

I was thinking more along the lines of “Unity: Golem” instead of “Unity: Freedom Six” (unless the new cards say that instead; I haven’t seen them yet). Not everyone who has the cards may know what their official name may be.

One advantage to consistent usage of colons is that it would allow me to easily compile aggregate statistics by deck.

consitent : are a go then.

And Ahh, i get what you mean now. I call them F6 but that isn't anywhere on the card.

So, probably something like

Absolute Zero: Freedom Six (Elemental Wrath)

seeing as I already have AA as

Argent Adept: Dark Conductor (Kvothe)

I like that a lot...

Some players probably call him by each name so by including both it should make it so that no one is confused at all.

Okay, the hero list is updated, and all turned over. Should be a lot easier to find what your looking for when inputing data.

MindWanderer - you said you also wanted the villain names to be different for Vengeance Style villains. Would it be easier if perhaps we made them all the same? Like:

Ambuscade (Vengeance)

Baron Blade (Vengeance)

Biomancer (Vengeance)

Bugbear ((Vengeance)

ect? Maybe (Team) instead of  (Vengeance), but Vengeance is consistent with the terminology I was using on the Wiki

Would that be easier to seperate out with your scripts, rather than as it is now?

Much, yes! Team or Vengeance are fine either way. OblivAeon is sort of a villain team, too, as I understand it… I’m not sure whether it would make more sense to combine the Vengeance villains with the Scions of OblivAeon, since I haven’t seen it in action yet, but I’m guessing not.

Now that’s an interesting format–JSON inside a spreadsheet. I should be able to work with that, and it’s nice that it uses aggregates for identical games instead of one game per row. Would the regular data be in this same format? If so, I can just modify my existing Google Sheets API pull, and a month’s worth of data at a time would be trivial to manage.

Okay! It will be Vengeance, I'll do that later tonight

It's in a Google sheet because I exported it directly from Google Analytics and that was one of the options. It's just a sample so you have an idea of what the data has and the format of each record.

Heh, you consolidated "(none)" and "(None)" which broke my script.  Fixed.

And more importantly, the calculator and user report tools are now up.  (Actually, the user report was up last Thursday, but I mentioned it in a post edit, which I forgot would get eaten.)

Next on the to-do list is reports by hero deck, since that's a thing I can do easily now, but it's low-priority so I might not get to it for a week or two.  Not sure how much difference it will make anyway, since the base variant of all heroes is always at least 3x more popular than the second-most popular variant, and usually much more.

Hey hey, how about that contributors page? :D I like!

Whoops! If anyone was trying the calculator earlier today, there was a small but catastrophic error in the function that would have produced gibberish results. It’s fixed now.

From now on, I’m going to start characterizing all my mistakes as “small but catastrophic errors!”

Anyway I’m glad you fixed the function that produces gibberish results.

It was an off-by-one error.  My randomly clicking around didn't reveal it because I happened to choose heroes/villains/environments that were pretty close to the one before them in most cases.  Then I tried Kismet and couldn't figure out why her predicted win rates were in the toilet...

I've also adjusted the "deviance" predictions and chi-square values for all comparisons involving a solo villain.  Looking through the stats, the choice of villain seems to be much more important as the choice of any one hero, the environment, or the team size, so I've weighted it twice in the prediction calculations.  It looks like even that might be underdoing it, but I'll leave it there for now until I have a chance to do some intensive math to see what the actual naive predictions ought to be.

I've un-adjusted those last adjustments.  The actual relationship is more complicated and not at all linear.  It looks like extremely good or bad heroes, villains, and environments have curvilinear relationships with other elements.  For instance, Kismet is really easy in most environments, but in a handful (Wagner Mars Base, Realm of Discord, Insular Primalis) she's quite a bit harder than you'd expect, and those are so extreme that they skew the averages.  Environments x Villains are actually interesting--in virtually all cases, only the villain matters, and the environment doesn't affect difficulty at all, but for some environments, the effect is significant.  Same with team size--adding an additional hero, unless it's the single worst hero in the game (Captain Cosmic: Prime Warden), will always make the game easier.  So I'm simplifying things and just using straight averages.  Users can figure things out as they will.

Also, so few interactions are statistically significant that I'll hold off on adding interaction terms to the calculator's model like I had planned.  Maybe I'll do that when I have Handelabra's data and my sample size goes way up.

Same with stats by hero deck.  Right now, except for Team Leader Tachyon, no variant is more than 1/3 as popular as the vanilla hero, and most are much less.  If I aggregate by deck, it'll just look like the default deck for everyone except her.  This might change once the complete variants pack is on store shelves, and digitally once Season 2 is out.  But for now it doesn't look like there's much point.  So the tool is basically done for now unless someone comes up with another good idea for a feature.

MindWanderer.

THANK YOU

Serously, thank you so very much for this awesome work. This is such a great bundle of tools that i am very impressed with your work and very very grateful for being able to take this to the next level

WOW. The new data results site is great to look at. And man I knew I had put in a lot of games over the last year or so because of digital but over 100 wow. Then there is AbNelson that dude is devoted.

I have a new and improved calculator and randomizer up here: http://mindwanderer.net/sotm/randomizer.html

Please note that since this is interactive, it's considerably more complex than the previous calculator, so right now please consider it in open beta.  Please give it a try and report any bugs to me--I'm sure there will be some.  And if a function isn't clear after reading the "about" text, let me know that as well.  You can try it on mobile, too; I've put in some responsive CSS, so this should be more one-size-fits-all than the original stats pages.

Also, this is all Javascript, which effectively makes it open source.  If anyone else wants to write tools based on this data or its underlying math, knock yourself out.

This currently doesn't support "target difficulty" the way the randomizer app does.  I'd like to do so but I have to think about how to design that efficiently.  In the meantime, it's probably faster to just try it a few times until you get something you like.

...How do you unlock heroes? c.c I feel kinda dumb.

They're unlocked by default.  You can lock or unlock them by clicking on the lock icon to their right, or clicking on the lock icon at the top of the table to lock/unlock all of them at once, or use the Quick Select buttons to lock/unlock all heroes, villains, and environments from a particular expansion.  The About link explains this, but I guess it wasn't clear?

It took me a while to find that link.

It would be helpful if this could bring the selected characters to the top of each column or something. I was expecting some sort of output list, until I noticed one of the environments was selected. With the limited display room and large number of heroes, it seems like it could be hard to find what's actually been randomized. Just my thoughts, I'm not great at internet sometimes. :C

You can mouse over the output after hitting Calculate to see what was selected (or touch it on a touchscreen).  I've also now added the ability to sort by checked status.

All right, so I've figured out this much.

The "Quick Select" buttons are for locking (or unlocking) whole expansions. Good idea, helps for the digital version especially.

I can't get any kind of output pop-up, but the sort by checked is working, so that's good. (I can also set it to show -1 entries in the drop-down, which gives me everything <.<)

Everything else seems to work correctly, though oddly if you deselect a hero and hit calculate, it won't until you've set the number of heroes to what's currently selected, or vice versa. That's kind of odd.

I think you've got a good randomizer here, though. :)

That's an artifact from my original concept.  I had planned to make it so that you could make an incomplete selection and it would show you what the range of possibilities are, if you chose the worst/hardest or best/easiest remaining unlocked choices.  In its current incarnation, I agree that it doesn't make much sense.  I'm not sure that feature is necessarily helpful, since you can sort by model coefficient and easily see what the most extreme options are, and just try them manually.  I'll go ahead and change it.

Also, FYI, Madame Mittermeier's isn't showing up on the randomizer, because it's hard-coded (to include expansion data) and it's misspelled in the Google form.

Out of curiosity, since we have all the normal villains we're ever going to have now, I took a look at where the difficulty levels stand.  Looking for obvious breaks, the difficulty levels seem to be about this:

Difficulty 1:

• Tormented Infinitor
• Unstable Kismet
• Chokepoint
• Ambuscade
• Gloomweaver
• Deadline

Difficulty 2:

• The Ennead
• Baron Blade
• Apostate
• Kismet
• Mad Bomber Blade
• Wager Master

Borderline difficulty 2/3:

• Akash'bhuta
• Miss Information

Difficulty 3:

• Spite
• Omnitron
• Grand Warlord Voss
• Plague Rat
• Citizen Dawn

Borderline difficulty 3/4:

• Skinwalker Gloomweaver
• Progeny
• Infinitor
• La Capitan

Difficulty 4:

• The Matriarch
• Agent of Gloom Spite
• The Dreamer
• The Chairman
• Cosmic Omnitron
• Vengeance Style (on average)

Difficulty 5:

• Iron Legacy
• Kaargra Warfang

Iron Legacy and Kaarga really do deserve a category of their own.  The gap between them and everyone else is tremendous.  It's interesting to see where >G got the difficulties right, and where they got them completely wrong.  The promos are largely way off, but not as badly as Deadline, Gloomweaver, and Omnitron.

Man, I never thought Kaargra was that hard. :O

Things can get out of hand quick with Kaargra simply because she can win just as easily with favor as she can by beating down the heroes.  

More easily, IMO.  I've lost to her several times, never due to incapacitation.  And every time, the heroes rack up the 20 favor while Kaarga is out, and actually have more favor than her when she reaches her 20.  If Kaargra herself had just a little bit less HP, or had one less "Get Back In There," I think she'd be on par with the 4's.  The fact that she has two of them out of only 15 cards makes that card, and Fickle Fans, much nastier than they should be.  If she didn't have "Get Back In There" at all, I think she'd be a solid 3.

Maybe it's just that I always enjoy fighting her. :B The difference in how you approach her as a villain versus everyone else just makes her games special.

New statistics added to the report: 

On the Heroes page, probability of each hero being incapacitated, and probability of going on to win given that the hero was incapacitated.

On the Villains page, probability that the villain will flip at least once during the game, and probability that the heroes will win given a game in which the hero does or does not flip.

On the Details page of each villain, the count of the number of times games ended in different ways.  Since we're on the topic of Kaargra: only 26% of Normal Kaargra's wins are due to hero incapacitation, and 17% of Advanced Kaargra's.

This is also the point at which error on the part of the contributors becomes extremely apparent.  For instance, a substantial number of people reported beating Baron Blade, Miss Information, The Dreamer, The Chairman, and Skinwalker Gloomweaver without flipping them.  However, the overall trends seem credible.

Added a button to the randomizer to toggle cards not yet present in the video game.

Also noticed something interesting, randomly: The members of Dark Watch have terribly synergy with each other.  Not many character-character interactions are statistically significant, but nearly all the Dark Watch ones are, massively so, and they're all negative.  It's not true for the actual Dark Watch promo cards, but the vanilla versions of Setback, Nightmist, Expatriette, and Mr. Fixer all do way, way worse with each other on their team.

Pretty sure I played a large part in creating that trend.

I battled each villain on advanced with the base form versions of each team until they either won once or lost 10 times…

The Freedom Five had a win rate of 82.9% in 36 games

The Prime Wardens had a win rate of 61.7% in 48 games

Dark Watch had a win rate of only 35.5% in 77 games

What's interesting is that the win rate for 4 hero teams is not very different from that of 5 hero teams which suggests to me that The Freedom 5 and Prime Wardens actually do work much more well together than Dark Watch and it's not just a matter of team size.

It's probably not just you.  I imagine a lot of people play a whole team of Dark Watch and then die a lot.

I guess the way to test that would be to make teams of mostly team members, then supplement them with one or two really good characters so it's not just a total suckfest.  It's unfortunate that most of the really good characters are members of the Freedom Five, but you could compare most of Prime Wardens to most of Dark Watch.  So, say, all current members of Dark Watch vs. Fanatic, AA, Haka, and Cosmic (Tempest being by far the most successful Prime Warden), both with, say, the Hunted Naturalist or the Sentinels, or Omnitron-X or Sky-Scraper in the video game, since those characters aren't of the "make good characters better" variety.  Or even three of the team members with two off-team support characters.  Dark Watch should still have a harder time, since they're worse, but they shouldn't have a worse time than their sum.

MindWanderer .  I've gotten your emails,  but it's coming up on my diet anniversary so. .  Yeah.  RL. 

You might be able to edit the form,  if you can go ahead and fix any mistakes you found.  If you can't post or email me and I'll try to at least find time to give you access

Stupid phone.  First anniversary .  Not diet. 

Nah, I don't have any privileges on that form, just the results spreadsheet.  No hurry, though, it's all minor stuff.

The celebration of a year of successful dieting is one of the most important times in a person's life!

In the difficulty calculator I noticed Advanced Bugbear missing from the Vengeance style list.

That’s because he appears nowhere in the data yet–no one’s fought Bugbear on Advanced and reported it.

Hadn't checked the Stats project in a while, it's looking really sweet!  I'm in the process of logging about 100 games I had screencapped on my iPad, so I'll have some more data in there soon.  Looking forward to seeing my aggregate data - I think there are a couple things that I'm going to have confirmed for me (I have a rough time with Omnitron IV, and an easy time with Mars Base, for example), but it'll be cool to see if there are some surprises in store...

Is there a way I can check what games I’ve submitted to the statistics project recently?

I accidentally threw out the paper I had written the results of some recent games on before adding them to my own spreadsheet where I’m keeping records.

I can put up a "most recently submitted games" on the user report.  In the meantime, your most recent game submitted was against Gloomweaver in Insula Primalis, the one where the game crashed, and before that was two identical games against Gloomweaver in Silver Gulch, and one against Gloomweaver in The Block.

Follow-up: I've added that report, and ran the update manually so it has your entries from eariler today.

Thanks, the games I'm missing are the 3 games before those… 2 games against Advanced Apostate (1 at H=4, and 1 at H=5) and 1 against Gloomweaver (at H=3). Specificly, I'm looking for who the heroes were and what the environment's were.

Also, those 2 identical games were really just 1 game which means I must have accidentally submitted it twice…  adding "most recently submitted games" on the user report would probably help to make sure that doesn't happen.

Thanks!

Unfortunately, the report is still only going to run once a day, so you'll normally have to wait until the next day to see what you just posted.  The amount of data is huge, so trying to process it from raw data live would mean sitting there for several minutes loading each page, and would probably irritate Google for hitting their API so often.

Looks like a lot of my games have been double counted actually..... that's problematic. My thought is the games must be being submitted again when I refresh my phone.  I'm going to submit all my games from my PC from now on to avoid this.

The new report looks great by the way... and it allowed me to fill in the missing environments and first heroes for those games.

I deleted the duplicate entries that were showing in the last 10 games.  It also turned out that having it include players' entire game history only added a few seconds to the processing time over including just the last 10, so it's all there now.

It looks like there might be some duplicates still.  If all those apparent duplicates are actual duplicates, let me know and I'll get rid of them.

Thanks!

I ran through the list of all the games I submitted to the stats project and found 31 errors. 28 games were duplicates, in 2 cases I listed the wrong environment and once I marked advanced on a game that wasn't.

Here's a pdf showing the timestamps of these games and what the error was:

Looks like three more environments will need to be added to the list, although they won't be available for several months yet.

Oh boy. xD

Once Christopher posts the information regarding them and I go to update the Wiki (or someone else does...) I'll update the form.

I realize I totally dropped the ball on getting you proper data. However I have picked the ball back up!

Attached you can find one week of data (2017-02-01 to 2017-02-08) from Google Analytics, representative of everything we're looking to share so far.

There are 3 platforms: Desktop (Steam), iOS, and Android.

There are 4 events: Game Start, Game Start (Team Mode), Game End, and Game End (Team Mode).

Each file includes the info about its context at the top, then a series of rows. Each row has a "dimension1" with JSON describing the game setup (and result for Game End events), and total_events which can be more than 1 if more than 1 player started or ended the same setup (esp common for the weekly one-shot setups).

Note that the Team Mode records are formatted differently than classic mode; this is due to some restrictions we have on the # of parameters we are allowed to record in some systems.

You will also see some challenge mode parameters; it turns out the data from beta testers is mixed in, which is a bit odd but shouldn't really be a problem I think.

It's up to you if you want to infer anything from the difference between Game Start and Game End events; note that the data doesn't tie anything between a particular user, so there will be some oddness in the data around report boundaries if you do so.

The file format can be changed; right now it's just dumping out the data from Google Analytics in Python. Please let me know if there's any simple transformation I can do that might help if you're having trouble importing it.

Once you're happy with the data format and interpretation and all that, I can generate all the historical data. And then we can coordinate on a schedule for data dumps and how to get them to you on an ongoing basis.

SotM Report 2017-02-01 to 2017-02-08.zip (245 KB)

One thing to watch out for when interpreting the data is that a multiplayer game will have the same events recorded for each player. For example, if me and two friends play a game, the total_events for that game will be incremented by 3 from our single game. It may make sense for a first stab to ignore any events with total_events > 1, or treat them specially.

Do you really have so few Android users?  You only have a dozen games recorded, none of them team games.

Do you have any idea how common multiplayer games are?  If it's a very small fraction of total games, it would probably be better to double-count them that to discount all duplicate setups.  On the other hand, discounting duplicates would also remove Weekly One-Shots, which might be harder or easier than random games with the same setup.  Another thought would be to do single days rather than weeks, since that would be even more likely to exclude multiple people just happening to play the same setup.

There is something weird going on with the Android report. On the Google Analytics site it says there should be at least a couple thousand events, but indeed I only get details for 12. I'll see what I can figure out there.

Within that period we have the following number of "multiplayer game start" events (note that this will count 1 for each player in the multiplayer game):

Desktop: 1355

iOS: 101

Android: 83

I can capture that information in the report if you like.

I could run the reports for individual days if you think that's useful, it would take a bit of extra scripting but not too much. I was planning to run them for a month's worth of data at a time.

The Android issue seems to be within Google Analytics so I've asked them about that. For now I would just ignore the Android data.

If you know how to cajole information about unique event logs from Google Analytics API, that would help with detecting and ignoring multiplayer or duplicate games. The best I've been able to come up with so far is what's in the report attached above.

Noticed the update to the environments list. Glad to have it sorted alphabetically, even if it did freak me out! XD

On that note, the Villains list could stand to be resorted...

Is the villains not Alphabetical?  Or are we talking VotM?

From where I'm looking (ie, the form itself) It seems the Villains list is Alphatbetica, as well as the Team Villains list.  - The exceptions being that at least there, I did not Alphabetize with 'The' as part of the name.

That, and I know Matriarch is under a couple other M villains she should be above.

I'm actually not concerned either way whether you alphabetize by "The". Consistency, on the other hand... :)

Heh. I’ll mess with it tonight.

All right, I have some sample pages with the data I have so far: 

http://mindwanderer.net/sotm/app-stats-full.html

That's just using the week's worth of data I got, and for simplicity it's only the desktop data, and only standard games (not Vengeance).  I'd like to have the file format finalized before I set it up to handle a whole array of input files instead of just one.  It also only counts completed games, as I'd have to figure out a whole separate way of handling abandoned games, and, as we discussed earlier, it treats multiple copies of the exact same setup and result as a single game (to handle multiplayer, and it also helps balance out weekly one-shots being overweighted).  Obviously, all the names of things aren't in their human-friendly format, as I'll have to manually map the "code" names to the common names.

So the next step would be 1) folks should check over those pages to look for obvious errors or data that don't make any sense, and 2) to figure out what schedule and method Handelabra can use to get the data somewhere I can get at it.  Again, I'd prefer to get just one day at a time; I can process each day on a schedule, and possibly just add each day as it comes in rather than recalculate everything (like I pretty much have to for the Google data), and also it helps deal with the multiplayer issue more cleanly.

Are you planning on keeping them seperate? (the manually entered doc and the video game doc that is) - I think it be best to do so, because a) the possible problems of the physical game (ie, rules mistakes) will be kept out of so called 'pure' data and b) double entries as people manually input games that they played online. 

It doesn't in the end matter to me, so I'm curious as to your take on it.

Not sure how much you read notes left in game entries, so I thought I'd mention here: There's no game-end option for the Earth being sentenced to destruction in the Celestial Tribunal. Happened to me yesterday! :B

If they’re separate, it would be fun to see how the two datasets compare over time… Am I the only one who finds comparing datasets fun?

We should set up a series of polls to find out.

LOL!

Anyway, yes, my plan was to keep them separate.

First of all, there's data in the custom form that Handelabra doesn't collect, such as whether villains flipped or heroes were incapacitated.

Second, the app is likely to be skewed, for instance, there's a lot of abandoned games, and there's no way of knowing why they were abandoned (a forfeit would count as a loss, but an "oops, I set this up wrong" or a "there's a new one-shot, I'd rather play that instead" wouldn't.

And third, the app will of course, for at least another year or so, be behind the physical game in terms of what content is available–and that's ignoring Sentinels of Earth Prime, which, if lynkfox agrees, I would also like to add to the stats project but which will probably never make it into the app.

You seem very pessimistic about how quickly we can implement content =)

I can make a wrapper script that generates daily data. Would a shared dropbox folder work for you for delivery?

Also, are you using all of the data files, or only some of them? If there are some I can avoid generating that will save some time.

Vengeance came out in 2014, so... yeah, if you can implement and release OblivAeon and all its promos within 12 months after it finally ships, I will be very surprised.  Not a dig at Handelabra, it's just a lot of work to get something like that working.

I'd prefer not to have to deal with Dropbox's authentication method--it looks like the only way to get API access to a shared folder is to create a key that would grant carte blanche access to the whole Dropbox account, so I'd have to create an account specifically for this purpose to not expose my personal files if the app was compromised.  But if that's the easiest way for you, I can work with it.

Assuming the Android data works correctly now, yeah, I'd definitely like all three platforms.  I'm still undecided whether I should handle abandoned games, in which case I'd need the start files as well as the end ones.  I can't really count them in the win/loss ratios unless I calculate all the stats twice (with and without abandoned games counting as losses).  I suppose I could do that.

Done!

Cool!

That's not really a fair estimator.  To say that it has taken Handelabra three years to make Vengeance would imply that they have spent those three years working on Vengeance - which they have not.  Sentinels was released on Steam in December of 2014.  Since then, they have released 4 expansions, 3 mini expansions, and Vengeance.

If you want to fairly estimate how long it took to make Vengeance, you have to look at the Season 2 Kickstarter.  The Kickstarter concluded in May of 2016.  Vengeance was released in December of 2016.  That's a half year turnaround time.  And sure, the OblivAeon format is probably trickier to code than Vengeance was.  But I would be very much surprised if it's half a year longer.  Especially as all the code is cumulative - the more features and effects they build into the game, the easier it is to use those effects on new cards without much effort.  Visionary Unleashed, for example, is probably only an hour of coding and a week of bugging Adam for art.  It's only the weird ones like Collector Guise or Maze of Mirrors that actually take significant time to create.  Also, although their focus is Villains that the moment, they know how the OblivAeon gameplay works and can get a head start on designing it and preparing for it.

So yes, I find your surprise surprising.

All right, fair enough.  They're coding faster than >G is developing.  But they do still have some catch-up to do, and OblivAeon has a lot of content, not even counting the new format.  I still think a year is probably a reasonable estimate.

And not a terrible one, either. I mean, if it took an extra year, would that be so bad?

[quote=MindWanderer]Vengeance came out in 2014, so... yeah, if you can implement and release OblivAeon and all its promos within 12 months after it finally ships, I will be very surprised.  Not a dig at Handelabra, it's just a lot of work to get something like that working.[/quote]

I agree it's a lot of work. But we didn't start working on Vengeance in 2014. We started focusing on it in September 2016, and it was released in December 2016. I expect you will be "very surprised" =)

> I'd prefer not to have to deal with Dropbox's authentication method

What alternative do you prefer? Creating new dropbox accounts is not generally a big deal, we do so for API access ourselves.

> Assuming the Android data works correctly now, yeah, I'd definitely like all three platforms.

I asked Google what's going on with the Android data, and got no response (surprise). We do also have Android/iOS data from Flurry that may be more reliable, if you're up for parsing another data file format.

> I'm still undecided whether I should handle abandoned games, in which case I'd need the start files as well as the end ones.  I can't really count them in the win/loss ratios unless I calculate all the stats twice (with and without abandoned games counting as losses).  I suppose I could do that.

So does that mean you want the start game events or not =)

I can live with that!  I doubt I'll get to play OblivAeon much with my group of friends, given how long a game will take.

All right, here we go!

http://mindwanderer.net/sotm/app-stats-lite.html
http://mindwanderer.net/sotm/app-stats-full.html

Everyone please take a look and sanity-check it.  Please bear in mind:

• This is only one month's worth of data, January 2017.  As a result, some elements which were in beta at the time (Challenge mode and a certain villain) have extremely small numbers.  It's still a data set of over 50,000 games!
• It would actually be even more games than that, but as discussed earlier in the thread, multiple identical games played in the same day are counted only once (per platform, because it was way easier that way), so Weekly One-Shots are typically counted as two wins and two losses every day the week they're active.
• Android data isn't included because the reports are messed up and I don't have the Flurry data yet.
• Game abandonment rate is extremely high.  Currently, the win rates and model coefficients only count completed games.  Even though I'm sure a good percentage of abandoned games are the players giving up, abandonment rates are far higher than even loss rates in our user-submitted data set, so it's clearly not reasonable to count abandoned games as losses.  Also, the win rates, surprisingly, match up pretty closely anyway.
• I took out the Fear Factor, because the logistic regression model captures the same concept pretty well anyway, and with a data set this large, it's pretty robust.
• The Sentinels are kind of screwed up.  Apparently you change between the Adamant and regular Sentinels the normal way, and that changes which "Instructions" card you use, which is reported in the data.  However, you can of course swap out individual team members on top of that.  For Vengeance games, I do have the data for which individual Sentinel was from which set, but I didn't bother trying to include it because A) it would be super annoying, and B) I don't have that data for normal games.
• Speaking of individual hero quirkiness, Completionist Guise is going to mess everything up a bit, since I determine game abandonment rates by comparing started games to completed games, but those won't match up if Guise swaps them.  I don't see a good way of dealing with this, but it's just one hero so I'm planning on ignoring it.

Huh.  I've only glanced through the stats, but if I'm reading it correctly, several of the villains are way easier to defeat in Advanced mode then in regular mode.

It's not that they're easier to defeat, per se.  The model doesn't take player skill into account, and people usually only take on Advanced villains when they're much more confident of their skill level.  So there's a significant selection bias.

Huh.  Maybe the different modes should be seperated (or at least seperable)?

On a different note, I find myself surprised as to how low Insula Primalis is in the Environments.  I would definately have expected it to be considered easier than Omnitron IV.  Maybe the original four environments have a new player bias as well?

Yeah, other reasons why I abandon games:

I screwed up the setup I wanted (wrong variant or hero order or whatever) and don't want to play through the existing game before redoing things.

I could probably win but it would be extremely tedious because of the state the game ended up in.

I have to pause the game and then when I come back I either just don't feel like continuing it or I'm silly and actually forget to continue it.

So will we no longer need to manually input digital games now? Cuz I kind of like having those games under my name. :B

Mindwanderer said earlier that the data from MigrantP and the manually input games will be separate. So I would continue inputting your games. 

Taking a closer look at the stats, when was this data compiled? Because, with minor exception, all my challenge games have been digital, and the only ones I haven't beaten are Miss Info and Cosmotron. Point being, Akash'Bhuta, Voss and Kaargra Warfang shouldn't come up as no games completed. :B Also, if you sort the solo villains by win rate in either version of the stats, the results do not make sense. D: Team villains come out just fine, but then they also don't have any 0's, 100's or N/A's. I guess it's being sorted alphabetically instead of numerically?

[quote="TakeWalker"]

Taking a closer look at the stats, when was this data compiled? Because, with minor exception, all my challenge games have been digital, and the only ones I haven't beaten are Miss Info and Cosmotron. Point being, Akash'Bhuta, Voss and Kaargra Warfang shouldn't come up as no games completed. :B  [/quote]

  [quote="MindWanderer"]

• This is only one month's worth of data, January 2017.  As a result, some elements which were in beta at the time (Challenge mode and a certain villain) have extremely small numbers.  It's still a data set of over 50,000 games!

[/quote]

So that is before Challenge mode was even implimented in the videogame (publicly).

I don't know if it gathers info from betatesters but if it does then I suspect a lot of  abandoned games come from encountering bugs or simple not encountering them when wanting to test out certain interactions.

I hate you Powerhound_2000, how dare you be 10 minutes faster than me yet not show up when I open the thread.

If a player were to start a game, not like the opening set-up and then press restart game 30 times until they got a start they liked would that count as 30 abandoned games?

If so, I think the high abandonment rate can possibly be attributed to a sub-set of players who restart games they don’t like the start of, until they get a start they like.

Looking villain by villain, I notice very high abandonment rates for villains like Iron Legacy, Chairman, and Dreamer all of whom have cards that can be awful if they do (or don’t) come out on turn 1 or during setup.

They are, using the filter field.  You can put "Advanced," "Challenge," or "Ultimate" there to show only those.  You can use negation–the filter supports regex–but the syntax is ridiculous.  I keep meaning to make that easier, haven't gotten around to it yet.

I'm sure it does.  Especially with the $0.99 sale they had a while back, I imagine lots of players have only the base set.  And I think " Sentinels: Learn to Play" is included as well, so Baron Blade and Insula Primalis are over-represented.

Yep, exactly.  Fixed by replacing "N/A" with a blank.

It would only count as 1, unless they did it on different days–and if they went on to complete a game, it wouldn't even count as abandoned.  In fact, and this just occurred to me, if they abandoned 30 times, lost at least once, and won at least once, it would effectively be -1 abandoned games!  Hm… I can't think of an elegant solution to this.

The learn to play edition is supposed to be recording to a separate store, but it looks like that might not be the case. Regardless you're going to be seeing a lot of tutorial games. You could potentially just ignore that particular setup.

Hurr, I am good at read :B

App stats added to the main page, http://mindwanderer.net/sotm/ !

Data includes pretty much everything now, from May 2015 to April 2017.  Over one million games recorded (not even counting the duplicate and playtest events that get dropped)!

Data is not currently added automatically.  Not only does MigrantP not have an automated update system in place (which looks like it might be burdensome on Handelabra's end), it's kind of burdensome on my end too.  I need to try to tweak my analysis tool to try to speed up adding data to the existing set.  (By comparison, I just re-analyze the whole user-submitted data set every day, and that takes about 13 minutes.  Doing the entire Handelabra set at once would take in excess of 2 hours, and uses a whole lot more CPU and RAM while it does so.)

Also, I added default regex to the villain tables that filters out the Advanced, Challenge, and Ultimate game modes.  It's confusing but it works; a more elegant solution using JS DataTables is certainly possible but would have been more work than I'm willing to put in right now.  Sometimes I do things just for myself.

Thanks MindWanderer - we're glad to help out and it's looking great!

It is very interesting how different the hero win rates are for the video game data vs. the human submitted data. It seems like the large disparity in number of abandoned games for different villains makes the villain data a bit suspect. 

The one that’s really throwing me for a loop is Tachyon’s win rate being so low. I usually think of her as one of the top 10 heroes.

I don't think Tachyon gets much use from players that have Team Leader and Super Scientific available to them.  They're substantially better in the majority of cases.  I believe she's the only hero whose variants, collectively, get used more than their base version, and not only that, but each variant gets used nearly as much as the base version all by themselves.  For most other heroes, no variant is even close; most get used an order of magnitude or more less often.

A lot of the differences also have to do with the Undo feature, I think.  Super-Scientific Tachyon, for instance--I bet she's so much better in the video game than in real life for exactly that reason.  I'm sure abandoned games have a lot to do with it too, but I'm not sure how they'd do so in a predictable way.

I would guess the self-submitted data skews more toward more advanced players, and those who wouldn't "cheat" on the tabletop. But the video game has players of all kinds.

One of the tricky things about "completion rate" is that it could consider a game started on Monday and finished on Tuesday as abandoned.

MindWanderer, how are you tracking games that have an "End Game" event but not a "Start Game" event on the same day?

Ah, good points. 

Is there an anonomized identifier that tells you which system submitted which games? If so, you could look at the stats just for devices that have been used for lots of games to get an approximation of experienced player's experiences. I bet Tachyon's and Nightmist's win rate would go up, for example. 

I don't!  There's no connection between start games and end games, and I didn't see a particular need to map them.  I just deal with the totals.  So if Legacy was in 100 started games and 50 finished games, I report his completion rate as 50%.  If I ran a whole report for a single day, yeah, I could get errors where there would be 1 completed game and 0 started games with a particular hero/villain/etc, but with aggregates that's not an issue.

Interesting fact I just noticed: One way to beat Heroic Infinitor is to get him banished by Unforgiving Wasteland while on his front side.  However, apparently this has never actually happened!  I'm guessing it's so much easier to beat him with the "no more manifestations" condition that it never comes up, but it's funny that apparently no one has ever even gone for it.

Not outside of testing versions at least. 

I actually get the testing data, too.  I'm filtering out any data that includes games with content that hasn't been released yet, but right now I'm reporting games played with Heroic Infinitor that took place before he officially went live.  I imagine there are tests performed on versions of the app that don't report their results at all, but beta testers' results are in there (unless they tested against, for example, Extremist Sky-Scraper--I'm not saying she's in there at all, but if she were, her games would be filtered out).

Hmm. That strategy would be good for challenge Heroic Infinitor, who is nigh impossible to beat on his front side.

He's hard, but I wouldn't say Nigh-Impossible.

Don't use an environment and heros with extra targets, and keep yourself healed, and he'll fizzle out.

I tried beating him with CC, and had no luck.

Freedom Tower, no small HP targets?  Just get a little DR and you're golden.

Testing data might mess the results. I might be the minority, but when I am testing, I don’t always play to win. Often, I am looking and testing particular interactions.

I had meant to say something along these lines.   Thanks for writing them for me about why I'm hesitant that they should be added.  

I already did say something!

Well I think I missed that one.   Thanks for showing your concern about it as well

MigrantP and I talked about that, but the number of test games which are actually completed is really small compared to all the legit "playing to win" data.  I'd be more worried about people aiming for achievements and unlocks, but even with that, it's just a small amount of noise in the larger signal.  Remember, we're looking at over a million games here.

Regardless of the number I'd prefer those be excluded. 

Well, I don't have any way of doing that.  They're not identified in any way.  I thought about setting up an elaborate check to see if any games were reported with characters that hadn't been released yet, setting up a whole table with the release dates of every hero, villain, and environment, but not only would that have been a great deal of trouble for very little gain, it still wouldn't necessarily catch all the playtester games.

And again, games in which people deliberately lose or do other strange things in order to unlock a particular character or achievement will be literally hundreds of times more common.  I started analyzing just January, before filtering out playtest games (challenge mode and Biomancer), and the filter ended up taking out only a dozen games or so.  At that rate, I estimate perhaps 0.03% of games being playtest games.  It's nothing compared to the other noise there will be in the data; it's less than the amount of clerical error in the user-submitted statistics project by far.

In other news, since there are so many games in the Handelabra data, I've added some more stats in the Details screen: the logistic regression model coefficient for each hero, villain, and environment assuming you've already selected that particular hero, villain, or environment. This makes it easier to see things like, for instance, the Mobile Defense Platform being unusually bad against Deadline and the Sentinels unusually good against Tormented Infinitor (common sense, but good to see numerical confirmation).  Some of the sample sizes are still really small, though, so use at your own discretion.

I did a few comparisons of the Video game stats vs. the Physical game stats. Here are some things I found interesting.

The completion rate ranking for villains tracks the win rate ranking pretty well, meaning I think people are more often abandoning games they think they will lose than games they think they will win. So, I think the win rate stats are biased by the completion rates - heroes with lower completion rates (36% like Argent Adept: Prime Warden) are probably "overrated" when you just look at win rates and heroes with higher completion rates (57% like Legacy:Young) are "underrated".

That said, some villains provoke abandonment more or less often than the win rate alone would predict. For example, more games against Wager Master and Kaargra get completed than expected, my guess is because they can win in a single card play. But fewer games against Akash and Spite get completed than expected, my guess is because they can be annoying depending on how the cards fall and that they can lock down hero damage so that people just don't want to finish even if they would eventually win.

The top 5 heroes that did better in the physical game than the video game (by rank, not win %) were Sky-Scraper (22 ranks) Tachyon (21) Visionary (18) Prime Warden Captain Cosmic (13) and Dark Watch Fixer (13). I have no idea why these heroes either underperformed in the video game (or overperformed in the physical game, depending on how you look at it).

The top 5 heroes that did better in the video game than the physical game were Setback (19) KNYFE: Rogue Agent (17) Mr. Fixer (15) KNYFE (14) and Parse (14). At least some of those seem to be due to the UNDO feature helping heroes with random base powers.

The top 3 villains that did better in the physical game than the video game were La Capitan (10) The Matriarch (8) and Cosmic Omnitron (4). The top 3 villains that did better in the video game than the physical game were Miss Information (12) Wager Master (9) and Tormented Infinitor (5). This is surprising to me, because I consider most of those villains in both categories to be easier to beat once you know their strategy, so I would have thought that would bias one way or the other.

The harder villains tend to have a much lower win rate in the video game than the physical game. I'd guess that this is because people reporting physical games are more likely to both 1) win and 2) report wins.

The top 5 most popular heroes in the video game are the base versions of Legacy, Tempest, Ra, Wraith, and Haka. Base Tachyon is the least popular hero from the first set, presumably due to her promos being very popular.

For Prime Warden Captian Cosmic I would guess it's because people would use it multiple times in a single turn and allow it to give the choice that many times. Which is very, very powerful. Dark Watch Mister Fixer may be people forgoing the destruction on it. 

If someone uses the undo feature to get a favorable result then I would also guess they would also check the decks in the physical to see what the result could be. My best guess is because people have had more practice with the heroes, thus allowing them to be more skilled by the time that hero was released in the Video Game. I think this is probably most true with Mister Fixer.

That got nerfed, you can't do that anymore.  Unless you think people are still applying the old interpretation? 

Like… cheating?  That doesn't make much sense.

I dunno, I find myself doing it more in the video game.  Somehow it seems easier to justify than if I were doing it in person.  I think that in a video game I'm more used to "anything the game allows me to do is legal" whereas in a physical game I know it's on me to not cheat.

There would still be games that were input that people would have used that. Also people would houserule that it still works like that.

I believe the proper term would be houserule.

In light of the abandonment issue, I’d love a “surrender” button to insta-lose. There are many times I get to a bad game state, where it’s pretty sure I’m going to lose, and it’s not worth spending the time to make sure.

Of course, a surrender button might be more accurate, but it also could just have the same problem, reversed?

If you had a surrender button, would you be more likely to use it only when you’re going to lose? Would you use it instead of abandoning and never abandon again? Or would you ignore it and continue to just abandon all games instead?

I know I abandon a bunch of Akash games. In many games, I have a tendency to find it difficult to stop then pick it up later. I often forget exactly what I was trying to do and the specifics of what is going on. This happens to many games, not just SotM.

Akash games tends to be longer. If I don’t finish it in one sitting, I very often abandon it instead of finishing it later.

Ah, good point. I hadn't thought of that.

Thanks for all the work on the Mindwander difficult pages.

On this page: http://mindwanderer.net/sotm/stats-full.html

there are logisitc regression coefficients for each villain.  Do those coefficiencts combine advanced and base versions?  If only base, is there a way to see the advance coefficients?

Advanced, Challenge, and Advanced+Challenge (Ultimate) are all separate, but filtered by default.  If you change the contents of the Search field you'll see the others.

Noticed that the data for the Stats project hasn't updated since 7-11.... wanted to flag it in case it wasn't an intended change to the frequency of the updates.

Thanks!  Fixed and updated.

Now that the SotM video game has a launcher API, I've adapted my calculator/randomizer to work with it: http://mindwanderer.net/sotm/random_vg.html

Please give it a shot and let me know if anything doesn't work as expected.  Especially for Steam, since I only own the game for Android.

Known issues:

• Challenge mode for team villains doesn't work at all.  This is on Handelabra's side and will be fixed in a future update.
• Variants released this month do not appear.  That's because I don't have any stats on them, so my calculator doesn't know what to do with them.
• Any variants released since the last game update won't load (you'll get the base version).  This is because the game loads the launcher-selected content before figuring out that the variants have been unlocked by Handelabra's magic button.  They'll work as soon as an update is released that includes them by default (although then there will be new variants that won't initially load).
• Because Chrono-Ranger and Sky-Scraper don't have any variants according to the default code, their behavior is a little wonky.  This will also sort itself out on the next update.
• Ultimate Trickster Kismet is in there for some reason.  Somehow someone managed to make the game think they played against her.  Obviously, selecting her for a game won't work.  Because of the sheer size of this database (over a million games), going in there manually to pull out one game is a pain in the butt and I'd rather just wait for her to become implemented for real.

Any other bugs or oddities, please let me know here.

Cool!

The Steam URL needs to be different - it's a steam:// URL that includes the sotm:// URL as a URL-encoded parameter. Right now both buttons have the same sotm:// URL.

When I click Randomize, the only thing that seems to happen is the text "Shuffled!" appears. It would be nice to see what the randomized setup is.

Fixed–there was code for that but it wasn't evaluating correctly.  At least, it's a steam:// URL now; I still can't confirm it functions correctly.

The radio buttons fill in.  If you then click Calculate, you can mouse over/tap the results window to see what the calculation is based on, or you can sort the tables by the radio button column to put the selected options at the top.

Just tried it out on Steam.... very cool!

Most of the radio buttons are not visible. It would be great if when clicking Randomize it would tell you what it did right away - perhaps instead of just saying "Shuffled!" in that space. Right now it takes 2 or 3 extra actions to find out, every time you randomize.

The Steam link is working now, thanks!

Hm... that would be a lot of information to cram into a small space.  I'd rather not make that space bigger, since its purpose is just to display one number or error message.  Right now it's just one extra click and a mouse-over, or two taps.  I could use a pop-up, but then you'd need that extra click just to dismiss it.  I suppose I could have it automatically float the chosen items to the tops of their respective tables, do you think that would be better?  Or I could have it immediately trigger the difficulty calculation, which would save the user that one click.

Definitely at the least float the chosen items to the top.

I would have never guessed in a million years that clicking Calculate + mousing over would show me that info. If you try to find out by sorting (which I also would never have guessed), it's 3 extra clicks the first time, and then 6 (!) extra clicks every next time you randomize.

IMO the results of what you randomized are the most important thing to show, and I would display them immediately in a prominent space. It could also just show the calculation result immediately as well without another click, whenever you randomize or change your selection. That operation seems to take no perceptable amount of time.

The calculate action does take a moment, especially on a slower phone.  It used to take a couple of seconds on my old Galaxy S5 and still takes a perceptible amount of a second on my new Pixel 2.  I suppose I could do it server-side asynchronously like I did with the old calculator for the tabletop version, but I'm shy about shunting too much to the server since I'm running this in a shared environment.

Anyway, I've changed it so that shuffling also results in it immediately calculating as well, and sorting the selected items to the top.

I'd like to think that you'd click on the "About" link or clicked on a table header with a sort icon on it within some time less than a million years...

Well I do write the greatest hyperbole in the known Universe after all =)

Definitely feels better to click that randomize button now!

Just checking this out. Very cool that there's an API. What are the Quick Select buttons supposed to do? I'm on Chrome/Win10 and nothing seems to happen when I click them.

Ah. I see the little lock buttons now. Recommend: Some sort of style change for each quick select button to show whether or not it's currently clicked/selected.

For each of the hero/villain/environment tables, clicking Name in the header sorts by name, but clicking the lock/unlock icon changes either unlocks or locks all rows. I found this inconsistency confusing, as I was trying to sort all of the unlocked rows to the top.

It would be nice to be able to exclude ultimate, challenge, and advanced modes.

Fair enough.  The column head now sorts, and the lock icon that used to be there is now in the section legend.

That I'm not going to do.  I'd have to check what should be highlighted every time any lock status was toggled, and with this framework, that could be resource-intensive on slower devices.

I had mixed feelings about this one.  One of my reasons for doing this was to encourage people to try those modes.  But I went ahead and did it, defaulting to leaving them available.

Thanks

Hmm... Something seems to be up with the stats project. The games I've logged after 2/12/18 aren't showing up. I clicked on a few other names in the contributers tab and didn't see any with games from after 2/12.

For a second I thought I had been banned from the project but now I see other contributers are not able to addi games afters 2/12. Does anyone know if the project is done updating?

Just checked the back-end spreadsheet, and yeah, after 2/12 there are no more entries. lynkfox will have to look into it.

Thanks for setting up that model and the auto updating. I really enjoy seeing the coefficients. Would it be a pain to merge the challenge mode for Agent of gloom spite and for skinwalker gloomy. Right now I am pretty sure that people are more likely to enter games as agent of gloom spite if they lose and skinwalker gloom if they win. The games are actually identical but I don't expect the coeficients to merge because it is sort of odd to enter the game as skin walker gloom if you never even got to that phase of the game and it is odd to enter the game as agent spite if it was hours ago that you defeated him and moved on to gloom. Like I said though, I really appreciate the work that has been put in.

i'm... stumped. I think it has to do with the new forms. its still getting responses (you can see them in the Individual section on the /form/ from the editors side - last active response beside my Tests is from 2/18 by abnelson ) but none of them are showing up on the spreadsheet.

i'll keep looking at it.

Doesn’t Google have a maximum size for their spreadsheets?

Oh. May need a new form address: https://goo.gl/forms/cQccMUcaXlqBumME3

(welll, the old link still works, thats just a shorter one)

Not since the new update, a few years ago. (we definitely ran into that issue at about 10k entries)

Its been accepting allt he data, but it hasn't been getting into the sheets. Anything from the last week is in there, but not in the sheet it seems. Going over it with MindWanderer now to see what I need to do or help in order to get the script working properly.

That link and the old one go to exactly the same place when fully resolved.

I don't think there's anything I can do if the data doesn't go into the sheet somehow.

Okay. It seems to be accepting new entries again. Seems I say, but its hard to tell.

Any entry entered between 2/12/18 after 11pm est and 2/18/18 about 2pmest  insn't going to be in the site Mindwanderer puts up. They are in the system, but not in the spreedsheet. I don't think that we'll get them either. No idea what happened. I'll keep an eye on it and see.

If anyone else sees something like that, just send me a pm so I can jump on it as early as posisble.

Seems like there may be an issue with new entries again.

Wanted to note that there still seems to be a problem where new entries aren't being recorded.

Hey all.

My daughter was born in Jan, so it has been a couple of very hectic months for me. I poked around, and the entries were all there - just the Google Form wasn't sending them into the Sheet for Mindwarnder to parse into his webpage.

Seems to be resolved for the moment (May even have got the missing stuff from the last time this happened, hah!)

I'm looking into learning SQL now to move us to something more permenant and better able to handle this number of responses. We are well into the 50k response range I believe, and its getting more and more clear we're running up against the limits of googlesheets. 

So if anyone out there is an SQL/Access/some other database admin who wants to donate some freetime to setting something up, let me know xD

As it happens, I can do SQL, too.  Not an expert, but I can get by.  I could just code up a whole new form and host it myself, if that's the direction you're going.  The advantage of GDocs is that it's easy to share and collaborate.  I don't know if it's possible or advisable to host any of this stuff on the wiki server, so it's not all just on me.

Congratulations lynkfox.

Data will be wonky today due to very old data being magically restored somehow.  This includes some of the old data format options.  I think I corrected them all but the website will take time to process the update and I won't be able to double-check the results until tomorrow.

Edit: It looks good to me now.  Please let me know if you see anything wonky.

I’m not seeing anything weird… data looks fine.

The wiki server could very well host it, i think. Wiki uses sql databases, and its just a normal webserver. public_html directory and everything. We could probably make that work without to much issue. 

Not that I know how to install a database onto a server without something like the mediawiki install wrapper but… xD

No hurry. what we've done should fix it for a couple more years, lol. We'll see what happens.

The system does appear to accepting new entries. It however appears that previously logged entries between 2/18/18 and 4/26/18 are now no longer logged. I know that in April they were there but presumably one of the fixes created a small bug that meant that data did not get included. Thank you for everything!

They disappeared for some reason, but fortunately I still had them.  They've been replaced and I'm re-running the data now.

Just a side note and I’m not sure how you’d filter this exactly but Ultimate Trickster Kismet is listed in video game stats.   However, we only have advanced rules at this point so those should apply towards records against just Advanced Kismet.  

One would think!  However, there are over a million games in the data set, and the JSON data I get from Handelabra doesn't make it easy to search for with any existing tools.  I'd have to write a custom script just to locate that one case, and it would probably take an hour or more to run.  So, sorry, but I'm just going to live with it.  It'll disappear as noise once her challenge mode is released officially.

I'm more worried about the fact that if it could happen to Trickster Kismet, it probably happened elsewhere in the data set as well.  But I have no way of knowing.

Just bringing it to your attentional it stuck out to me.  

Hmmm… looks like no new games have come in since you fixed the issue on 5/22.

Sigh. I believe they are all in the form, just ot getting transfered into the excel sheets. It will take me a few days but I’ll look into it when I can. Keep putting in games, for now

well, the results are in there:

https://docs.google.com/spreadsheets/d/1bVppJL4rC5lWULLGZ7AP5xH6YZLYsv86Wme1SpU6agE/edit?usp=sharing

21k results, with the latest as of today at the bottom (takes a while to load those responses, heh)

I think it has something to do with how the new google forms works. It stores the data in the form, but isn't automatically populating it to the spreadsheet. I'm not sure how to fix this (it may not be fixable, because of how much data we have) - but once i go and check on the form, the data gets updated.

You should see a change on mindwanderers statistics page tonight.

I'm starting to poke at SQL and setting up a proper database for this. This will take some time, but theres no hurry. The data is there, It just needs to be poked every now and then.

Hmmm...

Looks like Stuntman was never entered into the submission form.

Yeah, I e-mailed lynkfox about this today.

I'd be willing to bet a bunch of other people did, too. :B

hes there now. Enjoy.

Awesome! Thanks Lynkfox!

Lynkfox, do you think you can start poking the form regularly for a while?  It'll be exciting to see how the new data starts shaping up.  Should be a bunch for Benchmark and Stuntman already, and the rest of OblivAeon should be trickling in.

Well, stats for July, including Stuntman, Benchmark, and the Celestial Tribunal, are now loaded into the video game data.  Unfortunately, not the hero variants, since we haven't found them yet.

Thanks! I've been waiting to get some sense of where Stuntman and Benchmark fit in. I guess until they update in the other form entry model I will use Skyscaper as a stand-in for Stuntman when estimating difficulty and Naturalist for Benchmark. Both of those seem about right. I had the sense that Stuntman was a bit better than Benchmark and that they were both above average. I am very surprised to see that in the video game the tribunal is showing up as very difficult. The third hardest. That seems obsurd I wonder if video game players haven't caught on to the way the environment can finish you off so you do have to pay attention to it even though it doesn't do that much to hurt you. Previously it was registering as between Mobil defense and Freedom tower while now it is between Ruins of Atlantis and Omnitron IV. I have a hard time reconciling that discrepency.

I think it’s too easy to let the environment go on autopilot and end up with a dead representative before you realize it. In the detailed stats, you can see most of the losses are due to representative death, not hero incapacitation.

I poked it.

Sorry. took 5 classes for summer semseter, whichi s 6 weeks shorter than normal. That was a mistake.

All information up through today (8/5/18) is now in the spreedsheet.

https://docs.google.com/spreadsheets/d/1bVppJL4rC5lWULLGZ7AP5xH6YZLYsv86Wme1SpU6agE/edit?usp=sharing

(just to make sure that is the one you're using Mindwanderer. I think that is.... i just don't recall.

I can see my last couple games on that spreadsheet, but they're not showing up on my contributor page in MindWanderer's site. Just fyi. :B

I'd shut off the scraper because there hadn't been any new data to run it on.  Should be up-to-date now; moreover, it seems to be populating on its own again (for now).

We'll have to talk about how to handle OblivAeon games.  The format is so different it probably needs an entirely new form.  But it would be really nice to get stats on the shields and scions.

We can do it like I did vengeance style, it’s own page on the form.

I haven’t really played OblivAeon since very very early in the playtesting, so I’m not sure what all questions we’d need to ask however

It's a little more complicated than that.  Vengeance just adds an extra page where you can put in the individual villains.  OblivAeon would need to not only include new info (Scions and shields), but also multiple heroes per player and up to 5 environments.  There are also special, let's say circumstances, that would be good to include on a per-player basis (not to go into spoilers territory).

I can see that. There are a LOT of factors to consider when playing Oblivaeon, and considering how the game's played, there's also virtually no limit to how many characters can be used each game. Also, unlike a normal game, it's nowhere as big a deal if a character is incapacitated in Oblivaeon. In fact, having a bunch of incapacitated characters actually improves your team's chances, late game.

Hmm

We already did some corner cuts before (like can’t mix and match Southwest Sentinel or Skyscraper varients) due to complexity

Ugh.

I was thinking about it, and yeah, we need a whole separate form. That sucks because low data points until video game gets out but uuugh there are so many variables.

Which scions starred, entered play, flipped

Shields used

Missions completed /failed

Who got what rewards?

OblivAeon status, destruction track

Starting heroes and follow ups

Environments (x2) and followed after destroyed.

And I’m sure I’m forgetting something.

@MigrantP if you’re looking, make sure your video game recording can find Alll that

We also don't want to make people put down too much.  They may not remember every single reward anyone got, or the exact number of counters on the Destruction card.  I'm thinking just:

• Shield card used
• Which Scions made an appearance and whether they were fully defeated
• What heroes were used by each player
• What environments entered play
• What phase the game ended in
• The spoilery thing I'm not going to say in this thread

We can add fields for rewards but leave it optional, see how good the data we get there is.  Same for the Countdown.

That seems reasonable to me. 

I think the relative power levels of the heroes in OblivAeon fights is really quite different from any other fight, in that the best heroes are good at either 1) collecting rewards and having strong incap abilities early game or 2) using rewards late game. For example, in my opinion Bunker is in the stronger half of heroes versus OblivAeon. But... I'm not sure the stats will be that useful for determining hero power levels against OblivAeon because so much depends on the order in which you select heroes - if your first hero gets no rewards, you want to switch to another hero who is good at collecting rewards, not a hero who is good at using rewards. 

Mindwanderer, is there a JSON file available through your website that includes the number of games?  You kindly hooked me up to the JSON file that has the coefficients of the model, but I can think of ways to use the # of games as well for our local campaign (see the link for what I have been doing):

https://docs.google.com/spreadsheets/d/1IMCG6skD4yiRJgOcFW4w4L2hpkgUI-gWzAZC_s_XN3g/edit?usp=sharing

I can make that available to you, if you want it, but it's a large file so I'm not going to post it publicly.  Let me know whether you're using the card game or video game data and I'll PM you a link.

is everything still working here? xD

It's still logging games automatically as of this morning's pull. Yay!

Now we just need OblivAeon mode.

Here's some weirdness: I just ran August's statistics, which now include Action Hero Stuntman and S&D Benchmark, and AH Stuntman absolutely rocketed to the top of the chart!  There's bias there because only more avid players have even unlocked him, but I've never seen that before, especially since the model devalues results with small sample sizes.  S&D Benchmark is more where I expected him--in the top 10 but behind Legacy and Tachyon.

Meanwhile, the Celestial Tribunal is ranked as the 3rd-hardest environment, probably because people haven't figured out how it works.  Losses in the Tribunal are about twice as likely to be due to the Represenative being killed as to the hero team being incapacitated.

I suspected Action Hero Stuntman might do well but not that well. 

I'm not really surprised by Action Hero Stuntman - I always regarded him as insanely strong.

Benchmark Supply and Demand surprises me a lot more since in playtesting I only thought it was marginally better than the original.

I'm also surprised by how highly the base versions of Benchmark and Stuntman are ranked right now.

Yeah but to usurp TLT I don’t think anyone could guess that.   Though a lot of games have Action Hero Stuntman teamed up with TLT

After having gotten some real world experience playing him, in the video game (not just with my print and play playtester files) I can see how he can really get going. If he avoids some equipment destruction he is a beast once he has 3-4 hardware adn 3-4 software out. 

Same. I reported a bunch of games where he outdamaged the rest of the heroes combined and a game where he did 46 damage on turn 1. I called him OP, but others said I was just lucky or cherry picking games to make a point, so I feel a bit vindicated here (although to be fair, pretty much all of the playtesters did say that they felt it was a top tier promo as well). 

The real problem with balancing the base versus the promo was that the best way to do so was to nerf his card draw, which made him less fun, so it ended up with a fun but kind of OP promo.

EDIT - I would not have guessed that the published version of Action Hero Stuntman would beat TLT. It may come down over time. Also, b/c card draw is his one weakness, he is insane paired with TLT. 

In my last playtest game we nickamed s&d Benchmark, “Benchmark the Immortal” as he survived the whole game.

I feel nearly certain that somewhere in this thread my question is answered but the search function is not turning up something obvious, so...

Why are there listings for both Agent of Gloom Spite (Challenge + Advanced) and Skinwalker Gloomweaver (Challenge + Advanced).  Aren't they the same thing?

In the video game data, they're lumped together because they're functionally and technically identical--if you select either one you get the same game.  But in the hand-entered data it's manual, and they can be entered differently by the humans who filled out the form, so that separation is maintained.  Someone might enter a win against Challenge Spite but a loss against Challenge Gloomweaver, for instance.

Why can't they be aggregated for the manual games?  They are literally the same thing, aren't they? 

I get that you keep both entries in the form given the way the form works (first choose villain, then choose whether it is a challenge or not), but as far as I can tell whether you are playing with cards or the tablet, Agent of Gloom Spite Challenge is identical to Skinwalker Gloomweaver Challenge (and similarly with Challenge + Advanced).   I also get that this would be an extra coding step on the difficulty calculation end, and I may be seriously underestimating the complication of adding that step.  But in the difficultly values it would make sense to me to only include Agent of Gloom Spite (Challenge) and Agent of Gloom Spite (Challenge + Advanced) and then put a note in somewhere that just reminds the user that this is the same as Skinwalker Gloomweaver (Challenge) / (Challenge + Advanced).  

Don't get me wrong, you do a great job on this stuff.  I might be complaining, but its a minor complaint and I'm happy to be told "get lost".  :-)

EDIT: interestingly, the model difficulty value for the two Challenge + Advanced entries are very different, but that seems very likely due to the fact they are only based on 17 games each.  In reality, that's 34 games of the same thing.

It's because I don't know how people are answering the questions.  If they win both games, are they entering them once or twice?  If they win against Spite but lose to Gloomweaver, do they count that as one loss or one loss and one win?  If they beat Spite but run out of time to play for the day, do they enter the game at all?  If they lose to Spite, do they log a loss against Spite, Gloomweaver, or both?  What if a hero is incapacitated in one fight or the other?  The question is ambiguous, so aggregating them would make it even more unclear what's being described.  If we'd set it up to make it impossible to select them separately, that would be one thing, but as it is it's unfixable.

I personally have logged all my games only once and listed them as “Challenge Skinwalker Gloomweaver” even if I lost to Spiite before seeing Gloomy.

Maybe we should come up with a standard way of logging these games moving forward so that we can draw more meaningful conclusions from the data. If everyone’s filling it in differently it’s kind of not that helpful to track.

Ah, ok. I see the issue.  That does make sense.  I guess I would be inclined to simply drop those four villain entries (Agent Spite C, Agent Spite C+A, Skin Gloom C, Skin Gloom C+A) from the model entirely and not report them, given the concerns you raise above, but I get it.  

The only reason this matters to me, really, is that I have a spreadsheet set up now that randomly picks games for me to play on my iPad but weights the selection (villains, heroes, environment) by the inverse of the number of games that have been reported.  So I’m going to be doing a lot of Challenge + Advanced games over the next little while with obscure promo heroes.  :-)   

EDIT:  it looks like there are only 73 total games across those four villains.  If LynkFox were willing to alter the form in some fashion (even something as small as a note next to the Agent of Gloom Spite and Skinwalker Gloomweaver entries on the first page saying “if you played challenge mode use agent of gloom spite and record for the entire game played against both villains”) it would seem reasonable to me to drop those 73 games and start afresh.  But I’m glad other people are doing this work, not me, so I"m happy as is. 

As much as I would -love- that Dandolo, there are many many people who log games there who don't visit the forum, and even where I tried to standardize things, it still gets inputed wrong. :P

Always recall that any given coding project / interent thing can be broken because someone will not be thinking in the same way you are and will do something you never expected.

As it is, I'm currently learning SQL and Database management, so I plan on transfering the project into a far more robust and powerful system, which will hopefully help clean up a lot of this!

Well, that's exciting. :D

Mindwanderer, I've been pondering Fear Factor in the statistics, and I am confused by something.  All of the Villain fear factors are positive.  How does this come about?  I'm trying to wrap my head around both the math that makes that happen, but also what it means.  I think it implies that the win rates against ALL villains are somehow inflated, but that doesn't seem possible. Could you provide some insight?

Good question, and it took me a minute of thinking about it to figure it out, but it's actually simple: More successful heroes are more popular than less successful heroes overall, so every villain is more likely to have been challenged by "better" heroes than by "worse" heroes.  There aren't any villains for which people actually prefer to use "worse" heroes against, on average, so all the values are positive.

That’s interesting.  I really don’t see much correlation between win rate and number of games (even accounting for the age of the card), to directly support the idea that “better” heroes are played more often.  But its not clear you would see it, really.  All it takes is a tendency to have at least one good hero per game and the fear factors would then end up net positive.  The good heroes could be “carrying” the bad, essentially.

I had never really thought about it this way before, but this “carrying” idea matches with anecdotal experience.  I’ve been in many a game that was won, but one or more heroes in the game (usually the likes of Bunker, Absolute Zero, Argent Adept, heroes that take a lot of set up time) didn’t really contribute much overall to the game.  It might have been a five hero game, but really it was a three or four hero game with a little extra help.  To put it another way, it seems possible that “good” heroes increase the overall chance of a win more than “bad” heroes decrease it. 

This morning it occurred to me that there is another effect that could be affecting the fear factor for villains: "publication" bias.

Looking at the video game data, the completion rates for games can be as low as 25%.  Now there are all kinds of reasons why games would not be completed, but I think it is safe to assume that a large fraction of those games would have been losses.  Therefore, the win rates from the video game data are almost certainly inflated compared to some kind of theoretical true "win rate" which includes games that were given up as futile.  This is similar to publication bias in scientific reasearch; papers with positive results are more likely to be published.

Now, I personally will play a game to the bitter end nearly every time, but in hindsight I recognize that I am likely an outlier; most people are not such gluttons for tedious punishment in their spare time.  So it is very likely that the submitted games also are affected by this publication bias.  And since anecdotally games I have been inclined to give up seem to be far more frequently related to villain effects (e.g. the villain has generated a situation that seems impossible) rather than hero effects (e.g. all the heroes seem useless) this could affect the villain fear factor much more than the hero.  Also, I think a larger proportion of submitted games are coming from the app these days, and I suspect it is easier to end a solo game on the app for futility than it is to end a social game played with cards.

Does this seem likely?  Its really hard to judge, because the submission form does not allow for submission of incomplete games.  I suspect some submitters submit technically incomplete games where the end result was patently obvious.  

It’s an imperfect solution but you could count incomplete games as losses. I suspect you are right and most of them would have been had they been completed.

It might be worth doing the calculations just to see the effect

I actuallly did do that when I first got the video game data, to decide what the most appropriate way to handle incomplete games was.  Very little changed when I treated them as losses.  Obviously the win rates went down, but the relative rankings and the regression coefficients were basically identical.

How do you calculate the Fear Factor?

It's in the notes, although not expressed mathematically.  Basically, for heroes, for each game, look up the average win rate of the villain in that game.  Sum across all games and then divide by the number of games.  Subtract the average win rate of all villains.  For villains, do the same but use hero win rates.

Huh, that’s very interesting.  How could that be?  The regression coefficients are predicting win rate, and if win rate is lower, then something has to shift.  As an example, if you cut all win rates by half in the data, then something in the coefficients would have to change in such a way that the predicted win probs generated on the calculator and randomizer pages produced values that were half what they produced previously.    

I’m going to go out on a limb and say that I think you mean the relative coefficients for villains, heroes, environmenst remained much the same, and the only thing that changed was the intercept parameter.  A lower intercept would reduce ALL win rates at once across the board in a consistent (albeit non-linear) fashion.  For example, multiplying the current intercept (~-2.12) by a factor 1.1. reduces predicted win rates by about 1-5% (more for win rates closer to 50% and the less as you get towards 0% and 100%).  A factor of 1.5 reduces them by 10-20%. 

If I’m right and that’s what you meant, that means that completion rate is uncorrelated with choice of villain, hero, environment, etc.  That is interesting to me!  

Just to be clear, I really am not trying to make trouble here.  I LOVE that you and Lynkfox do all this work, and as far as I am concerned you can keep doing it exactly how you are doing it.  I’m just fascinated by this rich trove of data and all the things that can conceivably be learned from it about this game I love.  

EDIT: I would hypothesize that if there is any correllation between completion rate and villain (which I think you are saying there might not be), it would not be correllated with villain difficulty so much as villain “frustration factor”.  That is, villains that generate annoying situations where you have to do the same repeated actions over and over again in the app on the way to losing (or winning for that matter).  Villains that just take FOREVER to deal with.  Villains that generate obviously impossible situations but will still take a while to beat. Villains with low “frustration factor” are villains that are straightforward, or that play quickly, or that end the game in sudden, unexpected ways.  

Ten Lowest Completion Rate villains (Lowest first): Gloomweaver: Skinwalker, Spite, Citizen Dawn, Akash’bhuta, Spite: Agent of Gloom, The Matriarch, The Chairman, Miss Information, Gloomweaver, La Capitan

Ten Highest Completion Rate Villains  (Highest last): Ambuscade, Plague Rat, Deadline, Wager Master, Infinitor: Tormented, Baron Blade, Kismet, Chokepoint, Kaargra Warfang, Baron Blade: Mad Bomber

Looking at those lists, I feel like my frustation factor idea is born out, with emphasis on the sudden ending issue.  

You got it exactly right--the intercept going down was the major change.  IIRC there were some small changes to the other coefficients (i.e. the slope), but the rankings were the same.

There's probably a correlation between abandonment rate and length of game.  Unfortunately we don't have good data on length of game.  I'd have to compare the self-report data to the digital data, and the "number of rounds" field is optional (and I know I personally fill it in for short games, when I remember, and leave it blank for longer ones, where I lose track).  Looking at it ad-hoc, it certainly seems to hold true--games with abandonment rates disproportionate to win rates include Challenge/Ultimate Akash'Bhuta, Ultimate AoG Spite, and Ultimate Ennead.

Huh. I would’ve done it by measuring how far each hero is from the median of the win rate histogram. Interesting.

I love what you've done with the statistics results! The randomizer is AWESOME! I wanted to point out that stuntman and his variant havn't been added to the randomizer and calculator yet. Otherwise, it's really great! I love going through and playing games based on what is needed for the stats.

Whoops, don't know how I missed Stuntman!  Fixed.

Just a heads-up (and I mentioned it in the notes of a response I sent for a game I played earlier tonight in the environment), Celestial Tribunal's also missing, but only from the video game's randomizer. I figured now was a good time to bring it up, since I imagine the Void Guard (sans variants, for now) are getting plugged in there soon.

Bah, it was in the page, just missing the "The".  Fixed.

I noticed my games for the last couple days haven't shown up in the stats. Looks like that issue with the data sheets is cropping up again.

Edit: Nevermind, my games from 1/2 did show up. So I must have either forgotten to record my games from the previous days or made some other mistake in entering them.

Boom.

(Note that the data is necessarily limited.  A lot of games go incomplete and there are a lot of combinations.  In particular, trying to infer anything about Advanced is pretty pointless.)

Nicely done! :D Interesting how the Sentinels always die.

So is this all going to be generated from the digital game, then? Nothing for us to input? Mostly because I don't want to have to write down every little thing that happens in an Oblivaeon game. :B

Yeah, Lynkfox and I couldn't come up with a reasonable way of adding it into the existing Google form, plus there would probably never been enough data to make analysis useful.  There have already been 20 times as many games with the OblivAeon heroes in digital as in physical, just half a month after they came out.  Even as it is I don't have the greatest confidence in the reliability.

Says something that the Primary Objective is played more then all other shield combined

It’s the easiest shield, and the mode is already difficult?

Hm, TakeWalker actually seems to have caught a bug.  VGBM posted a screenshot in Discord that shows him winning a game with The Sentinels still active--although the screen was bugged because they'd replaced the Adamant Sentinels and was showing the HP of all eight of them.  It's possible it's the same bug, though.

When will the stats for these new heroes and environments show up for the other game play modes?  I’m not trying to press you on getting that done immediately as I’m sure what you put together for OblivAeon mode was quite the undertaking.  Just curious moreso to see those other stats.  

They're already there.

I don’t find them going to the lite stats page https://mindwanderer.net/sotm/app-stats-lite.html

Really interesting! Do the win rates at the bottom count unfinished games as losses or is that win rate the win rate of concluded games?

Especially interesting to see that only roughly 1 in 10 ObliVeon games are played to conclusion- I suppose that isn’t too surprising- the writing is usually on the wall long before you’ve actually lost and it’s a long game if you are fairly certain of defeat.

I’ve heard this before. Most of the games we’ve played, we believed we were going to lose. However, we still haven’t lost since it was released. It’s pretty amazing how fast the heroes can recover if they fight hard enough. ;-) 

(Mind you, I get it. For me, I don’t find OblivAeon as enjoyable of a game because it’s torturously painful to play through. I’ll stick with traditional or even team games for the most part!) 

Argeed.

I've only won 2 out of about 10 Oblivaeon digital games, but in both of the wins there were several moments when I thought I was FOR SURE going to lose, but played them out anyway. And then I won!

Do you have something cached maybe?  Because they’re there.

That must be it as if I use a private browser session they do show up.   I’ve not had any kind of  caching issue with that page before and the full stats page had them which is why this struck me as odd. 

I’ve found this to be true in Sentinels from the beginning. This is really a game where it’s not over 'till it’s over!

Oh, definitely- it’s very rare for me to quit a game even if it’s looking bad, as they can be some of the more interesting games because you’re on the edge of losing. I would imagine though due to the more ‘instant’ nature of digital play that there are definitely people who just rerack the game if it’s too much of a slog, and I guess it isn’t too shocking that OblivAeon mode increases that rate - it’s definitely a 3 course dinner rather than a snack, and time is precious. I’ve been enjoying the mode, but I’ve still only played it 4 times since release (2 wins, 2 losses) despite playing multiple digital solo games a day since then - it’s a commitment!

Hi MindWanderer:  I was looking at the "Details" pages recently, and I was surprised by the number of hero x villain combinations where there was a chi-square p-value of less than 0.05.  

I recognize it would not be easy to do, but I wonder if it would be possible to run the model once with the first order interaction terms?  I ask because with a lot more data than you may have had at first, some of those interaction terms may now have a significant relationship to the overall difficulty.  This would mean that the estimated difficulty in your calculator is over/under-estimating in some circumstances.

Or maybe not.  The chi-square test you are using on the deviations on the details page seems very sensitive; there are many cases where there are just a handful of games but a significant p-value.

Anyway, as usual thanks for all your work, and I understand if what I am asking for is impossible.  But I would love to see the co-efficients and p-values from those interaction terms at least once.

I'd love to do that, but unfortunately I don't have the computing power.  On the user-submitted data, the program takes over half an hour to run, most of which is reformatting the JSON data I get into dummy-coded dataframes.  On the video game data, when I initially ran the full data set, it took over 8 hours.  Fortunately, now I have that, so I just have to add a month's worth at a time, but recoding everything to add the interaction terms would take the program... what, probably a month to run at that point?  And a dataframe that large wouldn't even fit in memory, even if it were mostly zeros.  It probably could be done, but not using anything like my current method, and I wouldn't know how.

Good grief!  I withdraw the request.

Sounds like a highly parallelizable process.  Would there be any benefit to breaking the task into smaller subtasks that could be executed by forum members?

The recoding probably could be, but that wouldn’t help with the issue of the dataframe just being too big to work with without setting up a Hadoop cluster or something, which is way beyond my level of expertise and resources.  Even if it were, it would take a lot of work and organization, and frankly, to me, it’s not worth the amount of time it would take.  Plus it would require distributing Handelabra’s raw data to a bunch of people I don’t know, and I don’t have permission to do that.

The "impossible game" thread made me wonder if it would be feasible to have a page showing game setups that have never been won. Probably not including hero/villain/environment ordering, to make the combinatorics less outrageous. And maybe just game setups that have only been lost =)

The interesting aspect to games that have no wins would be that you’d also get to see setups that have never been attempted, but I guess that requires you to create all possible setups.

Well, I've written some code for it.  I'll try running it the next time I compile the stats and see what happens.

Anything interesting come of that with the May data, MindWanderer?

Unfortunately not.  In fact, including it in the general analytics script turned out to be a terrible idea, because trying to track all that possibility space was massively resource-intensive.  The script had to be killed, the data corrupted, and I had to cut back to using just the last 6 months' worth of data, at least for now.  Only got it working again a couple of days ago.  Fortunately it still seems to be generally representative of the whole--and might even be a good thing, as it's more responsive to the current "meta".

Oh no! Sorry.

I’m not sure if some data was lost but looking in lite and full video game stats the Sentinels only have 18 games to the Adamant having 2725 games.  

Thanks, that's an error in my code somewhere.

Edit: Actually, it was a change in how Handelabra encoded the data.  Formerly, when you played with The Sentinels, it would only specify if you used an Adamant member.  If you used a vanilla Sentinel it just wouldn't specify.  Now it does list the normal Sentinels.  I wasn't expecting to see that data, so I only picked them up if someone selected the vanilla Sentinels team but then manually swapped out all the individual members for Adamant.

Yeah, I know it really shouldn't do that, but usually people keep the team together, and figuring out all the combinations of each possible set of Sentinels members was more than I wanted to deal with.

Is this project still being updated?  If so, I was curious if data will be collected for the new challenge modes now available for the video game.

The spreadsheet still exists, and I know I've at least been logging games whenever I play legit ones. :)

It's worth noting a few team villains don't have challenge modes listed in the results. I can't remember now if it was just challenge or challenge-advenced, but it was a bunch of Villains villains.

Looks like ones submitted by form have been updated but from the digital version haven't been since 10/11/2019

The digital version changed its data format then, and I need to redo my code to account for it.  I didn't consider it a priority at the time because we had a decent amount of data already and new games weren't likely to change the spread.  Now that we have new challenge modes, I do need to take care of it, but I'm not going to have a chance to do that until the whole COVID situation changes.

O. M. G. Its been so long.

Did you know the Sentinels Statistics Project is still going? Still Accepting game data?

Did you know it has over 32,000 entries?

Did you know that i just spent all weekend coding a solution to get it out of the stupid potato GoogleSheets and into a Real database? Oh mommy, daddy, Sentinel Statistics is growing up right before our eyes!

Its not done yet, but!.. I need to write 3 more insert statement formats for 3 more sets of data, then all 32k can be uploaded to a mySQL server im hosting in AWS. I also have about 1/3rd of a resteful API in front of it to retrieve said data.

While I do plan on building a very limited front end, my React skills are pretty much crap. So! If you have any desire, I may be back around in a little while with some json schemas to you can use to start drawing data from the RDS to do whatever you want with it.

We’ll see. Lots of little bugs to fix still, but its going!

yo! this is exciting

Should add to this:

If you want to help out with the Front End, im planning a basic React web app, a node library for an SDK to go with the api, and maybe even a stand alone Electron “app” for use on desktop/mobile/ect.

if you want to help with those things, contact me on discord: lynkfox#4882 - you can find me in the Letters Page discord and Realm of Discord discord (heh) - feel free to toss me a message if you want to help out.

Let me know if you want access to the digital version data!

Does the sidekick app keep track of data like this? If that were an option, I would definitely opt in!

Sidekick has a link in it to the form submission, it doesn’t keep track of any data itself.

Hopefully ill be providing an api and SDK… (python and Node probably) if you wanted to make use odnthat in the future…

Also… maybe? On digital data. Not sure yet. This is at the momrnt mostly just for a better serving up of the user data

Just in case anyone is curious:

31298 entries retrieved in 4.0733235 seconds
… parsed in 16.9439467 seconds
… insert statements created in 0.2638696999999972 seconds
… inserted in 14.0668617 seconds

*Complete.

• 952 Users added.
• 25609 Unique Hero Team setups added.
• 2330 Unique Opponent Team setups added.
• 31222 Game data inserted.
  in 35.348001599999996 total seconds

So the interesting thing i see here is the 25609 Unique Hero Team Setups

(this has something to do with the way I am storing a given hero team for a given game, and creating a unique hash for it - so i can run analysis on it more efficiently.

However, what I did do was preserve the order of the team → so Bunker, Tachyon, Legacy is considered a different team than Bunker Legacy Tachyon, which is different than Legacy Bunker Tachyon … and so on.

Im not sure if this is worth recording or not. Thoughts?

image880×108 9.92 KB

hmmm. I think i need to do some validation…

Ah yes, the greatest hero team ever known. Absolute Zero, Absolute Zero, Absolute Zero and Absolute Zero.

Someone iced up the whole team.

Huh, I didn’t know that Null, the Inversiverse Captain Cosmic, had a deck. ; )

Yep. I still hate writing SQL queries. But… its the only way to make this kind of dynamic recall possible (my sql query is pretty brute force here, im sure there is a more elegant way to write these)

but this kind of query wouldnt be possible on the fly with a Dynamo (noSql) option so… here we are

image1265×546 35.6 KB

Progress! Api is working (ish)

image742×517 25 KB

Formatting work to go and obviously, its showing villain response stats instead of hero, but alas. Its something!

Aww yis.

And now. Time for bed! hah

Retrieval backend is just about finished. Still need a way to add games via api though.

If you’re someone who is a developer of front end sites and would like to help me out on building some, reach out! I’m starting the build in react, and my design skills are nil so yeah. Help is appreciated.

Or if you’d like to know how to access all this, well I’m not quite there yet, but I’ll be providing the retrieval link in the near future for anyone to use, and a private, API key enforced insert link (and a JSON schema for validating your input against) for those who want to hook into it directly.

image826×523 29.4 KB

image764×575 35.7 KB

image780×635 38.2 KB

image776×539 33.6 KB

image912×544 29.6 KB

look at the cool new things you can do!!

most popular teams (just by team members - position in play order does not factor into this)

like whats up there Haka, Legacy, Tempest, Visionary, Wraith - more popular even than the DW?

Is the CR, LA Comodora, Omnitron, Visionary the secret Time Traveler group?

Ahhh isnt data fun.

The Haka, Legacy, Visionary, Tempest, and Wraith likely has to do with cheese potential out of the base box. Viz and Tempest helps Legacy keeps his more powerful Ongoings out. Wraith and Viz help manage the villain deck. Haka can swallow villain targets with Savage Mana.

Thanks to arenson9 for help on an sql query, here is some more interesting data

and villains too!

obviously non team villains are not going to ever be anywhere other than pos1 but still!

and we can’t forget environments!

but what position is the environment in??? D:

I mean, you can find out for two out of five in OblivAeon mode. I assume, with absolutely no knowledge of coding.

While some of these make obvious sense (Legacy, GI Bunker, …) a bunch don’t! Why doesn’t Expatriette go last? What’s up with the Naturalist? Why so many games with Akash’Thriya in the lead?! I’m particularly curious why some characters’ distributions peak in the middle (i.e. usually neither first nor last), and why some aren’t uniform across the middle range but peak at, like, position #3. Is there any weighting applied depending on ?

I’m tempted to wonder if there’s a way you could capture whether a particular item is relevant for an unlock/achievement but I’m sure that would be a huge effort so don’t take it as more than idle wondering.

Its also important to remember that these have been collected since 2014 (there is actually 2 more years worth of data that is in a separate google sheet that i haven’t uploaded to the database yet) and so older heroes/villains/env have more time to get more representation. I will have to, when I start work on the Front End, plot out some of these things in interesting graphs - like : what density of entries over time - I would bet that there were far more games being added in the early days then there are now, meaning those early characters have a larger representation.

Edit: Oh and no weighting. Just straight raw counts.

I wonder what the most informative way to show order is. 3rd place in a 3-hero game means something very different from 3rd place in a 5-hero game.

I will definitely say that I’m seeing a strong indication that the games I played using thematic teams in specific orders had a big influence. For example, I played over 130 games with Akash’Thriya first in the order which accounts for more than half her games in that position.

Oh. Dandolo. Yes. you are also single handily responsible for this gem:

image695×325 18.8 KB

(those teams are in alphabetical order, not position based)

of the 145 games with that team, 130ish are tagged with your name

its one of the issues with self reported data. im hoping that when i finish everything i would like to do in the first pass that will make it much easier to input data. Ive already mentioned to MigrantP that there will be a POST api to enter games as well, which would let them add a button to Sidekick to just send the data straight in if they chose too (low priority for them tho)

3 different tables, for each team size.

Still plugging away here, slowly but surely. Adding games is now possible through an API - still have to build a front end for it. Started working on that today, and since it is far easier to build a front end for displaying data than for entering it, im starting there. In order to reduce the load on my wallet through AWS, i plan on having a days worth of “All data” available like win_loss percentage

Thing I love about this particular stat is that, in general, without regards to anything specific, there is a greater than 2/3rds, usually 3/4ths chance you’re going to win the game. And that there isn’t that big of a spread

And just so you dont have to dig and think:

Are the lowest and highest respectively.

And so we can be complete…

With

being the easiest and hardest villains respectively.

I would like to report a bug with the website. While the pages for the video game difficulty statistics and difficulty calculator both list data for Chrono-Ranger, Chrono-Ranger’s Multiverse page is completely blank. It seems to be the only one with this problem, but I haven’t actually tried every page yet. I assumed from the links that this would be the place to ask about bugs?

Thank you so much for building and maintaining this database. It is interesting and useful for helping me understand the game and gauge my own mastery of it—or lack thereof, given that I’m still quite new.

Thanks, just happened to notice this comment. Not sure how no one ever noticed that the multiverse considered Chrono-Ranger to be a variant of Ra! Fixed.

It looks like the video game difficulty calculator and randomizer is missing a few entries.
Heroes: Tachyon (Super-Scientist), Stuntman (Action Hero), Wraith (Freedom Five)
Classic Villains: Akash’Bhuta
The data is perfectly accessible elsewhere - it just doesn’t show up on the page for the randomizer.

Given that I have hundreds of games in the app across the past 20ish months, which are not part of the statistics unless they’re being harvested automatically from the app without my input, I will continue to regard these statistics as being limited in accuracy. I absolutely do not accept that Cosmic Omnitron is the hardest villain, for instance. No way in heck. Even excluding Oblivaeon, Vengeance, and Advanced or Challenge modes, out of the box COT doesn’t even come close to the danger level of Iron Legacy, Chairman, Progeny and Matriarch.

The data is harvested by Handelabra and is what is presented in the video game difficulty stats.

Win statistics derived from data provided by Handelabra. Currently displays data collected over the past 365 days.

As far as villains go the stats using the model number have Dreamer as the most difficult without using advance, challenge, or ultimate rules. Cosmic Omnitron is the harder once Challenge and Ultimate rules are used.

Thanks, it was due to name mismatches on those entries. Found and fixed.

Yes, they are, anonymously. I have an ID code which neither Handelabra nor I have any way of matching to you. This is how Unity Analytics (and pretty much every other analytics package) works.

It’s interesting, since I don’t think most people would peg Dreamer as hardest. I have a feeling it’s a combination of: 1) Most digital players play at Ⓗ=5 because they can, but that makes her much, much harder, 2) It’s really easy to play too quickly in digital, not pay attention, and accidentally kill her, and 3) Once she’s dead, you can’t undo.

Progeny might be biased because so many people played him to get the F5 unlocks and really mastered how to cheese him. But he’s also really not that hard for a 4. Matriarch… I don’t know why she’d be considered one of the hardest, honestly.

If you do count Challenge Modes, COT is a much more respectable case for being the hardest, because he has a really insane and unbalanced challenge effect, though on Ultimate I’m still inclined to think the various 4s probably beat him, along with the Ennead.

Matriarch is only really hard if she gets out both Huginn and Muninn before you can stabilize, and she’s very vulnerable to being cheesed. Legacy utterly shuts her down with Next Evolution as long as there are other Ongoings you can lose to keep it, and even a single point of DR makes her significantly easier. She’s hard if you don’t exploit any of her vulnerabilities, but her vulnerabilities are super easy to exploit, with only the exception of both Cohorts or an Indestructible one.

Progeny is usually not too hard compared to other 4s, but I’ve failed an astonishing number of times to beat the old Weekly challenge with CM Progeny in Rook City. Maybe that’s just because Dark Watch is kind of a crap team, but Progeny with bonus card plays and damage bonuses from the Environment is pretty devastating, though he’s not nearly as bad without those advantages.

Any plans to incorporate statistics for Sentinels of Earth-Prime, now that the digital edition’s released?