Wrath-of-the-Righteous-Combat-Log-Analyzer

Pathfinder: Wrath of the Righteous combatLog.txt Analyzer

This (Windows only, sorry Mac users) application analyzes die rolls recorded in the combatLog.txt (located in "%appdata%\..\locallow\owlcat games\Pathfinder Wrath Of The Righteous") created automatically by, well, the game by Owlcat Games called Pathfinder: Wrath of the Righteous. It can be purchased from Steam here or GoG here.

Limitations of combatLog.txt

combatLog.txt is not a reproduction of the ingame log. Its close enough that it is obviously coming from the same source, but... combatLog.txt has the following limitations:

1. Combatlog.txt only includes the following types of events:
   • Attacks, defined here as "Something that requires a standard d20 roll to make an attack." This includes touch and ranged touch attacks in addition to standard melee / ranged attacks.
   • Damage, but only damage that effects hit points. Attribute drain, level drain, and other similar effects aren't present in the log at all.
   • Healing, but only healing that increases hit points. Restoration, raise dead, or similr effects don't appear in the log at all.
   • Initative rolls
   • A number of other, minior, single line log entries such as "X dies", "Gain experience", "Beastiry Updates", "Combat [Started|Ended]" and the like. These lines are more of an annoyance than anything else, as they never contain die rolls.
   • Most signficantly, it does not contain spellcasting or saving throws. Spellcasting that results in a [ranged] touch attack, or savings throws that reduce hit point damage appear indirectly in the log file, but the actual die rolls are not recorded.
2. In addition to the above limitations, it also is somewhat buggy. In particular, sometimes attacks or damages simply don't make it into the log at all. You can verify that they are missing by comparing to the ingame log, or simply looking for "Attack 1 of 3" -- when this occurs, there should always be 3 consecutive attacks, numbered appropiately, in the log, and this isn't the case.
3. Even more problamatic, the log contains duplicate entries for both attacks and damages. This is most obvious with attacks, where you can sometimes see back to back entries that read "Attack 1 of 3" with the exact same roll to hit (without abilities to justify this) against the same target and so forth. These can also be verified by looking at the in-game log.

What this program does

It parses the combatLog.txt file, extracting almost all the information available, and:

1. Extracts all the die rolls from Initative, Attack, Damage and Healing events. These die rolls can be exported into a CSV file for the user to analyze as they see fit. I'd suggest Excel with a PivotTable, but that's just me. :) Note that while damage rolls are extracted it would be quite difficult, if not outright impossible, to analyze them. This is because the log only contains "4d6 = 12", not "4d6 = 3, 2, 5, 1 = 12". While the CSV file does tell you that the die roll is a d6, rolled 4 times, this doesn't help much when trying to analyze it. Further making life hard, the critical hit multiplier is applied before the sum of the die roll is recorded (e.g. "4d6 (x2) = 24") and this bonus would need to be backed out. The application extracts this modifier but it currently doesn't make it into the CSV file, largely because I can't figure out why anyone would try to analyze damage rolls. If you do, raise an issue and I'll add it.
2. Provides a built-in analysis feature, that can be filtered to look at full set of data loaded, a single file (if multiple files are loaded), or a single combat. A standard set of analysis features are provided, as described later.
3. Export as a "Prettyfied" version of the orignal combatLog.txt, which includes all of the data (including duplicates or unparasable lines) in a more consistant, easier to parse, format than the original. It is also valid HTML, so it can be viewed with a standard web browser.
4. Export as a "De-Dupped" version of the original combatLog.txt, which applies the de-duplication logic and adds "ID" numbers and is otherwise identiical to the "Prettyfied" version. This is useful for troubleshooting -- you can find an ID # in the application, then search for it in the de-dupped HTML file to see what preceeds / follows that event.

Analysis Features

1. All analysis is limited to d20 rolls -- that is, attacks and intiative rolls. Damage rolls, for reasons mentioned above, appear to be impossible to analyze, despite being present in the input data.
2. Automatically classifies characters into "Friendly" (includes the string "Companion" in the internal name), "Summon" (includes the string "Summon" in the internal name), and "Hostile" (everything else). Stats are generated seperately for each of these catagories, so you can (for example) see how many "1s" friendly units roll versus how many "1s" are rolled by hostile units.
3. Allows the user to reclassify characters as they see fit, between the three catagories listed above and another, "Unknown" category that is only included in the "All" statisics. These changes cannot, currently, be saved, so if you exit and restart the program you will need to reclassify the characters manually to get the same results.
4. Provides the following reporting (For "All", "Friendly", "Hostile", and "Summons" categories):
   • Frequency Analysis: How often does a 1 appear in the data versus how often a 20 appears. For each individual die roll, provides the cumulative binomial probability (Wikipedia, free online calculator) of rolling the number indicated or worse. Worse is defined here as "further away from the expected value", and "expected value" is defined as "The total number of die rolls displayed divided by 20." The delta from expected value is shown in parenthesis -- "95 (-8)" means that 103 5s were expected, but only 95 were found, a delta of -8, and the probablity shown is "What is the probablity of getting 95 or fewer 5s in 2,063 trials." This number should be close to 50%, although it is unlikely to ever be exactly 50%, due to the use of integer math in calculating the expected value. The expected value for 2,063 is 20.03, but 20 is shown in the applicaiton and use for the binomial calculations. The key thing to look for here are numbers < 10 % -- such numbers represent large anomlies.
   • P-Values (for Frequency Analysis): In addition to the cumulative binomial probablity described previously, I also report on the "P-Value" (Wikipedia) using the "Chi-Squared" formula (Wikipedia, Free online calculator). This shows the probablity of the entire dataset ("All", "Friendly", "Hostile", or "Summons" are seperate datasets) occuring by chance. Note that P-Value in general, and the specific formula that I'm using, is used commonly in scientific contexts for the same purpose -- but in those contexts, the desire is to prove that the results probably didn't occur due to chance, so low P-Values are good. Normally, a scientific paper will consider results "statistically signficant" if P < 0.05. In this context, though, higher P-Values are good, with 1.0 being the (never achieved) ideal.
   • Streak maximum length analysis: Simple a count of the longest unbroken streak of the same number. "1: 2" indicates that the longest sterak of 1s was 2 consecutive 1s. When broken down by one of the classifications ("Friendly" / "Hostile" / "Summons"), it only considers die rolls by characters classified in that bucket. Thus, a squence of "Companion attacks, Natural 1, miss" -> "Foe attacks, Natural 20, hit" -> "Companion attacks, Natural 1, miss" -> "Foe attacks, Natural 20, hit" results in a sequence count of 2 for "Friendly" for 1s, a sequence count of 2 for "Hostile" for 20s, and a sequence count of 1 (four times) for 20s and 1s for "All".
   • Streak distribution: This shows how frequenly streaks of a given length occur. This table shows how frequently streaks of a given length occur in the input data. Unlike most of the reported metrics, this is only available for the "All" dataset. The application knows, and calculates, steak lengths for each of the different catagories, but these tables are not shown currently. If there is a demand to do so, I can add this functionality -- raise an issue.
   • To Hit Margin analysis: This compares the value rolled on an attack versus the AC of the target and shows the margin achieved. Negative numbers are nominal misses, and positive numbers indicates nominal hits. There are a couple of things to know about this reporting:
     • There are a variety of effects (such as rolling a Natural 20 or 1, concealment effects, and mirror image effects) that will modify a "nominal" hit into a miss. This dataset ignores all such effects -- it just looks at the attack bonus (with all the various bonuses / penalties that apply) and the target AC (again, with all the relevant bonuses / penalties) to produce a target number, which is then compared against the actual value rolled.
     • -20 and +20 include all rolls where the margin was greater than or equal to the value shown. This is arguably a bug (and these should be "<= -20" ">= +20", but... Maybe I'll get around to fixing it someday. :)
     • The expected distribution, and the actual distribution, is a normal distribution, centered roughly on 0. I'm unsure of why this is the case, but it clearly is, so... If you can explain why, I'll add the explanation here.
   • Critical hit margin analysis: The exact same as previous but only covering critical hit confirmation rolls. In addition to the previous caveats (all of which still apply), note that this data is much sparser and may ommit altogether or report incorrectly critical confirmation rolls that include a natural 20 or natural 1. The issue is that the log doesn't include "critical hit bonuses" for natural 20s or natural 1s. In this case I try to use the attack bonus that applied to the inital attack roll -- but if the attack roll itself was a natural 20, then the "Target AC" information is missing. So when a critical hit occurs on a Natural 20, I can't calculate the margin for the critical hit confirmation result, regardless of what that die roll is (because the "Target AC" isn't reported). If a critical hit is threatend, and the confirmation roll is a Natural 1 or 20, then I can't calculate the bonuses that only apply to critical hit confirmation rolls, because that data isn't reported. This is, as you might expect, a pretty minor problem in the grand scheme of things, but something to be aware of...
5. You can export all of the above datasets (individually) to the clipboard via the "Steam" button in a format suitable for pasting into a Steam message board post. Hey, it was useful for me when developing this program to publish results as I went along... :)

Usage

1. Run the provided setup file (or compile the program for yourself, using the source files -- this is a Visual Studio 2017 application) to install the program -- an icon will be automatically created on your desktop.
2. Once the program is launched, use the "File -> Open... -> New" to open a combatLog.txt file for analysis. Note you can type "%appdata%\..\locallow\owlcat games\Pathfinder Wrath Of The Righteous" (without the quotation marks) into the "Filename" box and press enter to move directly to the correct directory.
3. Once the parsing is completed, the "All" node will be automatically selected, and metrics will be shown.
4. If you wish to export the data, use "File -> Save..." and select the appropiate target.

Advanced Usage

If you run this program and target "combatLog.txt" and launch the game, the program will detect this, automatically reset, and start analyzing the rolls in realtime as you play the game. In order to be useful you will need two monitors, so that the application results will be visible while the game is running.

If you archive your combatLog.txt files (you have to do this manually), you can select multiple input files using either the "File -> Open... -> New" (by simply control / shift clicking on multiple files) or via the "File -> Open... -> Append" options. Note that the order matters (for purposes of streak analysis), but if the files are numbered sequentially ("combatLog 1.txt", "combatLog 2.txt", etc.) then they will be opened in the correct order via a simple left click on the first file + double shift left click on the last file.

As the number of records increases, the amount of time requried to calculate statistics increases dramatically -- the "BigInteger" class is simply very slow and gets slower as the size of the numbers increases. And the numbers get absurdly large -- as part of calculating the cumulative binomial distribution, you need to calculate "N choose K" (Wikipedia, free online calculator) where N is the number of die rolls made and K is (for example) the number of 1s that was observed. While there are some tricks (which I use -- see here) to make this number more managable, the simple fact is that "100 choose 2000" is ~1.10×10¹⁷¹ and my full test dataset requires calculating "17784 choose 872" which is ~2.002×10¹⁵⁰⁹. This is only possible because of the Rational class, but that class depends upon the BigInteger datatype, and that... Well, it allows you to store numbers up to the limit of your available memory, but at the cost of making arithimitic very, very slow. And cumulative binomial probablity requires calculting the binomial probablity multiple times, and this has to be repeated for each die roll. This is very, very slow when N and K are large. Nothing I can do about that. As long asy you limit yourself to only a few combatLog.txt files, the peformance is pretty good -- and, even with 19 files loaded, it only takes ~1-2 minutes to calcuate stats. Enough to be annoying, but not "hours and hours".

Interperting the results

Be very wary of P-Value hacking (Wikipedia) when interperting the results. As you build up more and more results, the odds that some dataset, by chance alone, will have abnormal characterisitcs increases. And each dataset is automatically split into three groups ("All", "Hostile", "Friendly"), then by combat, and finally by file. That's a lot of datasets to hunt through looking for "suspicious" data -- if you start with the conclusion that the RNG must be unfair, then you'll likely find something that looks suspicious eventually. Some key tips to avoid this:

1. Determine what your end conditions will be before you start your analysis, and stick to that. For example, "I want 10,000 die rolls in my dataset" or "I want 20 combats", or "I want 10 hours of gameplay in Act 3". Whatever your "This is done" condition is, when it occurs, generate the results and only look at that dataset.
2. Determine what "suspecious" means before you generate the data. Is it a P-Value < 0.05? A cumulative binomial probablity less than 10% for 1s and/or 20s? Long streaks of 20s or 1s? You can pick several different items to be "suspicious", but you should decide in advance what qualifies.
3. Finally, reproducability -- if you find something suspecious, and it really is a problem with the RNG, then you should be able to reproduce the results with a new dataset that meets the same criteria as the orignal dataset. If you cannot, then the results that you received were, in the end, just the result of random chance.

Odds and ends

• The program knows about the various ablities that allow / force re-rolls of attack and critical confirmation rolls. When they occur, all rolls are recorded (not just the roll that the game uses), and all rolls are reported on as independent values. In the case of margin reporting, they are treated as multiple attacks, and in other contexts, they are just treates as additional d20 rolls.
• The program knows about the ablity that converts 1s into 20s (in the Trickster path). Such rolls are identified, and counted as 1s for all purposes within the program.
• The program knows that Natural 20s are automatically hits, and Natural 1s are automatic misses -- but the only time it cares is when reporting the "Total number of hits" that appears at the top of a stats block. Everywhere else, 1s and 20s are treates as "just another die roll result".
• The Trickster ability that forces all rolls to be Natural 20s isn't detected by the program. When active, it will produce very strongly skewed results. This is because this ablity doesn't replace a normal die roll with a Natural 20 -- no die roll occurs at all. From the program's standpoint, the "Natural 20" that results from using this ability is indistinguishable from a real "Natural 20". This does provide a good example of what skewed data looks like, though, so its handy for that. :)
• Sample data is available -- just download "combatLog SampleData.zip" from the root directory of this project. This starts at the beginning of the game and proceeds through the conquest of Drizen, and a bit beyond that. I probably won't update this file, so that's where it'll end -- but that's 10k+ d20 rolls to analyze. :)