overspend1/PathOfBuilding
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #3149 from PathOfBuildingCommunity/mod-parsing-tutorial

Browse Source 
Adding mod parsing tutorial doc
This commit is contained in:
Wires77
2021-10-20 20:56:30 -05:00
committed by GitHub
parent 043072d8df 2be6a654d9
commit e4615cf736
4 changed files with 233 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
CONTRIBUTING.md
View File

@@ -1,5 +1,15 @@
# Contributing to Path of Building
# Table of contents
1. [Reporting bugs](#reporting-bugs)
2. [Requesting features](#requesting-features)
3. [Contributing code](#contributing-code)
4. [Setting up a development installation](#setting-up-a-development-installation)
5. [Setting up a development environment](#setting-up-a-development-environment)
6. [Keeping your fork up to date](#keeping-your-fork-up-to-date)
7. [Path of Building development tutorials](#path-of-building-development-tutorials)
8. [Exporting GGPK data from Path of Exile](#exporting-ggpk-data-from-path-of-exile)
## Reporting bugs
### Before creating an issue:
@@ -147,6 +157,12 @@ Note: If you've configured a remote already, you can skip ahead to step 3.
git push -f origin dev
## Path of Building development tutorials
* [How are mods parsed?](docs/addingMods.md)
* [Mod Syntax](docs/modSyntax.md)
* [How skills work in Path of Building](docs/addingSkills.md)
## Exporting GGPK data from Path of Exile
Note: This tutorial assumes that you are already familiar with the GGPK and its structure. [poe-tool-dev/ggpk.discussion](https://github.com/poe-tool-dev/ggpk.discussion/wiki)

80
docs/addingMods.md Normal file
View File

@@ -0,0 +1,80 @@
In this tutorial, we'll step through the logic in `ModParser.lua`. This module takes care of parsing mods from any item, passive node, or pantheon in the game. This won't be comprehensive, since there are many patterns, and a lot of examples to work off of. If you'd like to follow along, the code starts [here](https://github.com/PathOfBuildingCommunity/PathOfBuilding/blob/master/src/Modules/ModParser.lua#L3453). Having a good understanding of [Lua patterns](https://www.lua.org/pil/20.2.html) will also be invaluable here (If you know regex, patterns will be similar, but less powerful and a different syntax).
#### Jewels
The first mods parsed are mods from jewels with a radius. Jewels with a radius care about the nodes around them. Since PoB doesn't know where these jewels will be placed when parsing the mod, a function is prepared for when the jewel has access to this information. Any variable data are passed along much like other mods via pattern capture (i.e. anywhere you see `cap1`, `cap2`, etc., corresponds to a captured number from the Lua pattern) See [`CalcSetup.buildModListForNode()`](https://github.com/PathOfBuildingCommunity/PathOfBuilding/blob/master/src/Modules/CalcSetup.lua#L76) for the code that handles actually calling the function to modify any nodes. There are several helper functions specific to each type of jewel, but they all boil down to this function (which can be used on its own, when necessary):
- `function(node, out, data)`
- `node`: The node that is being affected within the radius. Can use things like `node.type` to determine notable or keystone, for example
- `out`: Instance of `ModListClass`, used for altering or replacing the mod originally on the node
- `data`: other data that might be relevant. Mostly just used for preserving the source with `data.modSource`
There are 4 categories of jewels that have a helper function:
- Jewels that modify nodes in their radius (`jewelOtherFuncs`). Uses customized functions for each jewel most of the time.
- `getSimpleConv(srcList, dst, type, remove, factor)`
- `srcList`: List of stats that will be affected in the radius
- `dst`: Stat that those stats should apply to as well or instead
- `type`: The type of increase that should apply (usually "INC" or "BASE")
- `remove`: boolean that says whether or not to remove the original stats
- `factor`: If the conversion shouldn't apply the full value, apply this factor instead
- Jewels that modify themselves based on stats allocated in their radius
- `getPerStat(dst, modType, flags, stat, factor)`
- The first three parameters correspond to name, modType, and flags in the [mod syntax](https://github.com/PathOfBuildingCommunity/PathOfBuilding/wiki/How-does-the-Mod-syntax-work%3F), where only `ModFlag`s are used
- `stat`: The stat we're multiplying the `dst` stat by
- `factor`: If the stats aren't 1:1, use this factor to change that
- Jewels that modify themselves based on stats unallocated in their radius. This uses the same `getPerStat` function as above.
- Threshold jewels, which give stats after 40 of that stat is reached in their radius
- `getThreshold(attrib, name, modType, value, ...)`
- `attrib`: The attribute we need to reach a threshold for
- The rest of the parameters are covered in the [mod syntax](https://github.com/PathOfBuildingCommunity/PathOfBuilding/wiki/How-does-the-Mod-syntax-work%3F)
- Cluster jewels, which parse simply what mods to add to their nodes and which notables to add as well.
### Scanning for matching text
The next steps all use the `scan` function to match text. It looks for the "earliest and longest match from the [given] pattern list". If a match is found, it returns the value from the list, the remaining unmatched text, and any captures associated with the matched pattern. For example, passing in "15% increased fire damage", and a table containing the key "^(%d+)%% increased" would return the value corresponding to the key, "fire damage", and 15.
### Special mods - `specialModList`
This is the largest list of `ModParser` and it's a catch-all for mods that don't fit a standard format (and aren't numerous enough to change the parsing logic to accommodate their format). If the mod has a component captured via Lua pattern, a function can be used to capture the number(s) or other captured text. E.g. `["lose ([%d%.]+) mana per second"] = function(num) return { mod("ManaDegen", "BASE", num) } end,`
Otherwise everything in this list can be done using the standard [mod syntax](https://github.com/PathOfBuildingCommunity/PathOfBuilding/wiki/How-does-the-Mod-syntax-work%3F)
***
## Parts of a "standard mod"
The mod we'll demonstrate adding is "Attack skills deal 20% increased damage while holding a Shield" from the Solidity notable on the passive tree.
### PreFlags - `preFlagList`
The first step in the process is scanning the text for text that matches one in the preFlag list. In this case, we find `["^attack skills [hd][ae][va][el] "] = { keywordFlags = KeywordFlag.Attack },` and save that flag for later, continuing on with "20% increased damage while holding a Shield".
### Skill Tag - `preSkillNameList`
This looks for a skill name at the start of the line, usually used for enchantments. Our example is not based on a skill, so we continue with the rest of the line.
### Mod form - `formList`
This is the meat of the mod parsing logic. Most mods will match one of these styles. In our case, we match `["^(%d+)%% increased"] = "INC",`, store the 20% off to be eventually used as `modValue`, and continue with "damage while holding a shield".
### Mod Tags - `modTagList`
This logic is run through twice, so we can have up to two tags after a mod to restrict it. In our case, we only have one tag, and it matches on `["while holding a shield"] = { tag = { type = "Condition", var = "UsingShield" } },` and we continue on with "damage" as the only thing left of our line.
### Mod Name - `modNameList`
Finally we look through `modNameList` and match the remainder of our line on `["damage"] = "Damage",`, leaving us with nothing remaining in our line.
# Putting it all together #
Our example is fairly simple, as some mod forms will slightly alter mod names and values to be compatible with with names used elsewhere in PoB. Regardless, once we have the mod name, type, value, and tags, we can combine them all into a mod as defined in the [mod syntax](https://github.com/PathOfBuildingCommunity/PathOfBuilding/wiki/How-does-the-Mod-syntax-work%3F).
## Important notes and tips ##
- `ModParser.lua` is actually not where most mods really come from. When you refresh the dev mode version of PoB with `Ctrl` + `F5`, `ModParser.lua` runs and regenerates `ModCache.lua`, which stores the actual parsed version of the mod. `ModParser.lua` only gets used if the mod doesn't already exist somewhere when loading PoB (passive tree, unique list, or rare item list). If you hold left alt while hovering over a mod, you can see how it gets parsed: ![Parsed Mod](https://i.imgur.com/ArVupKs.png)
If you're missing something, you can also see what is unable to be parsed: ![Unparsed Mods](https://i.imgur.com/RiIH0u4.png)
- All mods get converted to lower case before getting parsed, so when adding a new one make sure it's also lowercase.
- When adding a mod, see if you can add a new flag, or mod tag, before adding it to `specialModList`. It makes the code much cleaner overall.
- When a mod changes, keep in mind backwards compatibility. Since mods in most builds contain just the raw mod text, and are not tied to the underlying mod from GGG's data, we need to keep old mod wordings intact so those keep working on older builds. This is especially true for mods that have gone legacy (i.e. can still exist in game)
### Miscellaneous
There are a few flags you might see attached to mods that are defined in [this section](https://github.com/PathOfBuildingCommunity/PathOfBuilding/blob/master/src/Modules/ModParser.lua#L3662). These have to do with enemy or minion modifiers, or mods that don't directly affect the player.

73
docs/addingSkills.md Normal file
View File

@@ -0,0 +1,73 @@
Skills in Path of Building are generated from what is called a template file. These template files are used when exporting data from the ggpk and can be found [here](../src/Export/Skills). While this tutorial will focus on the [combined data files](../src/Data/Skills) know that any changes will be overwritten unless added to the template files. The script that combines these template files with the game data can be found [here](../src/Export/Scripts/skills.lua)
## Template files
A template file for a new skill can consist of the following directives, which tell the export script what to put in the final file. Other fields that end up in
### #skill grantedEffectId [displayName]
This directive initializes all the data for the skill and emits the skill header. `grantedEffectId` matches up with the name defined in the ggpk, in the `GrantedEffects` table. `displayName` is optional, but can override the display name from the game data if needed
### #flags
Sets the base flags for an active skill, like projectile, attack, or minion, for example. These flags are then used, along with the SkillTypes gotten from the ggpk, to add different KeywordFlags and ModFlags to the skill. The available flags include the following, and add a KeywordFlag or ModFlag of the same name unless otherwise stated:
* melee
* attack
* spell
* projectile
* area
* hit
* brand
* totem
* trap
* mine
* chaining
* warcry
* duration
* curse
* hex
### #mods
This directive does nothing but ensure the mods that come with the skill are included after exporting. This will almost always be included
### #baseMod
This directive is used when there are mods associated with the skill that don't come in the statMap (see the statMap section below)
### #noGem
This directive is used when a new skill shouldn't have a gem associated with it. Some skills have a gem internally, but it's not one that can be socketed in gear, so this is used to disable it (e.g. Bone Armour)
## Combined data
The most important tables constructed from the game data are the `stats` table, and the `levels` table. Taking a look at just one row in `levels`, there is a list of numbers, followed by named entries, such as `levelRequirement`, `damageEffectiveness`, etc. Each of these stats are mapped to a mod in Path of Building either via `SkillStatMap.lua`, or if the stat is specific to this particular skill (e.g. `spectral_helix_rotations_%` would only apply to Spectral Helix) in `statMap` in this same table. If a mapping exists in both places, the one local to this skill will take precedence. The corresponding mod will have `nil` in place of its normal value, and that value instead comes from this row in the `levels` table. Notice that not all of the stats have a number in the first part of the `levels` row. These extra stats are usually for booleans/flags that are always true.
Notice how these stat numbers don't really align with damage numbers in any meaningful way for active skills. The stat numbers are interpolated by the numbers in the corresponding position in the `statInterpolation` table in the same row.
* 1 means take the number as-is. This is the most common interpolation
* 2 means apply a linear interpolation: `statValue = round(prevStat + (nextStat - prevStat) * (actorLevel - prevReq) / (nextReq - prevReq))`
* 3 means apply an effectiveness interpolation. Take this formula for current effectiveness and multiply by the gem level: `(3.885209 + 0.360246 * (actorLevel - 1)) * (grantedEffect.baseEffectiveness or 1) * (1 + (grantedEffect.incrementalEffectiveness or 0)) ^ (actorLevel - 1)`
The code for this can be found in `CalcTools.lua` starting [here](../src/Modules/CalcTools.lua#L166)
## Skill Parts
Many times a skill will have different components that each do different types of damage, or the skill can be used in more than one way that changes the damage output. To support these different modes or parts of a skill, add a table called `parts` to the skill that contains multiple entries of the form: `{ name = <Part name>, }`. Making mods based on the skill part the user chooses simply requires the `SkillPart` tag to be added to a mod: `{ type = "SkillPart", skillPart = 2 }`
## Pre Functions (preFuncs)
Some skills rely on knowing something more about the character before they can calculate damage or some other property. One example is Righteous Fire, as it has to know the player (or totem's) life and ES totals before running the calculation. To do this, a calculator will call a function, giving it `activeSkill` and `output` as parameters. This function will live alongside the skill and is completely custom. There are 4 preFuncs that can currently be used:
* initialFunc - This is called before anything else is done in CalcOffence, allowing for special mods to be added to the player
* preSkillTypeFunc - This is called before the flag-specific logic is called in CalcOffence. For example, if you needed to calculate ChainCount differently, you could add the mod here
* preDamageFunc - This is called before the final damage passes are done. This is the most used of all the preFuncs, so there are plenty of examples to search for.
* preDotFunc - This is run before damage over time is calculated. The only current example is Burning Arrow, for its 5 stack multiplier.
## Adding skills
1. Add `#skill grantedEffectId` to the appropriate template file for the skill
2. Add `#mods` below that
3. [Export the skills](../CONTRIBUTING.md#exporting-ggpk-data-from-path-of-exile) to combine it with the game data
4. If there are stats in the `stats` table that aren't recognized already in `SkillStatMap.lua`, add a `statMap` table to the template file to map them properly to a mod.
5. Add other directives/options if needed
## Adding minions
* The minion itself has to be added to `Minion.txt`. This file uses different directives to construct the data, but the most important ones are `#monster monsterVariety monsterName`, `#limit modName`, and `#emit`. `monsterVariety` uses the Id from `MonsterVarieties`, while `monsterName` will be referenced in the skill gem. `#limit` sets a limit on summoned monsters based on a multiplier calculated elsewhere on the character, and `#emit` works similarly to the `#mods` directive on skills.
* The only extra thing that needs to be added to the base skill is a table called `minionList` that contains the names of all the minions that skill can summon (`monsterName` from the previous step).
* Some minions have skills of their own. These skills can be added like any other skill to `minion.txt`.

64
docs/modSyntax.md Normal file
View File

@@ -0,0 +1,64 @@
This syntax is used all over the codebase, but there are two locations that hold the majority of them: [ModParser](../tree/master/src/Modules/ModParser.lua) and [Skill Stats](../tree/master/src/Data/SkillStatMap.lua).
The standard format of a mod looks like this: `mod(ModName, ModType, Value, source, modFlags, keywordFlags, extraTags)` See the function declaration [here](../tree/master/src/Modules/ModTools.lua#L20-L46)
### ModName
Used as a key, so you can reference this mod elsewhere in PoB. Can really be anything, but look around the codebase to find ones you need (e.g. "Damage", "Life", "PhysicalDamageGainAsLightning", etc)
### ModType
- "BASE": used for flat values that add to other base values (e.g. Flat added damage, flat life, flat evasion)
- "INC": used for increased and reduced mods that stack additively. Use a negative value to represent "reduced".
- "MORE": used for more and less mods that stack multiplicatively. Use a negative value to represent "less".
- "OVERRIDE": used when you want to ignore any calculations done on this mod and just use the value (e.g. "your resistances are 78%" from Loreweave)
- "FLAG": used for conditions. Value will be true/false when this type is used.
### Value
This represents the raw value of the mod. When it's used in the skills to map from the skill data, this will be `nil`, as it pulls the number from the gem based on the level.
### Source
This is where the mod comes from. Often it will be automatically filled in, coming from a tree node, gem, or item. If you do need to specify it for some reason, it's a string, and you can use "Tree:[nodeId]" as a special value to show a tree inset on hover.
### Mod Flags
These are bitwise flags that say what the mod can apply to. See a full list [here](../tree/master/src/Data/Global.lua) under `ModFlag`. If you want to use several flags at once, make use of `bit.bor` and `bor` (ModParser.lua uses this alias) to combine them. When combined, all of the flags have to match. If you only need one to match, use the "ModFlagOr" tag instead.
### Keyword Flags
These function similarly to the mod flags, and use the `KeywordFlag` group in `Global.lua`. These are usually based off of the flags on the gem itself. If you want to use several flags at once, make use of `bit.bor` and `bor` (ModParser.lua uses this alias) to combine them. When combined, only one of the flags has to match. If you need them all to match, use the "KeywordFlagAnd" tag instead.
### Extra Tags
Often a mod will only apply under certain conditions, apply multiple times based on other stats, etc. The syntax for that depends heavily on the first parameter, "type". There can be an infinite number of these tags at the end of a mod, so multiple can apply at one time. Some parameters, like `actor` or `neg` can be used on all of the types. Below are different types and the other parameters they need to function.
* Condition: Used for conditions on the player that need to be in place before the mod applies (e.g. CritRecently, Shocked, etc.)
* var: Contains the name of the condition
* neg: (defaults to false) Boolean that negates the condition
* In order to set a condition, use "Condition:[name]" as a FLAG mod
* ActorCondition: Used for conditions on an enemy or a minion.
* var: Contains the name of the condition
* neg: (defaults to false) Boolean that negates the condition
* actor: Can be "enemy" or "parent". "parent" is used when giving a mod to a minion that is based on a condition on the player (its controller). e.g. `mod("MinionModifier", "LIST", { mod = mod("Damage", "INC", num, { type = "ActorCondition", actor = "parent", var = "HavePhysicalGolem" }) }, { type = "SkillType", skillType = SkillType.Golem }),`
* Multiplier: Multiplies the mod by this variable
* var: mod to multiply by
* limit: The maximum number the mod can go up to
* limitTotal: boolean that changes the behavior of limit to apply after multiplication. Defaults to false.
* MultiplierThreshold: Similar to a condition that only applies when the variable is above a specified threshold
* var: name of the mod
* threshold: number to reach before the mod applies
* PerStat: Similar to Multiplier, but is used for character stats instead of arbitrary multiplier like number of sockets
* stat: The stat to multiply by
* div: Defaults to 1. Divide by this number after calculation, rounding down. Useful for mods that say "per 5 strength", for example
* StatThreshold: Similar to MultiplierThreshold
* stat: The name of the stat
* threshold: number to reach before the mod applies
* PercentStat: Used for mods based on percentages of other stats (e.g. Agnostic)
* stat: The name of the stat
* percent: value of the percent
* SkillType: This type is for mods that affect all skills of a certain type
* skillType: An enum value in Global.lua
* SkillName: Similar to SkillType, but specifies the name of the skill, usually for enchantments
* skillName: The English name of the skill (e.g. "Decoy Totem")
* GlobalEffect: This is used largely for buffs and curses that affect actors even when it's not the main skill
* effectType: Can be "Guard", "Buff", "Debuff", "Aura", "AuraDebuff", "Curse". These apply to you, you, enemies, you + minions, enemies, and enemies, respectively
* effectName: String to specify where the global effect comes from
* effectEnemyCond: Specify a condition so this mod applies to the enemy when that condition is fulfilled
* effectStackVar: Multiplies the mod by this variable (usually another mod)
* modCond: Apply the mod when the actor has this condition
* unscaleable: boolean that determines whether this buff can be scaled by buff effect
* DistanceRamp: A rare type that is used on skills and effects that do different things at different distances from the character
* ramp: Numbers to multiply the mod by at different distances. e.g. `ramp = {{35,0},{70,1}}` means the mod does nothing at 35 units, but has its full value at 70 units.
* ModFlagOr: Used when you only need one ModFlag to match, e.g. `["with axes or swords"] = { flags = ModFlag.Hit, tag = { type = "ModFlagOr", modFlags = bor(ModFlag.Axe, ModFlag.Sword) } },` needs `Hit`, but can use either of the other two flags
* modFlags: Use `bor` as if you were adding ModFlags normally
* KeywordFlagAnd: Used when you need all of the KeywordFlags to match
* keywordFlags: Use `bor` as if you were adding KeywordFlags normally