Archaeology Calculator

A tool that predicts Archaeology skilling trends in RuneScape

Upon the release of the Archaeology Skill in RuneScape on March 30, 2020, the RuneScape Wiki Administration Team was overwhelmed with work and research. Fortunately, one of the team members knew of my previous experience with JavaScript & Lua, so he notified me of an offer to head the lead research on the skill.

Archaeology Hotspot Calculator Site

The two-part project

Part 1: Data collection & analytics

The challenge of this project was that RuneScape game developers never released any core information related to the skill mechanics. This meant that every aspect of the skill had to be individually derived and calculated. So, from April to June of 2020, I conducted research and data gathering. For each skill mechanic, the following cycle was followed:

  1. Preparing experimentation for a core mechanic – To understand how the skill worked, an experiment was set up to understand how changes of a certain variable affect the outcome. We derived rates of excavation, chances of drops, material success chances, additive/multiplicative skill boosts derivations, and how core stats affected skill performance.
  2. Collecting data – Physically going to excavation locations and tracking the results, oftentimes placed in sheets. This took the most time.
  3. Analyzing data – Once data had been collected, it was analyzed for correlations. If results were conclusive over a large sample size, that trend was solidified. Otherwise, data collection resumed.
  4. Publishing results for peer review – The RuneScape Wiki is quite particular about spreading misinformation. For this project, each component of my results was reviewed by the Administration team to ensure that the data collected was correct and held conclusive results.

If you are curious, here are a few links used for analysis.


Mechanics & Results

Archaeology works similar to Woodcutting in the game: a player stands at an Archaeology Hotspot for a period of time, excavating a location until they collect an artefact. Progress to finding an artefact is shown in a bar above the player’s head. I found that the required amount of hits to collect an artefact remained the same. We called this value hitpoints. The tool used to excavate in archaeology is called a mattock, and different levels of mattocks exist with different strength values. Higher level mattocks have higher precision, which is effectively how much damage it does in each attack increment to the total hitpoints (HP) value. Higher level mattocks also have an additional stat called focus, which increases the % chance of obtaining a material per hit.

RuneScape is a tick-based game, where a tick is how often a program refreshes data values. RuneScape refreshes its data every 0.6 seconds. A player deals damage to a hotspot every 4 ticks, which is every 2.4 seconds. This means that a player deals damage to a hotspot 25 times per minute, or 1500 times per hour, which is the maximum threshold that the calculator allows. Thus, units of time were incorporated into the calculator (XP per hour, Materials per hour).

I also found that the artefact experience gained in hotspots could be estimated through a logistic curve with an R2 of 98. Until the precise experience values were collected at each hotspot, this curve was temporarily followed. The equation for the curve is displayed to the right. Additionally, I calculated that precision and focus multipliers were additive instead of multiplicative with a mere 1.3% error.

To this day, there remain three things that can not mathematically be derived without an incredibly large amount of data collection:

  1. Material base success chance per varying hotspot – While it is known that different level excavation hotspots have different material success chances, it is unclear of what this correlation is. Data gathered from previous experiments do not point to any reasonable correlation or equation with an R2 > 90. Until then, the calculator will use an averaged, flat base 21% success chance for all excavations and mattocks, which can be further increased from perks, focus, and other boosts.
  2. Level-based material success chance – There is yet to be a correlation developed between a player’s Archaeology level and material success chance. This is incredibly difficult to test since you can not lose experience or levels in the game.
  3. Soil screening – Success chance for materials from screening soil is capable of increasing and decreasing per change in archaeology level. A correlation couldn’t be created with approximately 500,000 soil screened, which is equivalent to 200 hours of sample size data. Thus, it has not yet been implemented into the calculator. Without knowledge of each of these trends for each material and soil, it can’t be added to the calculator without risk of displaying inaccurate results.

Part 2: Coding the calculator

Once all correlations and mechanics of the skill were derived, the coding development portion began. I started at around late June and finished in late August of 2020. The calculator used JavaScript for frontend and Lua for backend.

Frontend

In retrospect, the frontend was the simplest task of the calculator. Fortunately, the wiki had pre-made styling that could be used for the calculator. I merely introduced new input parameters and organized the visual structure.

Backend

This was where the brunt of the work was done. The backend was divided into two parts or modules: Archaeology Data, and the calculator code. The Archaeology data was merely a module formatted as a reference for the calculator module to extract archaeology information.

Understanding Wiki code

The first portion of the calculator module development began with understanding how the RuneScape Wiki outputs data from Lua modules. Through referencing other calculators, I realized that html objects could be created in the module lua code. For instance, creating a table full of data in Lua could be written as

local displayTable = mw.html.create("table")
	displayTable:addClass("wikitable")

Which allowed me to utilize one of the wiki’s styled tables. Additionally, variables returned from the calculator’s module can be retrieved by calling the appropriate template. Parameters are called with the param function, and variables are immediately called after the equals sign. As seen below, I call the hiscore variable, which allows me to access a player’s level through the RuneScape HiScores API.

<pre class="jcConfig">

template=Calculator:Archaeology Hotspot/Template
form = ArcheoForm
result = ArcheoResult

param=hiscore|Name||hs|archaeology_level,28,1;archaeology_xp,28,2

Afterwards, I loaded in variables for the (many) parameters that affected the skill. Once everything was preloaded, the conditional statements needed to be written. An example of this is written below. You may notice that I always include an or statement in case the calculator returns an error retrieving something from the calculator’s parameters.

local tick = .6
local tea = args.tea or 1
local monocle = args.monocle or 0
local manual = tonumber(args.manual) or 1
local brooch = tonumber(args.brooch) or 0
local cosmic = tostring(args.cosmic) or "None"
If statements
if tostring(methodtmp) == "Time Sprite Medium Intensity" or methodtmp == "Time Sprite High Intensity" then
	spritexpboost = 1.2
elseif tostring(cosmic) ~= "None" then
	spritexpboost = 1.1
else
	spritexpboost = 1
end


if tonumber(waterfiend) == 1 then
	waterfiendprice = gemw._price("Binding contract (waterfiend)")
	waterfiendchance = 1.05
end

The Archaeology Calculator is riddled with many small if statements. Rather than adjusting the final calculation manually with supermassive conditional if statements, I simply adjusted the multipliers themselves. For instance, if a multiplier is not selected in the input, its default value is equal to 1. This way, it won’t affect the calculations. Otherwise, the multiplier is applied as their appropriate value of 1.1, 1.2, etc. and the final calculation in the code remains the same.

Calculation

Once all of the necessary multipliers were loaded and conditionally adjusted, the skills’ mechanics could be written into code.

if args.method == 'AFK' or 'Time Sprite Medium Intensity' or 'Time Sprite High Intensity' then

	local d = r.failxp or 1*10^(0.0000965*(r.level^2))-- Finding the value of failure xp
	local a = (r.successxp or d*9.3)   -- xp for successfully getting a material
	local b = r.artefactxp or 100 * d  --xp for finding an artefact
	local c = artefactXP(r.artefacts[1][1])  --returns function, finds average experience of restoring one artefact

--	hitpoints calculation

	local e = r.hitpoints 
				
	if r.level >19 and r.hitpoints == nil then
		e = (17500/(1+(25*(1.035^(-2*(r.level-20))))))+3000 --dm boi#9999 for formula usage
	end		
				
				
				
	if skillcape == '120 cape' then --120 cape grants automatic 5-15% progress to next artefact, avg of 10%.
		e=e*.9
	end
				
--returns the hitpoints of one hotspot
	local g = r.artefacts  --returns list of artefacts from one hotspot
	local ggained = (havg)/(e/precision) + tony*((havg)/(e/precision))  --total avg artefacts gained per hour, calculated by taking hitpoints / precision (which is damage dealt)
	local sp = SoilPrice(havg,r)  --gets average profits for collecting soil
	if tostring(autoscreener) ~= 'None' then
		sp = 0
	end
	local materialboosts = 1+((roar-1)+(manual-1))
	local f = havg * s * materialboosts --f stands for total raw materials gained, before fiend and furnace apply. drop an f in the chat
				
			
	local totalPrice = TotalRestoringPrice(havg, r, precision)  --average price for restoring one artefact
	local mph = GetNumArtefactMaterials(havg,r) --* 2  testing here
	local chronoteX = GetChronoteValue(index)  --Gets average chronote value of one artefact from one hotspot
	local x = (ggained * (1+((waterfiendchance - 1)+(fortune-1)))) * (totalPrice)  --calculating the average cost of all materials required for restoring artefacdts
	local i = (ggained * b)*(1+(spritexpboost-1)+(pylonboost-1)+(effort-1)+(roarboost-1)+dxp+(tea-1)) --i stands for total xp gained from finding artefacts
	if skillchompa.Level>mattock.Level then
		i = ((i*skillchompa.Boost)-i)
	end
				
	local j = (ggained * (1+((waterfiendchance - 1)+(fortune-1)))) * c --j stands for total xp gained from restoring artefacts
	
	local q = (n*f*a*materialboosts)*(1+(spritexpboost-1)+(pylonboost-1)+(effort-1)+(roarboost-1)+dxp+(tea-1)+tony*((havg)/(e/precision)))  --q stands for total xp for finding mats with + without furnace perk. no furnace perk, remember n=1. cancels.
	if skillchompa.Level>mattock.Level then
		q = ((q*skillchompa.Boost)-q)
	end
	local l = (havg * z * d)*(1+(spritexpboost-1)+(pylonboost-1)+(effort-1)+(roarboost-1)+dxp+(tea-1)) --l stands for total failure xp. make sense? get that L? hahahaha... good one Pog
				
	if skillchompa.Level>mattock.Level then
		l = ((l* (skillchompa.Boost / 2))-l)
	end

	local mp = FindProfits(havg,r) --gets average profits for collecting materials


	local MAFKavg = l + q + j + i  --M means raw total xp/h without xp boosts. everything that follows the variable is logical.
	local boosts = ((1+(gmo-1) + (avatar-1) + (additional_xp-1)+ (torstol-1)+ (wise-1) )) --all xp multipliers, including ones that influence restoration xp
				
	local TAFKavg2 = 0--T means total xp/h including boosts.
	local hours = 0  --then this... this is part 2 from the equation i figured out just a few lines earlier.
	local TAFKavg = 0
				
	if remaining <= 0 then
		bxp = 0
		TAFKavg = MAFKavg * boosts
	elseif remaining < (bxp * boosts) + bxp then  
    		bxp = remaining/(boosts + 1) -- this DAMN formula took me 3 days to derive.. feeling frustrated and satisfied af that i figured this out. without algebra and fraction rules, i'd have never gotten here.
    		TAFKavg2 = MAFKavg * boosts--T means total xp/h including boosts.
		hours = (remaining - bxp)/TAFKavg2  --then this... this is part 2 from the equation i figured out just a few lines earlier.
		TAFKavg = MAFKavg * boosts + (bxp/hours)
	else
		TAFKavg2 = MAFKavg * boosts--T means total xp/h including boosts.
		hours = (remaining - bxp)/TAFKavg2  --then this... this is part 2 from the equation i figured out just a few lines earlier.
		TAFKavg = MAFKavg * boosts + (bxp/hours)
    	end   -- what it does is prevent ridiculously high amounts of bxp to accidentally overinflate xp/h rates. otherwise, it would use, say, the entire 900,000 bxp for a remaining xp amount of only 100,000. which isnt possible.
				
				
				
				
	local AFKProfit = ((1-(n-1))*f*(mp)) + tony*((havg)/(e/precision))*(mp) + (waterfiendchance*f - f)*(mp) + (f*fortune - f)*(mp) + ((ggained * (1+((waterfiendchance - 1)+(fortune-1)))) * chronoteX*chronoteprice*chronoteparam) - chargeprice  + sp -  x   - ((1-(dwarf-1)-(impsouled-1))*portablePrice*(1-(n-1))*f) - dwarfprice - powerburstprice - teaused - monocleused - ((havg * skillchompaprice) - (((havg * skillchompaprice) * skillchompaperk) - (havg * skillchompaprice))) - waterfiendprice -torstolPrice--TOTAL profit/loss calculation for the afk method!!!

This was split up into two different calculations: experience and profit. Due to the diversity of the skill, players look to a hotspot’s cost efficiency of varying training methods. Once I calculated raw values for how many materials and artefacts were gained, I could multiply those values by how much experience they give and how expensive it would be to fund their restorations.

Output

Once all values were calculated to a particular calculator input, an organized output needed to display with potential options. I chose to display results in a table, since it follows a convenient and consistent format. As mentioned before, this was quite easily done with a quick reference to html. Once a basic output was created, I began additional formatting to the calculator. For instance, I displayed the rows of hotspots above a player’s level range as yellow. I additionally formatted images to display for hotspots and hyperlinks for certain outputs which had wiki sites. 

Specifications

  • Three Modules
    • Calculator Module – 813 lines of code, a function that the Calculator references when a user submits input parameters
    • Calculator Display – 90 lines of code, the display which allows users to utilize the calculator
    • Data Module – 5529 lines of code, all hotspot and artifact data gathered to be referenced in the Calculator Module.
  • 10 functions
    • fnum() – Adds comma separators to numbers in the displayed output.
    • getArtefactData() – Retrieve’s all data for a particular artefact.
    • artefactXP() – Retrieves the restoration XP child for an artefact inside of the ArtefactData parent.
    • FindProfits() – Calculates a unit average profit from a hotspot per each successful material chance
    • GetNumArtefactMaterials() – Returns tabulated data for the chances for each material in a referenced hotspot
    • SoilPrice() – Each hotspot produces an additional byproduct called soil. This function retrieves the type of soil produced at a hotspot, and then returns its current active price.
    • TotalRestoringPrice() – The price to restore one particular artefact. This pricechecks each material that is needed for restoration, and it multiplies each cost by the quantity of material required.
    • getMattockName() – Retrieves the appropriate formal mattock name for the one used. The current dropdown list wasn’t capable of using spaces for parameter inputs.
    • remainingXP() – Converts an input level to an experience value if level is selected as a function input, or converts an input experience to a level value if experience is selected.