Template talk:DropTest

New guidelines

 * @User:Jello, User:Maltheser, User:Admiral_Titan, User:Tygra Dax, User:Eeb3: I have called you guys here because you have all either expressed interest in tracking the odds of item drops or have done work on the only other mathamatical template on this wiki (Template:SP). I would like you guys to review the guidelines I have created in Template:DropTest and would love to hear your opinions. CodeHydro (talk) 12:26, 19 May 2016 (CDT)

Do we list the training programs too?
Now that we get training programs on the missions, do we add them too? Kschang (talk) 01:33, 22 May 2016 (CDT)
 * @Kschang: I don't think this template would be the appropriate place to share that information. Hold onto that info for now. I've begun work on Template:WarpXP, though I'll need to know more about what happens at warp before I can finalize that template. How many of each training program did you get and for what mission? I'm curious also if it makes a difference in Ship Battle warps whether you have a 3-crew ship selected or a 2-crew. CodeHydro (talk) 11:07, 22 May 2016 (CDT)

A few hiccups
Hi there! This is a great template! :o) However, after my first tries (Template:DropTest/Not_A_Drop_to_Drink/1) I found a few strange things, and was wondering whether I missed something, whether it's a known issue, or whether this should be fixed either manually, or by extending the template:
 * The title appears on the template page, but the mission page Not_A_Drop_to_Drink only shows brackets. Curiously, the brackets are visible only in the edited one (which I created via the "Create one!" button - the unmodified ones (=elite+epic), which don't exist yet as individual pages, correctly show the title.
 * Is it correct I have to re-edit the mission page, and save it without modifications (it still contains just ) each time I update the template? Otherwise the mission page wouldn't update with new information (i.e., it still showed the first template version with 1 run, although I already was at 30). The template page itself always shows the correct, up-to-date version. Even when looking at the mission page with a different browser (to exclude browser cache problems), it still showed the old 1-run-version, until I re-saved the mission page. (this no-change-edit doesn't seem to appear in the page's history).
 * Hm, weird, this time (=50 runs) it worked without edit. I THOUGHT I was certain I actually saved the template, instead of looking at the edit preview... ^^ --Crunch (talk) 15:34, 23 May 2016 (CDT)

Cheers! --Crunch (talk) 14:30, 23 May 2016 (CDT)
 * In the description it is unclear to me how you define a "gap". Is playing one other mission in-between already a gap? Or do you mean a certain amount of time (1h, 24h, ...), or maybe restarting the app or something?


 * @Crunch: Thank you for your contribution! Regarding the broken title, I was experimenting with automatic title detection. Long story short, I wanted to eliminate the need to specify  in DropTest, but such proved impossible due to limitations in the MediaWiki language. I've added those params back into your template and in the "Create one!" form so you shouldn't have any more problems with the title. As for the gap, I did not mean time gaps, but rather sampling gaps; for example, let's say you did 50 runs, but only recorded runs 1-9, 15-7, 25-31, and 45-50--- the gaps would be the unrecorded runs. In other words, take care to avoid having any unrecorded runs between the first and last runs. Time gaps are fine; we'd never get enough runs if people had to do everything in one sitting due to lack of chronitons.CodeHydro (talk) 10:16, 24 May 2016 (CDT)


 * Thanks. Trying to add data to Template:DropTest/The Walls Have Ears/1, I noticed that the assignment of multiple contributors in  no longer works, presumably since the change in 17:33, 22 May 2016. I tried to mess around a bit (see Template:User and Template:)( - the idea was to use  ), but unfortunately it has been about 8 years since my last mediawiki template ^^ so it's not really working. The easiest (but not most beautiful) solution would probably be to add a number of optional parameters for additional users (  and so on). Maybe you can find a better solution? --Crunch (talk)

Category for Drop Tests
I created Category:Drop Test, to create a quick reference list of all DropTests. I encapsulated the link in, as otherwise every single mission page with a test would be placed in that category as well. However, when adding the same to Template:DropTest/preload, the link gets excluded in the preview, due to the  :-( (And in a perfect world, Template:DropTest/preload itself would not be included in that category - but that's just a minor glitch.) Any ideas? --Crunch (talk) 05:08, 26 May 2016 (CDT)


 * I just use Special:WhatLinksHere/Template:DropTest and restrict it to template namespace.CodeHydro (talk) 11:43, 28 May 2016 (CDT)

More rewards at Warp speed?

 * I just tried a few Warp 1's on a Space Battle mission, Long Distance Call, and noticed that I actually got 5 rewards each time, instead of the 3 I get when playing it regulary. Did they change something, is this intentional, or is it a bug? Maybe some people can run a few tests, whether this is new, particular to this mission, orSpace Battles in general, whether Away Team missions are also affected, and whether the number of rewards varies with different warp levels.
 * => Oops, my bad. The number of rewards is of course the same (=3), but you get 2x Basic Survival training programs at Warp 1. --Crunch (talk) 13:35, 26 May 2016 (CDT)


 * We should probably also create a page Warp, to collect the details/differences (creds, captain XP, no crew XP, reward numbers, etc.) --Crunch (talk) 13:21, 26 May 2016 (CDT)
 * @Crunch: I didn't see in the guidelines... does it matter if we are doing the missions to test drop chance on Warp or on Travel? We might want to record that in our data sheets in case later it is discovered that there are drop rate differences between the two. It has seemed that way to me a few times but I wasn't tracking so that is a very anecdotal experience. I would think that we wouldn't care until we know if we should care, but I wouldn't want to have to throw out results, if that does come to pass, where we weren't sure how they were gathered either.--Rodger (talk) 17:18, 2 June 2016 (CDT)
 * Good point, and I have been thinking about that myself. I seem to remember people at least placed a note ot remark if they did DropTests at Warp speed, but I don't know if the results have been kept separate consistently (for example, by using ). So until we know with some statistical certainty (is that an oxymoron? :o) ) one way or the other, we should probably do just that. Your thoughts, ? --Crunch (talk) 17:54, 2 June 2016 (CDT)
 * for some reason it didn't notify me that there was a post here; I have good confidence that they are the same. See Template:DropTest/A Logical Response/1 CodeHydro (talk) 14:27, 7 June 2016 (CDT)

Interesting test
I just thought that the differences in DropTest/Arms Race/1 between my drops and those of Crunch are very interesting. He got over double my amount in Spices and I got over seven times as many Test Cylinders per Chroniton (and infinitely more Authorization Code). Even more interesting is that the combined results for every item are all within the 2 standard deviation range given by the template before the new data was added. Considering the range is expect only to cover 95.5% of deviations, it's fascinating to think how different results in the 4.5% would be if results inside the range are like this! — CodeHydro 08:15, 14 June 2016 (CDT)

Procedures for determining whether old data is bad
I just reversed someone attempting to remove pre-May data. Please don't remove data without evidence that it is invalid. So you guys may see why, I have provided June only and May only data sets for that mission (identifiable information has been stripped to avoid embarrassing the data remover, who I believe acted in good faith, albeit misguided): And now I shall mathematically prove why the data May and June data sets are one and the same for this particular mission:
 * Step 1) Get cost ranges from BOTH data sets for an item rated "fairly reliable" or better. (If none, then we can't reliably determine anything. You can try "somewhat unreliable", but your results are going to be just that)
 * Step 2) Compare both sets of highs and lows and subtract the higher low from the lower high. This is your overlap amount for that item. (For item 3, the higher low is 3.68 and the lower high is 5.8, for an overlap amount of 2.12)
 * Step 3) Subtract low from high in one data set to obtain the item's cost range for that set. Divide the overlap amount (step 2) by the cost range and multiply by 100. This is the overlap percentage with the other set for that item.
 * (PS: If you follow these step for item 3 in the May only data set, you should have found %)
 * Step 4) Repeat step 3 for the same item with the other data set. Add overlap percentages of both sets and divide the sum by 2. This is the item's average overlap.
 * Step 5) Repeat steps 1-4 for all items rated "fairly reliable" or better in BOTH data sets.
 * Step 6) Get the average of all average overlaps. If this raw overlap value is greater than 50 percent, then you can safely conclude both sets are still valid as long as both sets are of similar size.
 * Step 7) If step 6 produces a value below 50 percent, you will have adjust for sample size differences. To find the adjustment factor:
 * Divide the number of runs from the larger data set by those of the smaller.
 * Take the square root of that quotient.
 * Add 1 and then divide the sum by 2
 * (For the above example, the adjustment factor would be )
 * Step 8) Multiply the raw overlap percentage obtained in step 6 by the adjustment factor of step 7. This is the adjusted overlap value. If this value is also below 50 percent, then you are justified in removing the old data; otherwise, keep both until more data has been obtained.

At step 6 for the above data sets, we find a raw overlap of %. That by itself should be enough to say the odds are one and the same. Once we divide that by the adjustment factor, we find an adjusted overlap of %

Thus, it is over 6 times more likely that both data sets reflect the same drop frequency than not!

Thank you for reading this and helping us ensure quality drop information. Live long and prosper! — CodeHydro 10:17, 15 June 2016 (CDT)
 * * Note: The above mission was not affected by the June 1 update when all missions stopped dropping multiple components at a time. — CodeHydro 10:20, 15 June 2016 (CDT)

% chance of success
Might it also be useful to calculate the % probability of receiving a given item in a single run? This could go in another column beside chronitons/unit. Yeah, it's technically derived from the same data, but perhaps a more helpful representation in certain circumstances. (unsigned comment by User:Cpt nichols. To sign type  after your comment)
 * I've thought of that, but the issue has been trying to keep the template from being too wide. Perhaps I can include it somewhere in the tooltip, or make hovering over the units column show something else since the average column is already pretty busy. Will look into it after I'm less busy with this event ;) — CodeHydro (msg) 14:43, 17 June 2016 (CDT)
 * Just noticed this topic now. I recently created a few helper templates to create sortable summary tables for DropTests of your choice, for example to quickly find the best drop chance mission for a specific item. See a (incomplete) list on my User page. You can easily create tables with that on your own, see the introduction at Template:DropTestTableRow. A few pros and cons:
 * You can keep track of your own DropTests, and the table has a column for the number of runs you did.
 * Each entry will auto-generate the link to the actual DropTest
 * Each entry will automatically compute the average chroniton cost per item, as well as the % value you are looking for
 * The disadvantage is, you'll have to keep the table up to date manually by updating the #runs and #items from time to time. --Crunch (talk) 21:30, 22 June 2016 (CDT)

A Tale of Confusing Lore


How in the world did I get 5 items from A Tale of Forgotten Lore? I have tried taking the same path dozens of times and can't duplicate it. I don't think there is a way to hit 5 reward nodes in one run. And this is obviously from the most recent version of the game because Changeling Bashir was released with the most recent update. I don't know what to make of this... could be an odd one-time glitch — CodeHydro (msg) 21:03, 22 June 2016 (CDT)



So I've been observing excess rewards in other missions. Only seen this 3 times out of thousands of runs.... so far only in away missions, not ship battles. Current hypothesis is that there is a tiny chance that a reward node gives double rewards. The code for variable reward amounts have been demonstrated for A Certain Formality in elite molde for crew training so it is not like DB has no precedent. Now we just need to see if it applies for ship battles and how it affects schematic drop multipliers.— CodeHydro (msg) 07:20, 24 July 2016 (CDT)



Now that's a first; 5 rewards from a 3 reward mission (4 node) mission.— CodeHydro (msg) 17:54, 24 July 2016 (CDT)

Should we add data from runs before rare drops claimed?
I think not, because before claim of rare (first time only) items the chance of dropping standard items must be different, than after claiming it. So I think that is combination of two different drop rates. Am I wrong? Example: Template:DropTest/Talarian Coup/1 --Koba44 (talk) 01:34, 7 July 2016 (CDT)
 * Well, whether the drop chance before or after claiming all rare items differs is impossible to tell without looking at their code - but it's likely that it's the same (the straight-forward way to program it would be something like "if(critical && CheckDropRare) DropRare; else RegularDrop;", which would keep drop rates for all standard items independent of one-time items). And either way, we'd have to discard all DropTest up until now, as we cannot reconstruct who had still rare items left, and who three-starred it.


 * What you noticed in the linked DropTest, though, is the inclusion of a run that contained a rare drop. As rare items are not listed, this will confuse the "per run" counter, and therefore produce an Error 1. So IMHO runs that contain a rare/one-time drop should be excluded completely. --Crunch (talk) 16:07, 12 July 2016 (CDT)


 * You may include runs before 3-starring a mission safely. The next release of DropTest will also include a way to include runs that had a rare reward as well. For now just create a pseudo-item in your raw data called "rare rewards" and tally your rare rewards as a separate item as a way to distinguish between runs where you legitimately got fewer than max rewards from those runs were you accidentally typed 1 for an item that you actually recieved 2 or more of. Do not retroactively apply this to runs with fewer than max rewards unless you specifically documented how many non-standard rewards you received. — CodeHydro (msg) 18:20, 21 July 2016 (CDT)

Next step - marking best drop ratios in item pages
So, there are some good data in wiki, so what about next step: Lets somehow inform in the item page which mission has best drop ratio. I need same things again and again and do not remember best mission for each item/rarity, so every time I must click every mission and compare ratios. It would be great to somehow mark which mission is the best for that item. My question is, if this functionality could be somehow automated via templates used in item pages.--Koba44 (talk) 10:35, 23 July 2016 (CDT)
 * The next version of DropTest has a single-item mode which can be placed directly on item pages in a compact manner. The code is done, but it is packaged with a new feature which has also been fully coded but needs to have the guidelines re-written. Alas, developing DropTest isn't as easy as template coding but also requires developing a uniform standard of testing to ensure the validity of results and such requires a lot of balancing... standards have to be strict enough to ensure quality but not so strict as to make it a burden... guidelines have to be detailed enough to be clear but not so detailed that people will see a only wall of text and decide not to bother reading it.— CodeHydro (msg) 07:03, 24 July 2016 (CDT)


 * In the meantime, I maintain best drop locations for many of the current event items on my page. --Crunch (talk) 08:23, 24 July 2016 (CDT)