Template:DropTest/doc

News
2016-7-26
 * Hovering your pointer over the Units column now shows a tooltip that gives information about average runs per drop and streak tendency of each item.
 * New, allowing items from multiple DropTested missions to be summarized in a sortable table.
 * Wrapping expressions  in #expr tags is no longer necessary for the parameters that commonly use them. (i.e. may use  instead of  )
 * Byline now links to contribution page instead of user talk.
 * New parameter  (for experienced testers only, see  for usage and rules).
 * Revised guidelines. Please re-read if you haven't yet ;)

2016-06-22
 * 

Tooltips and Color Indicator
There are two areas where you may hover your pointer to obtain additional information.
 * Units column
 * Cost / unit column

Hover your pointer over the number of units dropped of an item to see the average runs need per drop and some detailed "streak" information. Keep in mind those values are only as good as the reliability indicated in the next column.

You can hover over any average cost value for the range of costs at 2σ (~95.5%) confidence.

Note that reliability merely indicates how close the test average cost is likely to be to the "true" average cost given an infinite number of runs; it does not indicate that you could reliably obtain a single unit after spending that amount of chronitons or tickets. Funny thing about averages is that roughly half of all cases will be above average, and half will be below... almost none will be exactly average.

Usage
This template provides a standard form for presenting test results and performs many calculations automatically. It also has some built-in warnings to ensure the validity of test results.

It is strongly recommended that you list items in order shown in the "travel" window ([Warp 1] [Warp Max] [Travel]). Mission rewards are always shown in that order in the results screen, making data entry easier.

New data template should always be created via the "Create one!" links generated by Template:DropTest/multi. The code below is provided as a reference (and may be used to reset a template, though may not be synced with the current default preloader) 

Template Naming Convention
New DropTest templates should be named  where   is the name of the mission and   is replaced by a number 1, 2, or 3 which represent normal, elite, and epic, respectively. That is, Template:DropTest/The Wrong Crowd/2 is for Elite difficulty of The Wrong Crowd.

No need to do this manually. Use Template:DropTest/multi on any mission page and it will generate a "Create one!" link that will automatically create a template of the appropriate title and which has some parameters pre-filled.

Tips and guidelines
Be sure read the core guidelines before gathering any data.

Core standards

 * Record items in the order shown by the travel window; this matches the item order of the Success screen (but not in-mission reward crates). It just makes data entry easier and could help avoid/catch errors among similar-looking items.
 * Note: Travel and success order parallelism may break down when missions drop distinct items of the same short name and rarity, as may Highwaymen with ENT and VOY Starfleet Uniforms. For such cases, success window order may change from run to run. Continue using travel order.


 * Always resolve any Error 1 messages that show up after you edited a DropTest. Error 1 means there is incorrect data present (typos, failed nodes not giving a reward, etc.), so double-check everything, and if necessary, remove incorrect runs. You can ignore Error 2's, though, it's just a notification that a DropTest has only very few runs.
 * Avoid sampling gapsi.e., try not to have many unrecorded runs between the first and last recorded run. Also do NOT retroactively include results from memory. Both actions may introduce selection bias.
 * Though DropTest takes only totals, you should record each run individually. That way, if you get, you can just discard the anomalous run instead of the whole lot. (Just keep in mind each discard is a sampling gap!)
 * Due to "luck", it is inevitable that even properly-conducted tests may yield off-the-mark results. That said, it is important never to manipulate data to"fit" your expectations or pre-existing results. It is not unusual for one tester to have results that are wildly different from that of another even if both testers have done a large number of runs. Both are important to share.
 * If you have any doubts as to whether you entered a run correctly, delete the run from your data set. Do NOT delete a run just because it felt "too lucky".
 * Do not use the adjustments parameter unless your data was gathered pursuant to the higher standards required.

Combining data

 * When combining results from different testers, keep sources separate using expression notation   . The order of the data sets (separated by +) should match that of the bylinethat is, B is from by2 and C1 and C2 are both from by3. Example here.
 * Even though this template only displays the names of up to 4 testers, continue to use . Ideally the byline should be sorted by most runs to least, but simply arranging it so that the top 4 most runs are among the first 4 bys in any order is adequate. If by5+ has contributed less than 5 percent of the runs, you may discard that tester's data. (Larger, more contiguous data sets tend to be more reliable; thus, discarding weak sources may fortify results against selection bias.)
 * Please don't be a hero who refuses to take credit. The byline is not just for bragging rights but is an important tool for quickly identifying the source of each data set. (Makes adding and verifying data much easier).
 * Some wikis have a policy of adding data only if your drop rate differ by, say, 10% or more. However, our in-house Vulcan has deemed this illogical since, when published results are "normal", it selectively prevents other normal results from inclusion (and ensures only abnormal data is shared). Instead, if your experience is drastically different, consider doing additional runs before publishing but publish anyhow and mention it on this template's talk page. Live long and prosper!

Best practices
The following are voluntary (optional) guidelines:
 * Avoid "micro-publishing". Contributing very few runs may be more disruptive than helpful. Small data sets are highly susceptible to selection bias and clutter the data entry area, making it harder for others to contribute.
 * Minimums depend on mission costs. As a rule of thumb, the sum of your total chronitons spent and your total runs should be at least 60e.g. 12 run min for 4US$crn runs (4*12 + 12 = 60) or a 3 run min for 20US$crn runs (20 * 3 + 3 = 63 > 60). If less, just save your data elsewhere for now until you have a chance to do more runs. Disregard this minimum for limited-access missions such as cadet challenges or those locked by a trait.


 * If you expect to share additional runs in the future after publishing initial runs, try to record every run of that mission to avoid sampling gaps. If you were unable to document some runs since your first contribution, hold off publishing until you have X runs without omitted runs such that X is the minimum described in the previous bullet point.
 * Tip: If inconvenient to document some runs, take screenshots of the success screen so you may review them later.


 * When testing a mission that drops an item you want, we askto reduce selection biasthat you not end your test the moment you obtain it; consider deciding beforehand "I will do X additional runs after getting Y" and stick to it. (Personally, I shoot for at least 3 extra runs. This suggestion is less important once a mission has over 100 runs or so.)

Adjustments
The  parameter allows the inclusion of valid runs that normally trigger  and avoids creating sampling gaps. Because Error 1 is the first line of defense against typos, this parameter is not included in the standard preloaded form to discourage the inappropriate suppression of that warning.

There are strict ground rules for using it:
 * Only experienced users who have contributed to 10+ different DropTest data templates may use adjustments.
 * If you do not meet this requirement, do NOT modify the adjustments parameter (except as needed for keeping data in the same order as the byline); simply exclude runs which cause Error 1 from your published values. (Keep the runs in your raw data for now; you may include them in your published values once you qualify if they were recorded in accordance with every rule in this section.)


 * When gathering data, record adjustments as a separate pseudo-item (example) and be sure to take clear notes which explain why the standard reward count for that run is different. (In lieu of notes, you may have multiple pseudo-items such as "rares" and "failed/missed"). Valid reasons include:
 * A) Rare reward recieved instead of standard reward
 * B) Failed/missed a reward node due to locked trait or because getting a rare reward requires a sub-optimal path
 * Note: Ignorance of a better path is NOT a valid reason because one cannot distinguish runs where one received max rewards by accidentally stumbling across the best path from runs where one accidentally typed 2 instead of 1. (Remember: If in doubt, delete the run.)
 * C) A node gave double rewards. (Extremely rare, but possible.) If this happens, you may record receiving a "-1" adjustment pseudo-item in your raw data.
 * Note: Do NOT give yourself the benefit of the doubt if you find excess rewards in runs not accompanied by a note or other proof that the extra reward is not the result of a typo.


 * Devise some procedure to protect against the possibility that you may, for example, accidentally type "1" in the box for Rares instead of Clothing Pattern (or whatever item is next to your adjustment pseudo-item). Safeguards include putting an empty column/row between adjustments and standard rewards and/or redundancy (adding 1 to the reward box AND adding a separate note acknowledging your awareness of a rare reward).
 * The adjustments parameter should be placed below the runs parameter with comments as shown here:


 * runs =         152 + 79 + 13 + 5
 * runs =         152 + 79 + 13 + 5


 * adjustments = (3-1) + -1 + 0 + 0
 * }
 * If you find anyone using this parameter who might not be following the above rules, remove ALL data contributed by the user (or, if applicable, revert that user's data set to the version before he/she used adjustments).

As a final note, if only a handful of runs require adjustments and such runs are at the very beginning of your data set, consider simply omitting those runs from your published values instead of invoking the adjustments parameter. Remember. the main purpose of this parameter is to avoid sampling gaps between the first and last published run; thus, simply changing your starting point may be preferred for inexpensive missions.

Messages
{{DropTest/sandbox
 * mission=An Illogical Result|level=elite
 * cost=4
 * runs=3

tags to separate pre-update from new results and dividing old results by the former multiplier is acceptable. See example.
 * adjustments = 2
 * {{item|Seven{{!}}how'd I get 7 if it drops 9 at a time?}} | 7 | 9
 * {{item|item|common}} | 3 |
 * {{item|item|rare}} | 2 |
 * note = Of course, is it really an error if it was intentional?

Error 2
While not technically an error, this message warns readers when the sample size is extremely small. It may be avoided by doing either:
 * A) 40 runs or more.
 * B) enough runs for all items to have been rewarded at least 4 times (a plural drop counts as one time).

If the above minimums seem lax, it is because this template is relatively new and not yet widely deployed. Thus, at the moment, increasing the number of missions with test results is more important than quality for single missions.

Error 3
This message appears when the total number of adjustments exceed 1/6th of the expected total number of rewards (drops). Because the adjustment parameter has high risk of distorting results, testers are STRONGLY discouraged from adding data requiring an adjustment value that could trigger this warning for missions costing less than 14 chronitons. If you spot this message on an inexpensive mission, please examine the page history to determine which editor triggered this message. Unless the editor left a good reason (as listed in the guidelines, please remove all data associated with that editor. Then kindly leave a message on that editor's talk page explaining this policy.

Note that "I did 100 runs of 3 rewards before realizing that 4 rewards were possible" is NOT a valid reason. (Someone careless enough to do that probably was not particularly careful in recording data.) A valid exception could be if a reward node were locked by a trait such that the editor did not have a crewman of sufficient skill for that node (for example, node 2.A in A Father Figure); for this reason to be acceptable, the editor must share raw data and list "locked" as a separate item.

Standardized disclosures
Since DB seems to be making a habit of making changes to missions drops that do not affect drop frequency, the following are standardized disclosure messages for the sanctioned inclusion of outdated data using Template:DropTest/note (or, other warning messages):

Short
Pass  to a DropTest data template, where row is the relevant row. Place each additional data template on a new line and then pass them all as the first parameter of Template:DropTest/rows (no spaces in front of each template): 

Basic
The short form is preferred for item pages (example)

Long
Just set the first param as "+" of Template:DropTest/rows and in each DropTest data template insert "+" in front of the row value to include  [Expand] for full DropTest and last updated info. You may even add customized columns to further extend the table as much as you want using wikitable markup (though pipe characters will have to be escaped using  or   and equal signs with  ). 

Math mode
Add a "-" in front of the row value of each DropTest data template and it output only the average unit cost without tooltips or HTML. This also increases the output precision to 3 decimal points. The output may then be used in expressions.  Better: Better: