The following article is a technical write-up detailing the Risk-Limiting Audit (RLA) protocols which can be used for STAR Voting, and providing elections officials with detailed information on how to conduct a Risk-Limiting Audit.
For lay-person friendly information, click on the following links to learn about election security or auditing in regards to STAR Voting.
How does a Risk-Limiting Audit For STAR Voting work?
All Risk-Limiting Audits start by defining some amount of allowable risk, called "alpha." This is set arbitrarily by statute and is a percentage risk such as "5%" or "10%." For instance, if a jurisdiction sets the allowable risk at 5%, that means, under the pessimistic assumption that the election is NOT valid, there can be no more than a 5% risk of incorrectly validating the tabulation of the election.
A few parameters determine the size of audit needed. First, the smaller the allowable risk you set, the more ballots you will have to check.
The second parameter is the win margin. The closer the race, the more ballots you will have to check.
The third parameter for Risk-Limiting Audits is known as “gamma.” This is a safety factor, not for the final outcome of the audit, but for the chances you might have to check multiple ballot sets in sequential audit rounds, or even fall back to a full ballot count. The lower gamma is, the lower the number of ballots you will re-check in the initial round of the audit, but the higher the chances you might have to go back and re-check more. For STAR voting, you should use 1.1 if it is easy to add ballots to the audit (“pull new ballots”), and 1.2 if it is harder.
The formula for all Risk-Limiting Audits is complex (regardless of the voting method) and auditing for governmental elections is handled by professionals, but for those running non-governmental elections or for those interested in the matter, there is an online audit calculator for running Plurality RLAs, which can also be employed for auditing STAR Voting.
The main difference between auditing Plurality and STAR Voting using the online audit calculator is that you will run the calculations twice: once for the Scoring Round and again for the Runoff Round. Your audit's result — the number of ballots you will need to check — will be whichever round's output number is larger. Checking those ballots will allow you to limit the risk for both rounds.
(Note: The procedure used here makes a few “conservative” simplifications in order to keep the formulas involved relatively straightforward. That is to say, in practice, it will generally ask you to check a few more ballots than would be strictly necessary, and thus result in a risk even lower than the minimum you set — a better audit than the one you asked for. If necessary, a qualified statistician could design a procedure that required checking fewer ballots and didn’t have this extra safety margin, but the formulas for deciding how many to check would be substantially more complex. In particular, the procedure here for dividing the risk between the two rounds is conservative, as is the tallying step where you round up the one-vote and two-vote discrepancies for the score round.)
Instructions for using the Online Audit Calculator for STAR Voting:
Before you begin:
Decide on the overall Allowable Risk for your election. (See above for more information.) This number will be referred to as "alpha" for your audit. The calculator will need this as a fraction or decimal.
For example, if the overall alpha is 5% you would divide this into two parts: 3.33% for the scoring round, which would be input as “0.0333” in the calculator, and 1.67% for the runoff round, which would be input as “0.0167” in the calculator.
- Set the “gamma” to either 1.1 or 1.2, as explained above.
- Determine the total number of votes cast in the race to be audited.
Audit Election:
Calculate Size for Scoring Round Audit:
Step 1: Calculate the margin of victory for the Scoring Round, m:
a.) Find the second- and third-highest scoring candidates. Let's call those Candidate B and Candidate C, respectively.
b.) Find c, the difference in total scores between Candidate B and Candidate C.
c.) Calculate the minimum possible margin of victory, m, by dividing c by 5, and then dividing that result by t, the total number of ballots. (c/5)/t
d.) On the RLA calculator, you will use the second form (“Risk-limiting audit parameters: comparison audits”) to plan your audit. Input your m margin in the first field, where it says: "Margin of victory: the closest margin between a winner and a loser as a fraction of the total number of ballots for the given contest."
Step 2: Multiply your Allowable Risk (alpha) by 2/3 and input this number into the calculator where it calls for "Risk limit (alpha) as a fraction."
For example, if the overall alpha is 5% (0.05), the allowable risk of error for this scoring round is 3.3% (0.033 in the calculator.)
Step 3: Set overstatement and understatement rates. The default values (.001, .0001, .001, and .0001) are appropriate for machine-counted ballots; these correspond to error rates of 1 in 1,000 for mistaking valid votes for undervotes in either direction, and 1 in 10,000 for mistaking a vote for one candidate with a vote for another. If you are hand-counting ballots, you might want to allow for higher error rates, such as 0.01, 0.005, 0.01, 0.005 respectively.
Step 4: Click "Calculate" to determine the number of ballots needed to audit the Scoring Round.
Calculate Size for Runoff Round Audit:
Step 5: Calculate the margin of victory for the runoff round, n. This is calculated the same way as a "plurality-style" margin - the winning margin in the Runoff Round, divided by the total number of all ballots:
a.) Find d, the difference in total number of votes for the runoff winner, Candidate W, and the runner up, Candidate R.
b.) Calculate the minimum possible runoff margin of victory, n, by dividing d by t (the total number of ballots).
Step 6: Take the overall alpha for the RLA, and multiply by 1/3 for the alpha in the calculator.
Step 7: Leave other fields untouched for now. Overstatements and understatements should be set to 0. The field for "gamma" should be set to 1.1.
Step 8: Click "Calculate" to determine the number of ballots needed to audit the Runoff Round.
Perform Audit for Both Rounds Simultaneously:
Step 9: Identify ballots to audit:
a.) Compare the numbers calculated in steps 4 and 8. The larger of these two numbers is the total number of ballots needed for your initial Risk-Limiting Audit.
b.) Select a random sampling of ballots to audit.
Step 10: Evaluate the selected ballots for the scoring round, comparing scores for candidates B and C on each ballot to their scores on the cast vote record. Document any discrepancies you may find. Discrepancies should be recorded as overstatements and understatements. For STAR Voting, in the Scoring Round an understatement is the number of points a candidate was given in error which made their "margin of victory appear smaller than it really was," and an overstatement is the number of points a candidate was given in error which "make the margin of victory appear larger than it really was."
The overstatements and understatements can be kept in 4 running tallies:
- The total score discrepancy of the overstatements on ballots where the overstatement is less than or equal to 5 points. This total, divided by 5 and rounded up to the next whole number, is the number you’ll use for “1-vote overstatements” in step 12.
- The total score discrepancy of the overstatements on ballots where the overstatement is greater than 5 points. This total, divided by 10 and rounded up to the next whole number, is the number you’ll use for “2-vote overstatements” in step 12.
- The total score discrepancy of the understatements on ballots where the understatement is less than or equal to 5 points. This total, divided by 5 and rounded up to the next whole number, is the number you’ll use for “1-vote understatements” in step 12.
- The total score discrepancy of the understatements on ballots where the understatement is greater than 5 points. This total, divided by 10 and rounded up to the next whole number, is the number you’ll use for “2-vote understatements” in step 12.
For example: If a ballot had originally been counted as a score of 4 for Candidate B, but is in fact a score of 2 for Candidate B, then that would be counted as an overstatement of 2 for Candidate B because the difference between the original count and the audit count is 2 points, and because the error increased the margin of victory between Candidate B and Candidate C. This would go in tally 1 because 2 is less than 5. If this same exact error was found 11 times, the total number you would put in the calculator for 1-vote overstatements would be (2*11)/5=4.4, rounded up to 5.
Step 11: Evaluate the selected ballots for the runoff round, comparing the number of ballots with a preference for candidate W (the winner) vs R (the runner-up) to the corresponding preferences on the cast ballot record. Document any discrepancies you may find.
As above, discrepancies should be recorded as overstatements and understatements. Overstatements and understatements in the Runoff Round are tallied as 1-vote or 2-vote over/understatements. This number indicates how incorrect the original error was. In the runoff there are three options. A vote may be a vote for the winning candidate, Candidate W, a vote of No-Preference, or a vote for the runner up, Candidate R.
For example, a vote for Candidate W which was erroneously recorded as a vote of no preference represents a 1-vote change. A preference for Candidate W which was erroneously recorded as a preference for Candidate R is recorded as a 2-vote change. If the error increased the lead for A over B, then that is an overstatement. If the error decreased the lead for W then that is an understatement.
Check if audit is complete:
Step 12: Repeat steps 1-8, but use the upper block in the calculator (“Risk-limiting audit parameters”). In step 3, instead of setting approximate over- and under-statement rates, input the total numbers calculated in step 10 into the fields for 1- and 2-vote over- and under-statements. Similarly, in step 6, input the numbers from step 11.
Results: If the number of ballots the calculator tells you to recount for both rounds is not higher than the number you have already recounted, your audit is complete.
If not, then repeat steps 9-12, drawing only any new ballots needed. For instance, if you had previously recounted 100 ballots, and the new highest number needed was 150, then draw only 50.
A note on the audit parameters above:
The procedure used above makes a few “conservative” simplifications in order to keep the formulas involved relatively straightforward. That is to say, in practice, it will generally ask you to check a few more ballots than would be strictly necessary, and thus result in a risk even lower than the minimum you set — a better audit than the one you asked for. If necessary, a qualified statistician could design a procedure that required checking fewer ballots and didn’t have this extra safety margin, but the formulas for deciding how many to check would be substantially more complex. In particular, the procedure here for dividing the risk between the two rounds is conservative, as is the tallying step where you round up the one-vote and two-vote discrepancies for the score round.
Do you like this page?