WA Secretary of State Blogs
From Our Corner

R-71 update: ‘Error rate’ changes as process continues

by David Ammons | August 7th, 2009

confusedvoterConfused? So were we when the Referendum 71 webpage  posted a signature-rejection rate that was different — lower — than we had been seeing before, and in the daily postings. 

So here’s the scoop:  Elections Division has decided that a more accurate cumulative error or rejection rate (currently 11.63 percent) should reflect the sizable number of signatures that have been going from the rejected pile to the accepted stack after a master checker reviews the checker’s decision to reject. That is 409 so far. As previously discussed, also nearly 100 signatures that have been set aside because there was no voter signature on file will be shifted over to the “accepted” stack once the counties send the person’s electronic signature.     

The daily snapshots are accurate for that day, the numbers move around somewhat but subsequent review can change the error rate.  The lower cumulative error rate takes this into account.  Sorry this is confusing – it’s the first time they’ve been releasing daily tallies and the reasons for tentative rejection of signatures. It is very dynamic process and numbers move around before they are locked down.

We know that both supporters and foes of the referendum are anxious to project whether R-71 is headed to the ballot and look to daily information as a key indicator of how the full check will turn out.  All we can say is that it continues to look close, that the duplicate signatures will play a big role, and that the only number that really counts is whether they have 120,577 valid signatures confirmed by the end of the check.  Ordinarily, that would be done in a few days by random sample testing, but now it’s under the microscope for three weeks.

26 Responses to “R-71 update: ‘Error rate’ changes as process continues”

  1. So much for transparency.

  2. AJ –sorry about the confusion and my delay in answering questions earlier. i had to figure out what was going on myself. it was probably TOO much transparency — watching sausage being made! — as the signature-verification process is more fluid and a work-in-progress than i had realized. i have confidence in the integrity of the process, but the daily reporting (including by me) could use some work, or at least a more sophisticated breakout on what the different categories mean, and when the signatures are locked down as either accepted or rejected. that once seemed simple to me.

  3. David, I’m having a hard time reconciling the total checked compared to what’s in the daily stats. There seem to be 645 signatures unaccounted for in the daily checks versus what’s posted at the top of the page.

  4. Dave,
    This does add considerable messiness to the process. It would have been nice if the numbers of invalid signatures reflected the rejected signatures AFTER the “master checker” does the second stage check.

    Please consider posting the ACTUAL number of signatures that have been rejected in each category for each day. Specifically, projections of the number of duplicates requires knowing if any of the duplicates have been validated at the second stage of checking.

    Without this information, it is best not to release any of the number at all!

  5. John Colgan says:

    So what categories were the 409 newly deemed valid signatures originally rejected under?

    If they are coming from just the no signature on file and no match categories, that would mean that the vast majority of no match signatures would have to have been found to be a match. Which leads me to question whether there is any signature matching going on at all. With the petitions being withheld by court order, that means that Ref-71 backers could have simply pulled voter names from any source and forged the signatures and no one in the public would be any the wiser. The problem isn’t too much transparency but not enough!

  6. David,

    Here’s what is confusing. You said that 409 signatures got moved over. From what pool did these come from? From the
    counts of rejected signatures until this morning there were only 349 that were rejected due to mismatched signatures.

    Furthermore, if those 409 then went into the accepted pool, then the total checked would still be 29937 and the total accepted would be 26292.

    The 9 am update shows 30582 checked and 26994 accepted. How did the number checked go from 29937 to 30582? If the difference between the 29937 and 30582 accounts for signatures checked between 7 and 9 am, there is a discrepancy there. This implies that 645 signatures were checked in those two hours. However, the number accepted went from 26292 (the new total accounting for the 409 reversal) to 26994. That’s 702 additional accepted signatures.

    Bottom line is that none of these numbers are making any sense at this point. I’ve gone from feeling confident about this seemingly open process to doubtful and skeptical now. It would be really helpful if your math backed up your statements and you could accurately explain, in detail, what is going on.

  7. Sorry, but you’ve lost my trust in the process.

  8. John Colgan says:

    David,

    Does the master checker also review the checker’s decision to accept various signatures? Or is the process inherently set up to favor acceptance over rejection of signatures (i.e. acceptance is assumed valid, while rejection gets further scrutiny)?

  9. Stephanie says:

    I don’t understand the rejection numbers either on the R-71 website.
    The tallys for “not on state list of voters” and “signature appears more than once” is at 3613 (as of Friday morning at 0700)… that is greater than the 3559 quoted on the site as the rejected numbers… yet those tallied numbers shouldn’t change because of a second signature checker.

  10. David Johnson says:

    I suggest everyone just calm down. Mr. Ammons is doing the best he can under great scrutiny and time pressure. Also, as he noted, this is the first time his office is releasing signature verification information in real time. So if there are a few bumps in the reporting, let’s not overreact.

    Mr. Ammons, I would suggest that a simple solution is either wait until the master checker has signed off, or just create a category for signatures such as “pending final verification”. Then folks will know signatures in that category can shift either way.

    I do think that John Colgan has raised an interesting question above (about how the SoS can catch allegedly forged signatures, especially since the public is prevented from conducting its own review thanks to the TRO) and would appreciate seeing a response.

  11. John,

    David said “As previously discussed, also nearly 100 signatures that have been set aside because there was no voter signature on file will be shifted over to the “accepted” stack once the counties send the person’s electronic signature.” So this implies those 409 are only those that were rejected due to mismatched signatures. But, there were only 349 of these as of this morning. So where the 409 came from is a mystery.

  12. John Colgan says:

    David,

    I believe the 409 comes from 60 no sig (total was 89 site now says 29 so 60 were accepted) while the remaining 349 would have to mean that each and every signature rejected for not matching the sig non file have now been accepted. Like I said before this sort of changing numbers with no clear explanation makes it even more important that the lists be made public when all is said and done.

  13. As an accountant, I am troubled by the math. At present, a spreadsheet of the daily tallies adjusted for recategorized newly-valid signatures results in negative numbers in some columns. That should not happen.

  14. It would appear that the current totals as reported are:

    30,582 checked
    26,994 approved

    29 waiting for county
    113 duplicate
    3,506 not registered
    -60 signature not matching

    I have trouble with the minus sixty.

  15. John,

    Thanks for the clarification. So as far as the math goes, that still leaves us wondering why the number of checked went up this morning from 29937 to 30582 (difference of 645). At first I thought that was the number checked from 7 am to 9 am, but the number of accepted went up by an even higher amount. That sum went from 26292, which accounts for the 409 switches, to 26994 (difference of 702). This implies 645 signatures were checked, with 702 of them being accepted. Say what?

    And that also leaves me wondering how just about all (if not all) of those rejects have now been accepted.

    Some may say were being nit picky, but this may come down to just a handfull of signatures in the end.

  16. Echoing John Colgan, if the master checkers are only double-checking invalidated signatures and not validated ones, this would unfairly biases the process towards validating the referendum. Mr. Ammons, I hope you can let us know whether this is the case, and if validated signatures are also scrutinized by the master checkers, how do they decide how many and which ones to look at?

    Thank you!

  17. Darryl– you’re right re not posting invalid signatures until they’re master-checked. that usually wouldn’t be the same day as the original checker rejected, so i guess you’d have to have a categories called “accepted,” “rejected,” and “pending.”

    AJ — the top numbers include some new Friday checks and master-checks.

    John Colgan–Changes were made across all three categories of reasons to reject signatures, primarily the non-matching signatures. The only way the signatures get placed in the “accepted” pile is if they are legitimate–a Washington state registered voter on the state database, with a signature that matches what’s on file, and non-duplicated. The madeup name scenario couldn’t work. I assume that if one side or the other sues the result of the tally, the petitions will be available for inspection and analysis, at least by the parties to the suit. The issue of public release is something the federal judge is taking up Sept. 3. We, of course, continue to support release of the petitions as a public record.

    David–sorry your confidence is shaken, in terms of watching the continuing process of verification. As I have blogged from the start, the daily numbers and daily error rate are interesting, but not statistically valid in terms of projecting how this will turn out. Ordinarily, sponsors turn in enough that we can do a statistical sampling and it’s done in a few days. This time, it’s an every-signature check, with observers watching, and appealable to the courts by the losing side.

    John Colgan– the five supervisors check the close-calls and the rejects. Re the 409 signature status changes, this is a moving number itself and was a combination of the signature classifications that assistant elections Director Shane Hamlin obtained from the master checkers at mid-day Friday. That number will be changing all the time as determinations are made.

    David Johnson– thanks for your good thought, and for your good suggestion. We’re trying to release this in a helpful fashion, but it’s more complicated that anyone anticipated — and more a work-in-progress that’s hard to report in distinct silos.

    Timothy–the signatures are shifted at times between categories, thus the columns and the reconcilations change. i’m sorry this isn’t simpler — accept or reject.

    David — the morning update also included some new adjudications, counties reporting back with electronic signatures, etc. You’re right, if this comes down to a small number of signatures, each decision can come under scrutiny and second-guessing. Gregoire-Rossi all over again.

  18. Hi, David… I appreciate how complicated this can be. Thank you for your explanations. I know this day-to-day transparency is new, and there are bound to be mistakes. I suppose I would rather see the daily tallies and just have a better understanding of what the categories really mean, even if the reporting methods aren’t all worked out yet. I think we’re getting there. Thank you.

    Will you post today’s tallies before the end of the day? I’m curious what happened with the shift that ended at 3pm or even what happened after that. And it would be nice to see “Accepted” “Rejected but Pending” and “Rejected” with the last category signed off by the master checker.

    Thanks,

    Tony

  19. Thanks for the lengthy and detailed reply. It helps a lot. I still am confused about the 9 am morning total going up, though. I thought adjudications meant a switch between categories. This doesn’t explain the extra 645 checked signatures in the 9 am update vs/ the 7 am update.

  20. David,

    I just saw the updated figures over at the website. Is it safe to assume that the 116 rejected for signatures that didn’t match are going to be moved back over to the accepted column?

    Is there another column that can be added that shows how many of these rejected signatures from each category are added back in?

  21. So, I just saw the latest numbers, updated as of 4:32 p.m., and I am still confused. The total checked in the summary does not match the total checked in the daily tallies. I (now) understand why other numbers might not match up, but I wouldn’t think that number would change when some rejections are reversed.

    My calculator tells me that, per the daily totals, 35,863 signatures have been checked. The chart at the top says it’s 35,296. That’s a significant difference.

    Are rechecks counted as checked signatures each time they’re checked? Or why are those numbers so different? I would expect a unique signature to be counted as checked only once, not included in multiple days’ tallies. Can you clear that up for me?

  22. Dave Ammons,

    ‘…so i guess you’d have to have a categories called “accepted,” “rejected,” and “pending.’

    In fact, we also need the total number rejected in EACH CATEGORY on that top table. A statistical projection of the total number of valid signatures either has the measure making the ballot or not, depending almost entirely on how many of the stage-1 invalid signatures that were accepted in stage-2 are from the duplicates. Please, please, please, consider breaking out these totals.

    ” the daily numbers and daily error rate are interesting, but not statistically valid in terms of projecting how this will turn out.”

    As a professional statistician, I strongly disagree with this statement. Valid projections can be made, and can include the statistical uncertainty based on the sampled size.

    The reason the signatures must be validated is because the initial sampling was not definitive (i.e. sampling error was too large to definitively determine the outcome).

    Early in the validation process one can make probabilistic statements (with due caution for the non-random nature of the signature sampling). But at some point during the validation process, the sample size becomes large enough that definitive statements will typically be possible. That is why an accurate breakout of signature rejections is so important.

  23. Tony– thanks for your suggestion, which i’ll pass along. Assume you’ve now seen the new day-end blog with fresh numbers, and know about the r-71 webpage.

    David–yes, all or most of those missing a comparison signature are likely to be shifted to the “accepted” category. Re yr suggestion about a new way of displaying each category that gets added back in, the Elections Division will check and see if the software/IT folks can do that. it would be good detail.

    Brie–yes, there is a discrepancy and we need to track it down and fix. Shane Hamlin at elections couldn’t immediately answer whether a signature will show up more than once in the “signatures checked” if it is handled more than once, since the adjudication or getting the missing information likely would occur on separate day. will check it out, but it might not be til Monday.

  24. Dave Hershey– this from Elections Division on yr earlier queston: “It is possible that some signatures that are initially rejected as not matching the signature on file will later be accepted once they are examined a second time, but that should not be an expected outcome.

    “It is also possible that signatures that are initially rejected because the signer appears to be not registered to vote are later researched and found to be registered to vote. That too happens, but should not be an expected outcome.”

  25. David Ammons: to quote Lurleen: “Echoing John Colgan, if the master checkers are only double-checking invalidated signatures and not validated ones, this would unfairly bias the process towards validating the referendum.”

    That point about bias still seems not to have been addressed. Do the “close calls” that get checked include *accepted* signatures too, for example? If so, why do “rejects” get a second chance to qualify, but “accepts” don’t get a second chance to fail? Are all the changes ones that move sigs from “reject” to “accept”, or does it only go the other way? If the changes only go one way, that suggests bias. In fact, if anything other than a “close call” gets a second examination, that seems like bias unless *everything* gets double checked.

  26. I agree with the comments regarding rejects getting a second chance to qualify, but accepted signatures not getting a second chance to fail. This would, inherently, skew the counting in favor of qualified signatures.

    And if someone, such as a widow, signed her recently deceased husband’s name to a petition, and the signature looked plausible, what method would you use to disqualify that signature if the deceased is still listed as a registered voter? Right now, with the temporary restraining order (TRO), isn’t it true that the dead person would never be discovered by anyone, and the signature never checked a second time, as with the rejected signatures?

    At the local high school dedicated teachers send the failing students to summer school to help them qualify for graduation. I hope the summer school “no student left behind” philosophy isn’t being used with the failing signatures, too,

Leave a Reply

 


About this Blog

The Washington Office of the Secretary of State’s blog provides from-the-source information about important state news and public services. This space acts as a bridge between the public and Secretary Kim Wyman and her staff, and we invite you to contribute often to the conversation here.

On the Web

Comments Disclaimer

The comments and opinions expressed by users of this blog are theirs alone and do not reflect the opinions of the Secretary of State’s Office or its employees. The agency screens all comments in accordance with the Secretary of State’s blog use policy, and only those that comply with that policy will be approved and posted. Outside comments will not be edited by the agency.

Your Corner of Washington

Older Posts

Blogroll

Blog Contributors

Recent Topics