There have been some misunderstandings regarding the need to run so many simulations at this time. Hopefully this Paging Dr forum member's questions and the answers (provided by Lyndal Parker-Newlyn and "Henry Bukowski") clear up any outstanding concerns.
Question One
My understanding is that multiple simulations have to be run due to algorithmic/programming errors that create a faulty list. That's why each school is manually counting the ranks to see that the machine hasn't messed up. Then the programmers have to check the code, make modifications and rerun. I'm very thankful for all the long and stressful hours put in by the staff this year, and only hope the situation will run smoothly and pain-free next year.
Answer
Lyndal P-N: I don't want there to be an impression here that the programmers made a heap of errors, they didnt, the program is sophisticated and complex. There are multiple reasons that we must redo simulations and the programming is only one of them. Most often the changes are about quotas, place type preferences, minimum hurdles and other things that require human intervention. And rarely, to fix an algorithm.
Henry Bukowski: They're not programming errors per se. I didn't code the program and really know nothing about it, so everything I say is of course speculative. But the program is written to solve the problem. The code is often 'trained' on artificial data sets to ensure that the sorting algorithm performs as expected. As far as testing and validation for code goes, unless you get some actual data, it's very difficult to get the 100% answer on the first go. Especially with such an organic problem as this one. This explains why it's not a 'Simulation 1 fits all' process. It's also highly unlikely at this stage that there are any programming errors/faulty code. The code is the sorting algorithm. Adjustments to the code will not be the algorithm itself but perhaps weights given to variables. For example how much weight should be given to combine GPA/GAMSAT at one university compared to GPA/GAMSAT at another.
This also explains why the medical schools admissions staff cannot tell us how many simulations need to be run. As frustrating as it will be done when it's done can be, that's really the bottom line. To give an example of what could be happening between simulations, consider after simulation 5 is complete and all the universities have their list. For argument's sake, let's pretend all the universities were happy with their lists. In a dramatic turn of events, one university has unfortunately had to reduce the number of CSP places by one. This changes everything. Think of the follow-on effect. That one student who was at the 'bottom' of that university's list actually interviewed at a higher preference institution. So now they need to be matched against all their other preferences. Maybe their score is higher than someone else at a lower-preference university and so another person gets bumped off! Each time a change like this happens, the appropriate alteration is made in the code and the program is rerun.
I would also like to emphasise how labour-intensive the process is. I would say 99.9% of the process is not the GEMSAS magical matching system. It's the admissions staff. The program itself will take a very short time to run. Once the data is inputted the program itself probably takes five minutes to run. The output data then needs to be formatted appropriately and sent to all the schools. Each school needs to check all the information. They need to check that everyone on the suggested list meets the appropriate criteria for their university. They need to make sure that people who interviewed at their university, and have not received an offer, should not have received an offer. That's probably about 10% of it. Then you need to wait for all the schools in the consortium to contact GEMSAS with the green light or with problems with their list. This, I would argue, is the most important failsafe. Going through all interviewed candidates and checking that no one has been inadvertently left off the final list. I'm sure there are a number of people who will get offers this week who wouldn't have if the admissions staff weren't so dedicated and obliging.
Why are there still problems after 17 simulations? Any number of reasons. Obviously at this stage the schools aren't changing the number of places they have but there are a number of other factors to consider. Aside from clerical errors, you also need to consider offer quotas, minimum hurdles, place type preferences and so on. As far as multivariable problems go, this is it!
Finally I want to highlight the stress for the admissions staff. Firstly, they're dealing with prospective medicine students. The stakes are high to begin with. This year the whole system has changed. The work they have to do this year is I'm sure incomparably higher than previous years. And previous years was a lot of work! Think of the intensive data-checking. Looking at every student and comparing lists as described above! 17 times (and counting)! And to top that off, and this is the killer, they all went into this four weeks ago not knowing how long the whole thing would take!
Before someone chimes in and says why change it then?, this new system is the fairest possible system. When it comes to medicine admissions, a cost-benefit analysis wasn't even carried out. Cost is maximum (admissions staff labour and time), and the resulting fairness is maximum. And you guys are getting the benefit. For all the comments about the system, not one is about glitches/oversights/lack of fairness.
Question Two
I do have to wonder why so much of this is being completed so close to the deadline. Don't get me wrong, I'm extremely grateful for all the hard work that is going into this process, but shouldn't the engineering aspect been fine tuned out (maybe with fake numbers) a long time ago so that they won't experience so much stress now so close to the deadline?
Answer
Lyndal P-N: And re your comments about close to the deadline.. All schools uploaded admissions data on the 17th October. We have been at this for a month now. Everyone please stop with the implication that this is somehow last minute or badly planned. This has been running smoothly but is very complex and labour intensive, as I keep trying to explain.
Question Three
When medical school staff "manually" check the list, does this mean they must use eye-power to check that each candidate is in the correct rank? That no low-scoring candidate has somehow slipped into the list unintended? I can't imagine the pain of staring at the extensive lists of numbers for 18 iterations..
Answer
Lyndal P-N: Yes, each simulation means a few hours of us staring at spreadsheets. My eyes are very tired!


