Paging Dr

  • Increase font size
  • Default font size
  • Decrease font size

The timing of the simulations

There have been some misunderstandings regarding the need to run so many simulations at this time. Hopefully this Paging Dr forum member's questions and the answers (provided by Lyndal Parker-Newlyn and "Henry Bukowski") clear up any outstanding concerns.

Question One

My understanding is that multiple simulations have to be run due to algorithmic/programming errors that create a faulty list. That's why each school is manually counting the ranks to see that the machine hasn't messed up. Then the programmers have to check the code, make modifications and rerun. I'm very thankful for all the long and stressful hours put in by the staff this year, and only hope the situation will run smoothly and pain-free next year.

Answer

Lyndal P-N: I don't want there to be an impression here that the programmers made a heap of errors, they didnt, the program is sophisticated and complex. There are multiple reasons that we must redo simulations and the programming is only one of them. Most often the changes are about quotas, place type preferences, minimum hurdles and other things that require human intervention. And rarely, to fix an algorithm.

Henry Bukowski: They're not programming errors per se. I didn't code the program and really know nothing about it, so everything I say is of course speculative. But the program is written to solve the problem. The code is often 'trained' on artificial data sets to ensure that the sorting algorithm performs as expected. As far as testing and validation for code goes, unless you get some actual data, it's very difficult to get the 100% answer on the first go. Especially with such an organic problem as this one. This explains why it's not a 'Simulation 1 fits all' process. It's also highly unlikely at this stage that there are any programming errors/faulty code. The code is the sorting algorithm. Adjustments to the code will not be the algorithm itself but perhaps weights given to variables. For example how much weight should be given to combine GPA/GAMSAT at one university compared to GPA/GAMSAT at another.

This also explains why the medical schools admissions staff cannot tell us how many simulations need to be run. As frustrating as it will be done when it's done can be, that's really the bottom line. To give an example of what could be happening between simulations, consider after simulation 5 is complete and all the universities have their list. For argument's sake, let's pretend all the universities were happy with their lists. In a dramatic turn of events, one university has unfortunately had to reduce the number of CSP places by one. This changes everything. Think of the follow-on effect. That one student who was at the 'bottom' of that university's list actually interviewed at a higher preference institution. So now they need to be matched against all their other preferences. Maybe their score is higher than someone else at a lower-preference university and so another person gets bumped off! Each time a change like this happens, the appropriate alteration is made in the code and the program is rerun.

I would also like to emphasise how labour-intensive the process is. I would say 99.9% of the process is not the GEMSAS magical matching system. It's the admissions staff. The program itself will take a very short time to run. Once the data is inputted the program itself probably takes five minutes to run. The output data then needs to be formatted appropriately and sent to all the schools. Each school needs to check all the information. They need to check that everyone on the suggested list meets the appropriate criteria for their university. They need to make sure that people who interviewed at their university, and have not received an offer, should not have received an offer. That's probably about 10% of it. Then you need to wait for all the schools in the consortium to contact GEMSAS with the green light or with problems with their list. This, I would argue, is the most important failsafe. Going through all interviewed candidates and checking that no one has been inadvertently left off the final list. I'm sure there are a number of people who will get offers this week who wouldn't have if the admissions staff weren't so dedicated and obliging.

Why are there still problems after 17 simulations? Any number of reasons. Obviously at this stage the schools aren't changing the number of places they have but there are a number of other factors to consider. Aside from clerical errors, you also need to consider offer quotas, minimum hurdles, place type preferences and so on. As far as multivariable problems go, this is it!

Finally I want to highlight the stress for the admissions staff. Firstly, they're dealing with prospective medicine students. The stakes are high to begin with. This year the whole system has changed. The work they have to do this year is I'm sure incomparably higher than previous years. And previous years was a lot of work! Think of the intensive data-checking. Looking at every student and comparing lists as described above! 17 times (and counting)! And to top that off, and this is the killer, they all went into this four weeks ago not knowing how long the whole thing would take!

Before someone chimes in and says why change it then?, this new system is the fairest possible system. When it comes to medicine admissions, a cost-benefit analysis wasn't even carried out. Cost is maximum (admissions staff labour and time), and the resulting fairness is maximum. And you guys are getting the benefit. For all the comments about the system, not one is about glitches/oversights/lack of fairness.


Question Two

I do have to wonder why so much of this is being completed so close to the deadline. Don't get me wrong, I'm extremely grateful for all the hard work that is going into this process, but shouldn't the engineering aspect been fine tuned out (maybe with fake numbers) a long time ago so that they won't experience so much stress now so close to the deadline?

Answer

Lyndal P-N: And re your comments about close to the deadline.. All schools uploaded admissions data on the 17th October. We have been at this for a month now. Everyone please stop with the implication that this is somehow last minute or badly planned. This has been running smoothly but is very complex and labour intensive, as I keep trying to explain.


Question Three

When medical school staff "manually" check the list, does this mean they must use eye-power to check that each candidate is in the correct rank? That no low-scoring candidate has somehow slipped into the list unintended? I can't imagine the pain of staring at the extensive lists of numbers for 18 iterations..

Answer

Lyndal P-N: Yes, each simulation means a few hours of us staring at spreadsheets. My eyes are very tired!

Last Updated on Sunday, 13 November 2011 06:58
 

Why is the GEMSAS Match taking so long?

Remember the first time you tried to solve problems in the ACER Section 3 practice test? Did you solve every question correctly on the first attempt? Even if you know how to solve the problem, one small slip up in the calculation leads to the wrong outcome.

The GEMSAS matching process is far more complicated than a single GAMSAT question. Every university has a different criteria for getting an interview. Every candidate must be matched against the interview criteria FIRST before being placed on any of the university's lists of accepted applicants. Each university has yet again a different criteria for offering a place in medical school. Every candidate must be ranked according to the criteria.

There are going to be more candidates on each university's ranking list than there are medical school places. Those candidates, who are in excess, will be passed on to their next university preference, according to the candidate's preference list. This process continues until either the candidate is ranked high enough to receive an offer or all preferences are exhausted.

It's a lot more complicated than simply assigning people to a university course based on their Year 12 marks. Each university has two sets of criteria (interview and offer). Each candidate has up to four scores (GAMSAT, GPA, interview, portfolio) . Even the calculation of GPA and GAMSAT varies between medical schools, adding a further complication to the process.  Then throw in quotas for CSP, BMP, MRBS and rural candidates.

On top of all this is the internal differences between some schools. Some schools can approve a simulation at admissions level, some at senior academic level, some require Dean or even Vice Chancellor sign off depending on that university's policy. Add to this multiple time zones and about 50 very tired staff across the country and you get a glimpse of what the last month has been like for the admissions staff. This is why they cannot predict an exact date or time.  There has been an enormous commitment both in and out of work hours by amazing admissions staff everywhere.

It would be a miracle if they got this working in one simulation. The simulation is an iterative solving procedure. It is an engineering optimisation algorithm that takes multiple user-specified variables (as explained above) and weights them according to their importance (specified by each university). The iterative procedure then solves the multivariable problem.

The GEMSAS matching process went 'live' without any real testing. So the number of simulations needed for such a large problem were/are unknown. That is why the admissions staff tell you it's out of their hands - it really is.

Each simulation will run. A list will be outputted. Each university checks their list. If the cutoffs/weightings/other criteria are not what the university wants, they tell GEMSAS and the procedure is re-run accounting for this.

Any complex math problem involves an 'intial guess' and then iterating around that to get a final solution. That's what is happening here.

After each simulation, the results are carefully checked by admissions staff from every school. Even the schools, who were happy with the previous simulation, have to check the new data, in case something has changed. That's why we waited several hours after simulation 17 to hear the outcome. The admissions staff of each medical school must ensure that every candidate meets the university's interview and offer criteria. They also have to ensure all their quota requirements are fulfilled.  Then they have to get sign off from the appropriate person.

If just one medical school is not happy with the outcome, adjustments must be made to the GEMSAS matching process to meet this school's requirements exactly. It's like playing a game of pick up sticks. As soon as you adjust the algorithm to meet this school's requirements, you have to observe what would happen to the lists produced for the other schools.

If one school gains or loses a student, it (possibly) changes the list for all the other schools.

Finally, we need to be confident that nobody, who should get an offer, misses out. Some people have asked why can't they stop after a certain number of simulations? If they stopped their simulations runs now, would you like to be the one, who missed out on a place due to an error that nobody wants to fix?

Message From Lyndal Parker-Newlyn (University of Wollongong)

Simulation 18 will be run on Sunday ready for all schools to review on Monday. We are hopeful that the hold up from Friday has been resolved but we won't know until we review the outcome of the next simulation. 18 may be it or we may need more, really can't tell at this stage.

There appears to be a misunderstanding that a school was unavailable or couldn't/didn't respond on time. That is incorrect. All schools were online and communicating effectively and collaboratively, however for two schools there were unforeseen complications that delayed outcomes and have required further changes. Everyone scrambled to try and resolve it Friday, but ran out of hours in the day. No one was sitting on their hands.

Last Updated on Saturday, 12 November 2011 05:37
 

GEMSAS magical matching machine: how it works

Simulation = one iteration of the GEMSAS matching process. During a simulation, the standardised interview scores are loaded for each applicant. The applicants are matched with the university at which they interviewed.

Matching = adding applicants to a ranked list for each university. The ranking is based on the university's own offer criteria. At the end of the simulation, the top ranked applicants receive offers. However, the process does not stop here.

Every university interviews more applicants than they have places to accommodate. The applicants, who are not ranked high enough to receive an offer where they interviewed, will be "rolled down" to the next university on that applicant's preference list to see where they would be placed on their list. If they are again not ranked high enough to receive an offer, they will be "rolled down" to their next lowest preference. The process continues until either the applicant is ranked high enough to receive an offer or all preferences have been exhausted.

It is not possible for a applicant to be considered for a higher preferenced university than the one, where they interviewed. In other words, there is no "rolling up" the preference list.

When all the "rolling down" is complete, the lists of applicants are sent to the medical schools. The admissions gurus of each medical school examine the list to ensure it is correct. Each school has its own quota for MRBS, CSP, BMP etc places. When every school is satisfied with the list produced by the simulation, a virtual green light flashes to allow the admissions gurus to prepare and send their offer letters.

It is important to point out that GEMSAS does not decide when the offers are ready to be sent. The medical schools (represented by the admissions gurus) make this decision.

It is possible for all schools to issue their offers simultaneously. This is not likely to happen because some schools need to perform extra tasks with their lists. e.g. allocate students to clinical zones. Some schools send their offers via snail mail.

At this point, GEMSAS should also have a final list of applicants ready to send the GEMSAS email of death. If anyone receives the GEMSAS email of death, it means that all their preferences have been exhausted. The GEMSAS matching process is working through all your preferences for the FIRST round of offers.

Admissions gurus throughout Australia are putting their best efforts into finalising the lists for offer. They are each considering not just the applicants they interviewed, but those who unfortunately missed out on a place and are eligible for consideration elsewhere.  Everyone is working steadily to get the best outcome ASAP.

It is understandable that simulations might take a while.

Important notes for applicants

  1. Your preferences for place type (CSP, BMP, MRBS etc) cannot be changed.
  2. When your application is "rolled down", your application is first checked to see if it matches the minimum criteria for interview cutoff. The minimum criteria for interview cutoff = GAMSAT, GPA and portfolio (where applicable) scores of the lowest ranked applicant for interview. No matter what your interview score, you must meet the universities minimum criteria to be even considered for an offer.
Last Updated on Saturday, 12 November 2011 06:02
 
  • «
  •  Start 
  •  Prev 
  •  1 
  •  2 
  •  3 
  •  4 
  •  5 
  •  6 
  •  7 
  •  Next 
  •  End 
  • »


Page 1 of 7