Source: Pagalguy
Last month's announcement by the Indian Institutes of Management about the Common Admissions Test (CAT) going computer-based has left a lot of questions unanswered. While the July form notification will clear the air to a large extent, it will be interesting to see how the IIMs tackle certain issues that come packaged with an online test. Here are some aspects of the computer-based CAT that would make for good food for thought.
CAT 2009 will not exactly be an 'online' test
According to a high placed source at IIM Bangalore, “Test takers will not be required to be connected to the Internet at the time of the test. They will be attempting the test at terminals." The test will be already downloaded on the computer alloted to you and not administered via the Internet. Like the IIMs have said, it will be a 'computer-based test'. The testing environment therefore will not be hostage to slow Internet speeds or disconnections.
Increased cost of forms, applications completely managed by Prometric
The cost of taking the CAT 2009 is expected to be higher than that in earlier years. "The increase in examination fee is not expected to be in excess of 50 percent," says our source. Even if the CAT application form does cost 50 % more, it amounts to Rs 2,000 for the General category and about Rs 1,000 for the SC/ST category.
Not only are the IIMs outsourcing the CAT to the USA-based Prometric Services, they are also looking at laying the complete end-to-end responsibility of the CAT to the renowned testing agency that also administers the Graduate Record Examination (GRE) worldwide.
"In the first year the IIMs will be involved in processing of applications and once the model is established in the next two-three years, the only involvement of the IIMs will be in receiving final scores. Applications will be sent to Prometric and not to the IIMs as was the case in earlier years,” the source adds.
The IIMs also seem to be playing safe with the computer-based CAT in its first year.
“The results of the test will probably be declared after four-six weeks. It will not be possible for students to see test results immediately after attempting the test. We are being cautious as this is the first year of conducting the CAT as a computer based test,” says Prof Subhashish Gupta, Admissions Chairperson at IIM Bangalore.
"A decision has currently not been taken about whether CAT 2009 will be a computer adaptive test (that is, whether the difficulty of questions served by the computer will be adjusted according to accuracy of the test-taker). Students will in all probability be allowed to do rough calculations on paper provided at the test center and also go back and forth between different test sections while attempting the test," he further adds.
The computer-based test format
The bottleneck in a computer-based testing system is the rate of availability of infrastructure in the limited time in which the exam is going to be administered in. Since the IIMs have announced that the CAT 2009 is going to be held over a period of 10 days, it's a fair assumption to make that the questions are not going to be repeated beyond one testing session. Which means that the pool of questions is going to be very large and according to a few sources, the questions are going to be prepared by Prometric and not the IIMs?
However, the IIMs will have to put a very stringent procedure to make sure that every question paper administered in the 10 days of CAT 2009 is equal in difficulty level beyond doubt.
Prof Gupta says, “As is the case with the GRE, it will be statistically ensured that when the student attempts the test does not affect his test score. The experience of writing the test will be the same across the ten day period. We are trying to ensure that the computer based test is as similar as possible to the paper-pencil format.”
Infrastructure tightrope
In 2003, the Xavier's Admission Test (XAT, accepted by Xavier's Labour Relations Institute, Jamshedpur) made an abortive attempt at going computer-based. Barely 9 hours before the commencement of the test, the test was called off. The reasons for the cancellation were never well-known, though the official statement announcing return to the paper-pencil based test placed the blame on poor infrastructure.
Computer infrastructure in India has decidedly undergone a vast change since 2003, so has the number of trained computer network professionals. However, given that the interest in CAT amounts to over 2.5 lakh people across India, executing the CAT without any glitches is going to be a logistical challenge of gargantuan proportions, probably unheard of in India. If the IIMs and Prometric are indeed able to pull off a bug-free CAT, it would be an achievement to reckon with.
Assuming 2.5 lakh CAT takers over 10 days of testing, each day 25,000 MBA aspirants will take the CAT countrywide. Assuming 23 cities for test centers, it amounts to 1,086 tests per day per city. If the test duration remains 2.5 hours, then in a single day there would be three sessions of testing with 360 candidates in each. Which means, that each test center needs to have a working capacity of 360 working computers?
Of course, the unequal distribution of registered test-takers across small and large towns will make larger cities need higher capacities of computers.
We shall know in due course the kind of infrastructure Prometric is building, but will it be able to provide a uniform testing environment across the nation? Establishments housing desktop computers (computer training institutes, cyber cafes) have their own set of idiosyncrasies. Not everyone may get to take the test on a flicker-free flatscreen monitor that is not a strain on the eye. Likewise, computers might be slow at places or have partially malfunctioning mouse or keyboards. Even corporate establishments have to struggle to keep their computers virus and malware free. Technically speaking, even the most sporadic of cases of bad testing infrastructure would leave the CAT system vulnerable to litigation by test-takers.
It has been reported that the IIM-Prometric deal surrounding the CAT exam is worth $40 million. One expects Prometric to use the money to go on an overdrive putting a world-class computer testing facility across the country.
Too esoteric?
The CAT has traditionally been feared in smaller towns of India primarily because of the difficulty level of the English and Reading Comprehension sections. An unfamiliar testing format and increased form cost may discourage many to skip the CAT and instead concentrate on state-level management entrance exams or the Management Aptitude Test (MAT), thus resulting in only the more serious MBA aspirants taking the CAT 2009.
How MBA aspirants embrace the computer-based CAT on a large scale further depends on how easy the test preparation or coaching classes make it for them. Coaching class businesses in India are based on the franchisee structure, which puts a lot of the business expansion initiative on the local businessman who runs the city center of a test-prep agency. In smaller towns, franchisees will have to see enough business opportunity in training for an online CAT in order to make lakhs of rupees of investments in computer labs. The training courses also have to remain affordable for MBA aspirants.
Last month's announcement by the Indian Institutes of Management about the Common Admissions Test (CAT) going computer-based has left a lot of questions unanswered. While the July form notification will clear the air to a large extent, it will be interesting to see how the IIMs tackle certain issues that come packaged with an online test. Here are some aspects of the computer-based CAT that would make for good food for thought.
CAT 2009 will not exactly be an 'online' test
According to a high placed source at IIM Bangalore, “Test takers will not be required to be connected to the Internet at the time of the test. They will be attempting the test at terminals." The test will be already downloaded on the computer alloted to you and not administered via the Internet. Like the IIMs have said, it will be a 'computer-based test'. The testing environment therefore will not be hostage to slow Internet speeds or disconnections.
Increased cost of forms, applications completely managed by Prometric
The cost of taking the CAT 2009 is expected to be higher than that in earlier years. "The increase in examination fee is not expected to be in excess of 50 percent," says our source. Even if the CAT application form does cost 50 % more, it amounts to Rs 2,000 for the General category and about Rs 1,000 for the SC/ST category.
Not only are the IIMs outsourcing the CAT to the USA-based Prometric Services, they are also looking at laying the complete end-to-end responsibility of the CAT to the renowned testing agency that also administers the Graduate Record Examination (GRE) worldwide.
"In the first year the IIMs will be involved in processing of applications and once the model is established in the next two-three years, the only involvement of the IIMs will be in receiving final scores. Applications will be sent to Prometric and not to the IIMs as was the case in earlier years,” the source adds.
The IIMs also seem to be playing safe with the computer-based CAT in its first year.
“The results of the test will probably be declared after four-six weeks. It will not be possible for students to see test results immediately after attempting the test. We are being cautious as this is the first year of conducting the CAT as a computer based test,” says Prof Subhashish Gupta, Admissions Chairperson at IIM Bangalore.
"A decision has currently not been taken about whether CAT 2009 will be a computer adaptive test (that is, whether the difficulty of questions served by the computer will be adjusted according to accuracy of the test-taker). Students will in all probability be allowed to do rough calculations on paper provided at the test center and also go back and forth between different test sections while attempting the test," he further adds.
The computer-based test format
The bottleneck in a computer-based testing system is the rate of availability of infrastructure in the limited time in which the exam is going to be administered in. Since the IIMs have announced that the CAT 2009 is going to be held over a period of 10 days, it's a fair assumption to make that the questions are not going to be repeated beyond one testing session. Which means that the pool of questions is going to be very large and according to a few sources, the questions are going to be prepared by Prometric and not the IIMs?
However, the IIMs will have to put a very stringent procedure to make sure that every question paper administered in the 10 days of CAT 2009 is equal in difficulty level beyond doubt.
Prof Gupta says, “As is the case with the GRE, it will be statistically ensured that when the student attempts the test does not affect his test score. The experience of writing the test will be the same across the ten day period. We are trying to ensure that the computer based test is as similar as possible to the paper-pencil format.”
Infrastructure tightrope
In 2003, the Xavier's Admission Test (XAT, accepted by Xavier's Labour Relations Institute, Jamshedpur) made an abortive attempt at going computer-based. Barely 9 hours before the commencement of the test, the test was called off. The reasons for the cancellation were never well-known, though the official statement announcing return to the paper-pencil based test placed the blame on poor infrastructure.
Computer infrastructure in India has decidedly undergone a vast change since 2003, so has the number of trained computer network professionals. However, given that the interest in CAT amounts to over 2.5 lakh people across India, executing the CAT without any glitches is going to be a logistical challenge of gargantuan proportions, probably unheard of in India. If the IIMs and Prometric are indeed able to pull off a bug-free CAT, it would be an achievement to reckon with.
Assuming 2.5 lakh CAT takers over 10 days of testing, each day 25,000 MBA aspirants will take the CAT countrywide. Assuming 23 cities for test centers, it amounts to 1,086 tests per day per city. If the test duration remains 2.5 hours, then in a single day there would be three sessions of testing with 360 candidates in each. Which means, that each test center needs to have a working capacity of 360 working computers?
Of course, the unequal distribution of registered test-takers across small and large towns will make larger cities need higher capacities of computers.
We shall know in due course the kind of infrastructure Prometric is building, but will it be able to provide a uniform testing environment across the nation? Establishments housing desktop computers (computer training institutes, cyber cafes) have their own set of idiosyncrasies. Not everyone may get to take the test on a flicker-free flatscreen monitor that is not a strain on the eye. Likewise, computers might be slow at places or have partially malfunctioning mouse or keyboards. Even corporate establishments have to struggle to keep their computers virus and malware free. Technically speaking, even the most sporadic of cases of bad testing infrastructure would leave the CAT system vulnerable to litigation by test-takers.
It has been reported that the IIM-Prometric deal surrounding the CAT exam is worth $40 million. One expects Prometric to use the money to go on an overdrive putting a world-class computer testing facility across the country.
Too esoteric?
The CAT has traditionally been feared in smaller towns of India primarily because of the difficulty level of the English and Reading Comprehension sections. An unfamiliar testing format and increased form cost may discourage many to skip the CAT and instead concentrate on state-level management entrance exams or the Management Aptitude Test (MAT), thus resulting in only the more serious MBA aspirants taking the CAT 2009.
How MBA aspirants embrace the computer-based CAT on a large scale further depends on how easy the test preparation or coaching classes make it for them. Coaching class businesses in India are based on the franchisee structure, which puts a lot of the business expansion initiative on the local businessman who runs the city center of a test-prep agency. In smaller towns, franchisees will have to see enough business opportunity in training for an online CAT in order to make lakhs of rupees of investments in computer labs. The training courses also have to remain affordable for MBA aspirants.
