Inside Certification Development
Wednesday June 17, 2009
Certification Magazine has an interesting article about cert exam development. It's mostly a series of quotes from Paul Sorensen, global director of certification at Oracle University. Oracle has a large and growing certification program, so his comments are worth paying attention to.
One interesting nugget from the article is that it takes (Oracle, at least) "an average of 16 weeks to create an exam in beta, and industry numbers indicate it costs anywhere from $30,000 to $50,000 to build it."
It also discusses the fine line of making a certification too broad or too narrow, and the kind of interest the certified community must take in a credential to keep it going. This last point surprised me. From the article:
One interesting nugget from the article is that it takes (Oracle, at least) "an average of 16 weeks to create an exam in beta, and industry numbers indicate it costs anywhere from $30,000 to $50,000 to build it."
It also discusses the fine line of making a certification too broad or too narrow, and the kind of interest the certified community must take in a credential to keep it going. This last point surprised me. From the article:
“[We also need] an ongoing number of people — about 500 people per year — to participate in a certification for it to be a living, breathing project,” he said. “If we don’t have [sufficient demand], it’s hard to justify. Sometimes we’ll retire [a test] because it just doesn’t have the activity sufficient for us to reinvigorate it and build a new version.”I didn't realize that lack of interest can kill a certification; I always assumed it was because a new technology obsoleted the old one, or the technology just fell out of fashion. Interesting stuff.
This may or may not be relevant to this topic, but any certification testing should only be allowed to those who have proven work experience. To explain my statement; I have seen people cram for tests who have never done anything with computers and pass, and watch those same people afterwards not know how to do the basic of tasks. This task could be as simple as installing PCI cards or installing an OS. You would hope that everyone would do the labs or physically try the lessons in textbooks on their own but some people can cram enough to pass most tests (especially with practice test being made available). I am not against certifications, but I believe they should be available to those with proven work experience (varying on type of certification involved). It also levels the playing field for entry level to intermediate level positions, which may currently require certifications for equipment and/or software; nothing is better than hands-on training.
This is not a new concept for example, tradesmen have to have x number of years of both schooling and work experience to become a Journeyman and to be a licensed electrician one has to have x number of years of work experience to take the exam. These are only two examples, I have not seen many in the IT fields that require work experience to be proven before exams can be taken, and if things do not change soon certifications that do not require proven experience may not mean much in the future. Again, I have seen and heard of those obtaining certifications, which do not require proven experience, and not perform up to expectations of one being certified. Additionally, I have seen people without certifications who can perform task very well but are terrible test takers, at least for them work experience could help them overcome that problem.
If what I propose is realized then most exams can be honed over the years to adjust to changes and needs but the core would still be present. Additionally, with proven experience based certifications, nationally recognized titles can be formed to provide more uniformity to the IT industry. For example, exam levels could range like IT Computer Technician level I (one year work experience), II (two years work experience), etc and/or IT Administrator level I, II, etc, this type of nationally recognized title format could then lead to becoming prerequisites to taking vendor related exams. For instance, before one could take the LPI 101 exam they would have to have an IT Administrator level I certification, likewise, before taking the A+ exam one would need IT Technician level I certification. This would also allow vendors to eliminate common knowledge items out of their exams allowing them to focus on questions directly related to their exams. Furthermore, it would also make more budget sense if a company knows it has x number of personal up for exams it can better plan for the expanse of these exams annually. These are only my thoughts from my experiences and I am sure that I am not the only one who has ever proposed something like this, but this could definitely allow for more organization of IT personal, their job titles, and their job responsibilities. I have seen many titles given to IT personal but not many of those titles are backed by certifications, i.e. Jr. Administrator where is the certification for that title and what are its equivalents if there is a certification, (there would not need to be any equivalent titles if it was a nationally recognized certification with a title and cert for every position).
Thank you for your time in reading this.