Skill gap tests
Table of contents
IntroductionSkill gap tests aim at providing significant profiling information on the learner’s abilities and disabilities. These pieces of information enhance the organization’s strategic capability of building robust personalized learning paths according to company and learner needs. The skill gap tests interface offers a wide range of functionality, from comprehensive tests and questions creation, to correlations of question to skills and of test results to lessons and courses. This approach helps identify lack of skills and to can propose assignments of lessons and courses that would best cover this gap. These tests together with the subsequent Skill Gap Analysis comprise a methodology to identify the skills that users are currently missing and can potentially have a negative effect on their performance inside the organization. Results of skill gap tests can be then used for employee evaluation, job assignment proposition or courses assignment which is the point of particular interest within the LMS framework of eFront. In this respect, the skill gap related functionality of a LMS aims at:
Fulfilling these both requirements tremendously raises the level of efficiency of the learning procedure, by constituting it to a highly personalized and specific needs customized feature for each user separately. For example, assigning the “Intermediate Programming” lesson to all programmers within the company might seem reasonable. However, for the expert programmers who would not need attendance of this course, this action significantly debases the value of the learning procedure from their perspective, rendering it from a useful feature for personal and organizational growth to a mandatory chore. Consecutively, their approach to a lesson that would actually be useful to them might be less than acceptable, since their attitude towards the LMS is definitely prejudiced. The same might also apply for new users, for which the learning background is unknown or CV-claimed only, leading to possibly inaccurate or useless lesson assignments. The conclusion of these very common case examples is that separating different users according to their specific needs comprises an extremely important feature of a LMS, improving the user experience and thus enhancing the learning procedure altogether. To achieve the two aforementioned requirements in a scalable manner skill gap tests need to offer the following features:
Following these four guidelines, skill gap tests can be used in a variety of cases:
As will be presented next, eFront’s implementation of skill gap tests follows the above mentioned guidelines, and will be eventually in place to offer these and many more use cases. Skill gap tests involve administrators who create them and view their results and students who complete them. Main Skill Gap Test PageAll functionalities are accessed from the “Skill gap tests” link from the administrator Control Panel. The main skill gap test page for the administrator provides all necessary information for the effective management of this functionality. It comprises of two parts: the upper one which shows existing tests and questions and the lower one which shows the five most recently completed skill gap tests. By clicking on the icon on the top right corner of the recently tests, all solved tests will appear in a new page. Obviously, this interface allows the administrator to create new or review or edit existing tests and questions, while monitoring with one or two clicks the results of each particular test for each user completion.
Creation of a new skill gap test prerequisites in general the following four actions:
Skills are related to lessons either automatically (Educational version) and/or explicitly (Enterprise version). QuestionsIn the main skill gap tests page click on the tab “Questions” to display the list of all available questions. These are either associated with a particular lesson or with skill gap tests. From this, it should be evident, that existing questions belonging to lesson tests or just being created inside the framework of a lesson are valid choices for a skill gap test. More importantly, the eFront system takes care that all such questions are correlated with the specific skill of the lesson they belong to. Tip: An easy way to view all questions belonging to a certain lesson or being created solely for skill gap tests is by writing the corresponding text in the filter box on the bottom left corner of the table.
The implicated functionalities for each question are the following:
Add QuestionThe “Add question of type” select that appears on the upper left corner of the tab offers the functionality of adding a new question of a permitted skill gap type. Currently, the only type not allowed is the development question one, since it cannot be automatically corrected, which is fundamental for the wide applicability of skill gap tests. Clicking on the select and selecting a question type will lead to the following form, where a new question can be defined in the same manner as in a typical eFront lesson test. Edit questionEach question can be further edited after its initial definition by clicking on its text on the main questions tab or by clicking on the edit icon in that row. This leads to the two tabbed form presented below:
The first tab containing the questions’ information is again the same as in typical lesson questions. Changes on that tab need to be submitted and validated on the same question by clicking the “Save question” button on the bottom of the page or on a new question by clicking the “Save as new question” one. The second tab provides association capabilities of the edited question to skills. It is the same as the one shown as a popup when the button Tip: Skills can be associated to questions even after skill gap tests containing that question have been completed. This allows for fine tuning of the tests and inclusion of the new factors in the analysis even post-publishing. TestsA skill gap test is indirectly implicated with all skills associated with the questions which comprise it. All main tests management functionalities can be accessed through the upper table of the first tab “Tests” on the main skill gap test page.
This table, displayed on the image above, shows all currently defined skill gap tests. Notice that skill gap tests can be defined but not published, so as to be further processed by the administrator. Tests can be published either by clicking on the icon under the published column or inside the edit test form. The number of selected questions as well as the average score from all submitted solutions is shown for each test, providing overall management capabilities (like identifying general problems on a particular test). Finally the provided functionalities for each test are the following:
The table on the bottom part of the page contains information for the last five completed skill gap tests. This table has the typical “users to skill gap tests” interface that is also used in the complete list of all solved skill gap tests (which appears by clicking on the icon over the top right corner of that table) and in the form showing all submitted solutions for a single skill gap test (the icon on the main tests table). This interface offers three functionalities:
Add TestThe “Add skill gap test” link on the upper left corner of the tests tab on the main skill gap tests leads to the add test form shown below.
Though this form has several similarities with a typical eFront lesson test, it has the following differences:
By clicking on the “Save and select questions” button, the new test is created and the administrator is forwarded to the edit test page, which is explained in the following section. Edit testEach question can be further edited after its initial definition by clicking on its text on the main questions tab or by clicking on the edit icon in that row. This leads to the three tabbed form presented below:
The first tab contains the same form as the one presented in the previous chapter, where the test’s options can be changed and submitted by clicking the “Save test” or “Save as new test” buttons. The second tab which is shown in the picture above is used to correlate the edited test to questions. The table under this tab is very similar with the main questions table containing all questions from existing lessons as well as all skill gap defined questions. Any of these questions can be selected for this test by clicking on the check box of the corresponding row. The association is triggered immediately and the success of the operation is displayed with the appearance of the characteristic green check icon. Removals of questions for the test are done by deselecting that check box.
In the final tab captioned “Users” shows all existing assignments of the test and also offers capabilities for direct assignment of a test to a user, by clicking the check box on the corresponding row. Three states exist for each user and are denoted with the values under the “Check” column.
Skill Gap AnalysisThe skill gap analysis comprises the core of the entire functionality, by showing the user’s estimated skill possession according to his results on the particular skill gap test. A skill is supposed to be possessed by a user, if his total score on the test’s questions associated with that skill is higher than a threshold. On the initial load of this page, all thresholds for all skills are set equal to the general threshold defined for that test, during its creation.
As shown in the above picture, the score for each different skill related to the questions of this skill gap test is displayed under the score column. As mentioned above correlation of question Qi with skill S with relevance Ri means that if the Qi is answered correctly in the skill gap test it will count positively towards the possession of Si by a factor Ri. The opposite happens if Qi is wrongly answered. Therefore if correctness the Q¬i is Ci, for each of the N questions Qi related to S then the final score for the skill S is The missing skill set, as defined by the procedure above, will be used to identify proposed lessons and courses for this user. Stand-alone lessons and courses are automatically correlated with the “Knowledge of course/lesson Title” skill, while lessons that are only offered within a course do not offer any skill automatically - we assume that they are incorporated under the “Knowledge of course Title” of the course(s) they belong. Tip: eFront Enterprise administrators may also define they own skills and assign them in any way they choose to courses and lessons by the corresponding tabs in the Courses and Lessons management pages. Obviously, by correlating these skills to skill gap tests questions, users may very well include them in the skill gap analysis phase. The analysis then maps the missing skill set to the skills offered by courses and lessons. The higher the number of common elements between these sets (missing skills to lesson offered skills and missing skills to course offered skills) for each lesson and course, the higher the position this lesson or course will receive in the corresponding proposed assignments table, showed in the second tab. As displayed in the two pictures below, course “How to be a programmer (Instance#2)” is proposed for assignment for this student according to his results shown in the previous picture.
By clicking on the check box under the “Check” column, administrators may assign a lesson or a course to this student. Administrators are also able to validate this assignment but also view any existing lesson and courses assignments to this user in tables under the “Attending” tab. It goes without saying that lessons attended or completed by the user are not taken into account during the analysis. For example, if the student was already attending the “Internet basics” lesson, then no record would appear on the proposed lessons table. Administrators are also allowed to de-assign lessons from students, thus finding the best mapping of lessons to their students’ educational needs. Overall, the analysis interface enables the administrator to have a quick look at the lesson and courses currently assigned to the student and on the proposed assignments according to his results on this test. Moreover, thresholds for all skills or a particular skill may be altered, so as to enable administrator to deal differently for various tests or users. For example, a senior programmer in the company should be assigned a threshold of 80% for the “Algorithms Knowledge” while for a junior programmer a 50 or 60% would probably suffice. Of course these percentages should also take in regard the difficulty of the particular test, thus getting higher or lower for easier and more challenging tests respectively. StudentsWhen a student login he can see his assigned skill-gap tests on the left sidebar under the “Lessons” menu as well as links in the main lessons’ page. Note: Students will only see these links, if at least one unsolved skill gap tests exists for them. By clicking on the link, students are redirected a page containing all skill gap tests assigned to them (solved and unsolved ones). Solved tests are denoted with the double check icon and cannot be solved again (unless the administrator deletes current results ). As for the unsolved tests, which are denoted by the pencil icon, students are allowed to solve any of them, without any particular order.
Skill gap tests are solved in the same way as typical eFront lesson tests. Some differences here are that there exists no clock to time the results and tests cannot be paused. If the user has to close for any reason the test without submitting results, then his results will be lost and he will have to repeat the test from the beginning. Once the student submits a test, his results are sent to the system administrator, who can process them according to the process explained above. However, if the administrator has set the “Automatically assign lessons” option for this particular skill gap test, then the analysis will take place automatically. All lessons and all courses offering at least one skill from the missing skills set as estimated by the results of this test will be automatically assigned to this student, who can start studying them immediately after. Administrators retain of course the right to de-assign any of these lessons or courses after the automatic assignment. Files12
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
is clicked on the main questions table. Associations are made immediately either by clicking on the check box of each skill to be correlated, or by selecting a new Relevance value. The role of relevance is to denote how much the question and the skill are correlated. High relevance will lead to a higher impact of the result of this question during the skill gap evaluation by the system.
