Skill-Gap analysis
From EFrontWiki
Contents |
[edit] Introduction
This article introduces the concept and usage of skill gap tests provided for the Educational and Enterprise versions of the eFront platform (from release 3.5.2 and onward). Skill gap tests aim at providing significant profiling information on the learner’s abilities and disabilities. These pieces of information enhance the organization’s strategic capability of building robust personalized learning paths according to company and learner needs.
The skill gap tests interface offers a wide range of functionalities, from comprehensive tests and questions creation, to correlations of question to skills and of test results to lessons and courses. This approach helps identify lack of skills and to can propose assignments of lessons and courses that would best cover this gap.
[edit] Skill Gap Tests
This chapter provides a brief presentation on the concept of skill gap tests. These tests together with the subsequent Skill Gap Analysis comprise a methodology to identify the skills that users are currently missing and can potentially have a negative effect on their performance inside the organization. Results of skill gap tests can be then used for employee evaluation, job assignment proposition or courses assignment which is the point of particular interest within the LMS framework of eFront. In this respect, the skill gap related functionality of a LMS needs to aims at:
- Assigning lessons and courses to the system’s users, whose results demonstrate insufficient knowledge or possession of certain skills that are required (critical skills) or useful (non-critical skills) to perform their work adequately.
- Subsequently, but equally importantly, not assigning lessons to those employees who are proficient enough in the skills required.
Fulfilling these both requirements tremendously raises the level of efficiency of the learning procedure, by constituting it to a highly personalized and specific needs customized feature for each user separately. For example, assigning the “Intermediate Programming” lesson to all programmers within the company might seem reasonable. However, for the expert programmers who would not need attendance of this course, this action significantly debases the value of the learning procedure from their perspective, rendering it from a useful feature for personal and organizational growth to a mandatory chore. Consecutively, their approach to a lesson that would actually be useful to them might be less than acceptable, since their attitude towards the LMS is definitely prejudiced. The same might also apply for new users, for which the learning background is unknown or CV-claimed only, leading to possibly inaccurate or useless lesson assignments.
The conclusion of these very common case examples is that separating different users according to their specific needs comprises an extremely important feature of a LMS, improving the user experience and thus enhancing the learning procedure altogether.
To achieve the two aforementioned requirements in a scalable manner skill gap tests need to offer the following features:
- Identify possession or lack of skills through the corresponding tests
- Correlate skills to lessons and courses
- Provide enhanced automatism functionalities, so as to be scalable and easily assigned and corrected for every user of the system
- Present the results in a comprehensive manner offering further analysis features
Following these four guidelines, skill gap tests can be used in a variety of cases:
- Highly personalized lesson and courses assignments
- Skill-set identification and profile building for new and existing users
- Identification of the overall skill needs of the workforce and introduction of related lessons and courses
- Optimal current employee placement proposals according to their skills and to the ones their placements require
As will be presented next, eFront’s implementation of skill gap tests follows the above mentioned guidelines, and will be eventually in place to offer these and many more use cases. Skill gap tests involve administrators who create them and view their results and students who complete them.
[edit] Administrators
This chapter presents the basic administrative functions that relate to skill gap tests for eFront. All functionalities are accessed from the “Skill gap tests” link either from the left sidebar frame under the tab “Lessons” or from the administrator Control Panel.
[edit] Main Skill Gap Test Page
The main skill gap test page for the administrator provides all necessary information for the effective management of this functionality. It comprises of two parts: the upper one which shows existing tests and questions and the lower one which shows the five most recently completed skill gap tests. By clicking on the
Creation of a new skill gap test prerequisites in general the following four actions:
- Creating questions
- Assigning questions to skills
- Creating a new skill gap test
- Assigning questions to the new test
Skills are related to lessons either automatically (Educational version) and/or explicitly (Enterprise version). This is further explained in chapter 3.4.
[edit] Questions
In the main skill gap tests page click on the tab “Questions” to display the list of all available questions. These are either associated with a particular lesson or with skill gap tests. From this, it should be evident, that existing questions belonging to lesson tests or just being created inside the framework of a lesson are valid choices for a skill gap test. More importantly, the eFront system takes care that all such questions are correlated with the specific skill of the lesson they belong to.
Tip: An easy way to view all questions belonging to a certain lesson or being created solely for skill gap tests is by writing the corresponding text in the filter box on the bottom left corner of the table.
The implicated functionalities for each question are the following:
[edit] Add Question
The “Add question of type” select that appears on the upper left corner of the tab offers the functionality of adding a new question of a permitted skill gap type. Currently, the only type not allowed is the development question one, since it cannot be automatically corrected, which is fundamental for the wide applicability of skill gap tests. Clicking on the select and selecting a question type will lead to the following form, where a new question can be defined in the same manner as in a typical eFront lesson test.
[edit] Edit question
Each question can be further edited after its initial definition by clicking on its text on the main questions tab or by clicking on the edit icon in that row. This leads to the two tabbed form presented below:
The first tab containing the questions’ information is again the same as in typical lesson questions. Changes on that tab need to be submitted and validated on the same question by clicking the “Save question” button on the bottom of the page or on a new question by clicking the “Save as new question” one.
The second tab provides association capabilities of the edited question to skills. It is the same as the one shown as a popup when the button is clicked on the main questions table. Associations are made immediately either by clicking on the check box of each skill to be correlated, or by selecting a new Relevance value. The role of relevance is to denote how much the question and the skill are correlated. High relevance will lead to a higher impact of the result of this question during the skill gap evaluation by the system.
Tip: Skills can be associated to questions even after skill gap tests containing that question have been completed. This allows for fine tuning of the tests and inclusion of the new factors in the analysis even post-publishing.
[edit] Tests
A skill gap test is indirectly implicated with all skills associated with the questions which comprise it. All main tests management functionalities can be accessed through the upper table of the first tab “Tests” on the main skill gap test page.
This table, displayed on the image above, shows all currently defined skill gap tests. Notice that skill gap tests can be defined but not published, so as to be further processed by the administrator. Tests can be published either by clicking on the icon under the published column or inside the edit test form, described in chapter 3.3.2. The number of selected questions as well as the average score from all submitted solutions is shown for each test, providing overall management capabilities (like identifying general problems on a particular test). Finally the provided functionalities for each test are the following:
: Show all submitted solutions for this skill gap test
: Preview the test
: Print the test
: Change the order of question appearance through a drag-and-drop interface
: Edit the test, leads to the interface explained in chapter 3.3.2
: Delete the skill gap test
The table on the bottom part of the page contains information for the last five completed skill gap tests. This table has the typical “users to skill gap tests” interface that is also used in the complete list of all solved skill gap tests (which appears by clicking on the icon over the top right corner of that table) and in the form showing all submitted solutions for a single skill gap test (the icon on the main tests table). This interface offers three functionalities:
: View the solved test, where correct answers and mistakes are highlighted
: View the skill gap analysis for this user based on this test (as explained in chapter 3.4)
: Remove the submitted solution from the database, thus allowing a student to retake the test
[edit] Add Test
The “Add skill gap test” link on the upper left corner of the tests tab on the main skill gap tests leads to the add test form shown below.
Though this form has several similarities with a typical eFront lesson test, it has the following differences:
- General Threshold: This field denotes what should be the threshold (as a % percentage referring to score) under which a skill will be set as missing or over which it will be denoted as existing. For example, if a skill gap test has ten right/wrong questions (which are always either entirely correct or entirely wrong) all associated with skill S, then for a threshold of 50%, at least five questions need to be answered correctly, so that the system decides that the user possesses S. More on the subject will also be explained in the analysis part of this manual.
- Assign to all new students: If selected, each new student of the system will be automatically assigned this skill gap test, a very useful feature, considering the explicit assignment of the test for each new user.
- Automatically assign lessons: As will be explained in the chapter 3.4 the analysis of the tests is entirely automated. Therefore if this option is selected, then students will be automatically assigned the lessons and courses proposed by the analysis immediately after the completion of the test.
- Fields like duration, test repetitions, mastery score etc have been removed as irrelevant to skill gap tests.
By clicking on the “Save and select questions” button, the new test is created and the administrator is forwarded to the edit test page, which is explained in the following section.
[edit] Edit test
Edit test
Each question can be further edited after its initial definition by clicking on its text on the main questions tab or by clicking on the edit icon in that row. This leads to the three tabbed form presented below:
The first tab contains the same form as the one presented in 3.3.1, where the test’s options can be changed and submitted by clicking the “Save test” or “Save as new test” buttons.
The second tab which is shown in the picture above is used to correlate the edited test to questions. The table under this tab is very similar with the main questions table (section 3.2), containing all questions from existing lessons as well as all skill gap defined questions. Any of these questions can be selected for this test by clicking on the check box of the corresponding row. The association is triggered immediately and the success of the operation is displayed with the appearance of the characteristic green check icon. Removals of questions for the test are done by deselecting that check box.
In the final tab captioned “Users” shows all existing assignments of the test and also offers capabilities for direct assignment of a test to a user, by clicking the check box on the corresponding row. Three states exist for each user and are denoted with the values under the “Check” column.
- Not assigned
- Assigned but not completed
- Assigned and completed
: View the skill gap analysis for this user based on this test (as explained in chapter 3.4)
: Delete the results of the test and let the user complete it again. This will cause the table to reload, returning the user state to the “Assigned but not completed” state.
[edit] Skill Gap Analysis
The skill gap analysis comprises the core of the entire functionality, by showing the user’s estimated skill possession according to his results on the particular skill gap test. A skill is supposed to be possessed by a user, if his total score on the test’s questions associated with that skill is higher than a threshold. On the initial load of this page, all thresholds for all skills are set equal to the general threshold defined for that test, during its creation (chapter 3.2.1).
As shown in the above picture, the score for each different skill related to the questions of this skill gap test is displayed under the score column. As mentioned above correlation of question Qi with skill S with relevance Ri means that if the Qi is answered correctly in the skill gap test it will count positively towards the possession of Si by a factor Ri. The opposite happens if Qi is wrongly answered. Therefore if correctness the Q¬i is Ci, for each of the N questions Qi related to S then the final score for the skill S is 
.
If the bar of the test is green, this means that the score is higher than the threshold and the user is supposed to possess this skill. On the contrary, red bars denote a failure of the user to acquire an adequate score for this skill as defined by its threshold, inserting it into the missing skills set. A very interesting feature of this form is that the general threshold as well as the threshold per skill can be altered, thus providing different missing skill sets in the end, and being customizable for each test and each skill. The thresholds can be changed by typing a different percentage value on the corresponding text boxes and pressing Enter or clicking anywhere on the page (thus submitting the change).
The missing skill set, as defined by the procedure above, will be used to identify proposed lessons and courses for this user. Stand-alone lessons and courses are automatically correlated with the “Knowledge of course/lesson Title” skill, while lessons that are only offered within a course do not offer any skill automatically - we assume that they are incorporated under the “Knowledge of course Title” of the course(s) they belong.
Tip: eFront Enterprise administrators may also define they own skills and assign them in any way they choose to courses and lessons by the corresponding tabs in the Courses and Lessons management pages. Obviously, by correlating these skills to skill gap tests questions, users may very well include them in the skill gap analysis phase. Skill “Algorithms” in the picture above is such a case.
The analysis then maps the missing skill set to the skills offered by courses and lessons. The higher the number of common elements between these sets (missing skills to lesson offered skills and missing skills to course offered skills) for each lesson and course, the higher the position this lesson or course will receive in the corresponding proposed assignments table, showed in the second tab. As displayed in the two pictures below, lesson “Internet basics” and course “History course” are proposed for assignment for this student according to his results shown in the previous picture.
By clicking on the check box under the “Check” column, administrators may assign a lesson or a course to this student. Administrators are also able to validate this assignment but also view any existing lesson and courses assignments to this user in tables under the “Attending” tab. It goes without saying that lessons attended or completed by the user are not taken into account during the analysis. For example, if the student was already attending the “Internet basics” lesson, then no record would appear on the proposed lessons table. Administrators are also allowed to de-assign lessons from students, thus finding the best mapping of lessons to their students’ educational needs.
Overall, the analysis interface enables the administrator to have a quick look at the lesson and courses currently assigned to the student and on the proposed assignments according to his results on this test. Moreover, thresholds for all skills or a particular skill may be altered, so as to enable administrator to deal differently for various tests or users. For example, a senior programmer in the company should be assigned a threshold of 80% for the “Knowledge of lesson Internet basics” while for a sales manager, a 50 or 60% would probably suffice. Of course these percentages should also take in regard the difficulty of the particular test, thus getting higher or lower for easier and more challenging tests respectively.
[edit] Students
When a student login he can see his assigned skill-gap tests on the left sidebar under the “Lessons” menu as well as links in the main lessons’ page.
Notice: Students will only see these links, if at least one unsolved skill gap tests exists for them.
By clicking on the link, students are redirected a page containing all skill gap tests assigned to them (solved and unsolved ones). Solved tests are denoted with the double check icon and cannot be solved again (unless the administrator deletes current results as explained in chapter 3.3). As for the unsolved tests, which are denoted by the pencil icon, students are allowed to solve any of them, without any particular order.
Skill gap tests are solved in the same way as typical eFront lesson tests. Some differences here are that there exists no clock to time the results and tests cannot be paused. If the user has to close for any reason the test without submitting results, then his results will be lost and he will have to repeat the test from the beginning.
Once the student submits a test, his results are sent to the system administrator, who can process them according to the process explained in chapter 3.4. However, if the administrator has set the “Automatically assign lessons” option (see chapter 3.3.1) for this particular skill gap test, then the analysis will take place automatically. All lessons and all courses offering at least one skill from the missing skills set as estimated by the results of this test will be automatically assigned to this student, who can start studying them immediately after. Administrators retain of course the right to de-assign any of these lessons or courses after the automatic assignment.
