Right now the first prototype of the profiler agent has been implemented. This prototype clusters all Moodle users depending on different activities that they done/not done on the Moodle server. So, for example, it will analyse all comments made within a specific course. The analysis involves clustering the users into categories from the least active “comment’ers” to the most active, within the specific course.
There are 22 different course specific activities that are clustered and 1 non-course specific activity.
After analysing these the system then performs 3 overall analyses. One that clusters the users depending on their activity levels in all of the course specific activities, and another that clusters on the activities coming from the log, the comments made, forum discussion participation and general posts. The last one clusters users on the actual marks they have achieved.
Below is a table of the different activities that are analysed with a description.
[table id=2 /]
Reflecting upon the results the the system has produced on the training data there are a few issues or questions.
Here we are performing data mining on a separate training set than the real data. That is analogous to asking a blind man to be a goal keeper in football. Yes, he might be able to hear the shot and therefore estimate where the ball is going, but he cannot see the ball, and therefore might miss it. The system has been tested on two different Moodle installations, but there is no way of finding out if the results produced will be meaningful in the actual pilots. Then it could (and perhaps should) be argued that this is excatly why pilots are necessary. It is definitely expected that changes are needed after the pilots has run. Perhaps even throughout the pilots.
It is also clear that there are big differences between how Moodle is used by teachers. This will have an impact on the importance of different activities. The system is at the moment not changeable by teachers. A question is if it would make sense to create a profiler that would have such a feature?
A more practical problem is when the profiler should be started in a course. Obviously activities build up over time. It does not make sense to start it on day 0, because all students will look like they aren’t active. Perhaps the log activity or the users last access are more interesting in the beginning of a course. Just to give an indication of early activity. However this probably should not be used by any active alerter agent.
Any thoughts on these reflection?