Hidden learning spaces: What learning analytics cannot tell us
Authors: Biggins, D. and Kitchenham, A.
Conference: Association of Learning Technology Conference 2019
Dates: 3-5 September 2019
Abstract:Learning analytics is one of the most popular topics in higher education (Spring et al 2016) due to the potential benefits of improving the learning outcomes of students through the use of information on their engagement/performance during learning module/unit delivery (Pardo et al 2015; Persico and Pozzi 2015; Ferguson and Clow 2017). Underpinning learning analytics is the collection of information, much of which originates from the virtual learning environment (VLE). The analytics in learning analytics seeks to identify the activities and behaviours which benefit students’ attainment, satisfaction and retention and thus their outcomes (Biggins 2018).
The focus of this paper aligns with Conference Themes 1 and 3.
While there are identified benefits from learning analytics, successful implementation can be challenging. Sclater (2017) sounds a note of caution: 1. Institutions need to make good use of VLE tools to provide a basis for data analysis 2. Only a part of student learning takes place within the VLE 3. A student accessing the VLE does not mean that learning is taking place Recent analysis of VLE data at our institution has evaluated the association between the unit outcome in terms of the final mark with usage of the VLE across different units/disciplines. Using random forests machine learning techniques, the findings have been counterintuitive for they suggest that VLE usage has only a weak correlation with final marks. A key finding was that many units were not designed with learning analytics in mind, understandable given the recent introduction in our institution, and this no doubt explains some of the variance. However, even those units which did leverage a broader range of VLE functionality failed to display strong predictive ability. Of great surprise was the low correlation between VLE content usage and unit outcomes and this was to such an extent that the random forest algorithm would often omit content completion from its predictive models.
That students were achieving successful outcomes in units despite low content usage led us to conclude that student learning was taking place but that some/much of this was happening outside the VLE and outside institutional planning and control. The search for these hidden learning spaces had begun. Shoufan (2019) identified the reasons why students use resources such as YouTube and it became clear that there is a range of non-formal, social learning spaces inhabited by and used only by students. The authors commenced a study to capture information on the nature, extent and rationale for these learning spaces with the goal of augmenting the information gleaned from learning analytics to create a more complete picture of student learning, information that can be used to further enhance their attainment, satisfaction and retention.
In this paper, we report findings from on-going questionnaire/student-based primary research to shed light on hidden learning spaces and the key non-VLE factors identified, for example attendance. Our paper will be of interest and benefit to other institutions seeking to evaluate and optimise learning analytics to build a more holistic picture of student learning, thereby enhancing student outcomes.
Source: Manual