Lunchtime Seminar
Archive WiSe 2023/23 & SoSe 2023
Towards a Success Model for Automated Programming Assessment Systems Used as Formative Assessment Tool
Lecturer: Tobias Antensteiner - Researcher@QE
Date: 29.06.2023
Abstract:
The evaluation of source code in university education is a central and important task for lecturers of programming courses. Thereby, lecturers are confronted with a growing number of students, increasingly heterogeneous learners, a shortage of tutors and highly dynamic learning objectives and technologies. To support lecturers to meet these challenges, the use of automated programming assessment systems (APASs), facilitating formative assessments by providing timely, objective feedback, is a promising solution. Measuring the effectiveness and success of these platforms is crucial to understanding how such platforms should be designed, implemented, and used. However, research and practice lack a common understanding of aspects influencing the success of APASs. To address these issues, we have devised a success model for APASs based on established models from information systems as well as blended learning research and conducted an online survey with 414 students using the same APAS. In addition, we examined the role of mediators intervening between technology-, system- or self-related factors, respectively, and the users' satisfaction with APASs. Ultimately, our research has yielded a model of success comprising seven constructs influencing user satisfaction with an APAS.
Generate-Retrieve-Generate: A Novel Approach to Open-Domain Question Answering
Lecturer: Abdelrahman Abdallah - Researcher@DS
Date: 22.06.2023
Abstract:
Open-domain question-answering (QA) tasks require the retrieval of relevant information from a large corpus to generate accurate answers. In this paper, we propose a novel approach called generate-retrieve-generate (GRG) that combines document retrieval techniques with a large language model, and works by first prompting a large language model (LLM) to generate contextual documents based on a given question. In parallel, a dual-encoder network retrieves documents that are relevant to the question from an external corpus such as Wikipedia. The generated and retrieved documents are then passed to the second LLM, which generates the final answer.
By combining document retrieval and LLM generation, our approach addresses the challenges of open-domain QA, such as generating informative and contextually relevant answers. We conduct extensive experiments on benchmark datasets to evaluate the effectiveness of our approach. The results demonstrate that our GRG method outperforms existing state-of-the-art models, improving the quality and accuracy of open-domain QA answers.
Computational Models for Natural perception in Virtual and Augmented Reality
Lecturer: Fabio Solari - University of Genoa
Date: 15.06.2023
Abstract:
The widespread use of virtual and augmented reality (VR and AR) systems raises the need for better designing them to mitigate their perceptual side effects. Bio-inspired computational models of visual perception could guide such design improvements. The proposed neural model is based on space-variant mapping, by implementing paradigms of the visual processing streams. The cortical representation of visual information is directly exploited to infer features related to the real world without devising ad-hoc computer vision algorithms. Besides artificial vision applications, the proposed model can mimic and describe human behavioral data of both motion and depth perception and attention. By leveraging previous outcomes, we can employ the modeled perception to improve the experience in VR and AR environments. We show study cases, such as a foveated depth-of field blur to mitigate cybersickness.
Automatic Discovery of Collective Communication Patterns in Task Graphs
Lecturer: Fabian Knorr - Researcher@DPS
Date: 11.05.2023
Abstract:
In HPC cluster applications, MPI is the de-facto standard for inter-node data exchange. Its collective communication APIs give vendors the opportunity to optimize common data exchange patterns to best utilize interconnect and network topology, routinely outperforming hand-rolled communication. In this talk we present a new method for detecting these collective communication patterns in the parallelized task graphs of Celerity, a runtime system for accelerator computation. We extend Celerity's distributed scheduling mechanism by an online and coordination-free analysis step that replaces its default point-to-point communication scheme with collective MPI operations where appropriate.
Through a set of synthetic benchmarks as well as a strong-scaling experiment on a direct N-body simulation, we examine the performance improvement by this technique from multiple angles.
Deep Reinforcement Learning for Continuous Control: Exploration by Action Noise
Lecturer: Jakob Hollenstein - Researcher@IIS
Date: 20.04.2023
Abstract:
Deep Reinforcement Learning (DRL) has gained widespread attention since Deep Q-Networks achieved human-level performance in Atari games. DRL has also shown promise in robotics domains, where the available actions are continuous rather than limited to a set of distinct choices. However, because continuous action spaces cannot be fully explored, exploration schemes are more important than in situations where the actions are discrete. The most commonly used exploration scheme is action noise, which involves adding random variations to the actions chosen by the system. Despite its widespread use, action noise has not been thoroughly studied. In this talk, we present our findings on the impact of action noise on exploration and performance in DRL for continuous action domains (such as robotics), and we propose pink noise as a good default choice for this type of exploration.
Contrastive Distillation For Liver Segmentation In The Wild
Lecturer: Stefano Fogarollo - Researcher@IGS
Date: 30.03.2023
Abstract:
Automatic liver segmentation is a key component for performing computer-assisted hepatic procedures. The task is challenging due to the high variability in organ appearance, numerous imaging modalities, and limited availability of labels. Furthermore, strong generalization performance is required in real-world scenarios. However, existing supervised methods cannot be applied to data not seen during training (i.e. in the wild) because they generalize poorly. Why not "distill" knowledge from a powerful pre-trained model with a contrastive distillation scheme especially suited for the liver? Extensive experiments using common abdominal datasets, and several patient scans from the Innsbruck University Hospital, revealed the potential of the method for automatic liver segmentation in the wild.
Formalizing the Development Closedness Criterion for Term Rewriting
Lecturer: Christina Kohl - Researcher@CL
Date: 23.03.2023
Abstract:
Confluence is an important but undecidable property of term rewrite systems. However several criteria that ensure confluence of certain classes of rewrite systems are known and have been implemented in different tools. Many of them are based on joinability of critical pairs. One of the most powerful results in this area is the development closedness criterion by van Oostrom. I will present our formalization of this result in the proof assistant Isabelle/HOL as well as its integration into the certifier CeTA, which can be used to ensure that criteria like this are applied correctly by confluence tools. I also present our recent extension of the previous result to *almost* development closed critical pairs and illustrate difficulties we encountered while trying to follow the original paper proof by van Oostrom.
Uncle Maker: (Time)Stamping Out The Competition in Ethereum
Lecturer: Aviv Yaish - Hebrew University
Date: 09.03.2023
Abstract:
We present and analyze an attack on Ethereum 1's consensus mechanism which allows miners to obtain higher mining rewards compared to their honest peers. This attack is novel in that it relies on manipulating block timestamps and the Difficulty-Adjustment Algorithm to give the miner an advantage whenever block races ensue. We call our attack "Uncle Maker" as it induces a higher rate of uncle blocks. We describe several variants of the attack. Among these, one that is risk-free for miners. Our attack differs from past attacks such as Selfish Mining, that have been shown to be profitable but were never observed in practice: We analyze data from Ethereum's blockchain and show that some of Ethereum's miners have been actively running a variant of this attack for several years without being detected, making this the first evidence of miner manipulation of a major consensus mechanism. We present our evidence, as well as estimates of the profits gained by attackers, at the expense of honest miners. Since several blockchains are still running Ethereum 1's protocol, we suggest concrete fixes and implement them as a patch for Go Ethereum. These can be adopted quickly and mitigate the attack and its variants.
Smart Data Management Layer: Moving Towards Stateful Computing
Lecturer: Juan Aznar-Poveda - Researcher@DPS
Date: 02.02.2023
Abstract:
There is a visible trend to efficiently compute specific tasks on the edge before resorting to the Cloud. This compute continuum is notoriously complex due to the heterogeneity of data, resources, and latencies. In this context, applications are usually built based on distributed micro-services that are connected by control- and data-flow. Some of them are stateless and can be re-run without side-effects. However, the number of applications that require shared state, synchronization, and low latency among distributed micro-services is rapidly increasing, while existing solutions do not fully satisfy such needs. In this talk, I will present our novel data management layer to support stateful applications distributed across the cloud-edge continuum. The proposed data layer (SDML) is language independent, highly scalable, and it is expected to offer different levels of consistency and seamlessly collaborate with existing runtime systems to intelligently manage data placement for improved latency and performance.
Identifying image manipulations with analytical and learning-based methods
Lecturer: Benedikt Lorch - Researcher@SEC
Date: 26.01.2023
Abstract:
Verifying the authenticity of digital images is an important task in criminal investigations, journalistic fact-checking, and for insurance companies. To this end, a broad range of image forensics tools has been developed. These tools can broadly be categorized into analytical models and statistical learning. In the first part of this talk, I will present a model-based approach for image forensics with JPEG images. This approach exploits an artifact introduced by a popular JPEG library during chroma subsampling. The second part of the talk addresses one of the major challenges related to statistical learning. When machine learning tools are exposed to images that differ too much from the training data, they are prone to fail silently. We show how to mitigate silent failures using Bayesian detectors that can express uncertainty in their prediction.
Multi-timeline summarization: Improving timeline summarization by generating multiple summaries
Lecturer: Adam Jatowt - Researcher@DS
Date: 19.01.2023
Abstract:
Nowadays, online news articles are one of the most commonly accessed documents on the Web. However, due to the large amount of news articles available online, it is getting difficult for users to effectively search, understand, and track news stories. To solve this problem, a research area of TimeLine Summarization (TLS) has been established, aiming to help users better understand the news landscape. In this talk, we discuss a novel task, Multiple TimeLine Summarization (MTLS), which extends the flexibility and versatility of Time-Line Summarization (TLS). Given a collection of time-stamped news articles, MTLS automatically discovers different important stories, and generates a corresponding timeline for each story. To achieve this, we proposed a novel unsupervised summarization framework based on two-stage affinity propagation. We also introduce a quantitative evaluation measure for MTLS based on extending the previous TLS evaluation methods.
Toward more self-directed robots in the wild. With an example of how flexible robots can enable new regenerative farming concepts
Lecturer: Sebastian Blaes - Max Planck Institute for Intelligent Systems, Tuebingen
Date: 12.01.2023
Abstract:
In the first half of my talk, I will discuss a recent project in the field of intrinsically motivated reinforcement learning. The natural world is too complex to preprogram the behavior of robots down to every individual detail. At the same time, the world is too complex to learn everything from scratch purely from data in an end-to-end fashion. Toddlers and young children use directed, curious self-play to explore the world efficiently and build intuitive theories about the world that they can later use for goal-directed behavior. In this project, we equip an agent with a relational inductive bias in the form of an object-centric state representation and structured world models that lead to a similar curiosity-driven, structured self-play as observed in children if combined with an objective that maximizes for future novelty. During a free-play phase, our agent learns a model of the environment that it can later use for planning to solve goal-conditioned object-manipulation tasks in a zero-shot manner. In the second half of my talk, I will present a real-world application of robots in the wild. Specifically in the agriculture sector. Over the last century, industrial agriculture codeveloped with the invention of larger and larger single-purpose farming machines. As a consequence, farmland had to adapt to the requirements of these large machines instead of the machines adapting to the needs of the individual plants. This development led to the unstable, brittle, and unhealthy biological ecosystems that can be found in modern industrialized agriculture. With this project, we set out to replace large farming machines with swarms of small, flexible, mobile multipurpose robots that can attend to individual plans and become part of rich and diverse biological ecosystems. This opens up the avenue toward new regenerative farming concepts such as permaculture farming by automating many of the processes that, as of today, require intense human labor.
Recommender systems for music retrieval tasks
Lecturer: Eva Zangerle - Researcher@DBIS
Date: 15.12.2022
Abstract:
Music is ubiquitous in today's world and with the rise of streaming platforms, listeners now have access to more music than ever before. Music recommender systems aim to help users discover and retrieve music they like and enjoy. Such user-centric retrieval approaches need to capture aspects that influence the user’s perception of and preference for music. These aspects include music content (descriptors extracted from the audio signal, such as tempo or acousticness), music context (external factors describing the track or artist such as a track’s lyrics), user properties (comparatively stable, long-term descriptors of the user, such as general music preferences), and user context (short-term, dynamic factors describing the user, such as the current activity, occasion or emotional state). User models that incorporate these aspects allow for a better picture of user preferences and enable improved personalization. However, to date, comprehensive user models and recommender systems that allow incorporating rich user models are rare. In this talk, I will present our novel comprehensive user and item models to capture the characteristics of users, their context, and musical items and how these models can be leveraged in newly designed context-aware recommender systems. I will also discuss potential biases that are introduced by such an approach.
Engineering serverless application life-cycles in federated FaaS
Lecturer: Sashko Ristov - Researcher@QE
Date: 01.12.2022
Abstract:
The dynamic and heterogeneous nature of the underlying cloud platform and infrastructure introduces several deficiencies for agile development, automated deployment, and efficient and effective execution of cloud applications. In this talk, I will present our advances in several aspects of distributed cloud application life-cycles. The main focus will be given on development, modeling, and running serverless workflow applications or function choreographies (FCs) in federated FaaS. Firstly, novel programming models for building hybrid FCs with FaaSification and portable and scalable FCs at a high-level of abstraction will be described. Secondly, the novel FC simulation model for Federated FaaS will be detailed. Finally, I will present our novel abstrac FaaS resource model with three levels of abstraction and novel schedulers to optimize FC makespan, scalability, and resilience in federated FaaS.
Graph neural networks for session-based recommendation
Lecturer: Andreas Peintner - Researcher@DBIS
Date: 02.11.2022
Abstract:
Session-based recommendation (SBR) aims to predict the next item based on a set of anonymous sessions. Capturing user intent from a short interaction sequence imposes different technical challenges since no user profiles are available and interaction data is naturally sparse. Recent approaches rely on graph neural networks (GNNs) to model sessions and global item relations. In this talk we present two different approaches incorporating graph-knowledge into the session-based recommendation scenario. First, we create a feature-rich item co-occurrence graph and learn the corresponding item embeddings in an unsupervised manner. These embeddings deal as input to several SOTA sequential models to tackle SBR. As a second approach, we propose to explicitly model the information aggregation mechanism over multiple layers of GNNs via shortest-path edges based on knowledge from the sequential recommendation domain. This approach does not require multiple layers to exchange information and implicitly filters unreliable item-item relations. Furthermore, to address inherent data sparsity, we are the first to apply supervised contrastive learning at the session level by mining data-driven hard negative samples.
Understanding shoulder surfing
Lecturer: Pascal Knierim - Researcher@IGS
Date: 17.11.2022
Abstract:
Shoulder surfing is a social engineering technique that involves looking over the victim's shoulder to obtain confidential information such as personal identification numbers, passwords, or other data. Because shoulder surfing is often opportunistic and challenging to observe in a real-world setting, little is known about the attack patterns and behavior. In this Lunchtime Seminar, I will present our recent investigations of shoulder surfing leveraging the capabilities of virtual reality. We conducted a lab study and observed users' behavior in different waiting scenarios. Based on the findings, I will present factors influencing shoulder surfing, common attack patterns, and outline a behavioral shoulder surfing model. Finally, I will provide an outlook on our current efforts to replicate our results in the real world.
Causes and effects of unanticipated numerical deviations in deep learning frameworks
Lecturer: Alex Schlögl - Researcher@SEC
Date: 03.11.2022
Abstract:
For a fixed training model and fixed input data, inference results are not consistent across hardware configurations, and sometimes not even deterministic on the same hardware configuration. We performed a deep dive into a typical machine learning inference stack and identified algorithm selection, floating point accuracy, aggregation order, data parallelism, and task parallelism as causes for numerical deviations. I will present existential evidence for our identified root causes, highlighting the complex interaction between TensorFlow, CUDA, and the Eigen linear algebra library. I will also show how these factors combine to yield different results on a large set of different hardware configurations. Finally, I will briefly discuss the implications these deviations can have on forensics, machine learning security, and applications of machine learning.
Gradient Critic for Policy Gradient Estimation
Lecturer: Samuele Tosatto - Researcher@IIS
Date: 27.10.2022
Abstract:
In reinforcement learning, the policy gradient theorem prescribes the usage of a cumulative discounted state distribution under the target policy to approximate the gradient. In practice, most algorithms based on this theorem break this assumption, introducing a distribution shift that can cause convergence to poor solutions. In this talk, we argue that classic policy gradients are Monte-Carlo estimators; therefore, they suffer from high variance and become problematic when samples are off-policy. We propose a novel gradient estimator based on a gradient Bellman equation. This Bellman equation allows redefining the policy gradient in recursive terms and approximating it using temporal-difference
Formalising Mathematics in Isabelle/HOL
Lecturer: Manuel Eberl - Researcher@CL
Date: 06.10.2022
Abstract:
In this talk, I give a very high-level overview about some recent work concerning the formalisation of mathematics in the interactive theorem prover Isabelle/HOL. I start with a very brief look at what Isabelle/HOL is and what formalised mathematics looks like. Then I show two particular problems that I encountered and how I solved them, namely asymptotic estimates of real-valued functions and complicated integration contours arising in Analytic Number Theory.