Machine Learning for Offensive Computer Security (MALFOY)

ERC-funded research exploring how ML can be misused by attackers to reveal vulnerabilities and build automated threats.

Overview

The ERC MALFOY investigates how attackers could exploit machine learning (ML) models to craft automated, adaptive cyberattacks. Funded by the European Research Council and hosted at TU Berlin (MLSec group), MALFOY takes an offensive approach to better understand and eventually defend against future ML-powered threats. The goal is to anticipate tomorrow’s attackers and improve the robustness of digital systems by studying how ML can be used by adversaries.

Research Contributions

1. Human Feedback for Adapted AI-based Phishing Protection

One line of work explores the use of large language models (LLMs) for generating context-aware phishing content. This includes the design of controlled evaluation setups where LLM behavior is shaped by implicit user traits. We simulate phishing attempts that adapt based on the receiver’s profile, language, and perceived role, with the goal of supporting the development of human-centered, AI-based phishing awareness systems.

2. Cross-Source Attacks and Defenses in Mobility Tracking

Another strand of the project focuses on side-channel mobility and digital tracking assessment. We conduct a systematized study of cross-source attacks and defense mechanisms, analyzing how seemingly benign data sources—such as motion sensors—can be exploited to infer mobility patterns and unique user signatures. This work contributes to a broader understanding of how ML-based privacy attacks operate and how to mitigate them effectively.

Team & Collaboration

  • Host: Machine Learning and Security Group, TU Berlin (Prof. Konrad Rieck)
  • Role: Postdoctoral researcher and Visiting researcher