Structure-aware Deep Learning

Graph structures permeate the digital landscape in explicit and implicit forms. They connect or construct artifacts by combining semantic and structural information. We also observe them in the systems designed to process this data, in their learning algorithms and the very nature of the tasks they solve. At the same time, machine learning methods are extremely data-hungry, requiring petabytes of data for training. Due to their complexity, graphs remain an under-utilized resource in this regard. Many approaches cannot incorporate them due to being fully structurally unaware or not suited to the specific flavour of graphs encountered in some domains. This disconnect is sub-optimal from an effectiveness and efficiency perspective. We present methods that extend the scope of structure-aware deep learning through structural knowledge integration and enrichment, structural performance prediction, and synergistic transfer learning. Knowledge graphs organize information and make it directly available for querying. They provide a structured inference interface for manual and automated inspection, though they can suffer from data quality issues and require careful schema design. We rephrase the reconciliation of knowledge in knowledge graphs as a link prediction task, making it tractable with adapted graph neural networks, while also benefiting conventional link prediction tasks. We further combine textual semantics and structural expression for legal reference prediction via adapted heterogeneous graph neural networks operating on complex meta-information enriched graphs. Additionally, we explore methods for the integration of intermediary expressions in strongly typed heterogeneous graphs, improving prediction via meta-path-based processing. We also develop methods for automated machine learning workflow analysis and performance prediction. This includes the learning of salient representations for management as well as improvement of workflows through automatic suggestion and refinement of components. These are then extended to the prediction of Neural Architecture Search performance prediction, including adaptation to operation-on-edge spaces. Finally, we investigate the transfer capability of pre-trained attention structures for text-based prediction tasks and find it to be both inferior to directly optimized attention masks as well as highly dependent on inherent domain knowledge. We also show that the exploitation of hierarchical task formulation can improve prediction performance through joint learning in diverse learning domains, including link prediction, performance prediction, and specialized and general argumentation mining. The dissertation contains previously published or submitted texts: Wendlinger, L., Hübscher, G., Ekelhart, A., Granitzer, M. (2022). Reconciliation of Mental Concepts with Graph Neural Networks. In: Strauss, C., Cuzzocrea, A., Kotsis, G., Tjoa, A.M., Khalil, I. (eds): Database and Expert Systems Applications. DEXA 2022. Lecture Notes in Computer Science, vol 13427, p 133-146. Springer, Cham. https://doi.org/10.1007/978-3-031-12426-6_11; Wendlinger, L., Granitzer M. (2024). Informed Heterogeneous Attention Networks for Metapath Based Learning. In: SAC '24: Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing, p 458-465, ACM, New York. https://doi.org/10.1145/3605098.3635890; Wendlinger, L., Nonn, S.A., Al Zubaer, A., Granitzer, M. (2026). The Missing Link: Joint Legal Citation Prediction Using Heterogeneous Graph Enrichment. In: Wrembel, R., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Database and Expert Systems Applications. DEXA 2025. Lecture Notes in Computer Science, vol 16047, p 197-211. Springer, Cham. https://doi.org/10.1007/978-3-032-02088-8_14; Wendlinger, L., Stier, J., Granitzer, M. (2021). Evofficient: Reproducing a Cartesian Genetic Programming Method. In: Hu, T., Lourenço, N., Medvet, E. (eds) Genetic Programming. EuroGP 2021. Lecture Notes in Computer Science, vol 12691, p 162-178. Springer, Cham. https://doi.org/10.1007/978-3-030-72812-0_11; Wendlinger, L., Berndl, E., Granitzer, M. (2021). Methods for Automatic Machine-Learning Workflow Analysis. In: Dong, Y., Kourtellis, N., Hammer, B., Lozano, J.A. (eds) Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track. ECML PKDD 2021. Lecture Notes in Computer Science, vol 12979, p 52-67. Springer, Cham. https://doi.org/10.1007/978-3-030-86517-7_4; Wendlinger, L., Granitzer, M., Fellicious, C. (2023). Pooling Graph Convolutional Networks for Structural Performance Prediction. In: Nicosia, G., et al. (eds) Machine Learning, Optimization, and Data Science. LOD 2022. Lecture Notes in Computer Science, vol 13811, p 1-16. Springer, Cham. https://doi.org/10.1007/978-3-031-25891-6_1; Wendlinger, L., Braun, C., Zubaer, A., Nonn, S., Großkopf, S., Fellicious, C., Granitzer, M.: On the Suitability of pre-trained foundational LLMs for Analysis in German Legal Education, submitted to the proceedings of the International Conference on Machine Learning, Optimization, and Data Science 2025, preprint published: https://doi.org/10.48550/arXiv.2412.15902; Wendlinger, L., Kuhn, R., Mitrovic, J., Granitzer, M. (2025). Joint Learning for Efficient German Argument Mining. In: 2025 IEEE 37th International Conference on Tools with Artificial Intelligence (ICTAI), Athens, Greece, 2025, p 770-777. IEEE, Los Alamitos. https://doi.org/10.1109/ICTAI66417.2025.00111.

Level thesis:degree_level

thesis.doctoral

Grantor dc:publisher

Universität Passau

Year

2025

Author dc:creator

• Wendlinger, Lorenz

Contributors dc:contributor

• Granitzer, Michael
• Helic, Denis

• graph neural networks
• knowledge graphs
• structural knowledge integration
• structural performance prediction
• synergistic transfer learning
• link prediction tasks

Statement dc:rights

• Standardbedingung laut Einverständniserklärung

Repository record source_url

https://opus4.kobv.de/opus4-uni-passau/frontdoor/index/index/docId/1989

OAI identifier oai:identifier

oai:kobv.de-opus4-uni-passau:1989