{"id":{"repo_id":"ottawa-retro","oai_identifier":"oai:ruor.uottawa.ca:10393/51118"},"canonical_url":"https://search.dev.ndltd.org/etd/ottawa-retro/oai:ruor.uottawa.ca:10393/51118","repository":{"repo_id":"ottawa-retro","name":"University of Ottawa","base_url":"https://ruor.uottawa.ca/server/oai/request"},"display":{"title":"Automated Care Pathway Modeling Using Agentic and Knowledge-Aware LLMs","abstract":"Clinical pathways (CPWs) translate evidence-based guidance into stepwise care but are often disseminated as free text, obscuring control-flow semantics needed for clarity and computability. Formalizing CPWs as process models - e.g., in the Business Process Model and Notation (BPMN) - improves comprehensibility and enables downstream automation. This thesis designs, implements, and evaluates LLM4CPW, a pipeline for automatic guideline-to-BPMN modeling using large language models (LLMs). We compare two contemporary frameworks - MAO (agentic, multi-role orchestration) and ProMoAI (single-agent with self-refinement loop) - under controlled execution with standardized evaluation. Automated metrics combine node-level and structural similarity (after Dijkman et al.) with graph-edit distance; a clinician provides fidelity ratings with qualitative annotations. We then investigate knowledge-aware modeling via a categorization of recurrent errors and curated clinical statements, testing prompt-only injections versus a dedicated Knowledge Advisor agent placed at different stages of the workflow. Across four Ontario stroke Quality-Based Procedures (QBPs; n = 15 runs per framework), MAO attains higher node similarity (≈ 0.782 vs. ≈ 0.696) and structural similarity (≈ 0.630 vs. ≈ 0.585) than ProMoAI with large effects and p < 0.001, and exhibits markedly lower run-to-run variability; expert ratings also favor MAO. Knowledge-aware variants of MAO yield measurable gains: introducing a Knowledge Advisor after semantic review phase improves node similarity by > 4 points and reduces Graph Edit Distance by ∼ 13 (to∼ 97), with statistically comparable outcomes when placed before review; expert deltas likewise favor the advisor-based designs. Refining statement wording improves medians and interpretability without shifting means. Contributions. This thesis contributes (i) an auditable LLM4CPW pipeline and evaluation protocol; (ii) empirical evidence that agentic orchestration improves fidelity and stability; and (iii) a principled, deployable strategy for knowledge-enhanced modeling via a specialized advisory phase. Collectively, the findings demonstrate the feasibility of reliable, automatically extracted BPMN models from concise clinical guidelines and chart a path toward broader, clinically grounded automation.","abstract_html":"Clinical pathways (CPWs) translate evidence-based guidance into stepwise care but are often disseminated as free text, obscuring control-flow semantics needed for clarity and computability. Formalizing CPWs as process models - e.g., in the Business Process Model and Notation (BPMN) - improves comprehensibility and enables downstream automation. This thesis designs, implements, and evaluates LLM4CPW, a pipeline for automatic guideline-to-BPMN modeling using large language models (LLMs). We compare two contemporary frameworks - MAO (agentic, multi-role orchestration) and ProMoAI (single-agent with self-refinement loop) - under controlled execution with standardized evaluation. Automated metrics combine node-level and structural similarity (after Dijkman et al.) with graph-edit distance; a clinician provides fidelity ratings with qualitative annotations. We then investigate knowledge-aware modeling via a categorization of recurrent errors and curated clinical statements, testing prompt-only injections versus a dedicated Knowledge Advisor agent placed at different stages of the workflow. Across four Ontario stroke Quality-Based Procedures (QBPs; n = 15 runs per framework), MAO attains higher node similarity (≈ 0.782 vs. ≈ 0.696) and structural similarity (≈ 0.630 vs. ≈ 0.585) than ProMoAI with large effects and p < 0.001, and exhibits markedly lower run-to-run variability; expert ratings also favor MAO. Knowledge-aware variants of MAO yield measurable gains: introducing a Knowledge Advisor after semantic review phase improves node similarity by > 4 points and reduces Graph Edit Distance by ∼ 13 (to∼ 97), with statistically comparable outcomes when placed before review; expert deltas likewise favor the advisor-based designs. Refining statement wording improves medians and interpretability without shifting means. Contributions. This thesis contributes (i) an auditable LLM4CPW pipeline and evaluation protocol; (ii) empirical evidence that agentic orchestration improves fidelity and stability; and (iii) a principled, deployable strategy for knowledge-enhanced modeling via a specialized advisory phase. Collectively, the findings demonstrate the feasibility of reliable, automatically extracted BPMN models from concise clinical guidelines and chart a path toward broader, clinically grounded automation.","abstract_has_math":false,"creators":["Houshidari, Alireza"],"institution":"Université d'Ottawa / University of Ottawa","degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":["Van Woensel, William","Amyot, Daniel"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-12-01T20:26:20Z","date_published":"2025-12-01T20:26:20Z","updated_at":"2026-07-24T03:39:25Z","subjects":["LLMs","Process Mining","Knowledge-aware process mining","Process extraction","BPMN","Clinical Pathways","Clinical Guidelines"],"languages":["en"],"rights":["Attribution-NonCommercial 4.0 International"],"rights_urls":["http://creativecommons.org/licenses/by-nc/4.0/"],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["https://doi.org/10.20381/ruor-31573"],"render_values":[{"text":"https://doi.org/10.20381/ruor-31573","href":"https://doi.org/10.20381/ruor-31573","code":true}]}]},"links":{"outbound_url":"http://hdl.handle.net/10393/51118","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Van Woensel, William","Amyot, Daniel"]},{"key":"dc:creator","label":"Author","values":["Houshidari, Alireza"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-12-01T20:26:20Z","2025-12-01"]},{"key":"dc:publisher","label":"Institution","values":["Université d'Ottawa / University of Ottawa"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["LLMs","Process Mining","Knowledge-aware process mining","Process extraction","BPMN","Clinical Pathways","Clinical Guidelines"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Attribution-NonCommercial 4.0 International","http://creativecommons.org/licenses/by-nc/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/10393/51118","https://doi.org/10.20381/ruor-31573"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Clinical pathways (CPWs) translate evidence-based guidance into stepwise care but are often disseminated as free text, obscuring control-flow semantics needed for clarity and computability. Formalizing CPWs as process models - e.g., in the Business Process Model and Notation (BPMN) - improves comprehensibility and enables downstream automation. This thesis designs, implements, and evaluates LLM4CPW, a pipeline for automatic guideline-to-BPMN modeling using large language models (LLMs). We compare two contemporary frameworks - MAO (agentic, multi-role orchestration) and ProMoAI (single-agent with self-refinement loop) - under controlled execution with standardized evaluation. Automated metrics combine node-level and structural similarity (after Dijkman et al.) with graph-edit distance; a clinician provides fidelity ratings with qualitative annotations. We then investigate knowledge-aware modeling via a categorization of recurrent errors and curated clinical statements, testing prompt-only injections versus a dedicated Knowledge Advisor agent placed at different stages of the workflow. Across four Ontario stroke Quality-Based Procedures (QBPs; n = 15 runs per framework), MAO attains higher node similarity (≈ 0.782 vs. ≈ 0.696) and structural similarity (≈ 0.630 vs. ≈ 0.585) than ProMoAI with large effects and p < 0.001, and exhibits markedly lower run-to-run variability; expert ratings also favor MAO. Knowledge-aware variants of MAO yield measurable gains: introducing a Knowledge Advisor after semantic review phase improves node similarity by > 4 points and reduces Graph Edit Distance by ∼ 13 (to∼ 97), with statistically comparable outcomes when placed before review; expert deltas likewise favor the advisor-based designs. Refining statement wording improves medians and interpretability without shifting means. Contributions. This thesis contributes (i) an auditable LLM4CPW pipeline and evaluation protocol; (ii) empirical evidence that agentic orchestration improves fidelity and stability; and (iii) a principled, deployable strategy for knowledge-enhanced modeling via a specialized advisory phase. Collectively, the findings demonstrate the feasibility of reliable, automatically extracted BPMN models from concise clinical guidelines and chart a path toward broader, clinically grounded automation."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Automated Care Pathway Modeling Using Agentic and Knowledge-Aware LLMs"]}]}],"canonical_facts":{"dc:contributor":["Van Woensel, William","Amyot, Daniel"],"dc:creator":["Houshidari, Alireza"],"dc:date":["2025-12-01T20:26:20Z","2025-12-01"],"dc:description":["Clinical pathways (CPWs) translate evidence-based guidance into stepwise care but are often disseminated as free text, obscuring control-flow semantics needed for clarity and computability. Formalizing CPWs as process models - e.g., in the Business Process Model and Notation (BPMN) - improves comprehensibility and enables downstream automation. This thesis designs, implements, and evaluates LLM4CPW, a pipeline for automatic guideline-to-BPMN modeling using large language models (LLMs). We compare two contemporary frameworks - MAO (agentic, multi-role orchestration) and ProMoAI (single-agent with self-refinement loop) - under controlled execution with standardized evaluation. Automated metrics combine node-level and structural similarity (after Dijkman et al.) with graph-edit distance; a clinician provides fidelity ratings with qualitative annotations. We then investigate knowledge-aware modeling via a categorization of recurrent errors and curated clinical statements, testing prompt-only injections versus a dedicated Knowledge Advisor agent placed at different stages of the workflow. Across four Ontario stroke Quality-Based Procedures (QBPs; n = 15 runs per framework), MAO attains higher node similarity (≈ 0.782 vs. ≈ 0.696) and structural similarity (≈ 0.630 vs. ≈ 0.585) than ProMoAI with large effects and p < 0.001, and exhibits markedly lower run-to-run variability; expert ratings also favor MAO. Knowledge-aware variants of MAO yield measurable gains: introducing a Knowledge Advisor after semantic review phase improves node similarity by > 4 points and reduces Graph Edit Distance by ∼ 13 (to∼ 97), with statistically comparable outcomes when placed before review; expert deltas likewise favor the advisor-based designs. Refining statement wording improves medians and interpretability without shifting means. Contributions. This thesis contributes (i) an auditable LLM4CPW pipeline and evaluation protocol; (ii) empirical evidence that agentic orchestration improves fidelity and stability; and (iii) a principled, deployable strategy for knowledge-enhanced modeling via a specialized advisory phase. Collectively, the findings demonstrate the feasibility of reliable, automatically extracted BPMN models from concise clinical guidelines and chart a path toward broader, clinically grounded automation."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/10393/51118","https://doi.org/10.20381/ruor-31573"],"dc:language":["en"],"dc:publisher":["Université d'Ottawa / University of Ottawa"],"dc:rights":["Attribution-NonCommercial 4.0 International","http://creativecommons.org/licenses/by-nc/4.0/"],"dc:subject":["LLMs","Process Mining","Knowledge-aware process mining","Process extraction","BPMN","Clinical Pathways","Clinical Guidelines"],"dc:title":["Automated Care Pathway Modeling Using Agentic and Knowledge-Aware LLMs"],"dc:type":["Thesis"]},"updated_at":"2026-07-24T03:39:25Z"}