{"id":{"repo_id":"catalunya","oai_identifier":"oai:openaccess.uoc.edu:10609/153609"},"canonical_url":"https://search.dev.ndltd.org/etd/catalunya/oai:openaccess.uoc.edu:10609/153609","repository":{"repo_id":"catalunya","name":"Universitat Oberta de Catalunya","base_url":"https://openaccess.uoc.edu/server/oai/request"},"display":{"title":"Fine-tuning CLIP models for peripheral blood cell images retrieval based on morphological descriptions","abstract":"This master's thesis explores the fine-tuning of Contrastive Language-Image Pre-training (CLIP) models for retrieving peripheral blood cell images based on morphological descriptions, aiming to assist hematologists in diagnostic processes. The study addresses the limitations of manual peripheral blood smear analysis, which is time-consuming and prone to variability, by leveraging state-of-the-art AI techniques. The methodology involved preparing a dataset of lymphocyte images with textual descriptions of 12 morphological features. Pre-trained CLIP models (ViT-B/32, ViT-L/14, ViT-B/16) were fine-tuned using Cosine Similarity, Contrastive Loss, and Multiple Negatives Ranking Loss (MNRL). Performance was evaluated using a novel descriptor-specific recall metric. Initial zero-shot performance of the pre-trained CLIP ViT-B/32 was limited, with a Recall@10 of 0.080. Fine-tuning yielded substantial improvements. The MNRL function proved most effective, and the CLIP ViT-B/32 architecture offered a strong balance of performance and efficiency, achieving a Recall@5 increasing from 0.055 (baseline) to 0.356 and a Recall@10 of 0.660. Performance varied across descriptors, with features like 'Cytoplasmic Hairiness' being more reliably retrieved. A web application was developed to demonstrate these capabilities. The study concludes that fine-tuned CLIP models hold significant potential for specialized medical image retrieval. Future work includes exploring advanced data augmentation, refining captioning strategies, and clinical validation.","abstract_html":"This master's thesis explores the fine-tuning of Contrastive Language-Image Pre-training (CLIP) models for retrieving peripheral blood cell images based on morphological descriptions, aiming to assist hematologists in diagnostic processes. The study addresses the limitations of manual peripheral blood smear analysis, which is time-consuming and prone to variability, by leveraging state-of-the-art AI techniques. The methodology involved preparing a dataset of lymphocyte images with textual descriptions of 12 morphological features. Pre-trained CLIP models (ViT-B/32, ViT-L/14, ViT-B/16) were fine-tuned using Cosine Similarity, Contrastive Loss, and Multiple Negatives Ranking Loss (MNRL). Performance was evaluated using a novel descriptor-specific recall metric. Initial zero-shot performance of the pre-trained CLIP ViT-B/32 was limited, with a Recall@10 of 0.080. Fine-tuning yielded substantial improvements. The MNRL function proved most effective, and the CLIP ViT-B/32 architecture offered a strong balance of performance and efficiency, achieving a Recall@5 increasing from 0.055 (baseline) to 0.356 and a Recall@10 of 0.660. Performance varied across descriptors, with features like 'Cytoplasmic Hairiness' being more reliably retrieved. A web application was developed to demonstrate these capabilities. The study concludes that fine-tuned CLIP models hold significant potential for specialized medical image retrieval. Future work includes exploring advanced data augmentation, refining captioning strategies, and clinical validation.","abstract_has_math":false,"creators":["Piazza Amat, Ivana"],"institution":"Universitat Oberta de Catalunya (UOC)","degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-06-04","date_published":"2025-06-04","updated_at":"2026-07-27T19:06:49Z","subjects":["vision language models","blood cell morphology","medical image retrieval"],"languages":["eng"],"rights":["CC BY-NC-ND"],"rights_urls":["https://creativecommons.org/licenses/by-nc-nd/4.0/"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/10609/153609","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.other","label":"Dc Contributor Other","values":["Pérez-Millan, Agnès"]},{"key":"dc:creator","label":"Author","values":["Piazza Amat, Ivana"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2025-10-03T15:07:01Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2025-10-03T15:07:01Z"]},{"key":"dc:date.issued","label":"Date","values":["2025-06-04"]},{"key":"dc:publisher","label":"Institution","values":["Universitat Oberta de Catalunya (UOC)"]},{"key":"dc:type","label":"Dc Type","values":["info:eu-repo/semantics/masterThesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["vision language models","blood cell morphology","medical image retrieval"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["CC BY-NC-ND"]},{"key":"dc:rights.uri","label":"Rights URI","values":["https://creativecommons.org/licenses/by-nc-nd/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/10609/153609"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["This master's thesis explores the fine-tuning of Contrastive Language-Image Pre-training (CLIP) models for retrieving peripheral blood cell images based on morphological descriptions, aiming to assist hematologists in diagnostic processes. The study addresses the limitations of manual peripheral blood smear analysis, which is time-consuming and prone to variability, by leveraging state-of-the-art AI techniques. The methodology involved preparing a dataset of lymphocyte images with textual descriptions of 12 morphological features. Pre-trained CLIP models (ViT-B/32, ViT-L/14, ViT-B/16) were fine-tuned using Cosine Similarity, Contrastive Loss, and Multiple Negatives Ranking Loss (MNRL). Performance was evaluated using a novel descriptor-specific recall metric. Initial zero-shot performance of the pre-trained CLIP ViT-B/32 was limited, with a Recall@10 of 0.080. Fine-tuning yielded substantial improvements. The MNRL function proved most effective, and the CLIP ViT-B/32 architecture offered a strong balance of performance and efficiency, achieving a Recall@5 increasing from 0.055 (baseline) to 0.356 and a Recall@10 of 0.660. Performance varied across descriptors, with features like 'Cytoplasmic Hairiness' being more reliably retrieved. A web application was developed to demonstrate these capabilities. The study concludes that fine-tuned CLIP models hold significant potential for specialized medical image retrieval. Future work includes exploring advanced data augmentation, refining captioning strategies, and clinical validation.","Aquesta tesi de màster explora el fine-tuning de models Contrastive Language-Image Pre-training (CLIP) per a la recuperació d’imatges de cèl·lules sanguínies perifèriques basades en descripcions morfològiques, amb l’objectiu d’assistir els hematòlegs en els processos de diagnòstic. L’estudi aborda les limitacions de l’anàlisi manual de frotis de sang perifèrica, que consumeix molt de temps i és propens a la variabilitat, aprofitant tècniques d’intel·ligència artificial emergents. La metodologia va incloure la preparació d’un dataset d’imatges de limfòcits amb descripcions textuals de 12 característiques morfològiques. Es van modificar models CLIP pre-entrenats (ViT-B/32, ViT-L/14, ViT-B/16) utilitzant Cosine Similarity, Contrastive Loss i Multiple Negatives Ranking Loss (MNRL). El rendiment es va avaluar mitjançant una innovadora mètrica de recall específica per a descriptors. El rendiment inicial en la modalitat zero-shot del model CLIP pre-entrenat ViT-B/32 va ser limitat, amb un Recall@10 de 0.080. El fine-tuning va produir millores substancials. La funció MNRL va demostrar ser la més efectiva, i l’arquitectura CLIP ViT-B/32 va proporcionar un bon equilibri entre rendiment i eficiència, assolint un Recall@5 que va augmentar de 0.055 (baseline) a 0.356 i un Recall@10 de 0.660. El rendiment va variar segons descriptors, sent característiques com la “Pilositat Citoplasmàtica” recuperades amb major fiabilitat. Es va desenvolupar una aplicació web per demostrar aquestes capacitats. L’estudi conclou que els models CLIP modificats posseeixen un potencial significatiu per a la recuperació especialitzada d’imatges mèdiques. El treball futur inclou explorar tècniques avançades de data augmentation, refinar l’etiquetatge de les imatges i obtindre validació clínica.","Esta tesis de máster explora el fine-tuning de modelos Contrastive Language-Image Pre-training (CLIP) para la recuperación de imágenes de células sanguíneas periféricas basades en descripciones morfológicas, con el objetivo de asistir a los hematólogos en los procesos de diagnóstico. El estudio aborda las limitaciones del análisis manual de frotis de sangre periférica, que consume mucho tiempo y es propenso a la variabilidad, aprovechando técnicas de inteligencia artificial de vanguardia. La metodología incluyó la preparación de un dataset de imágenes de linfocitos con descripciones textuales de 12 características morfológicas. Se modificaron modelos CLIP preentrenados (ViT-B/32, ViT-L/14, ViT-B/16) utilizando Cosine Similarity, Contrastive Loss y Multiple Negatives Ranking Loss (MNRL). El rendimiento se evaluó mediante una nueva métrica de recall específica para descriptores. El rendimiento inicial en la modalidad zero-shot del modelo CLIP preentrenado ViT-B/32 fue limitado, con un Recall@10 de 0.080. El finetuning produjo mejoras sustanciales. La función MNRL demostró ser la más efectiva, y la arquitectura CLIP ViT-B/32 proporcionó un buen equilibrio entre rendimiento y eficiencia, logrando un Recall@5 que aumentó de 0.055 (baseline) a 0.356 y un Recall@10 de 0.660. El rendimiento varió según los descriptores, siendo características como la “Pilosidad Citoplasmática” recuperadas con mayor fiabilidad. Se desarrolló una aplicación web para demostrar estas capacidades. El estudio concluye que los modelos CLIP modificados poseen un potencial significativo para la recuperación especializada de imágenes médicas. El trabajo futuro incluye explorar técnicas avanzadas de data augmentation, refinar el etiquetado de las imágenes y obtener validación clínica."]},{"key":"dc:format.mimetype","label":"Dc Format Mimetype","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Fine-tuning CLIP models for peripheral blood cell images retrieval based on morphological descriptions"]}]}],"canonical_facts":{"dc:contributor.other":["Pérez-Millan, Agnès"],"dc:creator":["Piazza Amat, Ivana"],"dc:date.accessioned":["2025-10-03T15:07:01Z"],"dc:date.available":["2025-10-03T15:07:01Z"],"dc:date.issued":["2025-06-04"],"dc:description.abstract":["This master's thesis explores the fine-tuning of Contrastive Language-Image Pre-training (CLIP) models for retrieving peripheral blood cell images based on morphological descriptions, aiming to assist hematologists in diagnostic processes. The study addresses the limitations of manual peripheral blood smear analysis, which is time-consuming and prone to variability, by leveraging state-of-the-art AI techniques. The methodology involved preparing a dataset of lymphocyte images with textual descriptions of 12 morphological features. Pre-trained CLIP models (ViT-B/32, ViT-L/14, ViT-B/16) were fine-tuned using Cosine Similarity, Contrastive Loss, and Multiple Negatives Ranking Loss (MNRL). Performance was evaluated using a novel descriptor-specific recall metric. Initial zero-shot performance of the pre-trained CLIP ViT-B/32 was limited, with a Recall@10 of 0.080. Fine-tuning yielded substantial improvements. The MNRL function proved most effective, and the CLIP ViT-B/32 architecture offered a strong balance of performance and efficiency, achieving a Recall@5 increasing from 0.055 (baseline) to 0.356 and a Recall@10 of 0.660. Performance varied across descriptors, with features like 'Cytoplasmic Hairiness' being more reliably retrieved. A web application was developed to demonstrate these capabilities. The study concludes that fine-tuned CLIP models hold significant potential for specialized medical image retrieval. Future work includes exploring advanced data augmentation, refining captioning strategies, and clinical validation.","Aquesta tesi de màster explora el fine-tuning de models Contrastive Language-Image Pre-training (CLIP) per a la recuperació d’imatges de cèl·lules sanguínies perifèriques basades en descripcions morfològiques, amb l’objectiu d’assistir els hematòlegs en els processos de diagnòstic. L’estudi aborda les limitacions de l’anàlisi manual de frotis de sang perifèrica, que consumeix molt de temps i és propens a la variabilitat, aprofitant tècniques d’intel·ligència artificial emergents. La metodologia va incloure la preparació d’un dataset d’imatges de limfòcits amb descripcions textuals de 12 característiques morfològiques. Es van modificar models CLIP pre-entrenats (ViT-B/32, ViT-L/14, ViT-B/16) utilitzant Cosine Similarity, Contrastive Loss i Multiple Negatives Ranking Loss (MNRL). El rendiment es va avaluar mitjançant una innovadora mètrica de recall específica per a descriptors. El rendiment inicial en la modalitat zero-shot del model CLIP pre-entrenat ViT-B/32 va ser limitat, amb un Recall@10 de 0.080. El fine-tuning va produir millores substancials. La funció MNRL va demostrar ser la més efectiva, i l’arquitectura CLIP ViT-B/32 va proporcionar un bon equilibri entre rendiment i eficiència, assolint un Recall@5 que va augmentar de 0.055 (baseline) a 0.356 i un Recall@10 de 0.660. El rendiment va variar segons descriptors, sent característiques com la “Pilositat Citoplasmàtica” recuperades amb major fiabilitat. Es va desenvolupar una aplicació web per demostrar aquestes capacitats. L’estudi conclou que els models CLIP modificats posseeixen un potencial significatiu per a la recuperació especialitzada d’imatges mèdiques. El treball futur inclou explorar tècniques avançades de data augmentation, refinar l’etiquetatge de les imatges i obtindre validació clínica.","Esta tesis de máster explora el fine-tuning de modelos Contrastive Language-Image Pre-training (CLIP) para la recuperación de imágenes de células sanguíneas periféricas basades en descripciones morfológicas, con el objetivo de asistir a los hematólogos en los procesos de diagnóstico. El estudio aborda las limitaciones del análisis manual de frotis de sangre periférica, que consume mucho tiempo y es propenso a la variabilidad, aprovechando técnicas de inteligencia artificial de vanguardia. La metodología incluyó la preparación de un dataset de imágenes de linfocitos con descripciones textuales de 12 características morfológicas. Se modificaron modelos CLIP preentrenados (ViT-B/32, ViT-L/14, ViT-B/16) utilizando Cosine Similarity, Contrastive Loss y Multiple Negatives Ranking Loss (MNRL). El rendimiento se evaluó mediante una nueva métrica de recall específica para descriptores. El rendimiento inicial en la modalidad zero-shot del modelo CLIP preentrenado ViT-B/32 fue limitado, con un Recall@10 de 0.080. El finetuning produjo mejoras sustanciales. La función MNRL demostró ser la más efectiva, y la arquitectura CLIP ViT-B/32 proporcionó un buen equilibrio entre rendimiento y eficiencia, logrando un Recall@5 que aumentó de 0.055 (baseline) a 0.356 y un Recall@10 de 0.660. El rendimiento varió según los descriptores, siendo características como la “Pilosidad Citoplasmática” recuperadas con mayor fiabilidad. Se desarrolló una aplicación web para demostrar estas capacidades. El estudio concluye que los modelos CLIP modificados poseen un potencial significativo para la recuperación especializada de imágenes médicas. El trabajo futuro incluye explorar técnicas avanzadas de data augmentation, refinar el etiquetado de las imágenes y obtener validación clínica."],"dc:format.mimetype":["application/pdf"],"dc:identifier.uri":["https://hdl.handle.net/10609/153609"],"dc:language.iso":["eng"],"dc:publisher":["Universitat Oberta de Catalunya (UOC)"],"dc:rights":["CC BY-NC-ND"],"dc:rights.uri":["https://creativecommons.org/licenses/by-nc-nd/4.0/"],"dc:subject":["vision language models","blood cell morphology","medical image retrieval"],"dc:title":["Fine-tuning CLIP models for peripheral blood cell images retrieval based on morphological descriptions"],"dc:type":["info:eu-repo/semantics/masterThesis"]},"updated_at":"2026-07-27T19:06:49Z"}