{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/130700"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/130700","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Expresso-AI : a framework for explainable video based deep learning models through gestures and expressions","abstract":"We have developed a framework for Analyzing Facial Videos and applying it to Automatic Depression Detection. We also developed a video based models We have developed a framework to analyze the decisions of Deep Neural Networks trained on facial videos. We test this framework on Automatic Depression Detection. We first train Deep Convolutional Neural Networks (DCNN) pre-trained on Action Recognition datasets and fine-tune on the facial videos. We interpret the model's saliency maps by analyzing face regions and temporal expression semantics. Our framework generates both visual and quantitative explanations on the model's decision. Simultaneously, our video based modeling has improved previous single-face benchmarks of visual Automatic Depression Detection (ADD). We conclude successfully that we have developed the ability to generate hypotheses from a facial model's decisions, and improved Automatic Depression Detection's predictive performance.","abstract_html":"We have developed a framework for Analyzing Facial Videos and applying it to Automatic Depression Detection. We also developed a video based models We have developed a framework to analyze the decisions of Deep Neural Networks trained on facial videos. We test this framework on Automatic Depression Detection. We first train Deep Convolutional Neural Networks (DCNN) pre-trained on Action Recognition datasets and fine-tune on the facial videos. We interpret the model's saliency maps by analyzing face regions and temporal expression semantics. Our framework generates both visual and quantitative explanations on the model's decision. Simultaneously, our video based modeling has improved previous single-face benchmarks of visual Automatic Depression Detection (ADD). We conclude successfully that we have developed the ability to generate hypotheses from a facial model's decisions, and improved Automatic Depression Detection's predictive performance.","abstract_has_math":false,"creators":["Moreno, Felipe(Felipe I.)"],"institution":"Massachusetts Institute of Technology","degree_name":"Master","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","school":null,"contributors":[],"advisors":["Cynthia Breazeal."],"committee_chairs":[],"committee_members":[],"year":2021,"date_issued":"2021","date_published":"2021","updated_at":"2026-07-22T22:20:52Z","subjects":["Electrical Engineering and Computer Science."],"languages":["eng"],"rights":["MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided."],"rights_urls":["http://dspace.mit.edu/handle/1721.1/7582"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/130700","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Cynthia Breazeal."]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","EECS"]},{"key":"dc:contributor.other","label":"Dc Contributor Other","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science."]},{"key":"dc:creator","label":"Author","values":["Moreno, Felipe(Felipe I.)"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2021-05-24T19:52:20Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2021-05-24T19:52:20Z"]},{"key":"dc:date.issued","label":"Date","values":["2021"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Electrical Engineering and Computer Science."]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided."]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://dspace.mit.edu/handle/1721.1/7582"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/130700"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, February, 2021","Cataloged from the official PDF of thesis.","Includes bibliographical references (pages 95-102)."]},{"key":"dc:description.abstract","label":"Abstract","values":["We have developed a framework for Analyzing Facial Videos and applying it to Automatic Depression Detection. We also developed a video based models We have developed a framework to analyze the decisions of Deep Neural Networks trained on facial videos. We test this framework on Automatic Depression Detection. We first train Deep Convolutional Neural Networks (DCNN) pre-trained on Action Recognition datasets and fine-tune on the facial videos. We interpret the model's saliency maps by analyzing face regions and temporal expression semantics. Our framework generates both visual and quantitative explanations on the model's decision. Simultaneously, our video based modeling has improved previous single-face benchmarks of visual Automatic Depression Detection (ADD). We conclude successfully that we have developed the ability to generate hypotheses from a facial model's decisions, and improved Automatic Depression Detection's predictive performance."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["M. Eng."]},{"key":"dc:title","label":"Title","values":["Expresso-AI : a framework for explainable video based deep learning models through gestures and expressions"]}]}],"canonical_facts":{"dc:contributor.advisor":["Cynthia Breazeal."],"dc:contributor.department":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","EECS"],"dc:contributor.other":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science."],"dc:creator":["Moreno, Felipe(Felipe I.)"],"dc:date.accessioned":["2021-05-24T19:52:20Z"],"dc:date.available":["2021-05-24T19:52:20Z"],"dc:date.issued":["2021"],"dc:description":["Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, February, 2021","Cataloged from the official PDF of thesis.","Includes bibliographical references (pages 95-102)."],"dc:description.abstract":["We have developed a framework for Analyzing Facial Videos and applying it to Automatic Depression Detection. We also developed a video based models We have developed a framework to analyze the decisions of Deep Neural Networks trained on facial videos. We test this framework on Automatic Depression Detection. We first train Deep Convolutional Neural Networks (DCNN) pre-trained on Action Recognition datasets and fine-tune on the facial videos. We interpret the model's saliency maps by analyzing face regions and temporal expression semantics. Our framework generates both visual and quantitative explanations on the model's decision. Simultaneously, our video based modeling has improved previous single-face benchmarks of visual Automatic Depression Detection (ADD). We conclude successfully that we have developed the ability to generate hypotheses from a facial model's decisions, and improved Automatic Depression Detection's predictive performance."],"dc:description.degree":["M. Eng."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/130700"],"dc:language.iso":["eng"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided."],"dc:rights.uri":["http://dspace.mit.edu/handle/1721.1/7582"],"dc:subject":["Electrical Engineering and Computer Science."],"dc:title":["Expresso-AI : a framework for explainable video based deep learning models through gestures and expressions"],"dc:type":["Thesis"],"thesis:degree_name":["Master"]},"updated_at":"2026-07-22T22:20:52Z"}