Evaluating Large Language Models as Circuit Design Assistants

Large language models (LLMs) have exploded in capability in recent years. Previous attempts at AI systems for circuit design have had limited proficiency and been restricted in problem scope. LLMs, with their breadth of knowledge and reasoning ability, are a promising technology for a much more general-purpose circuit design assistant. We developed a dataset of electrical engineering problems and solutions with which to test an LLM-based system, since no such publicly available dataset exists to our knowledge; unmodified GPT-4 was able to solve 42% of the problems. We did a preliminary comparison of several knowledge bases to use for RAG knowledge injection, finding that a small, curated set of resources performed better than a larger, less-focused set of resources, though there were confounding factors which may have skewed the result. While this work is a start, significant future work is needed to continue developing an LLM-based circuit design assistant.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Cox, Matthew J.

Advisor dc:contributor.advisor

• Han, Ruonan

Statement dc:rights

• Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
• Copyright retained by author(s)

Licence dc:rights.uri

• https://creativecommons.org/licenses/by-nc-nd/4.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/164861

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/164861