2026 ASEE Annual Conference & Exposition

The recent advancement in generative artificial intelligence (AI) has transformed the way students and educators approach education by personalizing learning experiences, creating dynamic content, and offering efficient administrative support to educators. Recent studies have examined the integration of AI tools in Engineering education using various lenses like student surveys about AI tools and perceptions, classroom module implementations to teach responsible use of AI, analysis of AI-assisted writing in technical course, etc. While recently release AI models have shown performance improvement. OpenAI claimed ChatGPT 5 can provide PhD-level expertise in any topic. Elon Musk stated Grok performs “better than PhD level in everything" there is increasing concern among engineering educators about the reliability and effectiveness of these AI tools in solving technical problems, this area have received limited empirical investigation. While previous research has surveyed student perceptions of AI tools like ChatGPT across electrical engineering courses and others have studied student experiences using AI tools for writing assignments through comparative analysis, direct assessment of AI problem-solving accuracy in foundational engineering courses remains underexplored. Systematic reviews of AI in K-12 computer science education have focused primarily on programming platforms, automated grading systems and learning analytics, while engineering education reviews have emphasized implementation strategies and student engagement metrics rather than technical problem-solving accuracy. This study addresses this gap by directly evaluating AI performance on standard electrical circuit problems, simulating how students might use these tools for homework assistance. We investigated the reliability of two state-of-the-art AI models—Google’s Gemini 2.5 Pro and OpenAI’s gpt-5 in solving electrical circuit problems from Electric Circuits, 12th Edition by Nilsson and Riedel. A total of 360 questions were systematically selected from the 18 chapters of the book with 20 questions per chapter. The questions consisted of text-based questions and image-based questions containing circuit diagrams and plots. The questions were fed to Google’s Gemini 2.5 Pro via the web chat interface and that of OpenAI’s gpt-5 was done via API calls and the response saved for later evaluation.

Each AI response was evaluated across 6 categories:

​​Ability to read and understand circuit diagrams presented in images​
​​Ability to break down problem-solving procedures into clear, logical​ ​steps​
Ability to select appropriate formulas and conduct calculations
Ability to analyze circuit configuration and conduct calculations
​​Accuracy in the use of equations and numerical calculations​
Ability to read and understand plots presented in images

The responses were rated based on 5-point scale:
0 = N/A, 1 = Poor, 2 = Fair, ​3 = Good, 4 = Excellent​

Results show that both models perform strongly in analytical and computational tasks like formulas, equations, procedures, but struggle with visual interpretation in reading circuit diagrams. OpenAI’s GPT-5 outperformed Google’s Gemini-2.5-Pro in topics such as Circuit Elements (Ohm’s Law, Kirchhoff’s Laws, etc.) with 3.17 average performance against 2.72, Response of First-Order RL and RC Circuits (3.28 vs 3.10) and Simple Resistive Circuits (3.16 vs 3.09). On the other hand, Gemini-2.5-Pro outperformed GPT-5 in topics such as The Operational Amplifier (3.18 vs 3.11), The Fourier Series (2.57 vs 2.47). They both struggle with topics such as Introduction to the Laplace Transform, Balanced Three-Phase Circuits, and The Fourier Transform. The critical findings reveal the major weaknesses requiring attention as to how these tools are used and their reliable capabilities. These findings suggest that while AI models show promise as educational aid in circuit theory, their current limitations in visual comprehension and contextual analysis may affect their reliability in unsupervised learning environments.

Most existing studies have relied on student surveys or examined how to implement AI in the classroom. This work is different because it provides objective data on how well AI solves the kind of problems students regularly encounter in their coursework. The evaluation framework developed here can help educators understand where AI tools are reliable and where they fall short, which has practical implications for how these tools should be integrated into courses, labs, and online learning systems. The results also make it clear that certain types of problems still need human guidance, particularly those involving visual interpretation and circuit analysis.