This graduate-level course integrates hands-on training with research-driven exploration to prepare
students as future
experts in software reverse engineering and malware analysis. Students will gain practical expertise in
uncovering
malicious behavior on compromised systems and analyzing the techniques adversaries employ to evade
detection. The course
features extensive use of industry-standard tools such as Wireshark, Ghidra, IDA Pro, Dshell, Cuckoo
Sandbox,
Volatility, Metasploit Framework, Armitage, and Google Rapid Response. In addition to classical methods,
students will
engage with emerging approaches in automated reverse engineering, including large language model
(LLM)-assisted
techniques. The course also introduces cutting-edge strategies for red teaming Artificial Intelligence
(AI) systems,
focusing on black-box testing and adversarial evaluation.
Finally, the course emphasizes research engagement by surveying state-of-the-art developments in malware
analysis and
machine learning for intelligent malware classification and detection. Students will critically analyze
current
literature, identify open challenges, and design solutions that advance the field of reverse engineering
and
cybersecurity.
SRE involves deconstructing software to understand its components, functionality, and behavior without access to the original source code. As cyber threats become more sophisticated, SRE provides the tools needed to effectively dissect and mitigate cyber threats, ensuring the integrity and security of software systems across various industries. SRE is essential for identifying security vulnerabilities, understanding malware, ensuring compliance, and recovering lost or legacy code.
The rapid advancements in artificial intelligence (AI) and large language models (LLMs) have revolutionized many fields, including software reverse engineering. Automated reverse engineering powered by AI and LLMs can significantly enhance the efficiency and accuracy of analyzing complex software. These technologies can automate repetitive tasks, identify patterns, and provide insights that might be missed by human analysts. Now is the perfect time to study these advanced techniques because the tools and resources are more accessible than ever, and the demand for professionals skilled in automated reverse engineering is on the rise. As AI and LLMs continue to evolve, their applications in reverse engineering will become even more integral, making early adopters of these technologies invaluable assets to their organizations.
AI red teaming is an emerging field that focuses on testing and evaluating the robustness of AI systems. As AI becomes increasingly integrated into critical infrastructure and decision-making processes, ensuring its security and reliability is paramount. AI red teaming involves simulating attacks on AI systems to identify weaknesses and improve their defenses. This is crucial now more than ever because AI systems are being deployed in high-stakes environments where failures or vulnerabilities can have significant consequences. By studying AI red teaming, students can learn how to anticipate and mitigate potential threats to AI systems, making them more resilient and trustworthy.
Malware is continuously evolving, with new variants and sophisticated attack techniques emerging regularly. Staying ahead of these threats requires a deep understanding of the latest research and developments in malware analysis. By studying state-of-the-art malware analysis research, students can learn about the newest tools, techniques, and methodologies used to detect, analyze, and combat malware. This knowledge is critical for developing effective defense strategies and keeping up with the fast-paced nature of cyber threats. Engaging with cutting-edge research ensures that students are well-prepared to tackle current and future challenges in the field of cybersecurity.
Students are expected to have the following background:
Academic dishonesty is prohibited and is considered a violation of the UTEP Handbook of Operating Procedures (HOOP). It includes, but is not limited to, cheating, plagiarism, and collusion.
Any act of academic dishonesty attempted by a UTEP student is unacceptable and will not be tolerated. All suspected violations of academic integrity at The University of Texas at El Paso must be reported to the Office of Community Standards for possible disciplinary action. To learn more, please visit HOOP: Student Conduct and Discipline.
Permissive but strict. If unsure, please ask the course staff!
Some AI technologies or automated tools—particularly generative AI such as ChatGPT, Gemini, or DALL·E—can be useful during the early brainstorming stages of an activity, and you are welcome to explore them for that purpose. However:
Policy: You are not allowed to submit any AI-generated work in this course as your own. If you use information or materials created by AI technology, you must cite them like any other source and disclose the tool(s) used (including a link to your AI tool’s session or search history when feasible). Any direct use of AI-generated materials submitted as your own work will be treated as plagiarism and reported to the Office of Community Standards.
We welcome auditing requests from UTEP students and staff. As an auditor, you will have access to all course lectures but will not receive grades for labs, homework, or final projects. Due to limited resources, we are unable to provide feedback on assignments or projects for auditors. If you are interested in auditing this course, please contact the Computer Science department to make the necessary arrangements.
Please note that external requests for auditing will not be considered, as the course is conducted in-person on campus.
All course materials, including lecture slides, detailed notes, assignments, and final project instructions, will be made publicly available on the course website for your reference.
The course does not require any textbook.
Relevant books to the course (Optional):
