Automata for non-Markovian RL
Learning task automata from data: passive RPNI inference (HiPO), online refinement with confidence-weighted merging (DARP), and automaton feedback and preferences as a reward signal (RLAF).
PhD Candidate · University of Central Florida
Mahyar Alinejad
I build reinforcement learning agents that learn the structure of long-horizon tasks. They infer automata from experience and use them to plan, transfer and generalize. I come from power systems, and I bring these methods to energy and smart-grid problems.
- Peer-reviewed papers
- 10
- Citations
- 170+
- h-index
- 5
- PhD GPA
- 4.0
About
I am a PhD candidate in Electrical & Computer Engineering at the University of Central Florida, advised by Prof. George Atia in the Data Science Laboratory. My research is in reinforcement learning and neuro-symbolic AI. I study how agents can learn, represent and reuse the temporal structure of tasks whose rewards depend on history (non-Markovian tasks), using deterministic finite automata and reward machines.
My methods include passive and online automaton inference, automaton-based reward shaping and preference feedback, and knowledge distillation for sample-efficient transfer. Recent work adds large language models as a source of symbolic structure. This work has been published at AISTATS, ICASSP, TMLR and NeuS, and was carried out in collaboration with DARPA on transfer learning for autonomous decision-making.
Before ML, I trained as a power systems engineer (MSc, IUST). I worked on optimal scheduling of electric-vehicle charging and discharging in intelligent parking lots, V2G/V2V energy transfer, and battery state-of-charge estimation. That background shapes my current interest in AI for energy systems: I have applied deep RL to real-time energy flexibility in smart building communities and collaborated with Siemens Energy on AI-driven smart grids.
Open to research scientist and ML / RL engineer roles.
Research
Three threads connect my work: learn the structure, use it to learn faster, and apply it where decisions matter.
Neuro-symbolic transfer
Transferring knowledge from abstract models and automata into deep RL agents: bidirectional end-to-end transfer, automaton distillation, gated hybrid distillation (CADENT), and LLM-augmented symbolic reasoning (LANTERN).
AI for power & energy systems
Optimization and learning for smart grids: EV charging/discharging scheduling with V2G and V2V, battery SOC estimation, and deep RL for real-time energy flexibility in grid-connected buildings.
Highlights
- CADENT accepted at AISTATS 2026. Code released.
- Dynamic Automaton Refinement and Planning (DARP) accepted at IEEE ICASSP 2026. Code released.
- Automaton Distillation published in TMLR.
- LANTERN, LLM-augmented neuro-symbolic transfer, released on arXiv.
- Bidirectional transfer from abstract models at NeuS 2025 (PMLR 288). Code released.
- HiPO presented at IEEE ICASSP 2025. Code released.
Publications
-
AISTATS 2026
CADENT: Gated Hybrid Distillation for Sample-Efficient Transfer in Reinforcement Learning
International Conference on Artificial Intelligence and Statistics (AISTATS), 2026 · Paper · Code · Project page
-
ICASSP 2026
Dynamic Automaton Refinement and Planning for Non-Markovian RL
IEEE ICASSP, 2026 · Paper · Code · Project page
-
TMLR 2026
Automaton Distillation: Neuro-Symbolic Transfer Learning for Deep Reinforcement Learning
Transactions on Machine Learning Research, 2026
-
arXiv 2026
LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks
Under review · arXiv:2605.05478
-
Under review
Automaton-Conditioned Successor Features for Sample-Efficient Transfer in Non-Markovian RL
Under review, 2026
-
NeuS 2025
Bidirectional End-to-End Framework for Transfer from Abstract Models in Non-Markovian Reinforcement Learning
International Conference on Neuro-symbolic Systems, PMLR vol. 288, pp. 643–660, 2025 · Paper · Code · Project page
-
ICASSP 2025
Hybrid Offline Passive Grammatical Inference and Online Planning for Non-Markovian Tasks
IEEE ICASSP, 2025 · Paper · Code · Project page
-
arXiv 2025
RLAF: Reinforcement Learning from Automaton Feedback
arXiv:2510.15728, 2025
-
SCS 2025
Real-time Energy Flexibility Optimization of Grid-connected Smart Building Communities with Deep Reinforcement Learning
Sustainable Cities and Society, vol. 119, 106077, 2025
-
Sustainability 2022
A Charge/Discharge Plan for Electric Vehicles in an Intelligent Parking Lot Considering Destructive Random Decisions, and V2G and V2V Energy Transfer Modes
Sustainability, 14(19), 12816, 2022
-
J. Energy Storage 2021
An Optimal Management for Charging and Discharging of Electric Vehicles in an Intelligent Parking Lot Considering Vehicle Owner's Random Behaviors
Journal of Energy Storage, vol. 35, 102245, 2021
-
UPEC 2021
Optimal Scheduling of Electric Vehicles Charging and Discharging in a Smart Parking-Lot
56th International Universities Power Engineering Conference (UPEC), IEEE, 2021
-
ICIAS 2021
An Optimized Adaptive Estimation of State of Charge for Lithium-ion Battery Based on Sliding Mode Observer for Electric Vehicle Application
8th International Conference on Intelligent and Advanced Systems (ICIAS), IEEE, 2021
Complete and up-to-date list on Google Scholar.
Code
Open-source implementations of my papers, with configs, tests and documentation.
HiPO
ICASSP 2025
Passive automaton learning with RPNI and online planning on the product MDP for non-Markovian tasks. Includes QAS and L*-based AGI baselines.
DARP
ICASSP 2026
Online automaton refinement for non-Markovian RL: confidence-weighted RPNI, inconsistency and coverage-gap detection, UCB exploration and Q-value transfer.
CADENT
AISTATS 2026
Gated hybrid distillation for transfer RL: automaton-level and policy-level teacher knowledge combined through a state-action trust gate. Includes AD, PD, Kickstarting baselines and an ablation.
Bidirectional-RL
NeuS 2025
Learn the task automaton (RPNI) and a teacher simulator from offline data, then distil the teacher into Q-learning, DDPG and TD3 students with automaton distillation.
More repositoriescoming soon
Code for automaton distillation and the other papers is being released paper by paper.
Follow on GitHub →Experience
-
Graduate Research Assistant
Data Science Laboratory, University of Central Florida · Orlando, FL
- Developed RLAF, automaton-based reward shaping and feedback for non-Markovian RL tasks with formal temporal constraints.
- Designed DARP, online DFA learning with confidence-weighted RPNI and exploration bonuses for changing task specifications.
- Built CADENT, a gated hybrid distillation framework that improves sample efficiency over RL transfer baselines.
- Developed an end-to-end neuro-symbolic transfer framework that combines DFA inference and automaton distillation.
- Collaborated with DARPA on transfer learning for autonomous decision-making, and with Siemens Energy on AI-driven smart grids.
-
Graduate Teaching Assistant
University of Central Florida · Orlando, FL
- Graduate and undergraduate courses in machine learning, optimization and control systems. Delivered lectures, designed assignments and mentored student projects.
-
Research Assistant
Power Systems Laboratory, Iran University of Science and Technology · Tehran
- Optimal scheduling and control of EV charging/discharging in intelligent parking lots under stochastic owner behavior.
- Energy management and optimization for smart grids in MATLAB and GAMS; power-system analysis in DIgSILENT.
-
Teaching Assistant
Iran University of Science and Technology · Tehran
- Lab sessions and grading for Power Systems Analysis.
Education
-
Ph.D., Electrical & Computer Engineering
University of Central Florida · Advisor: Prof. George Atia · GPA 4.0/4.0
-
M.Sc., Power Systems Engineering
Iran University of Science and Technology · Thesis: Optimal scheduling for EV charging and discharging in smart parking lots
-
B.Sc., Electrical Engineering
Iran University of Science and Technology · Thesis: Evaluation of electrical power under unbalanced conditions
Skills & honors
Machine learning
Programming
Power & energy
Honors
- Top 0.1% in Iran's national university entrance exam; full scholarships for B.Sc. and the NODET program.
- Top 0.1% in the nationwide graduate entrance exam (M.Sc.).
Get in touch
I'm happy to talk about research collaborations and roles in reinforcement learning and applied AI.