(L to R) Preston Firestone, Sasa Misailovic, Gagandeep Singh, Shubham Ugare<\/p>\n
Firestone is a Master of Computer Science<\/a> student in the MCS Chicago program<\/a>. He admits that,\u00a0\u201chonestly, I felt like I had gotten away with something.\u201d<\/p>\n Misailovic says, \u201cPreston attended my CS 591 seminar on programming and AI safety in Chicago back in Spring 2024. He was looking to get research experience, and we figured out a possible idea where he could help my (then) PhD student, Shubham Ugare<\/a>.\u201d<\/p>\n Firestone is lead author of the paper, alongside Misailovic,\u00a0Meta research scientist Ugare (’25 Ph.D. Computer Science) and CS assistant professor Gagandeep Singh<\/a>.<\/p>\n Firestone says of the research team, \u201cThey are allowing me to contribute to science as a whole, or at least to my professional development. My primary goal for the research work, however, was to see whether I’d enjoy it, and to learn what exactly the work consisted of: I’d like to have more information about what it would entail before committing to a doctoral program. And I discovered that I do, in fact, enjoy it!\u201d\u00a0<\/p>\n \u201cPreston and Shubham identified that the problem appears when generating tokens for specific math formulas. Then they identified that the same issues also appear when generating human languages written in non-Latin scripts, including those written in Devanagari (for Indian texts), Cyrillic (for Slavic languages), and others.\u201d\u00a0<\/p>\n He continues, \u201cPreston then decided to study and develop a new theoretical framework to explain that the current abstractions LLM developers use when processing text with LLMs are not sufficient to shelter us from problems with character encoding. In this work, he connects many threads from machine learning, linguistics, programming languages and theoretical computer science communities. He also studied the existing empirical techniques for fixing the problem and devised a way to fix the issue in our system.\u201d<\/p>\n Firestone obtained his undergraduate degree from the University of St. Andrew\u2019s in Scotland, studying philosophy and theology. He says, \u201cAt the end of my bachelor’s, I decided to switch topics from philosophy and theology to computer science.\u201d<\/p>\n Learning that Siebel School of Computing and Data Science in The Grainger College of Engineering at the University of Illinois Urbana-Champaign offers the Illinois Computing Accelerator for Non-Specialists<\/a> (iCAN) program, he decided to enroll. \u201cMy plan always was to complete a master’s, and after iCAN, only the MCS is possible without the special assistance of a sponsoring professor. I chose Chicago because I was already living there.\u201d<\/p>\n Firestone met Misailovic in Chicago.\u00a0 Firestone notes that \u201cthe special characteristic of the Chicago MCS program<\/a> is that the courses are quite small and one thereby has direct and personal access to the professors, while being supported in the background by a large research institution. This combination of liberal-arts-style intimacy with R1-level resources gives excellent opportunities to those who aggressively profit from the access afforded to the professors. Had Sasa not caught me in the office one Thursday after I’d emailed him asking for a recommendation, I wouldn’t be here now. And if it weren’t for the intimate scale of the Chicago program, Sasa might not have known who I was, much less taken a personal interest in me to offer me the position on the project. And had I not made the effort to be in the office to socialize with and encounter professors and students, I wouldn’t have been known to Sasa by face and name.\u00a0\u201c<\/p>\n Upon hearing of Firestone\u2019s interest in pursuing a PhD, Misailovic connected Firestone with Ugare. \u201cI had always planned to continue in academia after the MCS,\u201d Firestone recalls, \u201cso I began asking professors for recommendations for doctoral programs during my last semester in the MCS. Sasa asked me what was on my resume and, realizing it was insufficient to qualify me for a PhD program, assigned me to Shubham’s SynCode project as a software developer. Luckily, I had already taken CS421, so I was prepared to wrangle LR parsers.\u201d<\/p>\n Now, Firestone has a research paper under his belt.\u00a0Misailovic concludes that \u201cWhat is impressive about Preston’s work is that it started as a side project when solving a practical systems problem with structured LLM tools and evolved into a general statement about many current frameworks for running LLMs. This work brings to attention the need to think systematically about how to improve abstractions when developing new LLM constrained generation frameworks.\u201d<\/p>\n Speaking of the MCS program, he says, \u201cOur Chicago program is growing, and it is bringing together students of diverse backgrounds, some from different disciplines (like Preston, who studied philosophy in the past) and many with professional\/industry experience. \u00a0I met other ambitious and creative students like Preston, who are willing to step outside of their comfort zone and make something new and exciting. The Chicago MCS program is helping those students discover new opportunities and skills — maybe even those they didn’t know they were capable of.\u201d<\/p>\n Grainger Engineering Affiliations<\/p>\n\u201cWe have been working on a novel approach for constrained generation as part of our Structured LLM<\/a> initiative, which controls large language models (LLMs) to generate text that conforms to user-defined rules,\u201d Misailovic explains. \u201cIn each step, an LLM generates tokens, which are typically words or several letters \u2014 parts of words; however, for non-Latin alphabets and some math notation, tokens can even be just a part of individual symbols. And as Preston was implementing the system we thought of, he encountered an unexpected problem.\u201d\u00a0<\/p>\n
<\/p>\n
<\/p>\n