LMQL

🔊 Recorded at PyCon DE & PyData 2025, April 23, 2025 https://2025.pycon.de/program/DQTMJB/ 🎓 This technical deep-dive explores how LMQL enables structured generation with open-source LLMs, from basic prompting to building evaluated RAG systems. Speakers: Christiaan Swart Description: In this presentation, Christiaan Swart explores advanced techniques for structured generation with open source Large Language Models (LLMs) using LMQL, a domain-specific language that enables interweaving of prompts and code. The talk demonstrates how LMQL's capabilities can enhance LLM outputs through logit biasing, type constraints, and control flow mechanisms. Swart examines practical applications including structured object generation, sentiment analysis, and retrieval-augmented generation (RAG) systems. Using a Wikipedia-based RAG implementation as a case study, he evaluates system performance through metrics including factual correctness, context recall, and faithfulness. The presentation highlights both the potential and limitations of small open-source LLMs, particularly in reasoning tasks. While LMQL itself is no longer actively maintained, the underlying concepts of logit processing and structured generation remain valuable, with alternative tools like Guidance and Outlines offering similar functionality. The key takeaway emphasizes the importance of workflow-based approaches over agent-based systems for production environments, with structured generation techniques providing more controlled and measurable outcomes. ⭐️ About PyCon DE & PyData: The PyCon DE & PyData conference unite the Python, AI, and data science communities, offering a unique platform for collaboration and innovation. The PyCon DE & PyData 2025 conference, provided an exceptional experience, fostering deeper connections within the Python community while showcasing advancements in AI and data science. Attendees enjoyed a diverse and engaging program, solidifying the event as a highlight for Python and AI enthusiasts nationwide. Follow us: • LinkedIn: https://www.linkedin.com/company/28908640/ • X: https://www.x.com/pyconde Links: • Conference website: http://pycon.de • Other sessions: https://2025.pycon.de/talks/ The conference is organized by • Python Softwareverband e.V.: http://pysv.org • NumFOCUS Inc.: http://numfocus.org • Pioneers Hub gemeinnützige GmbH: http://pioneershub.org If you enjoyed this session, please like, comment, and subscribe to our channel for more insightful talks and discussions. Share this video with your network to spread the knowledge! Hashtags: #Python #PyConDE #PyData #OpenSource #AI #DataScience #MachineLearning #SoftwareDevelopment #LLMs #Community Acknowledgements: Special thanks to all the volunteers and sponsors who made this event possible. About: Python Softwareverband e.V.: PySV is a non-profit that promotes the use and development of Python in Germany through events, education, and advocacy, fostering an open Python community. NumFOCUS Inc. supports open-source scientific computing by providing financial and logistical support to key projects like NumPy and Jupyter, promoting sustainable development and collaboration. Pioneers Hub gemeinnützige GmbH: is a non-profit fostering innovation in AI and tech by connecting experts and promoting knowledge exchange through events and collaborative initiatives. www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

Speakers: Marc Fisher: https://www.sri.inf.ethz.ch/people/marc Luca Beurer-Kellner (ETH Zürich): A Ph.D. student in the Department of Computer Science at ETH Zurich, under the supervision of Professor Martin Vechev. He holds a Bachelor's degree in Computer Science from Humboldt University of Berlin and a Master's degree from ETH Zurich. Luca's research focuses on the intersection of programming languages and machine learning. Most recently, he worked on controlled and constrained text generation with Large Language Models. He is the lead author of the LMQL (Language Model Query Language) paper in 2023, presenting an innovative query language for LLMs, enabling precise control over generated text. Abstract: Large language models have recently demonstrated outstanding performance on a wide range of tasks. However, to obtain state-of-the-art performance or adapt language models for specific tasks, complex task- and model-specific programs have to be implemented, which may still require ad-hoc interaction or laborious result parsing. To address this we implement LMQL (short for Language Model Query Language). LMQL is a programming language for language model interaction and generalizes natural language prompting, making it more expressive while remaining accessible. For this, LMQL builds on top of Python, allowing users to express natural language prompts as well as control flow, allowing the user to steer the LLM's reasoning process. In LMQL, users can specify high-level, logical constraints over the language model output. These constraints are then automatically converted into token-level prediction masks, which can be enforced eagerly during text generation. This allows to enforce constraints strictly, making it impossible for the model to generate content that does not satisfy the requirements. Try it now on https://lmql.ai