Why Our University Runs Its Own LLMs and Why Yours Should Too

Marwa Elsayed, Research Assistant at IIDS 

Most people encounter generative artificial intelligence through an application. They upload a document, ask a question, and receive an answer without having to think much about where the model is running or who controls the infrastructure behind it. But for a university, those questions matter. They shape what happens to institutional data and which models students, staff, faculty, administrators, and researchers can use. They also determine how much control the institution keeps over the systems it depends on. 

If you have been following this blog, you have learned about Vandalizer: an open-source AI document-processing platform built for research administration work. You have seen how it can turn tedious tasks into reusable workflow steps and help staff move from one-off document review to a more structured and repeatable process. The AI tasks in those workflows rely on large language models, or LLMs. Vandalizer connects to different model endpoints, including ones hosted on institutional infrastructure. The more I learned about that layer, the more I kept returning to a more fundamental question: where does the model behind it actually run? 

In our setup, a request goes through MindRouter, open-source software developed at the University of Idaho to route AI work across computing infrastructure. Vandalizer may be the tool a research administrator sees on the screen, but MindRouter is the layer underneath that decides which server handles the model request. It checks access, manages demand, and sends requests to servers equipped with graphics processing units, or GPUs, that run the selected model. I think of it like a campus traffic controller for generative AI use. 

MindRouter is available under the Apache 2.0 license, so other institutions can study the code, adapt it, and run their own version. The models have their own licenses. An open-weight model makes its trained weights available, but that does not necessarily make the whole model open source. The University of Idaho deployment has been in production since early 2026 and runs across a university-managed GPU cluster currently serving 63 open-weight models that had processed billions of tokens, the small chunks of text that these language models read and write. 

Because the infrastructure is university-managed, access can be governed through institutional authentication, quotas, audit logging, and monitoring. For research administration, the documents we work with often contain sensitive institutional data and proposal or sponsor information. MindRouter gives us an option for work that needs to stay within university-controlled infrastructure, while leaving the university responsible for who can access those records and how long they are kept. 

The output still has to be evaluated, and the use case still has to be appropriate. University guidance, updated in August 2026, lists VandalChat and MindRouter hosted by Research Computing and Data Services (RCDS) as approved for low-risk data. Moderate-risk use has interim approval only when accessed from an OIT-managed endpoint or server. Running a model locally does not, by itself, make every document appropriate to upload. 

There is also a cost difference. Commercial APIs often charge by usage, so repeated model calls can add up. For requests handled by its local models, the university can avoid an external provider’s per-token inference charge. The university still pays for hardware, power, cooling, maintenance, and staff. Those costs do not disappear because the software is open source. Whether local hosting costs less overall depends on how heavily the system is used and the models and capacity needed. 

I see that cost difference most clearly in my own research. I spend a lot of time evaluating whether LLMs follow a compliance framework as intended, which means running the same calls repeatedly and comparing the results. A model following a rule once does not tell me whether it will do so consistently across repeated runs or different settings. With MindRouter, I can run a script to make those comparisons across the available models without paying an external inference charge for every call. I still have to work within the system’s capacity and quotas, but repeated evaluation becomes much easier to plan. 

I also see the value as a student. VandalChat is a university-supported chatbot powered by MindRouter. Students can sign in with a Vandal account from the university network or through the VPN. As a student, I now have a chatbot I can use for everyday academic work without needing to start with a separate paid subscription. I use it to help me work through readings and think of related areas that I may want to explore further. When I am developing a paper idea, I can use it to look for blind spots in my thinking. Before a presentation, I can ask VandalChat to come up with questions I might be asked. I also use VandalChat for a final proofreading pass. I interact with it much like I would any other chatbot, but it runs on university-owned infrastructure. That makes the idea of running our own AI much less abstract. 

One feature I have come to appreciate is how MindRouter shares limited GPU capacity. The university can set token allowances and scheduling priorities for different groups while still making unused capacity available when demand is low. These are configurable features, so access can reflect institutional priorities instead of simply being first come, first served. 

Operating our own cluster makes its energy use something the university can examine directly. MindRouter records power use, giving the team information to work with when assessing demand and efficiency. Accounting for the whole service also means including cooling and other facility overhead, rather than looking only at the GPUs. This gives the university a basis for asking how much electricity the service uses and where improvements might be possible. Local hosting gives the institution a more direct role in measuring its use and deciding what to do about it. 

MindRouter was developed by Luke Sheneman in RCDS, within the Institute for Interdisciplinary Data Sciences. Making the software open source means other institutions do not have to start from a blank page if they want to manage local or institutionally controlled AI. They can adapt it to their own infrastructure and policies, provided they have the hardware and technical expertise to operate it. Each institution still has to decide which workloads make sense to host and how it will support them over time. 

Where the model actually runs is part of how a university chooses to use AI, and MindRouter gives the University of Idaho a way to make more of those choices for itself. 

AI4RA is about making what we learn useful beyond one office or one campus. If you want to learn more, you can explore the MindRouter code and documentation, look at the Vandalizer open-source project, or try Vandalizer.