In July 2023, Meta released Llama 2, an open-source large language model that represented a significant shift in how AI gets built and distributed. This wasn’t a marginal improvement to an existing tool; it was Meta betting that open-source AI could compete with OpenAI’s closed-ecosystem ChatGPT and GPT-4. Microsoft partnered on distribution, major tech companies started licensing it, and suddenly the conversation about AI shifted from “we need access to OpenAI’s API” to “we can run AI models ourselves, locally, with our own data, under our own control.”

This moment matters for your IT career, and not in the way you might expect. You don’t need to learn machine learning. You don’t need to understand how transformers work or spend six months studying deep learning. What you need to understand is what happens when organizations can run powerful AI models on their own infrastructure. That’s where the IT career opportunities actually are.

What Llama 2 Changed

The key thing about Llama 2 was that it was genuinely competitive with proprietary models, and it was open-source. You could download it, run it on your own infrastructure, modify it, fine-tune it, and use it however you wanted. That’s fundamentally different from ChatGPT, which is available only through OpenAI’s API. If you use ChatGPT, your data goes to OpenAI. Your queries are logged. There’s no privacy, no control, no ability to customize the model for your specific use case.

Llama 2 changed that calculation. Organizations could now ask a different question: would it be better to run this model ourselves? That opens up an entirely different set of possibilities. A healthcare organization could fine-tune a medical-specific model on its own data without sending patient information to a third party. A financial services company could customize a model to understand its specific terminology and regulatory requirements. A government agency could run AI without worrying about data leaving their infrastructure.

The trade-off is that running AI models yourself is computationally expensive. A large language model like Llama 2 needs significant GPU resources, memory, networking capacity, and the expertise to manage all of that. That’s where the IT opportunity lies.

The Infrastructure Demands Are Enormous

Let’s be concrete about what it takes to run a modern language model. Llama 2 comes in different sizes. The smallest version has 7 billion parameters, which is still a lot of computation. The larger versions have 70 billion parameters. A parameter is essentially a number that the model learned during training, and each parameter requires memory and computation.

To run even the smallest version of Llama 2 competently, you’re looking at GPU infrastructure. A single modern GPU, like an NVIDIA H100, costs around $40,000. Most organizations running models will run multiple GPUs, either in the same server or distributed across multiple servers. You need networking infrastructure that can move training data and model data quickly. You need storage that can handle the model files, the training data, and the outputs. You need cooling infrastructure because GPUs generate enormous amounts of heat. You need monitoring and observability to understand what the model is doing and whether it’s running efficiently.

And you need people who understand how to deploy, manage, and monitor all of this. That’s not generic IT infrastructure knowledge. That’s a specific specialty that sits at the intersection of infrastructure, DevOps, and machine learning operations.

MLOps as a Career Path

MLOps, which stands for machine learning operations, is becoming a category of engineering that organizations are actively hiring for. It’s basically the operations and infrastructure side of machine learning. If you understand Kubernetes, if you understand how to manage containerized applications at scale, if you understand networking and storage and monitoring, you already have most of the skills you need for MLOps work. You’re just applying those skills to machine learning infrastructure instead of traditional application infrastructure.

The difference is that machine learning infrastructure has some specific requirements. Models need to be trained, which is computationally expensive. Models need to be deployed and served, which requires the ability to handle variable load. Models need to be monitored for performance drift, which means tracking not just whether the service is up, but whether the model’s predictions are still accurate. Models need to be versioned and rolled back if something goes wrong. All of this is operations work, but with machine learning specific concerns.

Organizations that are running Llama 2 or similar models need people who can manage that infrastructure. They need people who understand that deploying a model is different from deploying a traditional application. They need people who can help them think through questions like: how much GPU do we actually need? How do we version models? How do we route traffic between different versions? How do we monitor inference latency? How do we handle model updates without downtime?

If you have a strong infrastructure background, this is actually your advantage. You understand infrastructure deeply. You don’t understand machine learning, and you don’t need to. What you need is to understand how machine learning workloads look different from traditional workloads, and how to build infrastructure that supports them. That’s learnable. That’s a 6 to 12 month deep learning path, not a career change.

Deployment and Monitoring at Scale

One specific area where the need is urgent is model deployment and monitoring. Organizations are asking questions like: how do we put a Llama 2 model into production in a way that’s reliable and scalable? What infrastructure do we need? How do we monitor inference quality? How do we handle updates?

There are tools emerging to handle this. Hugging Face has a platform for managing and deploying models. Replicate lets you deploy models with a simple API. Ray is a framework for distributed computing that can handle model serving. But these tools are new, and most organizations don’t fully understand how to use them yet. That’s an opportunity. If you can become expert in deploying and monitoring language models at scale, you’re in a space where organizations are willing to pay a lot for expertise.

Why IT Background Is Actually Your Strength Here

There’s a broader point worth making. A lot of IT professionals have imposter syndrome when they think about AI. They assume that working with AI requires machine learning expertise, which they don’t have. That’s wrong. Machine learning expertise is about building and training models. That’s what data scientists and machine learning engineers do. What organizations actually need right now is people who can operationalize those models. That’s infrastructure work. That’s IT work. That’s your work.

The distinction between knowing how to build models and knowing how to deploy and maintain them is critical. A data scientist can train a model that works great in a notebook. An MLOps engineer has to make that model work in production, handle edge cases, manage computational resources, monitor performance, and support model updates. Those are completely different skill sets, and the MLOps side is where the infrastructure people come in.

The Privacy and Compliance Angle

There’s another dimension to this that’s important for IT careers. Running models locally, on your own infrastructure, has significant privacy and compliance advantages. If you’re subject to HIPAA regulations, you can’t send patient data to OpenAI. If you’re handling financial data subject to SEC requirements, you need full audit trails and data control. If you’re in Europe operating under GDPR, you have strict requirements about where data can be processed.

Organizations that care about these constraints will choose to run models themselves, which means they need infrastructure people who understand compliance requirements and can build infrastructure that meets them. That’s you. That’s your domain. Understanding how to run machine learning workloads in compliance with regulatory requirements is a specialty that organizations will pay for.

The Skill Progression

If you’re an IT professional thinking about getting involved in this space, here’s a practical path: First, understand the current infrastructure you manage and how it would be different with machine learning workloads. Second, learn the basics of how large language models work and what their infrastructure requirements are. You don’t need to learn how to train them, just what they need to run. Third, pick a specific area to go deep: container orchestration for ML workloads, monitoring and observability for models, deployment automation for language models, or compliance and security for running AI locally.

From there, you’re positioned to take on work that’s actually valuable to organizations. You’re not trying to become a data scientist. You’re becoming an infrastructure person who understands machine learning specific requirements. That’s a much more straightforward path for someone with IT background.

The Bigger Picture

The release of Llama 2 fundamentally shifted the AI world from “you use an external API” to “you can run this yourself.” That shift creates infrastructure opportunities. Organizations that were comfortable relying on OpenAI’s API are now asking whether they should run models themselves. And if they decide to do it, they need infrastructure people who can help them figure out how.

This is similar to what happened with the IT skills shortage more broadly. There’s enormous demand for specific skills, and the supply doesn’t match the demand. But in this case, the demand is specifically for infrastructure people who are willing to learn machine learning specific operations. That’s a narrower market than general IT, but it’s also a market where people are willing to pay significant premiums for expertise.

The window for getting in early is open. In a year, everyone will be talking about MLOps and deploying language models. But right now, there’s still time to be early to the game, to pick up skills while they’re still relatively rare, and to position yourself as someone who understands this emerging space. For AI and IT professionals, the opportunity here is real. You don’t need to learn machine learning. You just need to understand infrastructure and be willing to learn how it applies to AI workloads.

That’s your edge. That’s where your career is heading if you want to be where the growth is.