Slash AI Costs: Run Open-Source Models for Pennies

Slash AI Costs: Run Open-Source Models for Pennies

Running advanced AI agents like OpenClaw can quickly become expensive. Users often spend tens or even hundreds of dollars monthly on API credits just to send messages. The reason for these high costs lies in the premium models these agents use, such as Claude and GPT. Even when switching to open-source models through services like OpenRouter, costs remain higher than necessary.

Cutting Out the Middleman for Savings

OpenRouter, for example, doesn’t run the AI models itself. Instead, it routes requests to servers with GPUs owned by others. This process adds a small fee on top of the actual computing cost. When you combine the compute cost, the routing fee, and the profit margin of the hardware operator, the expenses add up quickly, especially with frequent AI agent use.

The solution is to bypass these intermediaries. HPC AI offers a service that directly connects users to powerful open-source AI models running on their own GPU hardware. This means no routing fees and no platform markups. HPC AI owns and operates its GPU clusters, featuring advanced hardware like B200s, H200s, and B300s. They are also the creators of Colossal AI, a major open-source AI training framework available on GitHub.

Accessing Frontier Models Affordably

HPC AI’s new API service provides direct access to leading open-source models. Currently, they offer Minimax M2.5 and Kimmy K2.5. These are considered frontier models, achieving scores above 80% on the S-Bench benchmark. They also boast impressive multimodal context capabilities.

Comparing costs, using HPC AI can lead to significant savings. For Kimmy K2.5, users can expect to save between 65% and 70% compared to other providers. For Minimax, the savings are around 53%. These percentages translate into substantial reductions when used regularly.

How to Get Started with HPC AI

Setting up involves a few straightforward steps. First, visit the HPC AI website. You can choose to start a free trial, which guides you through signing up and accessing the model console. Alternatively, you can scroll down to view the available AI models and then follow a simple three-step process to get started.

To begin, click the “Get Started” button. You can log in using a Google account. Once logged in, navigate to the “Console” and then the “Models Console.” Here, you will see the available models. The next step is to go to “API Key” and create a new key. Name it something like “OpenClaw” and click “Create.” Copy this generated API key and save it in a secure document.

Setting Up a Virtual Private Server (VPS)

To run OpenClaw efficiently, it’s recommended to use a Virtual Private Server (VPS) rather than your personal computer. Services like Hostinger offer KVM VPS plans. When setting up, choose a server location and select Ubuntu 2.400 as the operating system. Generating a password and saving it is crucial. Enabling the Docker manager is also advised for easier future management.

While the goal is cost reduction, a VPS typically costs around $8 per month. This is significantly cheaper than running premium AI models directly. Choose a one-month plan to start and complete the payment. Once the VPS is ready, you will access it via a terminal.

Installing and Configuring OpenClaw

After your VPS boots up, open the terminal. First, upgrade the server using the provided commands. This process usually takes a couple of minutes. Next, install Node.js version 22. Verify the installation by checking the Node.js version; it should display 22.22.0.

With Node.js installed, you can now install OpenClaw. Run the installation command and wait for it to complete. After installation, verify the OpenClaw version. Then, install the OpenClaw agent by running the “install demon” command. This process will guide you through a setup menu.

Configuring OpenClaw for Free Access

During the OpenClaw setup, you’ll encounter several configuration options. For the security warning, use the arrow keys to select “Yes.” Choose “Quick Start” for the setup type. When asked about the model/provider, scroll down and select “Skip for now.” For other provider options, keep the current default settings.

For search engines, Duck.go is a good free option that doesn’t require an API key. When prompted to configure skills or enable hooks, you can choose to skip these for now and set them up later. Once the installation is complete, you will be prompted on how to hash your bot; this can also be done later.

Connecting to HPC AI and Testing

The crucial step is to switch the model provider to use HPC AI. You will need to replace a placeholder in the OpenClaw configuration with your HPC AI API key. If you’re unsure, you can ask an AI like ChatGPT to help format the key correctly. Copy your generated API key from HPC AI and paste it into the appropriate field in the OpenClaw configuration.

After updating the API key, restart the OpenClaw agent. Then, open the chat interface again. You might need to restart the TUI if your session closed. To confirm the connection, run the “openclaw devices list” command. You should see your devices listed, possibly requiring approval.

Approve the pending request using the provided request ID. Once approved, restart the agent again and launch the TUI. You can then type “hello” to test the connection. If successful, the AI agent will respond, indicating it’s using the HPC Minimax 2.5 model via the HPC AI API.

Switching Models and Cost Effectiveness

You can switch between available models like Kimmy and Minimax by typing “/model” in the chat and using the arrow keys to select your preferred model. For instance, switching to Kimmy allows you to interact with it directly.

The cost savings are remarkable. After using OpenClaw with HPC AI for several hours, the cost was only a few cents. Even with multiple interactions using different models like Kimmy, the expense remained negligible, often less than a single cent. This makes running advanced AI agents incredibly cost-effective.

Why This Matters

This development significantly lowers the barrier to entry for using powerful AI agents. Previously, the high cost of API calls made extensive use prohibitive for many individuals and small businesses. By providing direct access to frontier open-source models on affordable infrastructure, HPC AI enables users to experiment with and deploy AI solutions without worrying about escalating costs. This democratization of AI tools can foster innovation and allow for more widespread adoption of AI in various applications, from personal productivity to complex research tasks. The ability to switch between models also offers flexibility, allowing users to choose the best model for specific tasks based on performance and cost.

Future Outlook

HPC AI plans to offer more models in the future, all designed to be highly cost-effective. This trend suggests a move towards more accessible and affordable AI services, making it unnecessary to rely on more expensive alternatives.

Source: How To Run Openclaw For Free (YouTube)