Tether released a fine-tuning framework for Microsoft’s BitNet b1.58 LLM that works on any GPUs and consumer-grade handheld devices. For the first time, the model can be fine-tuned efficiently across multiple desktop and edge-device GPU architectures.
Techno-cynics’ argument against the AI revolution is its high infrastructure requirements. Electricity for training advanced models, water for cooling data centers, and the pressure on minerals for producing semiconductors and high-end processor chips. The popular “burning a bush with a single prompt” may be exaggerated, but not completely out of place.
For reference, running inferences on a half-precision (FP16) 1 Billion parameter Gemma 3 model (Gemma 3 1B) requires at least 2.2GB VRAM, obtainable in a $700 consumer-grade 16GB RAM HP Victus or Lenovo LOQ laptop with an 8 GB VRAM (like NVIDIA RTX 3050); this grows to a $1,500 pre-built gaming rig or custom PC with an RTX 5060 Ti (16GB) for running a 10 Billion model like the GPT-NeoX-20B. Not to mention, the cost of running inferences is only a fraction of the costs of training or fine-tuning a model with the same number of parameters.
In this arrangement, the price of machine intelligence is a proportionate demand for costly resources. A more intelligent model only means more training parameters and, consequently, exponentially growing costs.
The dilemma: higher-precision models or lower parameter models?
FP16 models are a resource-saving approach, using 16-bit representation to store weights in half the memory, reducing VRAM usage by 50% compared to FP32 (single-precision) models. However, it sacrifices precision and accuracy, but not to worry, because you could still get near-accurate responses for the majority of your prompts.
Another approach to taming the resource consumption of AI models is to run higher-precision models with fewer parameters. Higher-precision models with fewer parameters trade off high intelligence for accuracy, given the parameters they were trained with.
The premise of both approaches is either superintelligence achieved at the expense of resources or making do with significantly limited models that can run and be trained on mid-range devices. Either way, AI training, fine-tuning, and full utilization are still confined to multi-GPU servers dominated by NVIDIA hardware and the CUDA ecosystem.
The BitNet breakthrough
Microsoft’s 2024 research breakthroughs in quantization and the flagship 1.58-bit BitNet LLM are transforming AI and machine learning, significantly moving away from resource-hungry approaches to achieving machine intelligence. Ternary quantized models grow linearly efficient while using only a fraction of the usual (relative) resources, offering strong intelligence and resource efficiency. However, the ternary quantization optimizes BitNet LLM for CPU rather than GPU, since GPUs are built for floating-point, not ternary arithmetic.
Giving BitNet LLM an edge: Tether brings the BitNet revolution to user-grade devices.
Tether’s work on the open source BitNet development has yielded two remarkable results;
The first-ever demonstration of BitNet LLM’s efficiency on a GPU.
Perhaps the most remarkable development from Tether’s research is demonstrating that BitNet LLM can, in fact, run on consumer-grade GPUs with up to eight (8) times faster inference time compared to CPUs.
Tether introduced a novel Vulkan-based GPU backend, similar to the one used for the QVAC Fabric LLM fine tuning framework. The GPU backend expedites precise inference and fine-tuning for the 1.58-bit BitNet LLM on edge-device GPUs. It is GPU-agnostic, allowing BitNet to run efficiently on almost any graphics card and breaking the overdependence on NVIDIA processors and CUDA runtime.
The impressive GPU resource economy of the BitNet LLM
Tether’s AI research demonstrates that the BitNet-1B model uses up to 77.8% less VRAM than FP16 Gemma 3-1B and 65.6% less VRAM than FP16 Qwen3-0.6B. By replacing floating-point multiplications with ternary (-1,0,1) integer addition and subtraction, a 13 billion parameter BitNet LLM runs with 29% less GPU resources (VRAM) than a 4-bit quantized 4 billion parameter Qwen3 model. According to the benchmark, the 1.58-bit quantized model can run efficiently with only 2.8GB of VRAM, which is available on mid-range computers.
BitNet 13B LLM fine-tuning framework for edge devices
Tether is stepping up efforts to scale AI advancement on edge devices – everyday consumer gadgets and IoT devices. The main product of their research on BitNet is a local-first fine-tuning framework for the LLM, as announced in their latest press release. The BitNet LLM fine-tuning framework enables fine-tuning of different parameter variants of the BitNet LLM, up to the 13-billion-parameter model, on regular devices such as the Samsung S25, Google Pixel 9, and iPhone 16.
With the framework, users can fine-tune 125M-parameter BitNet models in ~10 minutes on a Samsung S25 (Adreno GPU) for a biomedical dataset of ~300 documents (~18k tokens). For larger models (such as one with 1B parameters), fine-tuning on the same biomedical data takes 1 hour 18 mins on the same device and 1 hour 45 minutes on the iPhone 16.
In addition to Vulkan runtime integration for the BitNet LLM, their work on the edge-first fine-tuning framework introduces a dynamic tiling algorithm to improve fine-tuning speed and memory allocation, reduce energy consumption, and maintain precision across CPU and GPU. The dynamic tiling algorithm breaks down large matrices into smaller ‘tiles’ whose sizes can be adjusted to fit perfectly into the GPU’s high-speed RAM (SRAM).
Tether’s BitNet fine-tuning demonstration is the first time a 13-billion-parameter model has been successfully trained on an iPhone 16 and similar devices; a procedure once reserved for high-end, centralized data centers can now run on consumer hardware.
The potential impact spans cost savings, ease of development, and adoption. Training more intelligent AI models on everyday devices unlocks superintelligence for the several billion users who rely on edge devices for their routine activities. For businesses, it offers significant cost savings and improved performance. On top of that, the privacy advantage: running models on devices means user data does not get uploaded to centralized servers.
AI technology is progressing in two ways: by scaling intelligence and applications, and by improving resource efficiency. Quantization and edge-focused research are pursuing the latter. Tether Data takes another giant leap toward local-first AI, positioning it as “the only way to bring superintelligence to billions of users”.
The future of tech is superintelligence. But billions of users and AI agents cannot coexist in shared data centers controlled by BigTech. Tether is committed to breaking conventions and developing a sustainable system for an infinite number of intelligent machines, AI agents, and netizens.
Read the full release on HuggingFace and follow Tether to learn more about its local-first AI solutions.
