State-of-the-art AI accelerators can be out of reach for most people, Partition AI enables expensive hardware to be shared without compromising privacy.
Old AI Accelerators (e.g. Google Coral) and Graphics Processing Units (GPUs) from gamers or blockchain miners can be upcycled by the community.
Examples of some AI hardware being made available to the community for AI acceleration:
Google has a range of TPUs, the following is an example of one of those such model being contributed the the community.
The Google Coral Dual Edge TPU is a low cost way of adding TWO AI hardware acceleration processors to compatible systems (hardware and software).
Due to the limited RAM capacity inside each TPU of only 8 MByte and the complexity of running multiple models inside at the same time, having multiple AI applications share a single TPU is not recommended.
Unfortunately, most computers do NOT have dual PCI buses wired to the same M.2 E-key connector, as required by the Coral Dual Edge TPU, so an adapter is needed in order for those computers to access both TPUs onboard.
Below is a Coral Dual Edge inserted into E-key to M-key adapter from Magic Blue Smoke which is in turn inserted into the spare M.2 M-key connecter inside in a Compute Station.
This adapter uses an Asmedia ASM1182e chip to expose both TPUs on the Coral Dual Edge TPU on to 2 separate PCIe x1 Gen2 buses.
01:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch [1b21:1182]
Kernel driver in use: pcieport
02:03.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch [1b21:1182]
Kernel driver in use: pcieport
02:07.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1182e 2-Port PCIe x1 Gen2 Packet Switch [1b21:1182]
Kernel driver in use: pcieport
03:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]
Subsystem: Global Unichip Corp. Coral Edge TPU [1ac1:089a]
04:00.0 System peripheral [0880]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]
Subsystem: Global Unichip Corp. Coral Edge TPU [1ac1:089a]
Although both TPUs on the device can be used by a single AI application at the same time, normally one TPU is fast enough for most applications, so the second TPU can be used by a different AI application.
For newer releases of Linux the following driver might be better:
The amount of VRAM and Compute Capability can be important for some AI applications.
| CUDA SDK | Maxwell | Pascal | Volta | Turing | Ampere | Ada | Hopper | Blackwell |
|---|---|---|---|---|---|---|---|---|
| 12.0 - 12.6 | 5.x | 6.x | 7.x | 7.5 | 8.x | 8.9 | 9.x | |
| 12.8 | 5.x | 6.x | 7.x | 7.5 | 8.x | 8.9 | 9.x | 12.0 |
| 12.9 | 5.x | 6.x | 7.x | 7.5 | 8.x | 8.9 | 9.x | 12.1 |
| 13.0 | 7.5 | 8.x | 8.9 | 9.x | 12.1 |
We suggest contributing GPUs with VRAM of at least 6 GByte and Compute Capability of at least 5.0 (at least Maxwell).
Following is a sample of some contributed GPUs:
| Nvidia Models | GByte | GB/s | CUDA | PCI | CC | Deployed |
|---|---|---|---|---|---|---|
| GeForce GTX 1060 | 6 | 192 | 1280 | 3.0 x16 | 6.1 | 1 |
| GeForce RTX 2060 Super | 8 | 448 | 2176 | 3.0 x16 | 7.5 | 1 |
| GeForce RTX 4060 Ti | 16 | 288 | 4352 | 4.0 x8 | 8.9 | 6 |
| GeForce RTX 5060 Ti | 16 | 448 | 4608 | 5.0 x8 | 12.0 | 4 |
CC = Compute Capability
Note currently, most open source software CANNOT use advanced hardware features like FP8 and Tensor Cores.
We need to spin the GPU fans up before it gets too hot
Need to edit the nvidia_fan_control.py file to set the fan curve before running it.
Suggested Fan Curve:
# Fan curve parameters
temperature_points = [30, 40, 60, 70]
fan_speed_points = [30, 60, 80, 100]