Note that the Turing Pi v1 has 7 slots for Pi CM/CM3/CM3+ modules, but it has been in short supply as each batch they made has sold out pretty quickly.
CM4 requires both boards to be stacked, so it doesn't work for something like this. IIRC there are also problems with the connector layout assuming zero tolerance.
Jeff Geerling! I'm excited for the video that I know is forthcoming, I'm haven't even looked at Turing Pi's official release because I want to hear your take on it :)
I'm quite interested in this but there seems very little explanation.
I found the wiki has more but I'd really like to see a good review of the cluster in use. Whether it is useful as more than a local self learning system.
https://wiki.pine64.org/index.php/SOPINE
I know it’s a HN cliché to comment on this, but it’s also true: I went several links deep, then looked at the homepage and the FAQ, and still have no idea what this product is or who it’s for.
Short version is that it is a custom board into which you can plug multiple RPi compute modules (and now some Jetson modules) to create a miniature version of a blade server system; this board is the backplane of that compute module blade system. Use it to create your own edge cluster system I guess; it does not seem particularly useful beyond being a neat curiosity when you put RPi CMs into it, but as a GPU/CUDA node filled with Jetson modules there is some interesting possibility there for people looking for a cheap local cluster for training ML models.
4x Jetson nano would cost $240, the Turing board will probably cost around $100 (it looks like they haven't decided yet) and you get 1,88 tflops; you can add 50 bucks and get GTX 1060 with 4.4 tflops and you can play games.
Not sure that's quite a fair comparison, because you'll need quite a bit more hardware to use the GTX 1060, and I think that the Turing board + Jetson setup would be all-inclusive (except a power supply and chassis, I suppose)?
Their two use cases are edge infra — horizontally scaled server applications on a power and cabinet space budget — or as a workstation for workflows that can benefit from distributed compute power.
I could imagine the latter might be handy if you’re doing CAD with rendering in the Amazon rainforest and don’t have 5A of power for x86_64 + GPU. Maybe.
It definitely seems like a solution in search of a problem. Happy to be proven wrong though.
There's a company here in Ireland, Cainthus, that does workplace wellbeing for dairy cows. It does so by continuously analysing video feeds and detecting behaviors that could indicate stress or other environmental factors that make the cows unhappy. Management could be done by the RPi while the inference could run on one or more Jetson boards. These little machines are very friendly for embedded work.
I'm hoping to get my hands on a board soon... I think they're still brushing up the prototype though, trying to get it to a more final production state.
Actually I wish they hadn't reduced the number of slots to 4, because part of the "fun" is dealing with the fact that with 7 nodes, using ssh and individual node management is no way to manage a cluster, so you're forced to treat it as a real cluster. I feel with 4, I might be tempted to individually manage each node. But I also understand why changes in the Pi Compute Module 4 made this necessary. The CM4 is physically much larger than the CM3.
Edit: Actually my real wish is for a compute module that has more I/O channels. I would love to build a hypercube-style supercomputer (like the Meiko Computing Surface) but these require 5+ high speed I/O interconnects say to build a 32+ node cluster. I wonder if PCIe offers a solution?
I too would be interested in playing around with more processor to processor interconnects. Just for fun I built a 16 way SAMD21 board that used the serial interconnects to make a hypercube arrangement and it was very cool to play with.
It would be possible to build an interconnect over PCIe, but of course it might just be better to use a 10g ethernet PCIe interface chip for each node and a local to PCB network.
They look to be great devices for edge compute. I can slap a Jetson in one slot and pi's in the other three. Cheap, easy to fix, no expensive support contracts. I'll be looking at throwing them out into rural locations/farms. I can get x86 nucs but tbh they lack the customisability in this has. This is awesome for industrial applications.
No, I saw that, I just don’t understand it. I can do all of that with a PC. Is this a PC? A more powerful Raspberry Pi? What does the ability to “learn concepts” even mean? I learn concepts from books, what does the hardware do?
This board is a very convenient way (maybe the most convenient one I've seen) to setup a bare-metal cluster of computers. Not just multiple cores, not just multiple VMs, four entirely separate ARM computers communicating over a real hardware network. One Alternative to boards like this is to connect multiple SBCs together, with all the wiring, and also some mechanical support. Another (more powerful alternative) is to install some kind of server rack at home. More expensive, too. Using multiple virtual machines is also not quite the same.
What people use it for? Mostly to learn how to deal with problems that arise from managing a cluster and running software on it. Can you build a website that tolerates getting one of the nodes or hard drives turned off?
Some people use such solutions for productive things, like a Home Server, but a store-bought NAS or a single PC is usually more performant. A PI cluster might be less power hungry in some scenarios.
Some people use them as build/test platforms for code that should run on ARM architectures. Others have used them to host a website from their internet connection (I know...).
Some people just have fun tinkering with such things....
Don't worry, it's not just you. I learn concepts by building and tinkering (and reading specifications), so you'd think I'm a target market. But when I wanted to get some hands on experience with a cluster file system, for a job, I spun up a cluster of 5 vms on... my normal computer.
4 seems like a very useless number to me. 4 raspis is more expensive and less useful than a used dual xeon on ebay. I could imagine maybe there's a use for something with 16 slots? or at least 8? But I don't get these cluster boards (or, for that matter, storage enclosures!) which presume I can do something fundamentally different with 4 small computes than 1.
The original Turing Pi had 7 slots (wish it had even more!). I do feel that was better, because it really forces you to manage it as a cluster.
Spinning up VMs is sort of fine, but they don't have quite the performance or management characteristics of a real cluster. The network is slow, the nodes individually are not very powerful, you have to work out how to image each physical machine, nodes break or have I/O errors, ...
> maybe there's a use for something with 16 slots? or at least 8?
You can always connect 2 or 4 of these together.
But I understand what you mean. A project I want to build one day, when I have the time and learn ethernet interfacing through PCB's is to build a single board cluster of Octavo SoM modules. They are individually inexpensive and it'd be relatively easy to build a board with a dozen of them connected to a switch chip.
Yeah exactly, my first thought is that a normal multicore PC is going to be not just more powerful, but more power efficient and cost efficient. It's a fun idea but I wouldn't be interested unless they publish some comparisons.
Basically everything here can be done on a single multicore computer (which is already a distributed system in many respects):
more powerful in most cases - yes. distributed - no.
learning about doing HA/Scale out via distributed systems is a really valuable skill, and projects like these make it sooo much more real, beyond even just basic networking.
Raspberry Pi has a compute module version: CM4, which is basically a pared down RPi4 with almost no IO options. This is a board for CM4 (and, apparently, NVidia Jetson) that lets you power, network and communicate with several CM4 boards in a cluster setup.
same here, their website should have a one-line description of what this board is/for. Though I did find a short answer on potential use cases from FAQ in one of the pages (I think it's the Turing Pi V2 product page?)
This could power a portable audio studio. Use the CMs for different VST instruments and sound effects. Zynthian is just using a single raspi, but 6 of these beasts would make up a complete audio studio with multi channel recording and real time effects and synths. Zynthian provides accurate emulations of classic instruments: Grand piano, Rhodes, Wurlitzer, pipe organ, Hammond organ, combo organ, Minimoog, DX-7, Oberheim OB-X, JX-10... In the 80s I would have given a limb for a machine like that. Put it into a 19 inch case, slap a large display and 16 Knobs and sliders on the front panel, give it 8 line ins and midi connectivity and you have the ultimate audio studio in a box. Sell it for 1000 bucks and it sells better than sliced bread.
I will get a Turing Pi ASAP to play with just this.
It should be noted however that even the bare bones Zynthian system is capable of doing a LOT of voices. Plus, you don't have to go with the original Zynthian hardware - I have a small stack of devices from Audiophonics (I-SABRE RaspTouch - [1]) that all run ZynthianOS perfectly well, and it is a very nice and easy way to add polyphony/DSP power to the setup.
EDIT: to note, this is just a perfect example of how open source projects inspire innovation - the ZynthianOS is free and amazing, and the Zynthian project itself has its own hardware designed for the purpose - but it can run on other peoples Pi-based hardware as well, and there is such a wide plethora of audiophile devices out there to pick and choose from. Having the TuringPi architecture available, I can imagine its only a matter of a few months before we see exactly the system - multi-core/expander-type synth modules - appear on the market, making ZynthianOS even more powerful ..
Reminds me of the good old “imagine a beowulf cluster of these” jokes.
For those who don’t remember, the origin of the joke came about shortly after the first beowulf clusters were announced - thereafter every time a new computer was announced on Slashdot, somebody would say "Imagine a Beowulf cluster of these".
Don't forget that one of the milestone things in Beowulf cluster was Stone Soup cluster, and the whole Beowulf cluster thing was heavily about "we don't have money for supercomputer, we hacked something from random junk"
My jam was rather Openmosix, but same vibe: throw whatever you have into the cluster and be happy with the inefficiency because those resources would sit unused otherwise !
Early early google seems to have used scavenged machines from university as soon as it outgrew what was essentially a phd project? I recall a photo of random assortment of Sun sparcstations and the like.
The ability to take Jetson cards is actually a really strong point in its favor IMO. Raspberry Pis by themselves aren't incredibly interesting in clusters except for pedagogical reasons, but with some Nvidia cards running whatever on CUDA, maybe with some RPis mixed in to do management/support tasks, and the built-in BMC, this could be pretty sweet for the right task.
I'm not sure these NVidia cards are very powerful. One decent GPU in a PC may blow several clusters of these out of the water. I haven't checked, though.
In raw performance, probably. The benefit these have (at least purportedly) is they're very energy efficient, consuming little power (and generating little heat) for comparatively large throughput.
So I can imagine someone wanting a few of these on a desk, running inference on some models or something, maybe as a small back-end for a hobby project. It may still be more power efficient to just use regular GPUs, but I suspect these win out because of the tight coupling between "CUDA cores" and the CPU.
Now, is that worth spending a bunch (many hundreds) of dollars on a carrier board and these Jetson modules? For me, no, but I at least see why it may appeal to some people.
I guess it is. But I suspect the price (as it's usually the case with "cool" rpi things) isn't going to look like one of a cable management solution. INB4: $200
Yes, exactly, thus tidying ('mother') PCB layout in the same way that a PCB tidies all the cables into a small arrangement with all discrete components fixed in the same plane.
These connect Pi Compute Modules, which are distinct from regular a Pi in a few ways (eg they don't function without a host board of some sort, so don't have certain things on board like network connectors, GPIO etc) but putting that aside, you'd get to a reasonably similar place if you hooked up some regular Pi's, it's simply more wires with the regular ones.
I think the biggest differentiator here is direct access to the PCIe bus and SATA that doesn't go via USB - that's something you can't get on a normal Rpi.
Pretty cool project, but on the other hand, if it ends up costing 200usd this is gonna be pretty expensive.
And tbh, what is the point of doing raspberry pi clusters in general?
NUCs/usff PCs are more powerfull, cheaper and easier to upgrade
One of the points is to have a cluster of separate hardware computers communicating over a hardware network. Not for performance reasons, but rather to learn to deal with the limitations.
One NUC is still one NUC. There may even be workloads where 4 CM4 modules (or even Jetsons) beat a modest NUC. Not sure where, though.
Ok, new NUCs might be a bit expensive.
But I can get lenovo tiny m72 or something like that, with core i3, 4GB of ram and 320gb hdd for <70usd. I would assume 4 of those are going to be similar in terms of size as turingpi build, much more powerfull and consume just a bit of power more (have a few similar machines, they consume something like 10W unless under heavy load)
I felt that expensive is subjective in this context. It is expensive as a full kit NUC, yes. However, barebone NUC is cheaper and I see a few of them are in $300 range with various of i3 and i5. I seen a very cheap $200 range barebone NUC last month, however it use low end cpu, i3 or Celeron.
I hope they keep the 20-pin ATX power connector. The latest Turing Pi V1 boards are missing it, and one must have an ATX-compatible PSU with the 4-pin motherboard connector instead. Most ATX PSUs have this, but not all 1U cases come with such PSUs. Supermicro's CSE-512 doesn't have one, and it's a shame that the V1 is not compatible with it, because they're the cheapest 1U Supermicro cases on eBay. No one is buying them, but had this point of compatibility been preserved, they could have gone to better use instead of the scrap heap.
Part of my reason for setting up a home lab was for cluster experimentation, so the suggestions of "just get a Mini-ITX computer" weren't a good fit. Yes, I could set up multiple VMs there, I wanted multiple chassis.
If the BMC is something good (thats supports standards like redfish / etc) and the managed switch is exposed using something standard, I think this could make a really interesting dev board for someone who wants to learn about bare metal "clouds".
Note, the page says "4x 1-gigabit ports are allocated for each computing node" which is incorrect if there are seven total ports as specified elsewhere. It should instead say something like "4x 1-gigabit ports are allocated, one for each computing node".
Faster and more of them. Unfortunately they're limited by what the CM4 exposes, which is gigabit ethernet, a single lane of PCIe gen 2, and a bunch of GPIO pins.
The cluster https://www.pine64.org/clusterboard/
The compute node: https://pine64.com/product/sopine-a64-compute-module/
The AI node: https://pine64.com/product/soedge-ai-neural-module/