Reconfigurable logic may be used to implement fairly small models, in applications where they're already employed and adding a coprocessor specifically for ML is either infeasible or doesn't make sense. As the paper mentions, you need hard logic blocks for arithmetic (if not floating point), and these are always in short supply. In DSP applications I've worked on we used the fpga for timing and i/o, to control jitter and sample from many ADCs in parallel - but apart from some filtering then ran the numbers on an adjacent non-reconfigurable core. You can get huge chips with a lot of hard logic built in, but they're relatively expensive compared to fpga + traditional coprocessor with shared memory. The high end application specific chips are shockingly expensive - worse than GPUs because there is comparatively little market for them. We had one evaluation board that was like $100K iirc.
Meanwhile others are trying to bring back analog CPUs for similar kinds of workloads, it is going to be interesting which ones end up winning in the long run.
For example, https://mythic.ai/products/m1076-analog-matrix-processor/
My pet project is to take these ideas and go to the logical end, arriving at a systolic array I call a BitGrid.
It's a Cartesian grid of 4 bit look up tables, with bits to/from each neighbor. This allows each output to be independent, maximizing utilization.
To solve timing issues, each cell would be clocked, with 2 phases for the grid, in a checkerboard pattern. This makes all inputs stable, and timing deterministic. Unlike an FPGA, latency is not the primary limit of performance, as everything is thus pipelined.
I wrote a simulator, and started learning VHDL in order to program an FPGA board I bought for prototyping the concept. My eventual goal is an ASIC through tiny tapeout.
The big software hurdle is compiling expressions into a directed graph of bitwise operations.
Because data only travels to the neighbors, all the lines in a chip are short, and it should be possible to use far fewer metallization layers in an actual chip than a CPU for example.
Dangs got some work cut out for him tonight
I'm glad to see this is being studied. I did a brief half-semester project summarizing the usefulness of accelerators after the death of Dennard's law and Moore's law (this was when Intel was still pushing out incremental improvements on 14nm). The short summary is that accelerators offered substantial performance per watt improvement, at the cost of longer development timelines, cost of manufacturing in the case of ASICs and ASIPs, and the inability to fit large algorithms on FPGAs. Though Microsoft found a way to daisy-chain FPGAs for Bing's page rank, it just didn't produce the order-of-magnitude improvement necessary to justify moving to FPGAs. With smaller algorithms or bigger FPGAs or occasional operation offloading (like what we do now with TPUs and GPUs), they could be good candidates for accelerators.
Of course the real killer is how easy it is(n't) to develop for them. AMD offers much better performance per dollar than Nvidia, but of course their poor drivers make using their hardware a fool's errand.
RADs sound like a good idea. We should be making reprogrammable hardware available and easy to use for everyone.
Is there any small risc-v soft-core with big ass SVE (scalable vector extensions)? I would like to play around with them but the only option seem to be cloud instances like gravitron and small (128bit) like the licheerv nano (c906) which also seems to only support a beta version of the standard.
Have them self-modify.
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
The big challenge when it comes to using FPGAs for deep learning is pretty simple: all of that reprogrammability comes at a performance cost. If you're doing something highly specific that conventional GPUs are bad at, like genomics research [1] or high-frequency trading [2], the performance tradeoff is worth it. But for deep learning, GPUs and AI ASICs are highly optimized for most of these computations, and an FPGA won't offer huge performance increases.
The main advantage FPGAs offer is being able to take advantage of new model optimizations much earlier than ASIC implementations could. Those proposed ternary LLMs could potentially run much faster on FPGAs, because the hardware could be optimized for exclusively ternary ops. [3]
Not to toot my own horn, but I wrote up a blog post recently about building practical FPGA acceleration and which applications are best suited for it: https://www.zach.be/p/how-to-build-a-commercial-open-source
[1] https://aws.amazon.com/solutions/case-studies/munich-leukemi...
[2] https://careers.imc.com/us/en/blogarticle/how-are-fpgas-used...
[3] https://arxiv.org/abs/2402.17764