Early in my career with data and analytics, one of my focus areas was architecture, which included installations and the occasional bit of firefighting when an installation wasn't behaving as it should.
When you get deep into how applications work, you also appreciate how they are designed, and the different components interacting with each other.
It was also a time when servers were equipped with CPUs, with no GPUs filling the racks in server rooms. Sizing, node placement and settings were all counted in CPUs and memory.
It may seem obvious, but analytics is about processing numbers, as well as presenting them in a way that ensures the user gets the information they need, rather than a matrix-style waterfall of numbers moving down the display.
During one of my nerd-sharing conversations with colleagues, I reasoned that moving the calculation and reporting engines (the bits that work out the report and dashboard layouts) to a GPU could improve performance.
After all, GPUs are good at processing lots of calculations in parallel, and surely that would be useful for something destined for viewing?
I probably even shared the idea with product managers at one point. Having repeated it several times over the years, the idea had lodged itself with me.
Fast forward many years, and that little seed has made its way back to the surface when I asked ChatGPT the question:
Why are GPUs so much more powerful than CPUs for certain workloads, and why hadn't CPU architectures adopted more of their design principles?
Fast forward many years...
The simple answer is that CPUs are good at doing lots of different types of tasks, and can be very efficient when doing so. GPUs are designed to perform many similar operations in parallel, making them particularly good at certain types of workloads.
Then, as I continued my research, I discovered there was another approach, with processors that can be configured to suit the tasks they need to perform.
Two examples are FPGAs (Field-Programmable Gate Arrays) and CGRAs (Coarse-Grained Reconfigurable Arrays).
Both allow the hardware to be configured for particular tasks, but at different levels. An FPGA can be configured at a fine-grained level, changing how individual logic elements are connected. A CGRA works at a coarser level, arranging processing elements and their connections to suit the work being performed.
The interesting part for me was that the hardware could adapt to the task, rather than relying entirely on a fixed arrangement of processing resources.
But that flexibility comes with its own challenges.
A touring theatre production
The way I like to understand it is from a theatre point of view.
A production goes on tour, but realises not every theatre has the same stage size, layout of wings, position of technical teams and equipment.
Part of the get-in and technical rehearsal is to ensure the show has been adapted to work in that theatre. Each venue will be different, and even the same theatre will have different productions visiting throughout the year.
The cycle of rearranging the show to fit the venue goes on.
It's a similar challenge when tasks need to be mapped to the underlying components of a processor. Each task has requirements that need to be accommodated within the available resources.
With many tasks running in parallel, working out where everything fits can become more complicated.
Once a task finishes, the next one may have different requirements, and the process of finding a suitable arrangement starts again.
This ability to reconfigure on demand gives these processors some interesting possibilities, but the work involved in mapping tasks and, where necessary, reconfiguring the hardware can introduce overheads.
And it was the time spent working out where things fit that caught my attention.
Using processing cycles to find a suitable arrangement is time that isn't being spent doing the actual work. Depending on the architecture, this can also have an impact on energy efficiency.
Down the rabbit hole
So, with this newfound understanding, I delved deeper, simply because I wondered if I could help.
I had an idea, researched whether it might work, which then took me onto another tangent.
Before long, I was reading research papers for prior art. This was a new term to me in this context, where you look at previously published work to understand whether others have explored similar ideas or arrived at related solutions.
There is a lot of great work out there, each looking at the problem from a different angle, and sometimes for a different purpose.
The more I read, the more familiar I became with the technical details, although none of the papers mentioned a touring theatre production!
Eventually, I started to believe that my approach might reduce some of the cycles currently spent finding a suitable arrangement for tasks.
I was also aware that it could mean a chip wasn't being utilised to its full capacity. That trade-off would need to be understood.
Time to share
But I don't have the knowledge or skills to go much further, and I wasn't about to launch a new side-study programme in chip design, so I needed help.
What happens next?
After reading the research papers and doing further online research, I selected four people or organisations and sent them my work.
I am not claiming I have solved it. Even I wouldn't make that claim.
But I wanted to show that someone from outside a research field can explore an idea, do some groundwork, and offer up a concept that may be worth investigating further.
Of course, they might rubbish the idea quite quickly, and they may never even respond to my email.
Even so, I will still open my inbox looking for feedback.
Perhaps I've stumbled on something that helps. An idea that started as a small thought many years ago, developed as I explored it, and eventually reached a point where I felt ready to send it onwards.
Its own touring production, ready for an audience.
Nothing wrong with dreaming, is there?
No comments yet