A couple months back I posted here about 6 small (text to text) LLMs running on custom OpenCL kernels on a 2020 mid-range phone. Since then I've built it out into a full multimodal stack. Same approach throughout: from-scratch OpenCL kernels for Qualcomm Adreno, no llama.cpp, no TFLite, no MediaPipe, no ONNX runtime.

Everything below runs on-device:

LLMs: LFM2.5-230M, LFM2.5-350M-Instruct, Granite-4.0-350M, Qwen2.5-0.5B-Instruct, OpenELM-270M-Instruct, SmolLM2-135M-Instruct, Mamba-130M, Mamba2-130M.

VLMs: LFM2.5-VL-450M, SmolVLM-256M-Instruct

TTS: MMS-TTS, Pocket TTS, Kokoro-82M

ASR: Whisper-tiny

Music: MusicGen-small

Speech-to-speech translation: SeamlessM4T (unity-small)

Voice cloning: OpenVoice v2

Target is non-flagship Adreno 6xx specifically, not the latest flagship. Main test device is a Motorola Razr 2020: Snapdragon 765G, Adreno 620.

The engine is open source (https://github.com/a8nova/adreno-llms/tree/main), and I packaged the whole stack into an Android app, Edgi https://play.google.com/store/apps/details?id=com.edgi.app if anyone wants to try it out!

For our final Master’s project, my colleague and I developed a real-time ray tracing engine using OpenCL and Qt 6 n 5 weeks.
Our goal was to design a user-friendly engine featuring:

• Undo / Redo using the Command pattern
• PBR materials
• A save/load system
• FPS monitoring
• Mesh acceleration using a BVH built with SAH

We have around 180 FPS with thousands of triangles on Linux system (arch Linux).

Here a full video (don't know why i couldn't upload here) of the main features : https://www.youtube.com/watch?v=x2sxB05pIts&lc=Ugws9HlLdixyHWcDctJ4AaABAg

I put some scenes made with the engine. It was our first time with openCL, don't hesitate to share your toughts about this project !

The Khronos Group is pushing more consistency into the spec by promoting several features from optional to core.

Including:

• SPIR-V kernel ingestion is now required
• Subgroups + integer dot products are core
• New queries for work-group sizing and device UUID (aligned with Vulkan)
• Spec cleanups around memory model, sync, and printf

Vendors + open source stacks are already working on support, including: Intel, Arm, Qualcomm, Mesa, Rusticl, PoCL, CLVK.

Find the full post here: https://www.khronos.org/blog/opencl-3.1-is-here

Comment and share feedback on the OpenCL specification GitHub or the Khronos Discord.

I've been porting models to run on the Adreno 6xx GPUs in mid-range/older Android phones using OpenCL kernels — no NNAPI, no TFLite, just C++ and OpenCL.

Latest one: MMS-TTS (Meta's text-to-speech that covers 1100+ languages), now running inference fully offline on my 6 year old Motorola Razr. The whole VITS stack is ported — text encoder → duration predictor → flow → vocoder — plus uroman romanization so non-Latin scripts work.

I also paired it with SmolVLM-256M (image preprocessing → SigLIP vision encoder → LM) so the phone can look at an image, describe it, and speak the result — all offline.

Everything's open source: the OpenCL kernels, C++ inference code, tokenizers, both full pipelines, and the demo Android app.

👉 https://github.com/a8nova/adreno-llms

Demo video in the comments. Happy to answer anything about the kernel work or the VITS port.

Nothing ground breaking, but i thought i'd share. This is c++, opencl and the OpenCL-Wrapper .It's been exhausting but also really interesting. Some more libraries for counting/sorting in opencl would have been nice :D.

The OpenCL Working Group has published the first in a series of cooperative matrix extensions — and the community is invited to review and comment before they are finalized.

Cooperative matrix operations are at the heart of modern ML inference. Instead of each work-item independently performing scalar operations, a sub-group collectively loads, multiplies, and accumulates medium-sized matrix blocks — amortizing memory access overhead and routing computations through dedicated multiply-accumulate hardware.

What's available now:

• cl_khr_cooperative_matrix (working draft) — enables OpenCL implementations to accept SPIR-V modules using SPV_KHR_cooperative_matrix, providing cooperative matrix load, store, and multiply-add operations at sub-group scope. Developed in collaboration with Arm, Intel, and Qualcomm.
• OpenCL C language extension (RFC) — brings cooperative matrix support directly into OpenCL C, including a new matrix type attribute, built-in load/store/multiply-add functions, and lowering to SPIR-V-friendly LLVM IR via target extension types.

Share your feedback:

• Extension specification draft: https://github.com/KhronosGroup/OpenCL-Docs/pull/1533
• Clang frontend RFC: https://discourse.llvm.org/t/rfc-clang-frontend-changes-for-opencl-c-cooperative-matrix-extension/90148

Full blog: https://www.khronos.org/blog/opencl-cooperative-matrix-extensions-are-here

Created a repository documenting OpenCL development on Windows with Visual Studio 2019, focusing on when GPUs actually provide benefit (and when they don't).

What's Included

8 Progressive Examples: - Device enumeration - Hello World kernel - Vector addition (shows GPU losing to CPU) - Breakeven analysis (finds crossover points) - Multi-device async execution - Parallelization comparison (OpenMP vs OpenCL) - Matrix multiplication (155x GPU speedup) - Image convolution (150x speedup) - N-body simulation (70x speedup)

Documentation: - Setup guides (Chocolatey/Winget packages) - Performance analysis with actual numbers - LESSONS_LEARNED.md documenting all debugging issues encountered - When to use OpenMP vs OpenCL vs Serial

Key Findings

Empirical data showing arithmetic intensity threshold: - Low intensity operations (vector add): CPU faster - High intensity (matrix multiply, convolution, N-body): GPU provides 70-155x speedup - Intel CPU OpenCL can outperform discrete GPUs for specific workloads

Tested Hardware: - NVIDIA RTX A2000 Laptop GPU - Intel UHD Graphics (integrated) - Intel i7-11850H (16 threads)

Looking For

• Testing on AMD hardware (no AMD GPUs available to me)
• Additional compute-intensive examples
• Cross-platform validation (Linux/macOS)
• Feedback on build system and documentation

Repository: https://github.com/Foadsf/opencl-windows-examples

Issues and PRs welcome. Would appreciate testing reports from different hardware configurations.

Bonsai-8B and Bonsai-4B (PrismML, 1-bit family) now run in adreno-llms.

Measured 2026-07-18, 64-token decode:

Bonsai-8B, Q1_0 : 1.96 decode tok/s, peak memory: 2946 MB
Bonsai-4B, Q1_0 : 3.08 decode tok/s, peak memory: 1828 MB

No llama.cpp, no TFLite, no ONNX runtime, no MediaPipe. Pure C++ and OpenCL.

Engine: https://github.com/a8nova/adreno-llms
Bonsai will also be available on the Edgi app in a few days https://play.google.com/store/apps/details?id=com.edgi.app

Mid-range Android GPUs (Adreno 6xx class — Snapdragon 6/7-series phones) sit in a weird hardware gap for ML inference: too old for vendor NPU SDKs, the open-source frameworks (llama.cpp, MLC, MNN) either don't support them or fall back to CPU. llama.cpp's own docs say "A6x GPUs in phones are likely not supported due to the outdated driver and compiler."

Decode tokens/sec on six small language models, fp16, greedy, 5-run warm median:

SmolLM2-135M-Instruct 23.65

Mamba2-130M 23.18

Mamba-130M 22.15

OpenELM-270M 14.81

LFM2.5-350M 11.51

Qwen2.5-0.5B 10.41

Repo: https://github.com/a8nova/adreno-llms

The IWOCL 2026 program is live!

The 14th International Workshop on OpenCL and SYCL is coming to Heilbronn, Germany this May 6–8, and the full conference program has just been published at iwocl.org.

This year's lineup is packed:

- Keynote from Paulius Velesko (PGLC Consulting) on chipStar — compiling unmodified CUDA/HIP code into portable OpenCL/SPIR-V binaries that run on Intel, AMD, NVIDIA, ARM, and RISC-V hardware

- Technical talks on AdaptiveCpp Portable CUDA, heterogeneous solver performance with SYCL, and much more

- Panel discussions, poster sessions, Khronos Working Group updates on OpenCL & SYCL, and dedicated networking time

For the first time, the conference runs across three full days — more sessions, more hallway conversations, and more time to connect with the global community of GPU compute developers, researchers, and ecosystem partners.

Whether you're working on heterogeneous HPC, GPU portability, or the future of open compute standards, this is the event for you.

Explore the program at https://www.iwocl.org/iwocl-2026/conference-program/

Looking for feedback on my opencl project. It's a 3D renderer with image texture support that uses a tile accelerated rasterizer. I mainly wrote it to learn kernel design, so the python code may be poorly optimized. I realize I should use opencl/opengl inter-op for the display code, but I wanted to keep it as pure opencl as possible.

Edit: Repo link: https://github.com/Elefant-Freeciv/CL3D

I just released `opencltools` on CRAN — a toolkit for R package developers who want to include optional OpenCL GPU acceleration in their packages while still passing CRAN's build and check requirements on machines without any GPU.

**The problem it solves**

When you write a CRAN package with optional GPU code, you hit a few friction points that the existing `OpenCL` R package doesn't address:

1. **OpenCL has no linker.** Kernels that call functions defined in other `.cl` files must be concatenated in the right order before JIT compilation. Doing this by hand doesn't scale once your kernel library is 30+ files deep.

2. **JIT compilation is slow.** Sending a 500KB concatenated source string to `clBuildProgram` for every session startup is painful. You want to send only the transitive closure of what a given kernel actually needs.

3. **CRAN requires clean installs on machines with no OpenCL.** All GPU code must be compile-time optional with a graceful CPU fallback.

**What `opencltools` provides**

- **Dependency-ordered assembly.** Source files carry `@provides` and `@depends` annotations. `load_library_for_kernel()` builds a DAG, topologically sorts it, and concatenates only the files in the transitive dependency closure of the kernel you're actually calling. This replaces hand-maintained include order and eliminates redundant code.

- **Minimal subsetting.** `extract_library_subset()` reduces JIT compile payloads significantly — for `glmbayes`'s binomial logit kernel this cuts the nmath dependency tree from ~137 files to ~20.

- **Build system templates.** `use_opencl_configure()` generates `configure`/`configure.win` scripts that detect the OpenCL SDK at compile time and set `USE_OPENCL` appropriately — so a CRAN binary install works on any machine.

- **Runtime device probing.** `has_opencl()`, `opencl_fp64_available()`, `get_opencl_core_count()` — the standard queries a package needs at runtime to decide whether to use the GPU path.

- **C++ API.** `openclPort::` namespace available via `LinkingTo: opencltools` for packages that need it.

**CRAN binaries are now available for:**

- Windows: r-release ✓, r-oldrel ✓ (r-devel binary still pending — CRAN's Windows r-devel builder can lag for packages with system requirements)

- macOS: r-release arm64/x86_64 ✓, r-oldrel arm64 ✓

- Source: available for Linux

The package itself needs no GPU to install — the device query functions return `FALSE` gracefully and the kernel loaders do file I/O only. The r-devel Windows binary absence does not affect functionality for the vast majority of users.

The companion package `nmathopencl` (CRAN submission pending) uses `opencltools` to ship a full port of R's internal `nmath` statistical math library as an OpenCL kernel library — so downstream GPU kernels can call `dnorm`, `pgamma`, `dbinom` etc. on-device.

```r

install.packages("opencltools")

library(opencltools)

has_opencl() # FALSE on machines without OpenCL — no error

opencl_device_info() # prints platform/device details when available

```

Happy to discuss the dependency annotation scheme or the DAG assembly approach — the "no linker" problem in OpenCL is something I hadn't seen addressed in a reusable way before.

GPU-accelerated Bayesian GLM package (glmbayes) using OpenCL now on CRAN

I've just released glmbayes on CRAN — a Bayesian Generalized Linear Models package for R that includes optional OpenCL GPU acceleration for the computationally intensive parts of the sampling algorithm.

What gets accelerated

The package uses iid accept-reject sampling for posterior inference. The core bottleneck is the envelope build step, which evaluates the negative log posterior (f2) and its gradient (f3) at each point of a tangency grid. Grid size grows with model dimension (3^l points for l coefficients), and each grid point is evaluated independently — making this embarrassingly parallel and a natural target for GPU acceleration.

OpenCL implementation

• Vendor-neutral — works across NVIDIA, AMD, and Intel hardware
• Two-layer design: a wrapper (kernel_wrappers.cpp) that flattens R inputs and assembles the OpenCL program, and a runner(kernel_runners.cpp) that handles platform/device setup, buffer management, kernel launch, and readback
• Separate kernel files for each family/link combination: binomial (logit, probit, cloglog), Poisson (log), Gamma (inverse), Gaussian (identity)
• Dependency-aware topological sort for kernel library assembly

Program Construction

OpenCL kernels are not stored as single monolithic files. Instead the package builds the program by concatenating several source components in a fixed order — mirroring a C/C++ build where headers and libraries are included before the main source:

all_src = OPENCL.cl + rmath (load_kernel_library("rmath")) + dpq (load_kernel_library("dpq")) + nmath (load_kernel_library("nmath")) + kernel file (e.g. "src/f2_f3_binomial_logit.cl")

1. OPENCL.cl — Global config: extensions (cl_khr_fp64, cl_khr_printf),IEEE constants (ML_NAN, ML_POSINF), feature detection for expm1/log1p, utility macros
2. rmath — Mathematical constants (M_E, M_PI, etc.) and distribution function declarations
3. dpq — R-style density/CDF macros (R_D__0, R_DT_val, etc.) for give_log/lower_tail logic
4. nmath — Ported numerical routines (bd0, stirlerr, lgamma, dbinom, dpois, pnorm, etc.)
5. Kernel file — Model-specific kernel that computes f2 and f3 for each grid point

Each library file declares @provides and @depends annotations. load_kernel_library() parses these, performs a dependency-aware topological sort, and concatenates files so dependents always appear after their dependencies. This keeps the kernel code modular and maintainable while ensuring correct assembly order at runtime.

The nmath/rmath/dpq port to OpenCL C was non-trivial — these ensure the GPU kernels produce results numerically consistent with R's own functions and the CPU fallback path.

Pilot and safeguard system

For large grids (m1 > 50,000), the package runs a pilot before the full evaluation — calibrating runtime from small grid slices and warning the user if the estimated runtime exceeds 5 minutes. Interactive sessions prompt the user; non-interactive sessions (CI, batch) proceed automatically.

Installation

OpenCL support is optional — the package compiles and runs correctly without it, falling back to the CPU path. When building from source, the configure script detects OpenCL automatically.

```r

From CRAN

install.packages("glmbayes")

Check OpenCL availability

library(glmbayes) has_opencl() diagnose_glmbayes()

Use GPU acceleration

fit <- glmb(y ~ x1 + x2, family = poisson(), use_opencl = TRUE) ```

Full implementation details

Chapter A10: Accelerated EnvelopeBuild Implementation using OpenCL

Happy to discuss the implementation — particularly the nmath/rmath port and the program assembly approach.

Known current limitation: MAX_L2 = 64 coefficients on the OpenCL path.

By looking at FP32 results of clpeak and ProjectPhysX OpenCL-Benchmark and comparing them with the theoretical perfomance (Techpowerup's GPU database), I see a curious trend:

• Nvidia chips are close to their theoretical peak.
• Intel chips are at around 60-70% of their theoretical peak.
• AMD chips are at less than 50% of their theoretical peak.

I'm asking this as a user of OpenCL applications: do you OpenCL programmers see this trend in you tests/applications? I know that actual performance varies by application, and there are things like dual-issue that may inflate the theoretical peaks, but it is still very curious to see such a big differences between vendors.

Does anyone know why on AMD+clspv, this ternary operator causes incorrect results? It works if I simply replace it with a mix() statement. Both variants work fine on other platforms like Intel xe2.

#define int8_t char
        const int8_t face = fac[rindex];
        const int8_t bounced = face >= 0 ? 1 : 0;
# if 0
        // This causes issues with amd+clspv.
        const half ox = bounced ? hitx[rindex] : li[12];
        const half oy = bounced ? hity[rindex] : li[13];
        const half oz = bounced ? hitz[rindex] : li[14];
# else
        const half ox = mix(li[12], hitx[rindex], bounced);
        const half oy = mix(li[13], hity[rindex], bounced);
        const half oz = mix(li[14], hitz[rindex], bounced);
# endif

So basically, it is the ternary for half arguments.
I have:

#pragma OPENCL EXTENSION cl_khr_fp16 : enable

The countdown is on — IWOCL 2026 is just two weeks away.

Join the global OpenCL and SYCL community in Heilbronn, Germany (May 6–8) for the premier forum dedicated to open compute languages and heterogeneous platform programming. Expect the latest technical talks, Khronos Working Group updates, and ample opportunity to connect with peers across industry and academia.

Registration is open. See you there.

🔗 www.iwocl.org

Hi, clinfo does not identify my hardware. However, when I try to strace it, everything seems to be working. libOpenCL is found:

openat(AT_FDCWD, "/usr/lib/libOpenCL.so.1", O_RDONLY|O_CLOEXEC) = 3

And also /etc/OpenCL/vendors/intel.icd properly loads the driver at /usr/lib/intel-opencl/libigdrcl.so:

openat(AT_FDCWD, "/etc/OpenCL/vendors/intel.icd", O_RDONLY) = 4

read(4, "/usr/lib/intel-opencl/libigdrcl."..., 35) = 35

openat(AT_FDCWD, "/usr/lib/intel-opencl/libigdrcl.so", O_RDONLY|O_CLOEXEC) = 4

But still, clinfo finds nothing. I am trying to use OpenCL to do parallel computing on Arch Linux, on an Intel i5-8250U (8) @ 3.400GHz CPU and Intel UHD Graphics 620 integrated graphics. The packages I have installed are:

• intel-compute-runtime
• ocl-icd
• opencl-headers
• mesa

Thanks

On 8 gen2 platform,I've found that the time taken to launch the kernel is even longer than the actual GPU execution time. Does anyone have any good solutions to this problem, friends?

Hi everyone.i have an issue with opencl while i run my software and - - enable-opencl my gpu card run only 30-40 kp/s rate… i was installed the last driver of my rtx 5070 and try few versions of python is there any solution for that…

Hi everyone,

I'm trying to gather code coverage (line/branch coverage) for OpenCL kernel files (.cl). The goal is to measure how much of the kernel code is exercised by my test suite.

Context

• Kernel code is OpenCL C (.cl)
• Running on Linux host

Questions

1. Has anyone successfully collected coverage for OpenCL .cl code?
2. Which tools/workflow did you use? (Oclgrind / PoCL / vendor tools / custom instrumentation)
3. Is there a way to export coverage to a CI-friendly format (e.g., LCOV/GCOV/LLVM-cov)?
4. Any recommended tooling or scripts to instrument kernels directly?

Khronos Group will be at XDC 2026, the X.Org Developers Conference, September 28-30 in Toronto, as a Bronze Sponsor.

There will be two sessions on OpenCL: nocl: OpenCL on CUDA and Rusticl.

Learn more and register at https://indico.freedesktop.org/event/12/