A chronological collection of things I’ve found interesting. Back to links.
2026-07-18
GPU & Accelerated Computing:
- Inside TPU and GPU Clusters: The Anatomy of Collective Communication | Aleksa Gordić
- Porting High-Performance HIP Kernels to FlyDSL | AMD ROCm Blogs
- Understanding ATen: PyTorch’s tensor library | Red Hat Developer
- MatX - GPU-Accelerated Numerical Computing in Modern C++ | NVIDIA
- Metal Compute Shaders And C++ (PDF)
Linear Algebra:
AI Compute Extensions (ACE):
- Intel Posts Initial GCC Compiler Patches For AI Compute Extensions “ACE” | Phoronix
- The AI Compute Extensions (ACE) for x86 (PDF, 147 KB)
Finite Elements & CFD:
- Templated FEM kernels: Benefits and Drawbacks
- From Particles on a Grid to the Wind Tunnel’s Digital Rival: The Rise of LBM in Defense Aviation | Dassault Systèmes SIMULIA
Programming:
- Story-time: C++, bounds checking, performance, and compilers | Chandler Carruth
- The Second Coming of the Command Line
Life Advice:
- Life Lessons from the First Half-Century of My Career | Communications of the ACM
- What the Middle Ages can teach us about preventing burnout | BBC Culture
2026-05-28
- Architecture & Systems are Changing: The Architect’s Role in the Era of Agentic Co-Design | Computer Architecture Today
- What’s new in Arm Performance Libraries 26.01 | Arm Developer Community
- perf-libs-sparse: Introducing a new open-source project for sparse linear algebra on Arm | Arm Developer Community
- NVIDIA Vera CPU Benchmarks: Olympus Cores Delivering The Best Performance Ever Seen On ARM | Phoronix
- Code Coverage Analysis for Fortran | Science and Technology Facilities Council Rutherford Appleton Laboratory (PDF)
- Why Japanese companies do so many different things | David Oks
- Advancing Computational Science with High-Order Finite Elements | Supercomputing Spotlights
- Human Bottlenecks | Fernando Borretti
- Strangely, Matrix Multiplications on GPUs Run Faster When Given “Predictable” Data! | Thonk From First Principles
- NVIDIA Took 20 Years to Ship 25 Lines of Python | Delanoe Pirard, Medium
- Can the Tile Paradigm Reshape the Competitive Landscape of the GPU Programming Ecosystem? | HyperAI
2026-05-22
2026-05-03
- Let Natural Selection Sharpen Your Writing | American Physical Society
- Writing is thinking | nature reviews bioengineering
2026-04-25
Object-oriented Design Principles:
Misc:
- C Traps and Pitfalls (PDF, 117 KB)
- New features in OpenMP 5.0 and 5.1
- New features in OpenMP 5.1 and OpenMP 5.2
- Driving a New Era of Accelerated Computing Intel® Fortran Compiler (IFX)
2026-04-21
Intel:
Misc:
2026-04-19
ArmPL:
- Arm Performance Libraries 25.04 and Arm Toolchain for Linux 20.1 Release
- Arm Performance Libraries 24.10
Batched Linear Algebra:
- The Design and Performance of Batched BLAS on Modern High-Performance Computing Systems (PDF)
- Batched BLAS APIs and Memory Layouts (PDF)
- batmat: Fast linear algebra routines for batches of small matrices
- Numerical Limitations for Compact BLAS and Compact LAPACK Routines
- Designing vector-friendly compact BLAS and LAPACK kernels
- INTEL® MKL Vectorized Compact routines (PDF)
- Adventures in Batched Linear Algebra in Intel® Math Kernel Library (PDF)
- Batched BLAS, SC24 BoF Session
- Introducing interleave-batched linear algebra functions in Arm PL
- BLASFEO - BLAS For Embedded Optimization
cuTile:
- Focus on Your Algorithm—NVIDIA CUDA Tile Handles the Hardware
- cuTile Python | Nvidia
- cuTile, the New/Old Kid on the Block: Python Programming Models for GPUs
- CUDA Tile IR
- cuTile Kernels
- cuTile.jl
- cuTile.jl: Bringing NVIDIA’s Tile-Based GPU Programming to Julia
- TileGym
- NVIDIA TileIR Internals: from CuTile to MLIR/LLVM to SASS
- CuTile on Blackwell: NVIDIA’s Compiler Moat Is Already Built
- CUDA Tile IR: Lessons from a Tile-Centric CUDA Dialect for MLIR
Interpreters:
- What opcode dispatch strategies are used in efficient interpreters?
- What are some common ways to optimise an interpreter?
- How not to make a virtual machine (label-based threading)
- Understanding Virtual Machine Dispatch through Duality
- Interpreters and virtual machines
- My experience crafting an interpreter with Rust
- Threaded Code
- Why Are My Bytecode Interpreters Slow? Hunting Truffles with VTune
- How to speed up dynamic dispatch by 20% using computed gotos in standard C++
- Computed goto for efficient dispatch tables
- Labels as Values
- Are Jump Tables Always Fastest
Unreachable:
- Portable assumptions
- Function to mark unreachable code
- Why do some languages not have an ‘unreachable’ function?
- Unreachable Switch Statement Default Cases
- __assume
- Is there a way to speed up a big switch statement?
- Most efficient method for large switch statements
2026-03-20
Interpreters:
- A Deep Dive into Dispatching Techniques | Jonathan Müller
- Parsing Protobuf at 2+GB/s: How I Learned To Love Tail Calls in C | Josh Haberman
- Subject: Re: Suggestions on implementing an efficient instruction set simulator in LuaJIT2
- The Implementation of Lua 5.0 (PDF, 145 KB)
- LuaJIT 2 beta 3 is out: Support both x32 & x64 (comment by mikemike)
- Template Interpreters | zackoverflow
- Cheaply writing a fast interpreter - Neil Mitchell
- Branch Prediction and the Performance of Interpreters - Don’t Trust Folklore
- Relative performance of interpreter implementations
2026-02-05
Misc:
- Establishing a Scalable Sparse Ecosystem with the Universal Sparse Tensor | Nvidia
- Tensor Algebra Processing Primitives (TAPP): Towards a Standard for Tensor Operations | arXiv
- I want a good parallel computer | Raph Levien
- Why didn’t Larrabee fail? | TomF’s Tech Blog
- Death and Life in Silicon Valley | Erik Lindholm (lecture recording)
Stencils:
- Compiling stencils in high performance Fortran
- Register Optimizations for Stencils on GPUs
- Evaluation of Programming Models and Performance for Stencil Computation on Current GPU Architectures
Batched BLAS:
- Batched BLAS | Innovative Computing Laboratory
- KokkosKernels: Compact Layouts for Batched Blas and Sparse Matrix-Matrix multiply
Runtime dispatch:
- The cost of runtime dispatch | Daniel Lemire
- Function Multiversioning | GCC
- ax, Qax (Compiler Option) | Intel oneAPI DPC++/C++ Compiler
- Function multi-versioning | MaskRay
- The - surprisingly limited - usefulness of function multiversioning in GCC
- Architecture Specific Code Generation and Function Multiversioning | Eric Christopher
- Generate code for multiple SIMD architectures
- Runtime CPU Detection | Nvidia Grace CPU Benchmarking Guide
- Intel® Intrinsics Guide
- CPU Dispatching: Make your code both portable and fast
- Instruction Set–Specific Dispatching on Intel® Architectures
- How does the CPU dispatcher work? | NumPy
- On CPU dispatch | AttractiveChaos
2026-01-12
Fortran Software Papers:
- Serious FORTRAN (1973)
- Serious FORTRAN—Part 2 (1973)
- Fortran 77 portability (1981)
- A fortran procedure for drawing some space-filling curves (1986)
- Module coupling and cohesion (COMP145, 2000) (see Team Software Engineering course homepage)
- Multibox Parsers (1994) (PDF, 702 KB)
- Multibox parsers: no more handwritten lexical parsers
Fortran Subsets (ELF90 and F):
- Loren Meissner, Little Giants - The New Fortran Subsets (1996) (PDF, 425 KB)
- Jerry Wagener, Fortran Reflections (1997)
- Fortran 90 subsets offer simplified environment for first-time users (1998)
Finite Difference Calculations:
- An extension of FORTRAN containing finite difference operators (1972)
- A Fortran program to generate finite difference formulas (1975)
- FORTRAN subprograms for finite-difference formulas (1976)
- Generation of finite difference formulas on arbitrarily spaced grids (1988) (formulas are implemented here, authors copy?)
- An abstract machine for partial differential equations (1994)
- Finite Difference Methods in CUDA Fortran, Part 1
- Finite Difference Methods in CUDA Fortran, Part 2
Make papers:
- Automatic generation of make dependencies (1984)
- A simple technique for automatic recompilation in modular programming languages (1989)
- Avoiding trickle-down recompilation in the Mary2 implementation
Misc:
- vpternlog: Signed Saturation
- Modal, GPU Glossary
- Fabien Sanglard, A History of Nvidia Stream Multiprocessor
- Heinz-Gerd Hegering, 50 Jahre LRZ (1962-2012)
2025-12-21
R history and other links:
- R: Past and Future History | Ross Ihaka (PDF, 92.5 KB)
- A Brief History of S | Richard A. Becker (PDF, 151 KB)
- History of S and R | John Chambers (PDF, 234 KB)
- History and Overview of R | R Programming for Data Science
- R behind the scenes: Using S the (un)usual way | Friedrich Leisch (PDF, 1357 KB)
(Gathered for my reply here.)
2025-12-20
Co-Array Fortran Resources:
- Coarray in the next Fortran Standard | John Reid
- Coarray Fortran on LRZ’s HPC systems
- Parallel programming with Fortran 2008 and 2018 coarrays | Anton Shterenlikht
- coarray-tutorial | Thomas König (look for
tutorial.md) - Introduction to PGAS | Cray Compiler Environment, HPE
Co-Array Fortran Articles:
- Writing a multigrid solver using co-array fortran (1998) | Numrich et al.
- A multi-platform co-array fortran compiler (2004) | Dotsenko et al.
- Co-arrays in the next Fortran standard (2005) | Numrich & Reid
- A parallel numerical library for co-array Fortran (2005) | Numrich
- Parallel numerical algorithms based on tensor notation and Co-Array Fortran syntax (2005) | Numrich
- A new vision for Coarray Fortran (2009) | Mellor-Crummey et al.
- An open-source compiler and runtime implementation for Coarray Fortran (2010) | Eachempati et al.
- A coarray fortran implementation to support data-intensive application development (2012) | Eachempati et al.
- Experiences at scale with PGAS versions of a Hydrodynamics application (2014) | Mallinson et al.
- Comparing Coarray Fortran (CAF) with MPI for several structured mesh PDE applications (2015) | Garain et al.
- MPI to Coarray Fortran: experiences with a CFD solver for unstructured meshes (2017) | Sharma & Moulitsas
- Accelerating Fortran codes: A method for integrating Coarray Fortran with CUDA Fortran and OpenMP (2025) | McKevitt et al.
Fortran debate:
- David Cann (1992). Retire Fortran? A Debate Rekindled (an earlier version is available in the Supercomputing ‘91 proceedings)
- Kuck & Wolfe (1984). A debate: Retire FORTRAN? No
- J.R. McGraw (1984) A debate: Retire FORTRAN? Yes
I found these at the bottom of the Fortran Wiki Articles page.
2025-12-18
Nvidia PTX Resources:
- Understanding PTX, the Assembly Language of CUDA GPU Computing | Nvidia
- Advanced NVIDIA CUDA Kernel Optimization Techniques: Handwritten PTX | Nvidia
- Inline PTX Assembly in CUDA | Nvidia
- What is Parallel Thread Execution? | Modal
- Tutorial: Understanding GPU Assembly with PTX | eunomia
- Parallel Thread Execution ISA Version 9.1 | Nvidia
- Parallel Thread Execution ISA Version 5.0 | Nvidia
- Parallel Thread Execution ISA Version 6.3 | Nvidia
- Parallel Thread Execution ISA Version 3.0 | Nvidia (PDF, 2.2 MB)
- PTX: Parallel Thread Execution ISA Version 1.4 (PDF, 2.5 MB)
- PTX: Parallel Thread Execution ISA Version 1.2 (PDF, 2.4 MB)
- PTX: Parallel Thread Execution ISA Version 1.0 (PDF, 558 KB)
- Nvidia’s PTX: Some background to the ‘virtual instruction set’ that underpins CUDA | The Chip Letter
- PTX Writer’s Guide to Interoperability | Nvidia
- A Gentle Introduction to CUDA PTX | Philip Fabianek (HN Thread)
- CUDA Binary Utilities | Nvidia
- CUDA PTX: GPU assembly language | Dr. Lawlor
- Maybe consider putting “cutlass” in your CUDA/Triton kernels | Henry Zhu
- ptx internals | redp
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels | Aleksa Gordić
2025-11-26
- Why Systolic Architectures? (PDF, 2.4 MB)
- Flang Documentation
- The Practitioner’s Cookbook for Good Parallel Performance on Multi- and Many-Core Systems | RRZE (PDF, 7.1 MB)
- Performance Analysis of the Apple AMX Matrix Accelerator | Jonathan Zhou (PDF, 1777 KB)
- Counting cycles and instructions on ARM-based Apple systems | Daniel Lemire
- Apple Firestorm/Icestorm CPU microarchitecture docs
- Finding and evaluating AMX co-processors in Apple silicon chips
- Apple vs. Oranges: Evaluating the Apple Silicon M-Series SoCs for HPC Performance and Efficiency
- A64 SIMD Instruction List: SVE Instructions
- High Performance Computing Class | FSU Jena
- Designing a SIMD Algorithm from Scratch | mcyoung
2025-11-20
- Comparing OpenBLAS and Accelerate on Apple Silicon for BLAS Routines | Frank Rosner
- Benchmarking and Testing | Tyler Sean Rau
- OpenMP* SIMD for Inclusive/Exclusive Scans
- Effiziente Nutzung von Hochleistungsrechnern in der numerischen Strömungsmechanik | Dr. Georg Hager (PDF, 599 KB)
- Automatic Translation of FORTRAN Programs to Vector Form | Randy Allen and Ken Kennedy (PDF, 2.8 MB)
- Multimedia vectorization of floating-point MIN/MAX reductions (2006) | Bik et al.
- Vectorization Essentials | Intel Software (PDF, 1913 KB)
- A comparison of 12 parallel Fortran dialects (1988) | Karp & Babb
- Extending MPI Correctness Benchmarking to the Fortran Language (2025) | Oraji et al.
- Investigating the performance of LLVM-based Intel Fortran Compiler (ifx) | Dhani Ruhela (PDF, 596 KB)
- Just Write Fortran: Experiences with a Language-Based Alternative to MPI+X (2024) | Rouson et al.