Working Student - LLM Inference Performance
Job description
A working student role on the LLM Inference Performance team, helping hardware and software engineers measure and improve how LLM building blocks run on Tensordyne silicon. The student writes and optimizes low-level kernels, benchmarks performance on real hardware, and builds profiling tools.
At a glance
- Employment
- Internship
- Work mode
- Remote
What you'll do
- Learn how Tensordyne silicon processes data, manages memory, and synchronizes operations, and how key LLM components such as attention, MLP, and Mixture-of-Experts (MoE) routing map onto the architecture
- Implement, test, and tune low-level kernels to improve the performance of LLM inference workloads
- Build analytical and roofline performance models, compare predictions against simulator and real-silicon measurements, and investigate performance bottlenecks
- Develop benchmark harnesses, profiling scripts, and trace visualizations that make performance easier to measure, understand, and improve
- Explore workflows where AI agents assist with kernel development, optimization, and benchmarking, and identify the tools and context they need to work effectively
- Work closely with compiler, runtime, and kernel engineers to translate performance insights into practical improvements
- Example project: develop a benchmarking and profiling harness for an LLM building block, such as decode attention, at two abstraction levels, compare measured performance against theoretical estimates, and make the tooling accessible to AI coding agents
What we're looking for
- Currently enrolled in a Master's degree program in Computer Science, Electrical Engineering, Mathematics, or a related technical field
- Strong programming skills in Python and at least one of C, C++, or Rust
- Understanding of computer architecture, memory systems, bandwidth, and parallel computing through academic coursework or personal projects
- Interest in low-level programming, hardware performance, and optimization
- Practical experience using AI coding agents in development workflows, with the ability to evaluate and improve their output
- An experimental mindset, with the ability to form hypotheses, analyze measurements, and communicate technical findings clearly
- Curiosity, initiative, and enthusiasm for learning new technologies
Nice to have
- GPU or AI accelerator programming
- Embedded systems or bare-metal programming
- Performance profiling, benchmarking, or optimization
- LLM inference fundamentals and architectures
Your team
LLM Inference Performance
Works directly with hardware and software engineering teams, including engineers working on compilers, runtimes, and kernel optimization.
Benefits
Learn from experienced engineers: work alongside specialists in hardware architecture, compilers, runtimes, and performance optimization
Explore AI-assisted development: gain practical experience building with AI coding agents and shaping how they interact with advanced hardware systems
Make a lasting impact: help develop performance tools and workflows that will support both current and future generations of Tensordyne technology
- Work on cutting-edge AI technology: contribute to the development of next-generation AI inference hardware and software
- Gain hands-on experience: apply your university knowledge to real silicon and challenging engineering problems
- Own meaningful projects: build tools and solutions that directly support our engineering teams
- Comprehensive benefits, competitive compensation, flexible spending options, and recognition programs
Apply
Direct apply at tensordyne
Opens the job posting at tensordyne
About Tensordyne
AI system solution company building very high-performance, low-power generative AI inference systems
Tensordyne is an AI system solution company that builds very high-performance, low-power generative AI inference systems. It is a well funded, fast-paced startup company with headquarters in both Sunnyvale, CA, and Munich, Germany, and many team members working remotely across North America and Europe.
Mission
Through the creation of custom silicon, hardware and software, to enable multimodal Generative AI inference acceleration at scale, with safe, sustainable, high-performance systems for hyperscaler and neocloud data center customers.
Key facts
- Sunnyvale, CA, and Munich, Germany
- AI hardware and software
- Headquarters in both Sunnyvale, CA, and Munich, Germany
- Many team members working remotely across North America and Europe
- Well funded, fast-paced startup company
- Builds custom silicon, hardware and software for multimodal Generative AI inference acceleration
- Serves hyperscaler and neocloud data center customers
What they value
- Put people first. We only succeed when our people succeed.
- Ethics and integrity always. Be open, honest, and respectful of everyone.
- Think Big. Be ambitious and have audacious goals.
- Aim for excellence. Quality and excellence count in everything we do.
- Own it and get it done. Results matter!
- Make each person better together than they would be as an individual.
- Embrace each other's differences, and embrace that there will be differences.