
Software Engineer, SI Infrastructure
Job Description
What to Expect
Tesla’s Super Intelligence Infrastructure team builds the software that turns GPUs and custom silicon into reliable, efficient training capacity for Autopilot, Optimus, Robotaxi, and Dojo. We own the systems layer between bare metal and researchers: cluster lifecycle, fleet automation, observability, scheduling interfaces, and the control-plane software that keeps multi-node training jobs healthy at scale.
In this role, you'll work as a software engineer in a high-performance environment, shipping production services and tooling that operate thousands of accelerated machines, not only configuring them by hand. You'll design, build, and operate software that provisions, monitors, heals, and optimizes our SI training fleets. You treat the cluster as a product: APIs, automation pipelines, reliability signals, and abstractions that make supercomputing capacity safe and fast for ML users. You’ll focus on distributed-systems and systems-software problems (hosts, accelerators, fabric, storage, schedulers), then encode fixes into software so the same failure class doesn’t require a hero page again. Scope and level scale with impact, from owning major automation surfaces to leading design across fleet-wide platforms.
What You'll Do
Keep the training fleet up; Drive reliability and efficiency of GPU clusters so research and production training progress is not gated on infrastructure toil or silent degradation
Ship automation as product; Replace manual ops with reliable, testable software, APIs, job pipelines, operators/controllers, and libraries other engineers and on-call can depend on
Own observability and remediation paths; Build metrics, logging, alerting, and automated remediation (e.g. Prometheus/Grafana-based stacks) that detect bad nodes, sick GPUs, fabric/storage faults, and scheduler failures and fence or repair safely at scale
Raise training reliability through software; Model failure domains, reduce blast radius of HW/SW faults on long-running multi-node jobs, and improve effective capacity (less idle accelerators, fewer silent stragglers)
Develop performance-oriented systems tooling; Build and extend tools that measure and improve throughput/stability across OS, network, storage, and runtime layers for real distributed training workloads
Integrate with scheduling and workload platforms; Engineer against and improve HPC/SI scheduling and orchestration surfaces (e.g. Slurm/LSF and related platforms; containers/K8s where used) so capacity is allocatable, debuggable, and efficient
Partner across the stack working with ML infrastructure users, networking, data center, and hardware/vendor partners to bring up capacity and expose clean software interfaces, not ad-hoc runbooks alone
Participate in a 24×7 rotation for critical training infra; invest in SLIs/SLOs, runbooks-as-code, and permanent fixes that drive pages down over time
What You'll Bring
Strong software engineering instincts applied to infrastructure: you write maintainable automation and services, not only operate tickets
Proficiency in Python and/or Bash (additional languages a plus); comfort building tools others will run in production
Deep Linux experience in production (boot/image or config management, networking stack, performance debugging, package/driver realities on real hardware)
GPU cluster operations or SI training infrastructure (NVIDIA stack, driver/CUDA ecosystem familiarity, multi-node job behavior)
HPC schedulers and workload managers (Slurm, LSF, or similar); exposure to Kubernetes for ML/infra workloads a plus
High-performance networking: RDMA, RoCEv2, InfiniBand, congestion behavior, and host-side tuning for collective-heavy training
Parallel / high-performance storage (Lustre, GPFS/Spectrum Scale, BeeGFS, or cloud/parallel equivalents) and its failure modes under training I/O
Experience with configuration management / IaC-style automation at fleet scale (e.g. Ansible, or equivalent)
Willingness to participate in a 24×7 on-call rotation for critical SI training infrastructure (cadence shared during hiring; escalation and automation are first-class goals of the role)
Degree in Computer Science, Electrical Engineering, or related field, or equivalent practical experience
#### Benefits
Along with competitive pay, as a full-time Tesla employee, you are eligible for the following benefits at day 1 of hire:
Company-paid Health, Dental, Vision, Life, and Disability insurances
RRSP with employer match, Employee Stock Purchase Plans, and other financial benefits
Employee Assistance Program
Parental Success Programs
Sick and Vacation time, and Paid Holidays
Tesla Babies program
Tesla products, fitness discounts, and perks program
#### Expected Compensation
Software Development Engineer - $114,095 - $164,095/annual salary
+ cash and stock awards + benefits for all levels
Pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. The total compensation package for this position may also include other elements dependent on the position offered. Details of participation in these benefit plans will be provided if an employee receives an offer of employment.
Tesla is an Equal Opportunity / Affirmative Action employer committed to diversity in the workplace. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, age, national origin, disability, protected veteran status, gender identity or any other factor protected by applicable federal, state or local laws.
Tesla is also committed to working with and providing reasonable accommodations to individuals with disabilities. Please let your recruiter know if you need an accommodation at any point during the interview process.
ApplyWave is not an employer or a recruitment agency. You apply to each employer directly — we find their contact and write your letter. Real employers never charge money for a job or a visa sponsorship.

ESDC LMIA records
Sponsored before
- 2022 · Transport truck driversHigh Wage · Calgary2
More from Tesla
Visa Sponsorship Data
AI Resume Tailoring
Example
Tailor your resume for Software Engineer, SI Infrastructure roles
- Scores your fit against the posting's own requirements
- Pulls the skills and keywords it names into your bullets
- Exports as PDF or Word, ready to submit
Reach hiring managers at Tesla
AI Cover Letters for Software Engineer, SI Infrastructure
Generate tailored cover letters, recruiter emails, and LinkedIn messages matched to your resume.
- Tailored to your resume & job
- Cover letters, emails, LinkedIn messages
- Professional tone, your experience
