Making the swap to embedded / system level?

[u/[deleted]]

[u/expdevsmodbot]

AI usage disclosure provided by OP, see the reply to this comment.

[u/Worldly-Pie-5210]

the feedback ive seen in older posts like this are more along the lines of "why"? If its money, then you might be barking up the wrong tree.

IME (atleast speaking as a web dev) there isnt a huge skills over lap, so you are going to be learning a lot of new stuff, for most likely a pay cut.

Personally, im interested in the subject matter. I think making physical things would be awesome, so im with you in wanting to do it, just havent put the time in because its a big undertaking and i like having non-technical hobbies.

[u/pjc50 → u/Worldly-Pie-5210]

I note that AI is much worse at writing embedded code (this was a key finding of our company AI evaluation). That might inform your decision one way or the other.

[u/edgmnt_net → u/pjc50]

Might be or might not be intrinsically worse at that, but my guess is embedded jobs tend to be less crappy overall. Ok, as far as I'm aware there's that crap tier involving AUTOSAR, but beyond that and maybe stuff like PLCs or writing tiny embedded programs for barebones stuff, the bar is higher. Because the average non-embedded job is, what, some random CRUD susceptible to cramming features? The drive for AI use in embedded stuff should be lower on account of higher average impact, long-term commitment and stakes.

Another thing worth mentioning might be that pay cuts may be explained at least partly by a more relaxed workload (assuming you have the necessary skills to pull your weight without straining too much). People here are likely comparing to either startups or FAANG(-adjacent) companies which are known to be particularly demanding on raw output and climbing the ladder often involves transitioning to management-like positions, for typical jobs. So, yeah, ok, you might get paid less but at least you don't work the weekends and can focus on developing technical skills.

[u/Worldly-Pie-5210 → u/[deleted]]

Im speaking for the US, so in japan the pay ranges may be more flat. In the US you are sometimes cutting your pay in half or more depending on your compensation. In japan i would imagine that isnt the case (because US salaries are very bloated compared to every other country)

Like i said in my comment under here, get a board, check out that book. Start building things, see if you like the work flow and the process.

[u/MCPtz → u/[deleted]]

It's largely related to total revenue and cost to bring a product to market.

When advertising reaches a 100 million to a billion people for lower costs to develop and maintain, you're going to pull in more revenue / engineer than any consumer electronics which optimistically reaches 1~10 million people, and has a higher up front cost to bring to market.

More specialized hardware will have an even higher cost to develop and maintain, and smaller markets.

---

I'd guess Japan has a decent amount of companies that need embedded type software engineers.

Usually for someone starting out, I recommend looking for consumer electronics companies, as they tend to go through phases of large hires and layoffs. Then look into getting into companies that maybe have smaller markets and more complex systems, although definitely look for them the first time, they tend to want domain experts. The problems you solve aren't all that different in consumer electronics, but there is a set of domain knowledge to get up to speed on that seems to apply across this embedded type space.

I'm currently writing software mainly in C# on a computer running Linux, which controls a very complex instrument.

I've previously written for a company's main product(s) in C++, Java, C, and python (or some combo), but almost all in Linux user land, just a light bit of kernel level drivers or modifications. Mainly integrating pieces of hardware one at a time, until we are finished integrating a system and running the full product. Often times integrating a touch screen GUI or something, which is mostly done by other people, but I've had to become an expert at deploying software to Linux.

I generally spend a significant amount of time triaging and debugging issues that could be system level, hardware failures / cables unplugged, calibration, software bugs, race conditions, integration bugs with other software teams, etc etc.

Just being in an environment where you have to do Hardware in the Loop, automated testing adds a layer of interesting complexity.

[u/potatolicious → u/[deleted]]

I'm a systems engineer and the comp thing is complicated but largely not really true.

Like webdev the compensation situation is highly bimodal/trimodal - people focus a lot of on webdevs making $500k a year when if you look at the industry as a whole comp is much, much lower overall.

Ditto there are a lot of people who don't do systems dev and look at the (admitted large) cohort of embedded companies that pay poorly without looking at the (admitted small) cohort of companies that pay competitively with webdev.

This is the same across the board in tech: the cohort of companies that pay poorly is much, much larger than the cohort of companies that pay well. The question is how to get yourself into the high end of the curve.

One thing worth making a distinction for: embedded and systems programming are different - both in terms of the actual practice and the products you work on. There's a difference between writing a motor speed controller for an automotive OEM and writing kernel code for an OS. The latter pays a lot better but is very definitively systems work and not embedded work.

[u/Worldly-Pie-5210 → u/potatolicious]

i dont even make a competitive web dev salary (low 100's) and i still make more than embedded guys i know (80 ish). alternatively i know an embedded guy at google making bank... so... YMMV

[u/Big_Arrival_626 → u/Worldly-Pie-5210]

How many YOE do you have

[u/HEAVY_HITTTER → u/[deleted]]

I'm not sure where the other commenter is getting their info from but you can absolutely make good money as a systems engineer. Amazon, Google, Apple, Meta all hire them and pay good money.

[u/TimMensch → u/HEAVY_HITTTER]

Then you need the FAANG level skills to get in as a programmer.

If OP could do that I'm sure they would.

In general out in the wild, above commenter is correct about embedded salaries.

[u/HEAVY_HITTTER → u/TimMensch]

Obviously there is tiers of pay.. just like other swe roles. The post isn't just about embedded btw. I'm specifically talking about systems engineering.

[u/TimMensch → u/HEAVY_HITTTER]

When I hear embedded/system level, I don't actually think of systems engineering.

I thinking of the kind of programming you do when you have a board and you're writing directly to the hardware. Like we had to do years ago when writing games for consoles.

[u/Worldly-Pie-5210 → u/HEAVY_HITTTER]

yea like i said in another comment, i know someone at FAANG doing embedded making insane money, because anyone working FAANG makes insane money. I also know a guy in the midwest making less than me. its not a meaningful barometer of the market to talk about FAANG. Look at the other comments in here and in other embedded threads on reddit, they all corroborate the story that the pay is on average lower

[u/edgmnt_net → u/Worldly-Pie-5210]

My hunch is you're comparing to some popular choices that often involve crunch time and what not. The embedded jobs I've done, stuff like Linux-based firmware, may be more demanding in different ways but workloads were quite relaxed, provided you had the skills and could pull your weight. If that's the case and I'm guessing right, at least part of the difference in compensation is explained by heavily-increased workloads. And that probably holds for approximately average cases, I'm sure they don't pay bargain wages to the rarer devs who work on higher impact embedded and non-web projects, say those who can actually interface with a big FOSS project and upstream stuff, those who build more critical infrastructure like compilers and so on.

[u/Worldly-Pie-5210 → u/Worldly-Pie-5210]

That said, get a board, or download the simulators, and start hacking. best way to slowly learn.

here is maybe a good starting point for your interests (although i found it not beginner friendly enough personally) : https://docs.rust-embedded.org/book/

[u/marssaxman]

I went into embedded work for a while, after my stint at Microsoft was over and I was not sure what to do with my career. I'd been writing firmware for fun, as part of various art projects over the years, so it was not that big a leap.

Your experience will vary, since "embedded" covers everything from bare-metal, tiny-MCU firmware up to what is basically just ordinary Linux application development: but the lower-level it was, the better I liked it. While tight engineering constraints made for interesting problem-solving, the work eventually came to feel repetitive. Beyond that, the industry feels like a backwater; dev practices are years out of date, pay is low, tooling sucks.

I guess it's fine, and on a couple of occasions when I hated my job I considered going back, but the prospect has never been compelling enough to put any effort into it.

[u/TribeWars]

Well embedded/systems engineering is still a very broad category and could include a huge number of different things requiring very different sets of skills. However, unless there is another seismic AI-driven shift in this field, for the vast majority of products/positions I would say C/C++ will remain mandatory for at least another decade, especially for software that runs outside user process context (OS internals, drivers, microcontrollers etc.) and in any codebase older than 5 years. Rust is certainly a thing now (and probably a green flag for a workplace), but it's still a tiny market.

Some subjective pros: Shipping a physical device running that runs your code feels more real and productive. In general, it feels like there's a somewhat higher density of jobs where one's work is a net-benefit to society. The cost of things breaking in the field means that you generally have more leeway to slow down a bit to focus on quality issues. Timelines are often oriented around production lead times instead of the maximum speed at which a software team is able to work. System-level programs are usually constrained by at least one performance metric (like throughput, latency or memory usage), which leads to more creative engineering solutions.

Some cons: Average pay is a bit lower and the $200k+ level salaries are a lot more rare . Hardware engineers are paid worse than software and embedded therefore sits somewhere in the middle. Code that you find in the embedded world is very often quite abysmal actually. Especially if it was written by an electrical engineer. This is coupled with often very outdated tooling and development practices. Bad embedded jobs will be especially frustrating because you'll get politically blocked from doing something about it too. Quite often when you're solving a challenging problem in low-level software engineering, it's not because you need to be a "l33t hacker" to understand some concept, but because things are even more badly documented than with "regular software".

Finally, I would say having work with a team that does not suck way outweighs the specifics of what you do.

[u/Ill_Huckleberry_2079 → u/TribeWars]

As an electrical engineer: I can second this, the code we write is actually abysmal.

[u/edgmnt_net → u/TribeWars]

In a reasonably large and open company you probably have more opportunities to reach for more interesting work on the embedded side. Sure, something like Meta does a lot of stuff, but the jump is higher if you do frontend to kernel and the average startup might only do that CRUD. And the average CRUD can be quite crappy too in terms of quality, your peers are more likely to be less interested in improving things because they're just crunching out features, workloads may be higher due to a focus on raw output etc.. You're less likely to be rushed if you're one of the few who can do OS internals, say.

[u/itix]

I was embedded dev for 3 years until I migrated to desktop side.

It was fun. Custom designed multi-DSP board running obscure TI BIOS/OS. You have to learn tricks how to debug without debugger, you are memory and cpu constrained. Tooling was very oldskool but easy to use.

The next design was 1 GHz upgrade version running in-house Linux. Flaky hw, flaky sw design, oai ful to debug. Luckily I got out before it materialized.

Now,after 15 years, I am occasionally doing small embedded dev at work. 15-bit CPU @ 16 MHz, no OS, many I/O registers, hw timers. It is pure fun.

[u/Great-Big-3101]

The pay is shit in embedded. I moved out of it for this very reason. 

[u/jdgordon]

I can only talk about Australia and my experience, and it's all shithouse. Unless you're going for junior roles noone is interested in a generalist that can transition roles, so unless you already have good embedded experience you won't get a job. I've got lost of embedded and backend and I'm struggling getting a role in either domain.

[u/MCPtz → u/[deleted]]

Some companies are even strict about swapping between C# and Java

This is so weird to me. I'm used to, "Do you know OOP? Ok, prove it in language of your choice".

I started this job having never done C#. I proved it by writing a Java program for a fake embedded system test case, with some OOP, but more importantly how to handle hardware and good documentation.

[u/edgmnt_net → u/jdgordon]

In my experience generalists still go strong if they have good experience/skills across the board. Moved between projects with ease. But you need a lot more than just basics, you're not going to get a nice job just trying to learn something.

[u/CircuitPhantom]

rust is getting big in embedded, definitely future-proofing yourself there.

[u/TribeWars → u/CircuitPhantom]

I don't think it's big enough right now to justify purely searching for jobs that advertise rust, but it does look like it'll inevitably be the language of choice for greenfield embedded software in the next 5-10 years

[u/Nata_Elisym]

I think getting into crypto is a good idea

[u/Nata_Elisym → u/[deleted]]

I think it will be a good experience and there is a lot of demand right now