Another repair attempt- this time for our out of warranty and out of support solar monitoring system that’s been stuck on “Starting Application” for a year now..

This one is a bit annoying- my Sunpower solar array uses ACPV panels with integrated microinverters. That allows it to handle partial shade better but also to provide panel level monitoring and diagnostics.

Here is an example of what each panel reports:

This data is very handy but requires a functional gateway box to talk to them over power lines and send data to SunPower backend as well as locally. In my case, the box has been flaky for a while now and since the system is about seven years old and SunPower and the installer no longer monitor it remotely, I had no idea the box was down until I happened to look at it’s display one day. I tried the usual “have you tried to turn it on and off?” method, and it would occasionally boot all the way and start working. A day later it would reset by itself and get stuck again.

Next was a call to installer who confirmed the gateway was out of warranty, obsolete and unsupported and to just use my house level power monitor to check on panel’s health. Yeah, not very useful.

Call to Sunpower was even more fun. “Have you tried power cycling your inverters?” was their only answer. Oh, and the SolarBridge company has been bought and sold a few times, so nobody had any ideas how to repair these things. . Well, technically they directed me to a new generation gateway that was not a drop-in and required rewiring the panel to allow it to sense total Solar current that I already do. Peachy

Looking around for used replacement yielded a few hits on Ebay but the part numbers all had a different abbreviation at the end, mostly SPM-101-AOU vs SPM-101-SPR I had. That seems to indicate they are customized for the other panel makers and may or may not work with SunPower back end. So scratch that idea.

Oh well, I guess we might as well take it apart.

The box looks well designed and sealed from elements but is pretty easy to take apart. Inside we find one board and a few flex cables to display/buttons membrane:

There is a hidden USB Host socket next to RJ45 connector that is not brought outside. Might be handy. Also a few debug headers in the lower left corner.

We also see something interesting- an SD card holder with a card and a piece of kapton tape holding it in place.

That gave me an idea to try- remove the card and see what happens on reboot. The screen displayed the usual welcome message and stayed on it.

It did NOT proceed to the next stages “Mounting filesystem” and “Loading application” that I’ve seen before:

That was a bit unexpected as it basically indicated that a sizeable amount of stuff (possibly including OS) resides on SD card and not the fairly large flash chip that’s on the board. That also meant that if the card was somehow getting worn out (not impossible for a 2012 vintage system since cards then may not have had sophisticated load leveling controllers), we may have a hope to revive this thing. But it also meant if that card failed completely It’ll be the end of it.

A quick google search later I was creating an image of the original card using an open source tool called Win32 Disk Imager . That took a few clicks and I soon had an image stashed away as a backup. Next I grabbed the nearest SD card I had on hand- Sandisk 16G microSD in and SD adapter frame and wrote the image back to it using the same tool. Reinserted the card and power cycled. The box got through the welcome and filesystem check screens in an instant but got stuck on “loading application”, again. It was responding to pings on local network though.

At this stage my next step would have to be chasing debug headers and console ports. but I ran out of time and left it till the morning, by which point it magically booted and proceeded to work just as if nothing happened..

I left it running thinking it’ll just reset again in a day or so, but a week later it was still running. At that point I called it good enough, closed up the box and reinstalled in its original place. I still periodically ping its web page on the internal network to see the reported up time.

Comm errors here are the lack of replies from sleeping inverters, but I am not clear why it’s sitting on 9719 untransmitted data points. That number has not changed since the power up.

Hardware details

While I had the board exposed, I took a look at components-see block diagram below. It’s basically an NXP I.MX253 system with the usual PMIC/memory bits/ETH PHY and an ST7540 FSK PLC modem to interface to inverters over AC power lines.

The modem pins are brought to test points and labeled nicely so it’s probably possible to see the traffic going to panels should somebody want to rewrite firmware for this box.

Summary

I still do not know for sure what the failure was and whether I did really fix it vs just gotten lucky on reboot. I guess time will tell. I can always go chasing console output later.
It’s a shame that Sunpower can just drop support on something as critical as the gateway to talk to its inverters. The warranty on panels/inverters is 25 years but the gateway only gets 3. They also very helpfully closed my support ticket once data started flowing back to the servers, without ever fixing anything..

Resources

Win32 Disk Imager

SolarPanelTalk thread