Empirical data meets theoretical

Over the years I’ve read enough citations that I’ve accepted it on faith that solar panel performance degrades about 2% in the first year and then 1% every year thereafter. Today I had a chance to test that.

Yesterday afternoon I hauled 6 new solar panels up onto my roof and wired them up. Today they’re producing so I spot checked against the production of my existing 290W panels from 2016. The rule of thumb above plus math suggests that those 8 year old panels have degraded about 9% and likely produce about 290W * 91% = 264W. The new panels are rated at 405W so I’d expect each old panel to produce 65% as much as a new one.

Today is an overcast day, which may affect production differently on the new versus old panels, but when I average the production of 10 old panels to 210W and divide that by the average of the 6 new ones at 320W, I get 65.6%. I find it very satisfying when reality matches theory so closely.

What is Net Metering?

A question I’m often asked about my solar is, “how exactly does Net Metering work?” Here in the state of Washington, the rules for Net Metering are enshrined in RCW 80.60.030. Go forth and read, if you’re the nerdy type. For everyone else…

Every year on March 31, my Net Metering kWh balance resets to zero. When the sun shines I harvest kWh. I also run the heat pumps, toaster oven, and charge my phone and car, all of which consume kWh. If I consume more than I produce, I pay my utility (Seattle City Light) for the kWh they provide, just as everyone else does. If I produce more kWh than my house and car consume, then SCL adds those surplus kWh to my Net Metering balance.

In practice, each year in April I produce as much energy as I consume. From May to September I generate surpluses and build up kWh credits. As winter progresses my solar production falls off and I start depleting the kWh credits. In most years, I exhaust my credits by January and have to start buying kWh.

On March 31st, if there’s any kWh credit balance, it gets wiped away as a free gift to SCL. Therefore, under net metering it rarely makes economic sense to produce any more kWh than one consumes.

My average consumption for the past couple years has been about 2,000 kWh more than I produce, resulting in bills of about $400 per year. That is why I just spent $900 on 6 new 405W solar panels to grow my array. With a rated capacity of 2,400 kWh, I expect them to produce about 2,000 kWh per year, getting me very very close to my target of Net Zero energy.

A new roof and an SPD in every panel

In the summer of ’24 I replaced my North roof (project info). A complication was that the roof was under my solar array. The following graph explains another complication, the cause of my desire to add more solar panels:

The blue bar is my household energy (all electric) consumption and green is electricity to power our electric vehicles. In most years, I produce enough solar (yellow) to nearly power the house. In the last two years the house power consumption has been less and that extra solar was car fuel. I’ve calculated my Levelized Cost Of Electricity at $0.05/kWh. That’s a substantial discount from grid power so it’s advantageous to buy more panels and produce more fuel for the cars.

As part of the roofing project, I removed and rebuilt my solar array (details here) and upgraded two of my subpanels. What I didn’t know when I did the work in the summer of 2024, was that the soon-to-be-adopted 2023 electrical code requires that all electrical panels have a SPD (Surge Protective Device). By the time I scheduled my final inspection, the newer code was in force and that was the only correction Inspector Friendly required of me.

Despite not being required (grandfathering), I had already installed a SPD in the main panel, since it seemed to me like really cheap insurance. I had also installed a Midnight Solar (MNSPD-300-AC) SPD in my PV combiner box, so now I have 4 SPDs. It feels a tiny bit excessive, until I think about how much it would cost to replace a panel full of AFCI/GFCI combo breakers.

PSA: crypto & pig butchering

First they befriend you online. They provide an extraordinary level of online attention and [usually] romantic interest. They casually “let slip” how much money they’re making from their crypto “investments.” They encourage you to also make money investing in crypto. After investing a little bit to test the waters, you earn unbelievably good returns. They have fattened up your confidence with time, a romantic relationship, and investment gains. After you invest more, they ghost you and both your money and relationship is gone.

The “pig” in this story is the victim and the butcher is a Chinese captive working in a forced labor camp in Myanmar or Cambodia. These scams particularly target the elderly, so do your part and raise awareness with those you love.

Lectric XP brake upgrade

I have a Lectric XP 2.0 e-bike I purchased in 2022. I ordered it with the cargo package (front and rear racks & baskets), intending to use it to reduce car trips for local errands and grocery shopping. Without any planning I can easily carry home two grocery bags on it. With a couple bungee cords, I can carry home a fair bit more.

The rather basic single-piston brakes with 160mm rotors that came with it are good enough to stop the bike and I on most terrain. However, I live in Seattle and we have good sized hills out here. When riding down a hill with a load of groceries, a fair degree of planning is required to bring the bike to a stop on time. It seems Lectric was made aware of this deficiency because the XP 3.0 has upgraded the brakes to 180mm rotors and hydraulic brakes.

Today I upgraded my XP 2.0 with a TEKTRO 203mm rotor on the front and I replaced the Zoom single-piston calipers with XTECH HB-100 hydraulic dual-piston calipers. The improvement in stopping power was immediately noticeable and I’m looking forward to testing my next grocery run.

PSA: shun TurboTax and H&R Block

Since I wrote in 2019, the biggest tax prep companies continue to spend tens of millions to prevent making tax filing simple and easy. They’ve been fined for bait-and-switch, hiding the Free File service they were paid to offer, and overcharging Free File eligible taxpayers. Instead of giving Intuit another nickel, use Cash.app taxes, a local preparer, or file paper returns.

4 Gallons

At my most recent donation, Bloodworks NW informed me I’ve now donated 4 gallons of blood. They tell me, “each gallon pin represents another eight lives saved.” I tried to corroborate that statistic and the nearest I found was from the American Red Cross who say that a single donation (1 pint) can save up to 3 lives. Each gallon is 8 pints (units).

It would be fun to know how much I’ve donated in total. I previously donated at Carter Bloodcare when I lived in Texas and the American Red Cross when I lived in Michigan.

Compared to rushing into burning buildings, donating blood is an easy and reliable way to save lives. You should consider joining us 2% of Americans who do so.

Goat Rocks Wilderness

At 5AM on Friday morning, we set out for the Snowgrass Hikers Trailhead. All proceeded according to plan until 26 miles off the paved road, a sharp rock tore a 1.5″ gash through one of my tires. I used my tire repair kit to insert 5 plugs, greatly slowing down the air leak. That was sufficient to hold air for about 10 miles, so we turned around and headed back towards civilization.

Upon hitting blacktop, we also got cell phone coverage. I called Tesla service and they would flatbed my car back to Fife, WA and replace the tire. Expecting that to take about 5 hours, I declined to explore my options. The two nearby rural tire shops had nothing in the correct size, so we headed towards I-5 and started calling every shop between us and Chehalis.

Les Schwab in Chehalis claimed to have a matching tire in stock. When we arrived, they had a Continental in 245/55R19, but it was the wrong series. They insisted they could only replace all 4 tires. My tires are nearly new, no thanks guys. Another call and Discount Tire in Lacy (27 miles North) had the correct tire, series, and size, so off we went again. The plugs held, Discount Tire replaced the tire, and shortly after lunch, we once again had 4 good tires.

For the second time, we set off for the Snowgrass Hikers Trailhead. As we got into the I-5 freeway, a car 5 car lengths ahead of us threw up something which left a quarter sized chip in my windshield. We pressed on and arrived at dusk. We camped overnight in Motel Tesla.

On Saturday morning we hiked over to Berry Patch Trailhead and headed North. We took the optional scenic detour to Goat Ridge and found the lookout is now ruins. We continued North and took the Lily Basin Trail #86. We stashed our packs and scooted out to Johnson Peak (7,257′) and tagged the summit at noon.

We descended almost bee-lining towards Heart Lake (a better route than our ascent), returned to our packs, and then visited Goat Lake. The camp sites were all full, the lake shore was crowded, but the lake was empty. I dove in and found out why: it was still ice-cold. We filtered drinking water, ate our first dinner, dried off in the sun, and then pressed onwards.

About a mile past Goat Lake we found an empty campground and set up for the evening. It was a lovely evening, cool and brisk and we slept well under my MSR tarp tent. In the morning we set our sights on the summits of Old Snowy Mountain (7,900′) and Ives Peak (7,920′).

Based on our previous days experience and the forecast, we again left our packs behind and headed for the summit of Old Snowy with just an extra layer and water bottles. Oops. As we ascended the wind picked up and the exposure made us regret the choice. To keep warm we picked up the pace and reached the summit before 10AM. We did not loiter and scurried right back down.

50° with 30-40 mph winds. Brrrr…

Being separated from our layers prevented us from hiking the ridge over to Ives Peak. We descended on the PCT, retrieved our packs, and then hiked out Snowgrass Trail #96 to the trailhead and our car.

haproxy dynamic TLS reloads

Haproxy has a great newer feature that lets one dynamically reload TLS certificates. I explored this today because I’ve had two instances in the past few months where haproxy stopped serving for time, at midnight when the cron job that renews TLS certs kicks off. I think it’s an edge case involving web sockets with a 1 hour timeout and a handful of TLS certs all renewing in close succession. Regardless, not having to reload haproxy at all sounded attractive.

The page above has this example for sending the certificate to haproxy’s admin API:

echo -e "set ssl cert /etc/haproxy/certs/site.pem <<\n$(cat ./new_certificate.pem)\n" | socat tcp-connect:172.25.0.10:9999 -

After exploring the vagaries of echo and echo -e and staring at the output for a few too many times, I finally determined the cause of the failure. The certificate I was attempting to send has a stray newline character. ??‍♂️ The solution is simple, assure your boundary character doesn’t match the data:

echo -e "set ssl cert /etc/haproxy/certs/site.pem <<\n$(grep . ./new_certificate.pem)\n" | socat tcp-connect:172.25.0.10:9999 -

FreeBSD portsnap vs git clone

FreeBSD is planning to deprecate portsnap in favor of git or svnlite repos. As a heavy git user, part of me thinks, “about time!” There’s a number of discussions about the change on the email lists and the FreeBSD forums. What I wanted is some data on how the change will affect daily use. Should I use a deep or shallow clone? Which git repo should I follow? So I ran a few tests:

portsnapgit clone (freebsd)git clone (github)git clone (freebsd, shallow)
disk used (M)8011,9602,940888
compressratio2.231.221.111.83
checkout time (real)3m 36s41m 49s7m 11s3m 12s
Disk Usage and Time Required to Download

This is a FreeBSD 13.2 host with /usr/ports on a ZFS filesystem with lz4 compression.

# zfs get compression zroot/usr/ports
zroot/usr/ports  compression  lz4

# rm -rf /usr/ports/* && rm -rf /usr/ports/.* && rm -rf /var/db/portsnap/*
# time portsnap fetch extract
Looking up portsnap.FreeBSD.org mirrors... 5 mirrors found.
Fetching public key from dualstack.aws.portsnap.freebsd.org... done.
Fetching snapshot tag from dualstack.aws.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Fetching snapshot generated at Sun May  7 17:30:05 PDT 2023:
aa65708d65765ca77e1756616d249f3512ebb192c1ef16         102 MB 8131 kBps    13s
Extracting snapshot... done.
Verifying snapshot integrity... done.
Fetching snapshot tag from dualstack.aws.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Sun May  7 17:30:05 PDT 2023 to Sun May  7 18:18:52 PDT 2023.
Fetching 5 metadata patches... done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
<snip thousands of lines>

# zfs list zroot/usr/ports && zfs get compressratio zroot/usr/ports
NAME              USED  AVAIL     REFER  MOUNTPOINT
zroot/usr/ports   801M  6.96G      813M  /usr/ports
zroot/usr/ports  compressratio  2.23x

# rm -rf /usr/ports/* && rm -rf /usr/ports/.*
# time git clone https://git.freebsd.org/ports.git /usr/ports
Cloning into '/usr/ports'...
remote: Enumerating objects: 5821903, done.
remote: Counting objects: 100% (942/942), done.
remote: Compressing objects: 100% (126/126), done.
remote: Total 5821903 (delta 923), reused 816 (delta 816), pack-reused 5820961
Receiving objects: 100% (5821903/5821903), 1.06 GiB | 488.00 KiB/s, done.
Resolving deltas: 100% (3512457/3512457), done.
Updating files: 100% (157219/157219), done.

real    41m48.997s
user    6m47.535s
sys 1m8.458s

# zfs list zroot/usr/ports && zfs get compressratio zroot/usr/ports
zroot/usr/ports  1.96G  5.80G     1.96G  /usr/ports
zroot/usr/ports  compressratio  1.22x