haproxy dynamic TLS reloads

Haproxy has a great newer feature that lets one dynamically reload TLS certificates. I explored this today because I’ve had two instances in the past few months where haproxy stopped serving for time, at midnight when the cron job that renews TLS certs kicks off. I think it’s an edge case involving web sockets with a 1 hour timeout and a handful of TLS certs all renewing in close succession. Regardless, not having to reload haproxy at all sounded attractive.

The page above has this example for sending the certificate to haproxy’s admin API:

echo -e "set ssl cert /etc/haproxy/certs/site.pem <<\n$(cat ./new_certificate.pem)\n" | socat tcp-connect: -

After exploring the vagaries of echo and echo -e and staring at the output for a few too many times, I finally determined the cause of the failure. The certificate I was attempting to send has a stray newline character. 🤦🏻‍♂️ The solution is simple, assure your boundary character doesn’t match the data:

echo -e "set ssl cert /etc/haproxy/certs/site.pem <<\n$(grep . ./new_certificate.pem)\n" | socat tcp-connect: -

FreeBSD portsnap vs git clone

FreeBSD is planning to deprecate portsnap in favor of git or svnlite repos. As a heavy git user, part of me thinks, “about time!” There’s a number of discussions about the change on the email lists and the FreeBSD forums. What I wanted is some data on how the change will affect daily use. Should I use a deep or shallow clone? Which git repo should I follow? So I ran a few tests:

portsnapgit clone (freebsd)git clone (github)git clone (freebsd, shallow)
disk used (M)8011,9602,940888
checkout time (real)3m 36s41m 49s7m 11s3m 12s
Disk Usage and Time Required to Download

This is a FreeBSD 13.2 host with /usr/ports on a ZFS filesystem with lz4 compression.

# zfs get compression zroot/usr/ports
zroot/usr/ports  compression  lz4

# rm -rf /usr/ports/* && rm -rf /usr/ports/.* && rm -rf /var/db/portsnap/*
# time portsnap fetch extract
Looking up portsnap.FreeBSD.org mirrors... 5 mirrors found.
Fetching public key from dualstack.aws.portsnap.freebsd.org... done.
Fetching snapshot tag from dualstack.aws.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Fetching snapshot generated at Sun May  7 17:30:05 PDT 2023:
aa65708d65765ca77e1756616d249f3512ebb192c1ef16         102 MB 8131 kBps    13s
Extracting snapshot... done.
Verifying snapshot integrity... done.
Fetching snapshot tag from dualstack.aws.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Sun May  7 17:30:05 PDT 2023 to Sun May  7 18:18:52 PDT 2023.
Fetching 5 metadata patches... done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
<snip thousands of lines>

# zfs list zroot/usr/ports && zfs get compressratio zroot/usr/ports
zroot/usr/ports   801M  6.96G      813M  /usr/ports
zroot/usr/ports  compressratio  2.23x

# rm -rf /usr/ports/* && rm -rf /usr/ports/.*
# time git clone https://git.freebsd.org/ports.git /usr/ports
Cloning into '/usr/ports'...
remote: Enumerating objects: 5821903, done.
remote: Counting objects: 100% (942/942), done.
remote: Compressing objects: 100% (126/126), done.
remote: Total 5821903 (delta 923), reused 816 (delta 816), pack-reused 5820961
Receiving objects: 100% (5821903/5821903), 1.06 GiB | 488.00 KiB/s, done.
Resolving deltas: 100% (3512457/3512457), done.
Updating files: 100% (157219/157219), done.

real    41m48.997s
user    6m47.535s
sys 1m8.458s

# zfs list zroot/usr/ports && zfs get compressratio zroot/usr/ports
zroot/usr/ports  1.96G  5.80G     1.96G  /usr/ports
zroot/usr/ports  compressratio  1.22x

Eternal Glaciers

The mountains of the Pacific Northwest are tangled, wild, remote, and high. They have the roar of torrents and avalanches in their throats.

Rock cliffs such as Kloochman rise as straight in the air as the Washington Monument and two or three times as high. Snow-capped peaks with aprons of eternal glaciers command the skyline–giant sentinels 11,000, 12,000, 14,000 feet high, such as Hood, Adams, and Rainier.

There are no slow-moving, sluggish rivers in these mountains. The streams run clear, cold, and fast.

Of Men and Mountains, Foreword, ix

Wouldn’t the authors be sad to learn that those eternal glaciers aren’t?

Marker Alpinist and Leashes

I’ve been using B&D ski leashes since Art Freeman recommended them several years ago. They’re a superb alternative to the itty bitty short leashes the binding makers sell. I managed to go several years without breaking a fuse link, and this year I’ve already managed to break two.

Although mine seem to have failed during traveling near other pairs of skis, the fuses are designed to break during spectacular yard sales and to prevent skis from being avalanche burial anchors. Bypassing the fuses is suicidal, touring without a leash isn’t awesome, and so I searched for an improvement.

Marker doesn’t document the feature but my Alpinist bindings have two itty bitty holes for attaching leash cords. I found an online comment telling me where to find them so I tossed my skis onto the bench and gave the bindings a closer look.

The active mount has the black cord included with the leash. I populated the second hole with orange 3mm accessory cord and a spare fuse. Next time a fuse breaks, I can just pop the leash onto the spare.


Solar Shenanigans

In early November I flew to Dallas and helped my buddy Ryan install a solar array. Since he had a large low-slope metal roof on his garage, we decided that was the right place to put them. Ryan’s dad also helped and in two days, the three of us had hoisted three large stacks of panels up onto the roof, bolted them down, and wired them all together.

Then we pulled the feed wires onto the roof, hooked up the three strings and then spent a bunch of time futzing with the inverters, replacing optimizers, and updating the locations of panels whose optimizers had invalid serial numbers on them (pro tip, check them all on the ground beforehand). Then we hooked up his eGauge meter so he can monitor total production and consumption.

On day 3, the inspector showed up, checked a few things, asked some questions, and then approved the final inspection. Since then, Ryan’s solar array had been making me jealous. Between the much higher solar factor in Dallas and having more panels, his winter production is ~10x mine.

e-cells e-bike brakes

In July of 2020, we purchased a 600 watt dual-motor AWD fat-tire e-bike from ecells.com. The bike has a motor in each wheel. The frame is super beefy and fairly heavy. It has been a hoot. Mostly we ride it on paved urban trails here in town, where the beefy frame and rack lets it excel at hauling home groceries. It’s more at home out at Meany Lodge where we ride it up and down forest roads in the mountains where the low-pressure fat tires provide abundant traction and good suspension. It can really haul on the loose gravel roads.

Last weekend I took it on a ride near Mailbox Peak with a group of friends. The bike did quite well at helping me ride up the mountain like I was 20 years younger and 20 pounds lighter. Where it wasn’t so awesome was blasting down the no-longer-maintained-and-sometimes-washed-out logging roads. I wanted to downhill hard and fast, like on my still-awesome Raleigh M-800 mountain bike. The E-cells brakes need to stop 70# of bike, 10# of gear, and me, while thrashing downhill at 30-35mph. I was experiencing significant brake fade and needing to plan my braking. The brakes lack authority. So I went shopping for upgrades and learned a few things.

Brake Pad Types

  1. metallic – longest life, greatest stopping power, more noisy
  2. organic/resin – quiet, good initial bite, glaze over / fade under heavy braking
  3. semi-metallic – combination of the two

The pads that came with my bike are Tektro A10.11, which is a sintered (semi-metallic) ceramic pad. That pad is no longer listed on the Tektro web site. The nearest OEM replacement is the E10.11 ($15), which is sintered organic. A higher performance metallic pad is the Tektro P20.11 ($24) which I have ordered. They provide a small boost in stopping power, but more importantly, they won’t fade under prolonged heavy braking.

Rotor Size

Bike rotor sizes start at 140mm and go up. The increased diameter of larger rotors provides more mechanical advantage so less friction is required to get the same stopping power. Larger rotors provide more thermal mass to absorb heat and more surface area to dissipate heat. Common e-bike rotor sizes are: 160, 180, and 203mm. Most e-bike forks are set up for 160mm rotors. Cheap ($10-15) adapters enable those forks to work with 180 and 203mm rotors.

The E-cells 600 comes with Tektro Aries mechanical disc brakes on 160mm rotors (front and rear) with adapters. Because the front wheel provides ~70% of the stopping power, it’s quite common to use larger front rotors: eg. 203mm on the front and 180mm on the rear. The higher spec E-cells 700 and 1000 models have exactly that setup with hydraulic calipers.

Disc Brake Types

• Mechanical disc brakes are inexpensive, reliable, and solidly better than rim and other brakes of yesteryear. Like legacy brake systems, they are cable actuated. They have a single moving piston which warps the brake disc into the other pad, compressing it and providing braking power.

• Hydraulic disc brakes replace the wire cable with hydraulic fluid (DOT or mineral oil) which provides equal force on two opposing pistons. The reduced friction and doubling of pistons provides more braking power with less effort. Testing (varies a LOT based on bike and system) shows a 40-70% reduction in braking distance with hydraulic disc brakes.

• Hydraulic 4-piston disc brakes are fairly new, fairly rare, and expensive. They are the go-to option for higher speed (22+ mph) and cargo eBikes. The piston engagement is progressive: you initially get two pistons braking and as the rider pulls harder, all 4 engage for massive stopping power. That much braking power would be dangerous on lighter bikes, but it’s needed for fast downhills on heavy bikes. 🚵 😁 😈 

• Hybrid: there exists a hybrid, the Juin Tech M1 cable actuated hydraulic brake. They’re intriguing, promising substantially better stopping power than mechanical disc brakes for a modest upgrade ($160) price and a very simple install. The only “not paid” review data I found is that they are an incremental improvement upon mechanical disc brakes, not a step-change improvement like going from mechanical to hydraulic.

e-bike brakes

Another layer of complexity added to e-bike brakes is that the brake levers need cutout switches that disengage the motor when braking. The vast majority of bicycle brake systems don’t have this feature.

Combine the newness of 4-piston brakes with the much smaller ecosystem of brake levers with cutoffs for e-bikes and the choices get very narrow. As in, the full list is: Tektro E-725, Magura MT-5, Magura MT-7. I opted for the Tektro because the Magura’s have plastic fluid reservoirs.

The switch to metallic pads and 4-piston brakes should suffice. If not, my next move will be upsizing the front rotor to 203mm, for another ~12% increase in braking power.

2022 Patrol Race, 9h:46

Shane and Doug, on a training day

Last weekend was the annual 20-mile backcountry ski race that I’ve been training for since December. We start at Snoqualmie Pass and follow the Pacific Crest Trail through the backcountry for 20 miles, ending just past Stampede Pass at Meany Lodge. I raced this for the first time in 2020, when Dan, Greg, and I completed the race in 12 hours and 13 minutes.

I raced it again in 2021, substituting Maud for Dan, after Dan suffered a ski injury. Our team, Mock Speed didn’t get an official time because a mile into the course, Greg’s binding ripped off his ski. Since it’s a team race, everyone has to finish. After a failed repair attempt, we escorted Greg to a bail out point and Maud and I finished the course in about 10 hours.

This year I recruited a couple “kids” my age (Greg is 10 years my senior and Dan is 20) to join Quaranteam: Smells Like Strong Spirits. My goal for Doug, Shane, and myself was to best the fastest Meany teams time of 10h:32m. This year we had great weather, with a fresh dump of mid-week snow and several days of consolidation (reducing avalanche risk) leading up to Saturday. We started skiing in the rain which transitioned to gentle snow as the day cooled. We had great weather, a great time, and crossed the finish line in 9h:46m.

2021 Michigan Road Trip

This summer we drove to Michigan to visit my dad and celebrate his 75th birthday. Because I’m still playing the Superchargers Visited game (2019 trip, 2020 trip), I had to choose routes that didn’t overlap with previous ones. Fortunately, that left lots of fresh ground to cover and we spent a lot more of this trip on highways and less time on the interstates.

The slower pace and smaller roads made road tripping more fun as I got to experience that joy of discovery that is rare on freeways. Sometimes the discoveries are, “huh, this route through Kansas is every bit as interesting as every other route through Kansas.” Other times, like driving through northern Wisconsin and Indiana, we stumbled through some lovely little towns and cities that were well kept and seem to have avoided the fate of so many rustbelt towns.

This trips haul was 86 unique superchargers visited over 9 days of driving.