Electronics My home solar energy generation and control system

[nsaspook]

TL;DR Summary

Solar energy battery hybrid, control system with home assistant

Still working on some aspects of this DIY home energy management system but all of the major parts are operational.
I'm using Home Assistant as the interface to several custom designed PIC 8-bit controllers and a Linux expert system to integrate data into a central MQTT server. It monitors utility energy flows from the utility and locally generated electrical energy into the house to reduce external usage requirements for HVAC, computer processing and general power usage.

Most the the individual devices controlled and monitored in the house use Matter/thread as the connection fabric. Matter version 1.3 has been a great improvement over the original Matter 1.0 devices. Many of the original devices can get firmware upgraded to 1.3. This system is totally local for home device control, with no cloud control or monitoring of the energy devices with the exception of Home Assistant cloud for remote access and control. The cloud access in not needed but it is nice to see what's happening at home from anywhere in the world, securely.

Most of the original build and design is here: https://forum.allaboutcircuits.com/...ge-controller-datalogger.194146/#post-1826013

A few screen shots of the HA main control panel in my home electrical shop.

The Linux C energy control system logger.

Some of the MQTT JSON formatted data sent to the server from various energy interface device PIC18 controllers.

 

[nsaspook]

The system also integrates the home automation and media with local and cloud access to the home AV server, security IP cameras and outdoor garden plant monitors.

The Home Assistant Iphone APP running on the house local wireless network.

One last thing I'm playing with now is voice control using the Home Assistant Voice Preview Edition device.
https://www.home-assistant.io/voice-pe/

We will match and then surpass other voice assistants. Supporting languages that big tech ignores - while being private, open, and fully customizable. Running inexpensively on local hardware.

The voice device and a remote energy monitor (PIC18 Q84 design) running on the home CANBUS network.

Building, testing the PCB and embedded software for the remote energy displays.

All of the voice processing is local, no need for cloud server access (optional cloud processing in included with a sub to the HA cloud server, but I don't use it for that) for Speech-to-text. It's local, using faster-whisper processing on the home server and the internal FPGA on the Voice Preview Edition device.
https://github.com/SYSTRAN/faster-whisper

 

[Rive]

....really well done

Though - based on my personal experiences - at this point you better start to make proper documentation too.
Not much later that these homebrew do-it-all projects reaches the point where they really can do it all and no interaction is needed anymore the owner tends to forget the fine details what keep it tickin'

 

[nsaspook]

....really well done

Though - based on my personal experiences - at this point you better start to make proper documentation too.
Not much later that these homebrew do-it-all projects reaches the point where they really can do it all and no interaction is needed anymore the owner tends to forget the fine details what keep it tickin'

I'm professional electrical engineer with decades of experience building life-safety systems but thanks for the advice about documentation. This is mainly designed first in my head, documented for the build process and corrected with as-built corrections. I know the forgetting process on complicated projects. The entire project is backed up on archive servers (one remote from the main building) in the house and online with details of the software (documented using Doxygen), hardware flowcharts for interconnects, wiring diagrams, from the PCB BOM to the embedded software to C source in great detail. GIT is used to maintain version control of all documentation and software.

Some early design charts.

You're right, this is about the time to update the design database with the latest changes while they are still fresh in the head.

I've got the master book and two grown kids with CS credentials to take over, Right.

 

[nsaspook]

The complex part of this project is data collection from many devices, all with completely different interfaces and protocols. Lot's of that required reverse-engineering weird, strange, odd-ball data formats, encoding methods, non-standard physical interfaces. All of that data is converted to JSON format for storage and distribution on the MQTT mosquito server process on the home computer center that's mainly powered by the solar energy control system.

Running mainly surplus HPE DL360 and DL380 enterprise servers.
https://www.hpe.com/psnow/doc/c04346229.pdf

Yes, it eats power and needs cooling but the energy control system can reduce that energy usage to near zero in full sun and reduce it by shifting saved battery energy directly into the server at night.

The first step was getting Solar energy data from the charge controller. FLEXmax Charge Controller is a classic and bulletproof design but has a proprietary data interface. I'm not the only person that's needed to hack the protocol so it was easy to find design information online.

They use a 9N1 proprietary interface protocol that's been reversed engineered several times.
Most of the protocol and interface adapter information came from here.
https://github.com/jorticus/pymate

That software was written in Python so I re-implemented it using XC8 for the PIC18 series of controllers so I can pull important system data for logging to the local system without the expensive and limited Outback MATE remote display that has an ASCII port.
https://www.outbackpower.com/downlo...ment/mate/mate_serial_communicationsguide.pdf
By connecting directly to the charge controller I have complete control and access to the device.

Next step is AC utility power monitoring. For that two devices were used. One device was RS-485 with good documentation, easy

and the other device is the IAMMETER 3 and 1 phase wireless systems. Very good. I use three of these on the current system. https://www.iammeter.com/
All processing and communications is local with these devices. No cloud access is needed.

So I designed a few prototype 8-bit controller circuit boards and wrote the software in C to integrate the data to a common interface using CANBUS as the backbone.

REV 1.1 PCB in black and gold, withFM80 interface, TTL serial, CANBUS and MODBUS interface channels with SPI LCD display capability.

Testing a two board network. LCD screen SCM: FM80 in Sleep mode, Canbus online, Modbus online.

Github software mateQ84 branch for those types of boards: https://github.com/nsaspook/Q84vtouch/tree/mateq84

Source code, schematic designs in Eagle/Fusion360 with documentation.
https://github.com/nsaspook/Q84vtouch/blob/mateq84/mateQ84.X/mate_a_sch.pdf
Source code example in C for the 8-bit controller. https://github.com/nsaspook/Q84vtouch/blob/mateq84/mateQ84.X/main.c

 

[nsaspook]

This is the last major sub-system. Called the Dump Load controller. This provides for AC-AC hybrid power coupling and energy time-shifting.

It's basically a completely dependent Solar energy system with charge controller, with a separate 5120 Wh battery but without solar panels. The power is supplied by a 1KW switching power supply that gets AC from an off-grid inverter (that also supplies computer rack power) powered from the separate solar panel charged battery 5120 Wh battery.

This is the dump load system controller. Two 8-bit controllers. One controller, a PIC18F1320, talks to the charge controller via MODBUS and shares that data with the main energy controller, a PIC18F47K42.

The main controller monitors the state of charge of its battery and receives/sends control data and information from/to the Linux energy master controller using the system standard MQTT JSON data via Ethernet using a serial port interface.

The main Linux controller looks at the total energy balance between both systems, the incoming solar power, the outgoing electrical energy, the expected total energy collection for that day and decides what to do. First priority is to top off the main energy battery from solar energy, when that's done, energy is directed via the switching PS to the dump load battery. Once those systems are at the calculated needed set-points for the day, solar energy is directed to grid tie inverters to feed AC power into the home.

The home energy flow levels are calculated using standard PID control feedback to keep a slightly negative net balance at full solar power generation, so the batteries are slowly drained until the energy set-points for later power shifting.

Early morning, all power to recharging batteries, positive energy balance. 5120Wh LFP batteries for 5000 to 10000 cycle charge/discharge lifetimes.

Battery near full energy balance.

When the sun sets, the dump load controller is commanded by by the Linux program to return today's stored energy in it's battery pack to the house and the main solar battery is kept in reserve set-point unless the energy prediction (using a Home Assistant solar plugin https://www.home-assistant.io/integrations/forecast_solar/ ) from the next day forecasts we can also use it to dump energy into the house at via it's own grid-tie inverter system to the energy shift set-point.

 

[nsaspook]

Software documentation uses Doxygen. It makes the source code also the source of the software documentation.
https://www.doxygen.nl/

The Linux control program documentation PDF: https://github.com/nsaspook/ha_energy/blob/fsm_main/bmc/latex/refman.pdf
You will need to download the file if you want to use the pdf file links.

Example screenshots of the code documentation pdf.

 

[nsaspook]

This is the granddaddy of the current system from 15 years ago. Mainly from surplus parts, left-overs from work projects and decommissioned equipment.

Then

Now
https://forum.allaboutcircuits.com/...ic-controlled-battery-array.32879/post-204585
https://github.com/nsaspook/mbmc/tree/new_solar
It's still running as a Zombie Attack backup power system. Uses a 12 VDC battery bank to be compatible with a car during emergencies, with a 2 KW off-grid inverter that's kept the forced air gas heat AC motor blower running in winter during a few blackouts in ice-storms.

The current system switching energy modes.

 

[nsaspook]

New DAQ board design for a remote panel energy monitor. A 8-bit pic controller for a Orange pi 3 Linux using that machines SPI Master.

 

[nsaspook]

I have the beta custom DAQ board for the Orange PI in operational testing. 8 24vdc, 2 5vdc digital inputs. 16, up to 24vdc low side switches, 7 12-bit ADC inputs and 1 8-bit DAC output. The analog and connections to the OPi3 are handled by a PIC18F47Q84 controller firmware. SPI2 is the SPI slave for the OPi3 Master controller. SPI1 handles the display and the two DIO devices that transfer data via SPI2 per Master requests for status and changes.
On the Orange PI and custom kernel driver was designed (still needs optimization to use CPU threads effectively) to interface with the Comedi DAQ API. This allows for specific hardware device functions to be abstracted into a common user API so different hardware DAQ frontends can be used with the same applications software.

https://github.com/nsaspook/mqtt_comedi/blob/main/secs_q84 v2_sch.pdf
https://github.com/nsaspook/mqtt_comedi/blob/main/secs_q84 v2_brd.pdf

Early Beta software: https://github.com/nsaspook/mqtt_comedi
Applications program: https://github.com/nsaspook/mqtt_comedi/tree/main/bmc
Q84 firmware: https://github.com/nsaspook/mqtt_comedi/tree/main/Q84/bmc_slave.X
Linux kernel protocol driver: https://github.com/nsaspook/mqtt_comedi/blob/main/daq_bmc/daq_bmc.c
Requires Linux kernel patches to add the needed device-tree and Kconfig to the new user protocol driver source into the standard kernel build process. Then add the new protocol driver spi_bmc instead of the system standard spi-dev in the Orange PI
device tree-config file.
Boot config file: https://github.com/nsaspook/mqtt_comedi/blob/main/daq_bmc/orangepiEnv.txt
Device-tree source file: https://github.com/nsaspook/mqtt_comedi/blob/main/daq_bmc/sun50i-h616-spi1-spibmc.dts

BMC1 MQTT data. The ADC channel 0 and 1 are the raw voltage reading on the ADC pins for the RS232 line voltages (that have been normalized and scaled on the Q84 display to the +- 15 volts range)

DAQ testing RS232 raw ADC values displayed on the Home Assistant Shed PV voltage cards. The Sys voltages are live system values.

Second line: The first number is the error count, second ADC conversions, third master data packet requests.
Third line: ADC channel 1 raw value, ADC channel 2 raw value.
Last Line: SPI2TCNT, SPI2INTF SPI2 registers and program state flags. The number 1, on the end means, spi_comm_ss.REMOTE_LINK is true.

RS-232 Jack status. Voltage and Mark/Space or Open line condition.

Last line: Input line buffer from the SPI2 and UART links.

data_in2 = SPI2RXB;
serial_buffer_ss.data[0] = data_in2;
char_rxtmp = UART1_Read();
serial_buffer_ss.data[1] = char_rxtmp;

Line data scope shots:

MOSI -> MISO ADC data transactions.

One ADC value request transaction.

Q84, all channels ADC conversion and buffered processing time.

Q84 ADC all channel update rate with the display update gap timing. We only monitoring slowly changing DC signals for this applications, so the variation in sampling rate is not important.