> Home Assistant already knows which bins (trash cans, for the international readers) are due for collection. [...] What it can’t tell me is whether I’ve actually done anything about it.
My first gut reaction to this was: Have you tried using a vision-language model?
I used to have an elaborate Home Assistant setup with tons of zigbee sensors here and there telling me if front door is not closed properly, if the rice cooker is left on (via current monitoring), if the garage door is left open (with a magnetic contact sensor), if window is left open, if the soil is dry, etc. I had to keep changing batteries of all these things and manage flaky Zigbee connections in the past.
More recently, I've found the absolute most reliable, ~zero-maintainence solution to be an LLM. Home Assistant already has an LLM plugin, you can wire it up to a local LLM if you want, and you can just feed it an image and ask "Is this garage door open or closed" and I found that to be much better for my sanity than changing batteries every 6 months and dealing with flaky sensors sending me false alerts.
I even found it useful for "Did the cat vomit on the floor" (if so notify me and pause the cleaning robot schedule) and "Do any plants look like they need attention"
For TFA's use case, if you already have a security camera facing front, you can just feed it that image and ask "Are the garbage bins out"
Also, if you're worried about cameras sending home images to the cloud, there are non-cloud options. (I use Unifi cameras and Home Assistant gets a RTSP stream over LAN.)
> how often do you have to change your cameras' batteries, or are you accepting ugly wiring for powering from mains
Meh (a) I want security and cat-monitoring cameras anyway (b) I'm willing to take the effort to install cameras neatly since they're capable of serving as 100 different sensors
> constantly sending images of one's private space to a cloud LLM is Orwellian
I agree. I have a local server with a GPU. The security side of things records to a local NVR. Nothing goes to the cloud.
> IP cameras are flaky, too
haven't had nearly as many problems here
> are you polling your LLM minutewise "Did the cat vomit on the floor"?
Kind of, but it only re-polls if the camera detected motion, and then every 15 minutes as a catch-all.
Hmmmm. Idt UWB needed for this. Just good old Bluetooth beacons/rssi. Oh and I think BTLE 5.6 or ish actually has UWB like positioning built in down to centimetres AFAIK. And they're SUPER low power.
If the author is reading this, I did a ctrl-f in your writeup for "battery" and "life" and didn't find anything specific about how long the batteries on these last. Can you elaborate a bit more on the battery types in use and lifespan? Charging? Since use lithium? 18650? CR2032 or CR2477 size coin cells? Any other details would be very interesting to read.
My biggest barrier to doing something like thins in my home automation setups is that I already have an extensive schedule of things that need periodic disposable battery changes (like CR2477 cells in zigbee humidity/temperature sensors), and I'm really hesitant to add even more into my house. In addition to things like battery-backed smoke detectors in every bedroom, etc, which need periodic 9V replacements on a schedule.
In fact I've started looking at going the opposite direction, where I can run a cat5e cable from my utility room, down and across the ceiling of my basement, and come up in a location where I can install an 802.3af to arbitrary DC voltage DC-DC converter (3.3, 5VDC, 12VDC out, or similar) as a PoE splitter, to make some of these things not dependent on batteries. Same idea as how you can find 9V or AA battery to direct DC wiring "dummy battery" battery eliminators now. Obviously such an approach wouldn't work for garbage bins but that's how much I am trying to eliminate having dozens of batteries to replace, as they're already all over the place with window and door sensors.
You can also find variants of them that are microUSB or USB-C power out. Finding ones that are natively 3.3V or 5VDC out is more difficult and rare, it seems the most viable option if I do this is to buy a very tiny DC-DC converter that's 12VDC in and then 3.3 or 5VDC out, and glue/stick it to the poe splitter, perhaps in an enclosure of some sort. Then wires out to the battery dummy eliminator.
The thing above is a 'dumb' device so any monitoring of the power you would have to do from your 802.3af PoE switch, entirely depending on what model/capabilities your switch has for port status, wattage, etc.
edit: Google "USB AA battery eliminator" and you'll find products which have a USB-A male input, expect standard 5V USB input, and output either 1.5V, 3.0V or 4.5V or 6V (1S, 2S or 3S or 4S AA battery voltage). You could then plug one of those into the many 802.3af to USB-A female (5V2A) poe splitters which exist on the market. Example: https://www.amazon.com/Battery-Eliminator-Supply-Adapter-Rep...
I forgot to mention that in some cases if replacing a single AA battery as a 'battery eliminator' you would need native 1.5VDC out. Tiny 12V to 1.5V DC-DC converter do exist. Same idea for 12V to 9V battery. It looks like a fair number of the small ones I'm seeing on mouser and digikey are meant to be soldered onto a board as part of a larger project, but I don't see why they wouldn't work if soldered directly to a couple of wires coming from the 12V out of a poe splitter such as that.
That’s all well and good until the truck driver operating the mechanical arm drops your bin into the truck. Spoiler alert: the bin will not be fished out, it is now garbage.
So those tags are going to have to be a little cheaper before I’ll use them. :-)
Not the whole bin, but the operator managed to scrape off the hinged lid and both wheels from my recycling bin a few weeks back. Sent them photos and they sent me a new one.
Yeah, our city’s new garbage contractor is kinda crap. After three weeks of no new bin (“You’ll get a new one on Tuesday.” “Which Tuesday?”), I sent the city solid waste manager after them. Next day, new bin. :-)
Thanks. I regret not taking more pictures during the initial testing indoors. I should document these things better, I only think about writing them up when it’s too late
> Or the bin’s precise location - how many fixed reference points would I need to triangulate it on a 2D plane?
Back in 2019 I worked for a company (in San Diego) that added UWB-based positioning to the 7 train line in NYC.
The code I wrote ran on what was basically a combined Raspberry Pi 3 compute module connected to a UWB device installed into the train's bonnet which would ping UWB devices set along the track side to determine the position of the train.
In most situations I could get a unique position based on only two beacons, but I had the benefit of the trains being on one of a limited number of fixed tracks which severely limited where they could practically be in space... well, assuming there hadn't been a train derailment.
The system didn't work in a 2D plane, though, our surveyed track data and track side UWB positions were all 3D encoded as ECEF coordinates.
In some situations depending upon track layout you needed more than 2 beacons to determine the correct position if you were just sampling single points in isolation, but since you're working with a continuous stream of changed positions as the train ran along the track at predictable speeds, you could pretty easily figure out which of multiple possibilities it had to be based on what came before even if you had potential multi-positional results for the current sample.
Of course, we would use as many of the track side beacons as we could see to strengthen confidence in the positioning, and the beacons were placed so we could generally see a good 4-6 of them most of the time.
Snow was the biggest challenge for the system, in a heavy thick snow the UWB signals bounce all over the place and you get severe multipath effects that can make it pretty difficult to determine which is the true line of sight signal for your time of flight calculations.
Fun job, I had to get certified to be able to walk the NYC train tracks because we installed a lot of the track side beacons ourselves, and I spent a lot of late nights doing live testing runs on trains that were empty other than a couple of people from our company and some MTA workers assigned to our test trains.
> Home Assistant already knows which bins (trash cans, for the international readers) are due for collection. [...] What it can’t tell me is whether I’ve actually done anything about it.
My first gut reaction to this was: Have you tried using a vision-language model?
I used to have an elaborate Home Assistant setup with tons of zigbee sensors here and there telling me if front door is not closed properly, if the rice cooker is left on (via current monitoring), if the garage door is left open (with a magnetic contact sensor), if window is left open, if the soil is dry, etc. I had to keep changing batteries of all these things and manage flaky Zigbee connections in the past.
More recently, I've found the absolute most reliable, ~zero-maintainence solution to be an LLM. Home Assistant already has an LLM plugin, you can wire it up to a local LLM if you want, and you can just feed it an image and ask "Is this garage door open or closed" and I found that to be much better for my sanity than changing batteries every 6 months and dealing with flaky sensors sending me false alerts.
I even found it useful for "Did the cat vomit on the floor" (if so notify me and pause the cleaning robot schedule) and "Do any plants look like they need attention"
For TFA's use case, if you already have a security camera facing front, you can just feed it that image and ask "Are the garbage bins out"
Also, if you're worried about cameras sending home images to the cloud, there are non-cloud options. (I use Unifi cameras and Home Assistant gets a RTSP stream over LAN.)
Kinda liking this idea :)
I’ve not played with vision models much. I did some experiments with OpenCV and Processing many years ago, but times have changed.
I think you just wrote me next weekend project ;)
- I have to change batteries of zigbee devices maybe every three years or so
- how often do you have to change your cameras' batteries, or are you accepting ugly wiring for powering from mains?
- how many cameras do you have installed to cover your home?
- IP cameras are flaky, too
- constantly sending images of one's private space to a cloud LLM is Orwellian
- are you polling your LLM minutewise "Did the cat vomit on the floor"?
> how often do you have to change your cameras' batteries, or are you accepting ugly wiring for powering from mains
Meh (a) I want security and cat-monitoring cameras anyway (b) I'm willing to take the effort to install cameras neatly since they're capable of serving as 100 different sensors
> constantly sending images of one's private space to a cloud LLM is Orwellian
I agree. I have a local server with a GPU. The security side of things records to a local NVR. Nothing goes to the cloud.
> IP cameras are flaky, too
haven't had nearly as many problems here
> are you polling your LLM minutewise "Did the cat vomit on the floor"?
Kind of, but it only re-polls if the camera detected motion, and then every 15 minutes as a catch-all.
Hmmmm. Idt UWB needed for this. Just good old Bluetooth beacons/rssi. Oh and I think BTLE 5.6 or ish actually has UWB like positioning built in down to centimetres AFAIK. And they're SUPER low power.
If the author is reading this, I did a ctrl-f in your writeup for "battery" and "life" and didn't find anything specific about how long the batteries on these last. Can you elaborate a bit more on the battery types in use and lifespan? Charging? Since use lithium? 18650? CR2032 or CR2477 size coin cells? Any other details would be very interesting to read.
My biggest barrier to doing something like thins in my home automation setups is that I already have an extensive schedule of things that need periodic disposable battery changes (like CR2477 cells in zigbee humidity/temperature sensors), and I'm really hesitant to add even more into my house. In addition to things like battery-backed smoke detectors in every bedroom, etc, which need periodic 9V replacements on a schedule.
In fact I've started looking at going the opposite direction, where I can run a cat5e cable from my utility room, down and across the ceiling of my basement, and come up in a location where I can install an 802.3af to arbitrary DC voltage DC-DC converter (3.3, 5VDC, 12VDC out, or similar) as a PoE splitter, to make some of these things not dependent on batteries. Same idea as how you can find 9V or AA battery to direct DC wiring "dummy battery" battery eliminators now. Obviously such an approach wouldn't work for garbage bins but that's how much I am trying to eliminate having dozens of batteries to replace, as they're already all over the place with window and door sensors.
What hardware do you recommend for 802.3af to arbitrary DC power? Granular configuration and remote monitoring are considerations in my use cases.
These same ones from China are sold under like 40 different brand names:
https://www.amazon.com/Splitter-Compliant-Surveillance-5-5x2...?
You can also find variants of them that are microUSB or USB-C power out. Finding ones that are natively 3.3V or 5VDC out is more difficult and rare, it seems the most viable option if I do this is to buy a very tiny DC-DC converter that's 12VDC in and then 3.3 or 5VDC out, and glue/stick it to the poe splitter, perhaps in an enclosure of some sort. Then wires out to the battery dummy eliminator.
The thing above is a 'dumb' device so any monitoring of the power you would have to do from your 802.3af PoE switch, entirely depending on what model/capabilities your switch has for port status, wattage, etc.
edit: Google "USB AA battery eliminator" and you'll find products which have a USB-A male input, expect standard 5V USB input, and output either 1.5V, 3.0V or 4.5V or 6V (1S, 2S or 3S or 4S AA battery voltage). You could then plug one of those into the many 802.3af to USB-A female (5V2A) poe splitters which exist on the market. Example: https://www.amazon.com/Battery-Eliminator-Supply-Adapter-Rep...
Many thanks.
I forgot to mention that in some cases if replacing a single AA battery as a 'battery eliminator' you would need native 1.5VDC out. Tiny 12V to 1.5V DC-DC converter do exist. Same idea for 12V to 9V battery. It looks like a fair number of the small ones I'm seeing on mouser and digikey are meant to be soldered onto a board as part of a larger project, but I don't see why they wouldn't work if soldered directly to a couple of wires coming from the 12V out of a poe splitter such as that.
How much of this was written with/edited by an LLM? Some parts of it feel somewhat Claude-ish, and I’m very curious if that’s a false positive.
Super cool work! I had no idea UWB tags had gotten so cheap, or that CR2077 was a size.
I don't think CR2077 is a size, it may be a typo for CR2477 which is very common?
CR2477 is high enough capacity that it's used in a number of zigbee or zwave800 based products that have no wired interface.
https://www.google.com/search?client=firefox-b-d&q=cr2477+ba...
https://aichiplink.com/blog/CR-Battery-Sizes-and-Specificati...
That’s all well and good until the truck driver operating the mechanical arm drops your bin into the truck. Spoiler alert: the bin will not be fished out, it is now garbage.
So those tags are going to have to be a little cheaper before I’ll use them. :-)
I don’t think I’ve ever heard of that happening to someone!
Luckily these are only £5 each https://www.alibaba.com/x/1lBJtin?ck=pdp
The majority of cost is the wrover and the programming jig (which I think the manufacturer built to order, it’s very… home brew
Not the whole bin, but the operator managed to scrape off the hinged lid and both wheels from my recycling bin a few weeks back. Sent them photos and they sent me a new one.
Yeah, our city’s new garbage contractor is kinda crap. After three weeks of no new bin (“You’ll get a new one on Tuesday.” “Which Tuesday?”), I sent the city solid waste manager after them. Next day, new bin. :-)
Enjoyed this, thank you for writing it up and posting
Thanks. I regret not taking more pictures during the initial testing indoors. I should document these things better, I only think about writing them up when it’s too late
> Or the bin’s precise location - how many fixed reference points would I need to triangulate it on a 2D plane?
Back in 2019 I worked for a company (in San Diego) that added UWB-based positioning to the 7 train line in NYC.
The code I wrote ran on what was basically a combined Raspberry Pi 3 compute module connected to a UWB device installed into the train's bonnet which would ping UWB devices set along the track side to determine the position of the train.
In most situations I could get a unique position based on only two beacons, but I had the benefit of the trains being on one of a limited number of fixed tracks which severely limited where they could practically be in space... well, assuming there hadn't been a train derailment.
The system didn't work in a 2D plane, though, our surveyed track data and track side UWB positions were all 3D encoded as ECEF coordinates.
In some situations depending upon track layout you needed more than 2 beacons to determine the correct position if you were just sampling single points in isolation, but since you're working with a continuous stream of changed positions as the train ran along the track at predictable speeds, you could pretty easily figure out which of multiple possibilities it had to be based on what came before even if you had potential multi-positional results for the current sample.
Of course, we would use as many of the track side beacons as we could see to strengthen confidence in the positioning, and the beacons were placed so we could generally see a good 4-6 of them most of the time.
Snow was the biggest challenge for the system, in a heavy thick snow the UWB signals bounce all over the place and you get severe multipath effects that can make it pretty difficult to determine which is the true line of sight signal for your time of flight calculations.
Fun job, I had to get certified to be able to walk the NYC train tracks because we installed a lot of the track side beacons ourselves, and I spent a lot of late nights doing live testing runs on trains that were empty other than a couple of people from our company and some MTA workers assigned to our test trains.