Updates from October, 2014 Toggle Comment Threads | Keyboard Shortcuts

  • Urban 20:50 on 31 Oct. 2014 Permalink |  

    Smart thermostat shenanigans 

    Winter’s coming and this year I set my mind on installing a smart thermostat. I’m used to unexpected problems with tech, so I wanted to have it set up before it actually gets sufficiently cold outside that experimentation will cause me discomfort. Which means it’s now or never.

    I briefly looked into 3 options:

    • Nest, the epitome of a smart thermostat (acquired by Google)
    • Netatmo Thermostat, made by the French Netatmo, most known for its weather stations
    • Tado Smart Thermostat, made by Tado, a german HVAC startup
    • I also checked a bunch of other (locally controlled) options, of which nothing captured my attention

    To distill some of my findings (please note I’m doing this from an EU perspective):

    • Nest
      • [+] definitely tries to be smart; uses a motion sensor and a learning algorithm to find patterns in your away-from-home time
      • [+] the API looks pretty comprehensive & well done
      • [-] but it’s bulky and heavy
      • [-] and optimized for US heating standards & boiler interfaces (runs on 24V AC)
    • Tado
      • [+] supposed to work with most EU heaters
      • [+] the app reports distance from home to a server; when you get closet to home, it gradually increases the temperature (pretty clever)
      • [+] also supports plain old daily schedules for people without smartphones
      • [-] no activity detection
      • [-] no display or buttons, entirely app-controlled
      • [-] the most expensive of the bunch, but rental option available [+]1
      • [-] no API (none that I could find)
    • Netatmo
      • [+] supposed to work with most EU heaters
      • [+] relatively cheap
      • [+] nice design
      • [+] has an API
      • [-] the dumbest of the bunch (no sensors, no AI)
      • [-] just a web-based and app-basaed heating schedule editor

    I found out pretty quickly that Nest won’t cut it with my heating system. I don’t have 24V AC on my heater, and no AC power near my old thermostat mount location. In addition, the device itself felt pretty heavy/bulky. Last but not least, it’s Google-owned, so all the data about your physical presence and heating schedules will reside on the Google’s servers.

    So it was in fact a run-down between Tado and Netatmo.

    Tado

    I liked Tado’s approach, but didn’t like the design of the device: no display, just a single “emergency” button to let it know that you’re home (e.g., if you happen to forget your phone at work). Monitoring the GPS position of all your family members to determine when to turn on heating is pretty clever and probably brings the largest possible amount of savings: even if you deviate from schedule, Tado still knows when you’re coming home.

    However, I can’t help but think about what can go wrong if you let your heating be controlled by a stack of unreliable technologies:

    • you have to have the phone on you (if you forget it at home, heating stays on; if you forget it at work, it stays off; nonetheless, that’s a pretty minor issue, since I rarely forget my phone)
    • but my phone battery dies relatively often; what then?
    • or what if you don’t have connectivity
    • or if the app’s GPS logging fails (i’ve experienced a lot of this with GPS logger apps, at least on iOS)

    On the plus side, Tado seemed to have some smarts regarding taking into account the outside temperatures, pattern detection (not much info online there), and it did specifically support my 3-pin (7-8-9) terminal of the Vaillant heater (which I saw in the installation video, no other documentation found).

    Netatmo

    Netatmo looked quite nice, but the situation regarding available information online was even worse than with Tado. It seemed the dumbest of the three. The site shows nothing but HLB marketing, without any technical details. There’s no info about the features,  about what happens if you lose WiFi connectivity, or if it integrates with the Weather station. They do have a demo GUI that allowed me to play with a fake thermostat to get a feeling about what’s possible. I also had to dig deep to find it has an API that allows you to add some brains on top of it with your own app. And the fact that it took the original Netatmo Weather Station website quite some time to switch from HTTP to HTTPS was not really demonstrating the security savviness you’d like to see, if you want to let other people control your heating.

    However, in the end I did order this one–primarily based on the low price, appearance, and *claims* that it supports my heater brand (there was just a logo on the website). Not much info online here, just the video installation instructions that show it has a simple 2-terminal on-off switch, which I thought could be somehow connected to my Vaillant heater 7-8-9 terminal.

    Netatmo installation

    The first surprise was that I couldn’t get it to work in place of my old room thermostat. As it turns out, Vaillant 7-8-9 terminal is a potentiometer that gradually changes the voltage from about 11V (off) to 21V DC (full throttle) to regulate heating. This terminal cannot be easily linked to a single-pole switch. Arguably it could be connected to a 2 pole switch to have either off or full throttle.

    Luckily, the package comes with everything necessary for another mode of installation: It can be connected as a relay directly to the heater.

    In fact, the wifi relay comes with 2 backplates:

    • one is a socket plug, to connect it directly to mains. In this case, the “relay” acts only as a WiFi-to-radio relay and sends commands to the room thermostat module. In this case, the battery-powered room thermostat turns heating on and off.
    • the other one is a 4-wire connector that can be connected directly to the heater. Two wires (240v L+N, blue and brown) are connected to mains power (inside your heater), and the other 2 wires (black and grey) are connected to the relay connector of your heater.

    My heater (as probably every other Vaillant, I’ve checked a lot of online manuals) has a 240V relay connector on pins 3 and 4; they are normally bridged with a wire, which means that this 3-4 “switch” is always on, and heating is controlled through a room thermostat on 7-8-9. However, if you disconnect your room thermostat (i.e., all 7-8-9 wires are in the air, which means ON), the heating is controlled only through 3-4.

    vaillant3
    Bottom line, I connected Netatmo relay adapter directly to the 3-4 terminal, and the power wires (240V) to L+N (both inside the heater). I also left my old room thermostat in place. This means both need to be active for the heating to turn on (logic AND). In my case this is easily solvable, because my old room thermostat can be set to “always on”, and this would mean the heating is controlled by Netatmo alone. But in addition, it gives me some peace of mind, because I can limit max temperature on the existing room thermostat as well, and prevent Netatmo from going berserk, turning on and heating the house to 30 degrees. (hey, it’s a cloud-driven thermostat, anything could happen)

    Post-install and first days

    I was pleasantly surprised by some of the features that are not mentioned anywhere on the web (in Netatmo’s defense, they do have a forum that I only found later: http://forum.netatmo.com/):

    • It does work when WiFi’s down; it just follows the last schedule it knows
    • It does integrate with your weather station; it fetches the temperature from the outside module of the same account and uses it for plotting charts and doing predictive heating
    • It *can* have the polarity reversed, which means I could even connect it to the room thermostat post (off = all wires in air = full throttle; on = bridged control wire and low voltage)
    • It has 2 modes for heating: hysteresis and PID; PID means it learns the time constant of your home, takes into account the outside temperature and starts heating earlier or later accordingly

    Below are some screenshots from the web app Q&A that are not accessible from the demo web app linked above.

    2b6d6eda394b7aa134d663fc54e18d2c ce022b5f046edfa67dda51c5ed592af4b7a44523628e9746aca1a44de7eb9f16   6daef66bdeb8bb3fca9f6ead8bb9ec0d1e7d9f5de56e194e34f1985dfef560a5

    1. It seems Tado just dropped the price by 50 EUR []
     
    • Julia 13:21 on 8 Jan. 2016 Permalink

      has anyone had any trouble with it? Mine doesnt seem to want to control the temperature. my heating on my other box has to be on all the time, but even when my thermostat says its off, it still heats my home to somewhere between 20 and 25. I thought I had it working the other day but it seemed to turn the heating on when i set the desired temperature UNDER the current temp, and off when desired temp is over current….so far its costing me money not saving it!

    • Urban 15:13 on 8 Jan. 2016 Permalink

      Hey!
      From what you say, it seems that your heater uses reverse logic (meaning that you have to open the circuit to turn on the heating). Try changing the “polarity” under settings (in your app, go to settings -> your thermostat (at the top) -> Advanced settings (at the bottom) -> Polarity –> set to Reversed). If this is the problem, it should fix it.

    • Shambolic 12:06 on 11 Feb. 2016 Permalink

      I’m having a problem with the thermostat not turning the boiler on first thing in the morning. My current workaround is to have it switch on and off twice in the morning, trouble is sometimes it catches on the first one and therefore comes on a lot earlier than I would like. have it currently in the place of my old thermostat, with the relay just plugged into a mains socket. Maybe if it was hard wired into the boiler it would sort things?

    • Urban 23:26 on 11 Feb. 2016 Permalink

      Hmm, that’s hard to say.. I’m assuming the temperatures check out ok, with the thermostat actually measuring a lower temp than you’ve set? (If not, you can set the offset under advanced settings).
      Maybe there could also be a connectivity problem between the relay and the unit? To check this you can try changing the temperature in the app manually, and if that reliably works, this also can’t be the problem.
      Other than that, I imagine something like the heating prediction could mess things up (this tries to be smart and take into account the outside temperature, which could be wrong). You can try turning this off if you use it..
      Lastly, hard wiring it to the boiler might help, in case there’s some kind of hardware problem in the thermostat’s circuitry..

    • Shambolic 12:34 on 12 Feb. 2016 Permalink

      It’s a bit of a mystery. Temperature is fine, it works manually and after the first turn on the timed setting are working but it just misses that first timed switch on. I’ve switched off the predictive feature, no change. All that’s left to try is hard wiring to the boiler.

    • Enis Erkul 17:41 on 21 Jul. 2017 Permalink

      Hello! I know its a little bit old entry but I bought a Netatmo Thermostat. Now I want to replace my old Vaillant VRT40 thermostat with the new Netatmo. I couldn’t really understand from your post, am I able somehow connect Netatmo using cables 7-8-9 from the old thermostat? Maybe I can boiler, I can swap cables from 7-8-9 to 3-4? I dont really want to connetct netatmo through a boiler. Are there any chances I could use 7-8-9 cables?
      Thank you advance!

    • Enis Erkul 18:52 on 21 Jul. 2017 Permalink

      Replying for my own question 🙂 Netatmo Forum, Official support guy answer:
      You will just respect this process:

      disconnect your current thermostat from the 3 wires of the wall.
      disconnect the 3 wires of your boiler (they should be connected on ports 7, 8 and 9 of the boiler).
      remove the bridge between the ports 3 and 4 on your boiler.

      To use it with existing wall-wires:

      connect 2 of the 3 wires you have disconnected on the first step on the ports 3 and 4 (no matter the cables you pick, there is no polarity), and insulate the third one,
      connect the Thermostat on these 2 wires on your wall.

    • Urban 19:06 on 21 Jul. 2017 Permalink

      Hey! This is just talking from memory, but I think the 7-8-9 is the low voltage connector, meant to be used with the battery-powered Netatmo room thermostat module. 3-4 is high voltage (240V), meant to be used with the separate 240V relay module, which is also a bridge between WiFi and the room module. (At least I have these 2 modules, I’m not sure if they still make it this way).

      In any case, if this is from the support guy it’s probably right 🙂

    • Scott Pattinson 10:01 on 23 Nov. 2017 Permalink

      Vaillants own product Vsmart is based on netatmo. The difference is that their product uses the 7 8 9 terminals so the thermostat can modulate the boiler according to need and outside temp. this helps to maintain the temp at s constant value the boiler can run at a low setting which must be better for comfort and boiler life. Using the terminal 3 and 4 you effictively are only able to have the boiler off or running full blast.

    • Peter 22:55 on 13 Dec. 2017 Permalink

      It seems Vsmart is for newer Vaillant boilers, not the old analog 7 8 9.

  • Urban 19:16 on 1 Oct. 2014 Permalink |  

    If you want something new, you have to stop doing something old

    Peter Drucker
     
  • Urban 02:35 on 1 Dec. 2012 Permalink |  

    625 days of walking 

    I’ve had my Fitbit for over 625 days. It records walking activity with 5 minute resolution and I’ve carried it around almost everywhere.

    So here’s my past 1.7 years of walking in one picture (click to enlarge).

    Every column represents five minutes (288 columns from 00:00 to 24:00) and every line represents a single day (625 lines from March 15 2011 to November 28 2012).

    Green means no activity and red means high activity (highest intensity red is 706 steps in a 5 minute interval).

    What’s immediately visible is when I wind down and go to bed. Then there’s some displacements where I was obviously traveling in a different time zone.

    There’s also a trend that I’m not fond of: I seem to be rising earlier and earlier, since the active (yellow/red) band is moving slightly to the left as time progresses.

    A quick note on how this was compiled: the data was extracted from the Fitbit website, using the undocumented API that fuels the flash charts (the cookie that serves as auth was copied from the browser’s inspector into the script). I looped through the entire date range and saved every day into a separate XML file. Another quick and dirty script combed through the files with a regex and concatenated all 5-minute intervals in a day into a single comma separated line. The resulting CSV file was then imported into Excel where simple normalization was done, as well as some color coding using conditional formatting. Yes, this is extremely lame, but it was all finished before I could even decide what other tool to use.

     

     

     
  • Urban 09:00 on 14 Nov. 2012 Permalink |  

    Smart bulbs (and other musings) 

    As a gadget enthusiast I instinctively clicked “Back this” when I saw the Lifx project on Kickstarter. I was torn, however, when I saw the public outcry regarding the founder and his alleged incapability to ship a cardboard box. I hesitated until the last day, not sure whether to keep the pledge or cancel it.

    Meanwhile, I happened to stumble upon another similar project at the Mini Maker faire at Strataconf NY–the Visualight. It instantly caught my eye and one of the founders explained to me how he was just finishing the writeup when the Lifx project came online.

    So I said, “convince me that you’re better and I’ll cancel the Lifx pledge and back yours instead.”

    He did give me a pitch with plenty of differentiation, saying that Visualight is a great data visualization tool which sports open APIs for all the communication. Who needs disco effects and music visualization, when you can have the light change color according to weather, stocks or subway service (kind of like the Nabaztag / Karotz).  He also showed me a working prototype.

    But right there, I couldn’t decide which one had a better premise. It all boiled down to the “smart bulb, stupid network” (Lifx) vs. “smart network1, stupid bulb” (Visualight) dilemma, and I got an instant case of analysis paralysis.

    I started thinking that I’ve seen the story many times before.

    For example, in computers.

    In computing we started off with a centralized design (mainframes) and dumb terminals. Then the brain moved to the local box (PC), and now, finally, it’s moving back to the network (cloud), with the clients getting more and more stupid once again (just take a look at Chromebook).

    Something similar seems to be happening in mobile phones, with the brain first moving from the network to your iPhone, and now slowly creeping back into the datacenter (Siri, anyone? Or maps with server-computed turn-by-turn?)

    But right now the infrastructure is not quite there yet. It’s not infallible and 99.999% robust, and it pisses us off when Siri can’t take a simple note. Imagine you can’t turn on your light at 2AM because your server’s down.

    So that’s what I was thinking while standing there, staring blankly into empty space. I decided that (at least my) world might not be ready for a remote controlled stupid bulb.. yet.

    And a couple of days later, Philips announced the Hue. It’s severely limited (iOS only, and the bulbs are not self-sufficient; it uses an additional ethernet-connected gateway which communicates with bulbs via ZigBee). However, with its market cap, lighting expertise, reputation and virtually the same price point, Philips might have just eaten the lunch of every other lighting startup.

    Then there’s another issue where Philips wins: safety. A product like that, done wrong, can easily burn down your house. I’ve already seen the remains of an exploding Chinese USB charger, and this is indeed a great concern. Compared with cheap Chinese LED bulbs that I bought en masse years ago, such a smart bulb has to be always on to benefit from its embedded computer. If you switch it off, it’s dead.

    So we’ll have to wait and see who’s going to be the winner here. The race is long. In fact, it’s never-ending.

    And no, I haven’t cancelled the pledge.

     

     

    1. and here, by “smart network” I mean “a server” []
     
  • Urban 20:00 on 13 Jun. 2012 Permalink |  

    Backing up large zpools 

    I’ve just recently heard the term “data gravity”, which implies that large amounts of data are next to impossible to move. I’ve experienced this with my ZFS NAS, which makes heavy use of compression and deduplication to cram tons of backup VMWare images to a 2TB mirrored volume.

    To back it up I simply did what seems to be best practice around the Internets: I attached a USB volume, used zfs send to synchronize only the changes between the most recent common snapshots, and that was it. However, the process took a long time.

    Fast forward a couple of months, when I naively played with the thing and tried virtualizing it (VMware + raw device map), in the process corrupting the zpool so badly, that it had to be restored from backup in its entirety.

    If I thought syncing the changes to USB took long (days), this took weeks. That’s when I learned about mbuffer to speed up zfs sync, but dedup and compression still took their toll. Plain uncompressed and undeduped file systems synced with normal USB read speeds (about 15MB/s), while dedup slowed zfs send down by two orders of magnitude.

    So I’ve resorted to a little advertised approach, which has so far also proved to be the most fool-proof. It’s using the zpool resilvering mechanism for what’s called a split mirror backup.

    It goes like this: you create a 3-way mirror. Pull out one disk and shelf it as backup. When you want to “refresh” your backup, plug it back in and put it online using something like zpool online tank c5t0d0.

    A great surprise was how smart the resilvering process is. It doesn’t rewrite the entire disk, but only copies the changes. There’s a lot of advantages over zfs sending, at least from my point of view:

    • it’s fast, without the need to fiddle with mbuffer or copy each filesystem individually
    • no snapshots needed to serve as reference for incremental zfs send
    • creates an exact copy (entire zpool, with all the properties intact), so if your server burns, the backup drive can immediately serve as a seed for a new server
    • if you pull out a disk during resilvering, there seems to be no harm and the process just continues next time

    The only drawback is that your pool is always in a degraded state, because a disk is missing. But that’s really a minor inconvenience.

     

     
    • Damian Wojsław 09:38 on 15 Jun. 2012 Permalink

      Hi
      I’d recommend not using deduplication, as it can get you in a lot of trouble if not used carefully. What is often suggested solution is to create a separate zfs filesystem for golden image of a virtual machine and then provision creating zfs clones. You have a kind of a cheap deduplication, without the additional trouble of DDT.

    • Urban 12:03 on 16 Jun. 2012 Permalink

      Hi, thanks for your comment 🙂

      I’m more and more aware of the troubles of dedup, and have already disabled it on most filesystems where the performance penalty outweighs the benefits. 

      However, I have a single zfs serving as destination for remote backups of vmware images; since the data is highly redundant, the dedup factor is large; there’s also not that much unique data, so the ddt can easily fit in ram. Last but not least, the performance of this archiving process is absolutely *not* critical, and if everything blows up, it’s just backups 🙂

      But I do have important data on other filesystems in the pool and I still want to back up the entire thing to an external disk. Do you see any potential problems with the split mirror backup I described? What mechanism do you use?

    • Collin C. MacMillan 04:41 on 8 Jul. 2012 Permalink

      Beware this form of backup as it will not survive disk corruption (i.e. non-recoverable hard or soft errors) on the “backup” disk. It is also no good for more complex pool configurations… 

    • Urban 15:07 on 8 Jul. 2012 Permalink

      Thanks for your comment,

      actually, I agree on both counts. I’m aware it only applies to a very narrow set of mirrored pool configurations, and I’m also aware this is no enterprise-grade solution. However, for a home server it provides a quickfix solution without the hassle and cost of remote replication or a tape unit.

      And on the plus side, soft errors will at least be detectable..

    • Asdf 23:04 on 10 Oct. 2012 Permalink

      Instead of dedup, you can put one VM in a separate zfs filesystem, and then you clone it (via snapshot). Each clone will read from the master zfs filesystem but store changes on it’s own filesystem. This way you can have many clones of one VM. For instance, setup Windows2012 server as a master. In one clone you install SAP, in another you install Oracle DB, etc. So all clones will read from the win2012 master snapshot.

      Kind of dedup.

c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel