Fixing this household’s DNS problem

I’m very irritated at Netgear’s Orbi. I’m not sure if that irritation is directed to the right place, but it’s what I have.

For a long while now, our Android phones have had issues with various apps:

  • The Twitter app won’t load images or videos inline unless you let it sit for 30 seconds to a minute.
  • The Guardian app won’t show images unless you let it sit.
  • The Cronometer app starts with a blank screen for 30 seconds before fully loading.
  • The Google Play Store takes a Very Long Time to download and install new apps.

From what I can gather, there is some sort of DNS lookup that happens inside these apps that runs into some kind of issue with the Orbi’s built in dnsmasq service. I don’t see the issue on the laptop because I’m able to set my DNS servers to something that is not the router, an option not available on the phone.

I’ve tried to address this before, but I solved my most annoying pain point by switching to Twitter’s mobile website and deleting the app. The PWA doesn’t have any of the same issues loading images or videos. My other issues were apparently not annoying enough to make me do anything other than grumble.

Until today!

I received my replacement phone and went through the process of copying all of my data over. For some reason, Android likes to download all of the apps from the Google Play Store rather than just transferring them between phones. This left me with a phone that was completely setup except for 130 apps that needed installing.

After about an hour, I checked the phone and 17 apps had installed. Each one was apparently suffering from an issue I at this point assume must be related to the Orbi and its handling of DNS.

I grumbled louder. I dug out the trusty 8 year old Cisco (Linksys) E4200 that served us well for years before I replaced it with the Orbi to reach corners of the house the old one couldn’t. It has the same SSIDs assigned, so all I had to do was unplug the Orbi and plug in the Cisco.

My phone installed the remaining 123 apps in 30 minutes. 🤔

So now I definitely blame the Orbi. I have no idea why it would have such a problem, but it does. Normally I would order new electronics and replace the whole setup, but generating new electronic waste without good reason is a bummer.

My new plan: use the E4200 to provide the network—DNS, DHCP, routing—and the Orbi to provide the mesh as a pass-through access point. We have a weather monitor that has trouble as well, so I may even setup the 2.4ghz network on the Cisco and let the Orbi handle just the 5ghz.

Do you believe in magic? I think I might.

It took about 20 minutes, but I was able to juggle through a handful of setting SSIDs here and setting SSIDs there and made the swap. Now:

  • The Cisco is connected to the cable modem and serving DHCP and proper, good older router “here’s a DNS server to use”.
  • The 5ghz network on the Cisco is disabled.
  • The 2.4ghz network on the Cisco is enabled and the SSID I had previously assigned to the guest network on the Orbi is now there.
  • Our main SSID is broadcast by the Orbi and one satellite and is working as intended: 125mbps down in that far, far corner.
  • The guest network on the Orbi that I thought was on 2.4ghz before, but was really dual mixed whatever is now disabled. The side benefit of this is that our weather monitor (don’t buy a La Crosse) started working again! It’s been relying on a 2.4ghz network that apparently never properly existed.

All of this and all of our electronics through the house are just working. And everything on the phone is responding quickly as one would expect in the year 2020!

I feel like I performed open heart surgery on the house and succeeded. 😅

Time to go watch some basketball!

Strange Google Pixel DNS things

I’ve had this off and on issue for a while in which images and videos don’t load on Twitter with my Google Pixel 3 on both Android 9 and Android 10. If I wait long enough—maybe 60 seconds–the current timeline populates and things generally work.

This makes Twitter even more annoying than its baseline. I often use this as an excuse just to stop using Twitter for the hour or night.

Recently, I noticed that images in the Guardian app weren’t loading either. The full text of the articles would come in, but there were blank boxes everywhere. These aren’t always necessary for the story, but that they’re missing is a little distracting and makes me wonder what else isn’t working on the phone.

While I was out of the house and on LTE today, I noticed that images and videos in Twitter and the Guardian were loading fine. Something finally clicked in my head and I decided there must be some kind of funky problem with my network setup at home.

I’ve had the Cloudflare DNS servers configured on our router for quite a while and haven’t given them too much thought since they seem to work just fine. Today I made a temporary change to Google’s public 8.8.8.8 and 8.8.4.4 DNS servers to see if it made an impact. Sure enough, Twitter images and videos started loading immediately.

I then changed to OpenDNS’s public 208.67.222.222 and 208.67.220.220 DNS servers and the issue came back—no Twitter images or videos. Switched back to Google’s DNS and they started working.

The strange part at this point is that Guardian images still aren’t coming in.

So then I switched back to the DNS my router receives from the ISP and everything immediately started working. Imagine that!

This is great and all, but ISP DNS is generally disgusting. When I try to go to a domain that doesn’t exist, the request is automatically routed to a “non-existing domain landing service” that is branded by Spectrum, Time-Warner, and Charter. That there’s not just one company name associated shows how careful the friendly conglomerate is with branding (and customers’ data).

An opt-out link is at the bottom of the page, but it doesn’t actually work.

So what to do. None of this has any impact on our MacOS laptops. I should do some actual testing, but the DNS lookups still seem faster to 1.1.1.1 than anywhere else. I tried the Cloudflare 1.1.1.1 app on the phone to do DNS over TLS, but it has trouble with Twitter images and videos too. Android doesn’t appear to let you override the DNS, so all requests are made through the router. I’m guessing that ends up being part of the real issue.

I’m going to go with the Google DNS servers for now and see how it feels for the next few days. Then I guess start diving into Netgear Orbi forums to figure out how it could be screwing things up. 🤷🏼‍♂️

Update, 6:55pm November 24, 2019:

A couple issues combined here:

  • There should be a way to manually set DNS in Android, but that doesn’t appear to be available to me. For this reason, the DNS is set to 192.168.1.1 rather than any DNS server I configure on the Netgear Orbi.
  • It should be possible to tell the Netgear Orbi to pass DNS servers via DHCP other than its own IP. The Orbi has dnsmasq running, so it apparently assumes that will be the best option and now disables all others.

So now I need to figure out how to get the DNS actually set to the best provider on my phone without downloading one of those shady DNS apps.

Update, 7:11pm November 24, 2019:

From what I can tell, Android 10 no longer allows you to set custom DNS for a specific network. But! There’s a private DNS option available in the general network settings that allows you to configure DNS over TLS.

I first set this to 1dot1dot1dot1.cloudflare-dns.com, but the issue persisted. I then set it to dns.google and images started loading immediately in both Twitter and the Guardian. Videos are still a little funky, and I’ve found a few images that don’t load, but this may be a good short term fix.

The long term fix needs to be making the router not improperly handle DNS queries for the network.

Update, 7:29pm November 24, 2019:

I’m irritated enough at the router for running its own dnsmasq instance that I’ve now reached the hack or replace stage of router ownership.

Update, 8:09pm November 24, 2019:

I have shell access to the router and have updated the dhcpd configuration to send 8.8.8.8 as the DNS server to DHCP clients rather than the router’s IP.

This is excellent because it solves my irritation that the router was running dnsmasq poorly. This is less excellent because my actual issue with images is still there. 😂

Update, 8:13pm November 24, 2019:

There’s a chance that things are fixed? I’m not really sure, but I should switch to something else. I’ll add an update after a bit once I’ve actually had a chance to use it.

Various Networking Configurations in VVV

I dug in to some different configurations in VVV today and decided to write them up as I went. This will be posted in some form to the VVV wiki as well. There are other networking configurations available in Vagrant, though I’m not sure that any would be useful in development with VVV.

I would recommend using default settings for initial provisioning as things can get quirky inside the VM when trying to access outside sources. Run vagrant reload to process any network configuration changes.

Private Network (default)

config.vm.network :private_network, ip: “192.168.50.4”

This is the default configuration provided in VVV. A private network is created by VirtualBox between your host machine and the guest machine. The guest is assigned an IP address of 192.168.50.4 and your host machine is able to access it on that IP. VVV is configured to provide access to several default domains on this IP address so that browser requests from your host machine just work.

Outside access from other devices to this IP address is not available as the network interface is private to your machine.

Port Forwarding

config.vm.network “forwarded_port”, guest: 80, host: 8080

One option to provide other devices access to your guest machine is port forwarding. Uncommenting or adding this line in VVV’s Vagrantfile and then running vagrant reload will cause any traffic on port 8080 directed at your host machine to instead communicate with port 80 on the guest machine.

This configuration will work with private or public IP configurations as it deals with port forwarding rather than the IP of the virtual machine itself.

An immediate way to test this once configured would be to type your host machine’s IP address into a browser followed by :8080. With port forwarding enabled, something like http://192.168.1.119:8080 would bring up the default VVV dashboard.

Of course, this doesn’t do you much good with the default WordPress sites, as you’ll be stuck adding port 8080 to every request you make.

The easiest hack around this is to setup port forwarding on your router. Point incoming requests for port 80 to port 8080 on the IP address of your host machine. Requests through the router will then traverse ports 80 (public IP) -> 8080 (host) -> 80 (guest) and your development work can be shared with devices inside and outside of your network.

Say my router’s public IP is 14.15.16.17 and my computer’s local IP is 192.168.1.100.

  • Enable port forwarding in Vagrantfile.
  • Configure router to forward incoming port 80 to port 8080 on 192.168.1.100.
  • Visit src.wordpress-develop.14.15.16.17.xip.io on my phone, connected through LTE.

There are other things you can do on your local machine to reroute traffic from 80 to 8080 so that it forwards properly without the use of a router. Sal Ferrallelo has posted steps to take advantage of port forwarding directly in OSX using pfctl.

Public Network

config.vm.network “public_network”

Replacing our default private network configuration with a public network configuration immediately provides access to other devices on your local network. Using this configuration without specifying an IP address causes the guest machine to request an address dynamically from an available DHCP server—likely your router. During vagrant up, an option may be presented to choose which interface should be bridged. I chose my AirPort interface as that is what my local machine is using.

==> default: Available bridged network interfaces:
1) en0: Wi-Fi (AirPort)
2) en1: Thunderbolt 1
3) p2p0
4) bridge0
5) vnic0
6) vnic1
    default: What interface should the network bridge to? 1

Once the guest machine receives an IP address, access is immediately available to other devices on the network.

  • vagrant ssh and type ifconfig to determine the IP address of the guest – mine was 192.168.1.141.
  • Visit src.wordpress-develop.192.168.1.141.xip.io on my phone, connected to the wireless network.

To me this is most desirable as it provides access to devices on the local network, not to the outside. If you are using public wifi or another insecure network, be aware–this does open your machine up to other devices on that network.

config.vm.network “public_network”, ip: “192.168.1.141”

The same configuration would be available without DHCP by specifying the IP address to use. If you know what subnet your network is on, this may be a shortcut for providing access without having to use ifconfig inside the guest machine.